2024-06-15T00:47:11.145 INFO:root:teuthology version: 0.0.1.dev316+gc1b06c2 2024-06-15T00:47:11.145 DEBUG:teuthology.run:Teuthology command: teuthology --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143 --verbose --name yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/orig.config.yaml 2024-06-15T00:47:11.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-15T00:47:12.067 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143 branch: wip-yuri5-testing-2024-06-08-0902-reef description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: yweinste@redhat.com first_in_suite: false job_id: '7756143' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri5-testing-2024-06-08-0902-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-reef-drop-centos8 sha1: e3daf20513a97cf7004cd9e7bfdba8f5a015acbc owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 302 sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: wip-reef-drop-centos8 suite_path: /home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa suite_relpath: qa suite_repo: https://github.com/cbodley/ceph.git suite_sha1: e3daf20513a97cf7004cd9e7bfdba8f5a015acbc targets: smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/GCLsAA5D+mFCyRJAgW6nVLrhO3uVeg0QAIQJ3M8sYnFEKlxcAP2ttddj0uEpocIS8qKjJE6QaSUJB4lzZisI= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCCRNGIpTAah/q15cR1CqzPeLTpxMY0Mj6lR4zvOp6D0Oid36dxL3wDbbcIQ3p0hgjZ7BXhn4/dNPJ0EvT+t3Q= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: c1b06c2b5d3a02a3583265a115f09e052855bcf2 timestamp: 2024-06-14_18:29:32 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-06-15T00:47:12.067 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa; will attempt to use it 2024-06-15T00:47:12.068 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks 2024-06-15T00:47:12.068 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-15T00:47:12.069 INFO:teuthology.task.internal:Checking packages... 2024-06-15T00:47:12.089 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6f4b09560ed399c3f139402a02f389c2fc8bb2cc' 2024-06-15T00:47:12.089 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-15T00:47:12.089 INFO:teuthology.packaging:ref: None 2024-06-15T00:47:12.089 INFO:teuthology.packaging:tag: None 2024-06-15T00:47:12.089 INFO:teuthology.packaging:branch: wip-yuri5-testing-2024-06-08-0902-reef 2024-06-15T00:47:12.089 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:47:12.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri5-testing-2024-06-08-0902-reef 2024-06-15T00:47:12.312 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1443.g6f4b0956 2024-06-15T00:47:12.313 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-15T00:47:12.320 INFO:teuthology.task.internal:no buildpackages task found 2024-06-15T00:47:12.320 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-15T00:47:12.327 INFO:teuthology.task.internal:Saving configuration 2024-06-15T00:47:12.347 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-15T00:47:12.353 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-15T00:47:12.375 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-15 00:42:11.364477', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/GCLsAA5D+mFCyRJAgW6nVLrhO3uVeg0QAIQJ3M8sYnFEKlxcAP2ttddj0uEpocIS8qKjJE6QaSUJB4lzZisI='} 2024-06-15T00:47:12.408 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-15 00:42:11.365920', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCCRNGIpTAah/q15cR1CqzPeLTpxMY0Mj6lR4zvOp6D0Oid36dxL3wDbbcIQ3p0hgjZ7BXhn4/dNPJ0EvT+t3Q='} 2024-06-15T00:47:12.408 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-15T00:47:12.415 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-06-15T00:47:12.416 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-06-15T00:47:12.416 INFO:teuthology.run_tasks:Running task console_log... 2024-06-15T00:47:12.471 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9bea9bbbe0>, signals=[15]) 2024-06-15T00:47:12.471 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-15T00:47:12.477 INFO:teuthology.task.internal:Opening connections... 2024-06-15T00:47:12.478 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T00:47:12.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:47:12.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-06-15T00:47:12.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:47:12.635 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-15T00:47:12.643 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-06-15T00:47:12.660 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-06-15T00:47:12.660 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-06-15T00:47:12.717 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-06-15T00:47:12.717 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-06-15T00:47:12.717 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-06-15T00:47:12.717 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-06-15T00:47:12.717 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-15T00:47:12.718 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-15T00:47:12.719 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-06-15T00:47:12.736 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-06-15T00:47:12.754 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-06-15T00:47:12.754 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-15T00:47:12.811 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-15T00:47:12.812 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-15T00:47:12.812 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-06-15T00:47:12.829 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-15T00:47:12.838 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-15T00:47:12.844 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-15T00:47:12.844 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-06-15T00:47:12.846 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-06-15T00:47:12.867 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-15T00:47:12.873 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-15T00:47:12.873 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-06-15T00:47:12.902 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-06-15T00:47:12.979 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-15T00:47:13.019 INFO:teuthology.run_tasks:Running task kernel... 2024-06-15T00:47:13.032 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-15T00:47:13.032 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-15T00:47:13.032 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-15T00:47:13.033 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-15T00:47:13.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T00:47:13.049 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-06-15T00:47:13.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T00:47:13.050 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-06-15T00:47:13.105 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-437.el9.x86_64 2024-06-15T00:47:13.106 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.14.0-437.el9.x86_64 2024-06-15T00:47:13.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum install -y kernel 2024-06-15T00:47:13.107 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-437.el9.x86_64 2024-06-15T00:47:13.107 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-437.el9.x86_64 2024-06-15T00:47:13.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-06-15T00:47:16.477 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.1 MB 00:01 2024-06-15T00:47:16.857 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-06-15T00:47:18.578 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:00 2024-06-15T00:47:18.641 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-06-15T00:47:22.833 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.3 MB 00:00 2024-06-15T00:47:23.781 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.3 MB 00:01 2024-06-15T00:47:24.521 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 17 kB 00:00 2024-06-15T00:47:25.254 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 17 kB 00:00 2024-06-15T00:47:26.638 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-06-15T00:47:30.194 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-06-15T00:47:31.927 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-15T00:47:33.991 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-15T00:47:33.992 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-15T00:47:34.099 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: kernel x86_64 5.14.0-457.el9 baseos 989 k 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core x86_64 5.14.0-457.el9 baseos 16 M 2024-06-15T00:47:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules x86_64 5.14.0-457.el9 baseos 35 M 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core x86_64 5.14.0-457.el9 baseos 29 M 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 81 M 2024-06-15T00:47:34.112 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 124 M 2024-06-15T00:47:34.113 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:47:34.429 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): kernel-5.14.0-457.el9.x86_64.rpm 3.5 MB/s | 989 kB 00:00 2024-06-15T00:47:35.204 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): kernel-core-5.14.0-457.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-06-15T00:47:35.405 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-06-15T00:47:35.504 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): kernel-modules-5.14.0-457.el9.x86_64.rpm 26 MB/s | 35 MB 00:01 2024-06-15T00:47:35.821 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): kernel-modules-core-5.14.0-457.el9.x86_6 21 MB/s | 29 MB 00:01 2024-06-15T00:47:35.821 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:47:35.821 INFO:teuthology.orchestra.run.smithi142.stdout:Total 47 MB/s | 81 MB 00:01 2024-06-15T00:47:36.523 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:47:36.864 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:47:36.864 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:47:37.461 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-15T00:47:37.461 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-15T00:47:37.562 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:47:37.571 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:47:37.571 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:47:37.573 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:47:37.573 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-457.el9 baseos 989 k 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-457.el9 baseos 16 M 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-457.el9 baseos 35 M 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-457.el9 baseos 29 M 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 81 M 2024-06-15T00:47:37.574 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 124 M 2024-06-15T00:47:37.575 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:47:37.825 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:47:37.826 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:47:37.981 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-457.el9.x86_64.rpm 3.7 MB/s | 989 kB 00:00 2024-06-15T00:47:38.782 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-457.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-06-15T00:47:39.482 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-core-5.14.0-457.el9.x86_6 19 MB/s | 29 MB 00:01 2024-06-15T00:47:39.798 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-5.14.0-457.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-06-15T00:47:39.799 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:47:39.799 INFO:teuthology.orchestra.run.smithi137.stdout:Total 36 MB/s | 81 MB 00:02 2024-06-15T00:47:40.234 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:47:40.499 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:47:40.829 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:47:40.830 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:47:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-core-5.14.0-457.el9.x86_64 1/4 2024-06-15T00:47:41.005 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:47:41.775 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:47:41.775 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:47:43.649 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:47:43.824 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:47:44.185 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:47:44.818 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-457.el9.x86_64 1/4 2024-06-15T00:47:45.007 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:47:47.659 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:47:47.802 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:47:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:47:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:47:52.557 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:47:52.616 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:47:53.336 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:47:57.384 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:23.126 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:23.126 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-5.14.0-457.el9.x86_64 1/4 2024-06-15T00:48:23.126 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:48:23.126 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core-5.14.0-457.el9.x86_64 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-5.14.0-457.el9.x86_64 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core-5.14.0-457.el9.x86_64 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:48:23.887 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:48:24.299 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-06-15T00:48:24.383 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:24.383 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-15T00:48:24.383 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-457.el9.x86_64 2024-06-15T00:48:24.383 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-15T00:48:24.383 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-15T00:48:24.384 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-06-15T00:48:24.384 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-06-15T00:48:24.384 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-06-15T00:48:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:24.436 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-15T00:48:24.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-06-15T00:48:24.466 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-06-15T00:48:24.466 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-06-15T00:48:24.466 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-06-15T00:48:24.467 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:48:24.468 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-06-15T00:48:24.469 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-15T00:48:24.469 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 5.14.0-457.el9.x86_64 2024-06-15T00:48:24.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-15T00:48:25.080 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-06-15T00:48:26.488 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-15T00:48:26.508 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-06-15T00:48:26.510 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-06-15T00:48:26.542 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.zvycTPqJvw 2024-06-15T00:48:26.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zvycTPqJvw 2024-06-15T00:48:26.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.zvycTPqJvw 2024-06-15T00:48:26.739 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.zvycTPqJvw is 6KB 2024-06-15T00:48:26.788 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.zvycTPqJvw 2024-06-15T00:48:26.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-06-15T00:48:26.864 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-15T00:48:26.864 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-15T00:48:26.864 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-15T00:48:26.864 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64.conf 2024-06-15T00:48:26.865 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64 2024-06-15T00:48:26.886 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:27.015 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-06-15T00:48:27.230 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:27.230 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-457.el9.x86_64 1/4 2024-06-15T00:48:27.230 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-15T00:48:27.230 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-15T00:48:28.000 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-15T00:48:28.000 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:48:28.000 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:48:28.001 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:48:28.350 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-06-15T00:48:28.420 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.420 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-15T00:48:28.421 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-457.el9.x86_64 2024-06-15T00:48:28.421 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-15T00:48:28.421 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-15T00:48:28.421 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-06-15T00:48:28.421 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-06-15T00:48:28.421 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-06-15T00:48:28.473 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-15T00:48:28.474 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-15T00:48:28.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-06-15T00:48:28.504 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 68.el9 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8264254 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-06-15T00:48:28.505 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:48:28.506 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-06-15T00:48:28.507 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-15T00:48:28.507 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-457.el9.x86_64 2024-06-15T00:48:28.507 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-15T00:48:29.163 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-06-15T00:48:30.640 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-15T00:48:30.667 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-06-15T00:48:30.669 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-06-15T00:48:30.686 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.BEE4f8INrD 2024-06-15T00:48:30.686 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BEE4f8INrD 2024-06-15T00:48:30.752 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.BEE4f8INrD 2024-06-15T00:48:30.890 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.BEE4f8INrD is 6KB 2024-06-15T00:48:30.939 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.BEE4f8INrD 2024-06-15T00:48:30.953 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-06-15T00:48:31.014 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-15T00:48:31.014 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-15T00:48:31.014 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-15T00:48:31.014 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64.conf 2024-06-15T00:48:31.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64 2024-06-15T00:48:31.166 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-06-15T00:48:57.043 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-06-15T00:48:57.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:48:57.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:01.173 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-06-15T00:49:01.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:49:01.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:15.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-06-15T00:49:19.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-06-15T00:49:24.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:49:24.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:27.618 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-06-15T00:49:28.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:49:28.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:31.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-06-15T00:49:39.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:49:39.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:43.716 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:49:43.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:49:58.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-06-15T00:50:02.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-06-15T00:50:13.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:50:13.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:50:13.473 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-06-15T00:50:13.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:50:13.862 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-457.el9.x86_64"... 2024-06-15T00:50:13.863 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-06-15T00:50:13.878 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-457.el9.x86_64 2024-06-15T00:50:13.878 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-457.el9.x86_64 vs 5.14.0-457.el9.x86_64 2024-06-15T00:50:13.878 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-15T00:50:13.878 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-15T00:50:14.879 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-06-15T00:50:14.879 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-15T00:50:14.951 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-06-15T00:50:15.003 DEBUG:teuthology.parallel:result is None 2024-06-15T00:50:17.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:50:17.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:50:17.470 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-06-15T00:50:17.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:50:17.817 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-457.el9.x86_64"... 2024-06-15T00:50:17.817 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-06-15T00:50:17.872 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-457.el9.x86_64 2024-06-15T00:50:17.872 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-457.el9.x86_64 vs 5.14.0-457.el9.x86_64 2024-06-15T00:50:17.872 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-15T00:50:17.873 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-15T00:50:18.874 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-06-15T00:50:18.874 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-15T00:50:18.957 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-06-15T00:50:19.008 DEBUG:teuthology.parallel:result is None 2024-06-15T00:50:19.008 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-15T00:50:19.017 INFO:teuthology.task.internal:Creating test directory... 2024-06-15T00:50:19.017 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-15T00:50:19.019 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-15T00:50:19.038 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-15T00:50:19.049 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-15T00:50:19.066 INFO:teuthology.task.internal:Creating archive directory... 2024-06-15T00:50:19.066 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-15T00:50:19.078 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-15T00:50:19.127 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-15T00:50:19.133 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-15T00:50:19.133 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-15T00:50:19.137 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-15T00:50:19.176 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-15T00:50:19.186 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-15T00:50:19.193 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-15T00:50:19.205 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-15T00:50:19.206 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-15T00:50:19.213 INFO:teuthology.task.internal:Configuring sudo... 2024-06-15T00:50:19.213 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-15T00:50:19.234 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-15T00:50:19.275 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-15T00:50:19.286 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-15T00:50:19.287 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-15T00:50:19.303 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-15T00:50:19.331 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-15T00:50:19.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-15T00:50:19.450 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-15T00:50:19.529 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-15T00:50:19.578 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:50:19.579 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-15T00:50:19.646 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-15T00:50:19.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-15T00:50:19.744 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-15T00:50:19.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-15T00:50:19.872 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:50:19.872 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-15T00:50:19.939 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-06-15T00:50:19.941 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-06-15T00:50:19.996 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-15T00:50:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-15T00:50:20.541 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-15T00:50:20.550 INFO:teuthology.task.internal:Starting timer... 2024-06-15T00:50:20.550 INFO:teuthology.run_tasks:Running task pcp... 2024-06-15T00:50:20.562 INFO:teuthology.run_tasks:Running task selinux... 2024-06-15T00:50:20.571 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-06-15T00:50:20.572 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-06-15T00:50:20.662 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-06-15T00:50:20.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-06-15T00:50:20.710 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-06-15T00:50:20.712 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-15T00:50:20.712 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-06-15T00:50:20.753 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-06-15T00:50:20.753 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-06-15T00:50:20.790 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-06-15T00:50:20.790 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-06-15T00:50:20.790 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-15T00:50:20.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T00:50:20.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-15T00:50:20.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T00:50:20.859 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-15T00:50:20.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-06-15T00:50:20.895 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-06-15T00:50:20.926 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-15T00:50:20.935 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-15T00:50:20.938 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-15T00:50:21.063 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-15T00:50:21.074 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-15T00:50:21.075 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi137.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-15T00:54:45.730 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-06-15T00:54:45.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:54:45.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:54:45.807 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-06-15T00:54:45.892 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-06-15T00:54:45.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:54:45.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-15T00:54:45.972 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-06-15T00:54:46.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-06-15T00:54:46.046 INFO:teuthology.run_tasks:Running task clock... 2024-06-15T00:54:46.057 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-15T00:54:46.057 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-15T00:54:46.058 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-15T00:54:46.060 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-15T00:54:46.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-15T00:54:46.087 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-15T00:54:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-15T00:54:46.116 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-15T00:54:46.124 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-06-15T00:54:46.130 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-15T00:54:46.135 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-06-15T00:54:46.150 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-15T00:54:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-06-15T00:54:46.163 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-15T00:54:46.165 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-06-15T00:54:46.179 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-15T00:54:46.193 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-15T00:54:46.223 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.227 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.259 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.262 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T00:54:46.263 INFO:teuthology.run_tasks:Running task install... 2024-06-15T00:54:46.271 DEBUG:teuthology.task.install:project ceph 2024-06-15T00:54:46.271 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'}} 2024-06-15T00:54:46.272 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'} 2024-06-15T00:54:46.272 INFO:teuthology.task.install:Using flavor: default 2024-06-15T00:54:46.277 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-15T00:54:46.277 INFO:teuthology.task.install:extra packages: [] 2024-06-15T00:54:46.277 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-06-15T00:54:46.278 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:54:46.278 INFO:teuthology.packaging:ref: None 2024-06-15T00:54:46.278 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:54:46.278 INFO:teuthology.packaging:branch: None 2024-06-15T00:54:46.278 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:54:46.542 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-15T00:54:46.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-15T00:54:46.544 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-06-15T00:54:46.544 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:54:46.544 INFO:teuthology.packaging:ref: None 2024-06-15T00:54:46.544 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:54:46.544 INFO:teuthology.packaging:branch: None 2024-06-15T00:54:46.544 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:54:46.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-15T00:54:46.689 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-06-15T00:54:46.689 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-06-15T00:54:46.720 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-15T00:54:46.720 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:54:46.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-15T00:54:46.747 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-15T00:54:46.747 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:54:46.747 INFO:teuthology.packaging:ref: None 2024-06-15T00:54:46.747 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:54:46.748 INFO:teuthology.packaging:branch: None 2024-06-15T00:54:46.748 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:54:46.748 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-15T00:54:46.759 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-06-15T00:54:46.759 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-06-15T00:54:46.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-15T00:54:46.788 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:54:46.788 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-15T00:54:46.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-15T00:54:46.816 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-15T00:54:46.816 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:54:46.816 INFO:teuthology.packaging:ref: None 2024-06-15T00:54:46.816 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:54:46.816 INFO:teuthology.packaging:branch: None 2024-06-15T00:54:46.816 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:54:46.816 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-15T00:54:46.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-15T00:54:46.888 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-15T00:54:46.953 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-06-15T00:54:46.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-06-15T00:54:46.959 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-15T00:54:47.023 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-06-15T00:54:47.024 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-06-15T00:54:47.191 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-06-15T00:54:47.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw 2024-06-15T00:54:47.261 INFO:teuthology.orchestra.run.smithi137.stdout:45 files removed 2024-06-15T00:54:47.287 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw 2024-06-15T00:54:47.643 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 513 kB/s | 76 kB 00:00 2024-06-15T00:54:47.721 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 480 kB/s | 76 kB 00:00 2024-06-15T00:54:47.844 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 65 kB/s | 9.4 kB 00:00 2024-06-15T00:54:47.896 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 76 kB/s | 9.4 kB 00:00 2024-06-15T00:54:48.019 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-06-15T00:54:48.055 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 19 kB/s | 2.2 kB 00:00 2024-06-15T00:54:48.661 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-15T00:54:49.874 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-06-15T00:54:50.579 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-06-15T00:54:53.306 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 20 MB 00:02 2024-06-15T00:54:54.921 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.3 MB 00:00 2024-06-15T00:54:56.313 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 17 kB 00:00 2024-06-15T00:54:57.690 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.3 MB 00:00 2024-06-15T00:54:57.703 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-15T00:54:59.708 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 17 kB 00:00 2024-06-15T00:55:01.656 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-06-15T00:55:02.899 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-15T00:55:05.312 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:05.346 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-15T00:55:05.347 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-15T00:55:05.348 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-06-15T00:55:05.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout:Install 28 Packages 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 2 Packages 2024-06-15T00:55:05.350 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:05.351 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 53 M 2024-06-15T00:55:05.351 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:06.108 INFO:teuthology.orchestra.run.smithi142.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 28 MB/s | 5.2 MB 00:00 2024-06-15T00:55:06.141 INFO:teuthology.orchestra.run.smithi142.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 722 kB/s | 24 kB 00:00 2024-06-15T00:55:06.175 INFO:teuthology.orchestra.run.smithi142.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-06-15T00:55:06.208 INFO:teuthology.orchestra.run.smithi142.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-06-15T00:55:06.342 INFO:teuthology.orchestra.run.smithi142.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 18 MB/s | 7.7 MB 00:00 2024-06-15T00:55:06.359 INFO:teuthology.orchestra.run.smithi142.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-06-15T00:55:06.384 INFO:teuthology.orchestra.run.smithi142.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-06-15T00:55:06.410 INFO:teuthology.orchestra.run.smithi142.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.4 MB/s | 161 kB 00:00 2024-06-15T00:55:06.468 INFO:teuthology.orchestra.run.smithi142.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 17 MB/s | 4.5 MB 00:00 2024-06-15T00:55:06.493 INFO:teuthology.orchestra.run.smithi142.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-06-15T00:55:06.519 INFO:teuthology.orchestra.run.smithi142.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-06-15T00:55:06.544 INFO:teuthology.orchestra.run.smithi142.stdout:(12/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-06-15T00:55:06.744 INFO:teuthology.orchestra.run.smithi142.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-06-15T00:55:06.786 INFO:teuthology.orchestra.run.smithi142.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 400 kB/s | 107 kB 00:00 2024-06-15T00:55:06.811 INFO:teuthology.orchestra.run.smithi142.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 599 kB/s | 160 kB 00:00 2024-06-15T00:55:06.837 INFO:teuthology.orchestra.run.smithi142.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-06-15T00:55:06.895 INFO:teuthology.orchestra.run.smithi142.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-06-15T00:55:06.921 INFO:teuthology.orchestra.run.smithi142.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 506 kB/s | 42 kB 00:00 2024-06-15T00:55:06.946 INFO:teuthology.orchestra.run.smithi142.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-06-15T00:55:06.993 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-06-15T00:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 696 kB/s | 111 kB 00:00 2024-06-15T00:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-06-15T00:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-15T00:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout:(23/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.4 MB/s | 309 kB 00:00 2024-06-15T00:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-15T00:55:07.189 INFO:teuthology.orchestra.run.smithi142.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 5.6 MB/s | 192 kB 00:00 2024-06-15T00:55:07.231 INFO:teuthology.orchestra.run.smithi142.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-06-15T00:55:07.290 INFO:teuthology.orchestra.run.smithi142.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-15T00:55:07.348 INFO:teuthology.orchestra.run.smithi142.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 28 MB/s | 3.3 MB 00:00 2024-06-15T00:55:07.407 INFO:teuthology.orchestra.run.smithi142.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-06-15T00:55:07.465 INFO:teuthology.orchestra.run.smithi142.stdout:(30/30): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-06-15T00:55:07.470 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:07.470 INFO:teuthology.orchestra.run.smithi142.stdout:Total 25 MB/s | 53 MB 00:02 2024-06-15T00:55:07.536 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:07.572 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:07.572 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:08.093 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:08.093 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:09.416 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:09.422 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:09.430 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:55:09.430 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:09.430 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-15T00:55:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-06-15T00:55:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout:Install 28 Packages 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-06-15T00:55:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:09.434 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 53 M 2024-06-15T00:55:09.434 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:55:09.488 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-06-15T00:55:09.529 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-06-15T00:55:09.579 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-06-15T00:55:09.901 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-06-15T00:55:09.994 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:10.086 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:10.176 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-06-15T00:55:10.177 INFO:teuthology.orchestra.run.smithi137.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 26 MB/s | 5.2 MB 00:00 2024-06-15T00:55:10.194 INFO:teuthology.orchestra.run.smithi137.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-06-15T00:55:10.212 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:10.227 INFO:teuthology.orchestra.run.smithi137.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-06-15T00:55:10.261 INFO:teuthology.orchestra.run.smithi137.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-06-15T00:55:10.275 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:10.304 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-06-15T00:55:10.347 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-06-15T00:55:10.370 INFO:teuthology.orchestra.run.smithi137.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 20 MB/s | 7.7 MB 00:00 2024-06-15T00:55:10.389 INFO:teuthology.orchestra.run.smithi137.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-06-15T00:55:10.412 INFO:teuthology.orchestra.run.smithi137.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-06-15T00:55:10.413 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-15T00:55:10.437 INFO:teuthology.orchestra.run.smithi137.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.4 MB/s | 161 kB 00:00 2024-06-15T00:55:10.439 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:10.471 INFO:teuthology.orchestra.run.smithi137.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 321 kB 00:00 2024-06-15T00:55:10.505 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:10.538 INFO:teuthology.orchestra.run.smithi137.stdout:(10/30): librgw2-18.2.1-0.el9.x86_64.rpm 16 MB/s | 4.5 MB 00:00 2024-06-15T00:55:10.572 INFO:teuthology.orchestra.run.smithi137.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.9 MB/s | 297 kB 00:00 2024-06-15T00:55:10.597 INFO:teuthology.orchestra.run.smithi137.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.6 MB/s | 99 kB 00:00 2024-06-15T00:55:10.612 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-06-15T00:55:10.647 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-06-15T00:55:10.688 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-06-15T00:55:10.720 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-06-15T00:55:10.747 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-06-15T00:55:10.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-06-15T00:55:10.814 INFO:teuthology.orchestra.run.smithi137.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-06-15T00:55:10.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-06-15T00:55:10.898 INFO:teuthology.orchestra.run.smithi137.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 328 kB/s | 107 kB 00:00 2024-06-15T00:55:10.923 INFO:teuthology.orchestra.run.smithi137.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-06-15T00:55:10.965 INFO:teuthology.orchestra.run.smithi137.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-06-15T00:55:11.015 INFO:teuthology.orchestra.run.smithi137.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-06-15T00:55:11.074 INFO:teuthology.orchestra.run.smithi137.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-06-15T00:55:11.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-06-15T00:55:11.124 INFO:teuthology.orchestra.run.smithi137.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2024-06-15T00:55:11.145 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-15T00:55:11.157 INFO:teuthology.orchestra.run.smithi137.stdout:(20/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-06-15T00:55:11.210 INFO:teuthology.orchestra.run.smithi137.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-06-15T00:55:11.233 INFO:teuthology.orchestra.run.smithi137.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-06-15T00:55:11.235 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-15T00:55:11.250 INFO:teuthology.orchestra.run.smithi137.stdout:(23/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-06-15T00:55:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-06-15T00:55:11.275 INFO:teuthology.orchestra.run.smithi137.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-06-15T00:55:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-06-15T00:55:11.309 INFO:teuthology.orchestra.run.smithi137.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 5.6 MB/s | 192 kB 00:00 2024-06-15T00:55:11.342 INFO:teuthology.orchestra.run.smithi137.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-06-15T00:55:11.418 INFO:teuthology.orchestra.run.smithi137.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-06-15T00:55:11.485 INFO:teuthology.orchestra.run.smithi137.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 23 MB/s | 3.3 MB 00:00 2024-06-15T00:55:11.518 INFO:teuthology.orchestra.run.smithi137.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-06-15T00:55:11.577 INFO:teuthology.orchestra.run.smithi137.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-06-15T00:55:11.582 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:11.582 INFO:teuthology.orchestra.run.smithi137.stdout:Total 25 MB/s | 53 MB 00:02 2024-06-15T00:55:11.650 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:55:11.684 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:55:11.684 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:55:11.703 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-06-15T00:55:11.731 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-15T00:55:11.786 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-15T00:55:11.853 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-06-15T00:55:11.884 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-06-15T00:55:12.204 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:55:12.204 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:55:13.510 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:55:13.593 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-06-15T00:55:13.624 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-06-15T00:55:13.682 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-06-15T00:55:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-06-15T00:55:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:14.123 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:14.216 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:14.296 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-06-15T00:55:14.324 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:14.371 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:14.408 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-06-15T00:55:14.451 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-06-15T00:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-15T00:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:14.600 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:14.609 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:14.658 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-15T00:55:14.717 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-06-15T00:55:14.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-06-15T00:55:14.753 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-15T00:55:14.753 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-15T00:55:14.753 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-15T00:55:14.753 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:14.785 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:14.792 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-06-15T00:55:14.832 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-06-15T00:55:14.868 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-06-15T00:55:14.904 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-06-15T00:55:14.937 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-06-15T00:55:15.236 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-06-15T00:55:15.283 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-15T00:55:15.357 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-06-15T00:55:15.388 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-06-15T00:55:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-06-15T00:55:15.824 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-06-15T00:55:15.852 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-15T00:55:15.924 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-06-15T00:55:15.992 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-06-15T00:55:16.022 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-06-15T00:55:18.128 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:18.160 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:18.714 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-06-15T00:55:18.756 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-15T00:55:18.853 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-06-15T00:55:18.853 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-15T00:55:18.853 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-15T00:55:18.853 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:18.890 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-06-15T00:55:24.266 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-06-15T00:55:24.267 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-06-15T00:55:24.267 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:25.027 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-15T00:55:25.053 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:25.054 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-15T00:55:25.138 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-15T00:55:25.138 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:26.247 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:26.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-06-15T00:55:26.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-06-15T00:55:26.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-06-15T00:55:26.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-06-15T00:55:26.248 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:26.249 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-06-15T00:55:26.250 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.119 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: fmt-8.1.1-5.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-15T00:55:27.120 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-15T00:55:27.121 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:27.122 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:27.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-test 2024-06-15T00:55:28.041 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:26 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:28.164 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-06-15T00:55:28.165 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:28.166 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 40 M 2024-06-15T00:55:28.167 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 166 M 2024-06-15T00:55:28.167 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:28.474 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 165 kB 00:00 2024-06-15T00:55:28.577 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-06-15T00:55:28.578 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:28.908 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): ceph-test-18.2.1-0.el9.x86_64.rpm 72 MB/s | 40 MB 00:00 2024-06-15T00:55:29.291 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 78 kB/s | 64 kB 00:00 2024-06-15T00:55:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-15T00:55:29.375 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:29.377 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-15T00:55:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-06-15T00:55:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:30.350 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): socat-1.7.4.1-6.el9.x86_64.rpm 152 kB/s | 304 kB 00:01 2024-06-15T00:55:30.352 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:30.352 INFO:teuthology.orchestra.run.smithi142.stdout:Total 18 MB/s | 40 MB 00:02 2024-06-15T00:55:30.380 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:30.399 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:30.399 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-06-15T00:55:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-06-15T00:55:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-06-15T00:55:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-06-15T00:55:30.714 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:30.714 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:31.178 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:31.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-15T00:55:31.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-06-15T00:55:31.290 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:31.290 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-06-15T00:55:31.290 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: fmt-8.1.1-5.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-15T00:55:31.291 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.292 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.293 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:31.293 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-06-15T00:55:31.293 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-15T00:55:31.293 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:31.293 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:31.323 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-15T00:55:31.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-15T00:55:31.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-test 2024-06-15T00:55:32.235 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:26 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:32.359 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-15T00:55:32.360 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 40 M 2024-06-15T00:55:32.361 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 166 M 2024-06-15T00:55:32.362 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:55:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.7 MB/s | 165 kB 00:00 2024-06-15T00:55:32.976 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): ceph-test-18.2.1-0.el9.x86_64.rpm 69 MB/s | 40 MB 00:00 2024-06-15T00:55:33.000 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 127 kB/s | 64 kB 00:00 2024-06-15T00:55:33.292 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): socat-1.7.4.1-6.el9.x86_64.rpm 342 kB/s | 304 kB 00:00 2024-06-15T00:55:33.295 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:33.295 INFO:teuthology.orchestra.run.smithi137.stdout:Total 43 MB/s | 40 MB 00:00 2024-06-15T00:55:33.323 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:55:33.342 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:55:33.343 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:55:33.658 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:55:33.659 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:55:34.127 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:55:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-15T00:55:34.270 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-15T00:55:34.294 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-15T00:55:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-15T00:55:35.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-15T00:55:37.232 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-15T00:55:37.232 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-06-15T00:55:37.232 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-06-15T00:55:37.232 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-15T00:55:37.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-15T00:55:37.509 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:37.509 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:37.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:37.509 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-06-15T00:55:37.510 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:37.510 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:37.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph 2024-06-15T00:55:38.022 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-06-15T00:55:38.117 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-15T00:55:38.378 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:36 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:38.505 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:38.508 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:55:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:38.508 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-15T00:55:38.509 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-15T00:55:38.510 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:38.511 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout:Install 33 Packages 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 30 M 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 110 M 2024-06-15T00:55:38.512 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:39.417 INFO:teuthology.orchestra.run.smithi142.stdout:(1/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 16 MB/s | 2.1 MB 00:00 2024-06-15T00:55:39.442 INFO:teuthology.orchestra.run.smithi142.stdout:(2/33): ceph-18.2.1-0.el9.x86_64.rpm 40 kB/s | 6.4 kB 00:00 2024-06-15T00:55:39.559 INFO:teuthology.orchestra.run.smithi142.stdout:(3/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 5.2 MB/s | 1.4 MB 00:00 2024-06-15T00:55:39.584 INFO:teuthology.orchestra.run.smithi142.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.6 MB/s | 242 kB 00:00 2024-06-15T00:55:39.643 INFO:teuthology.orchestra.run.smithi142.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-06-15T00:55:39.818 INFO:teuthology.orchestra.run.smithi142.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 47 MB/s | 18 MB 00:00 2024-06-15T00:55:40.010 INFO:teuthology.orchestra.run.smithi142.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 556 kB/s | 106 kB 00:00 2024-06-15T00:55:40.069 INFO:teuthology.orchestra.run.smithi142.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 523 kB/s | 253 kB 00:00 2024-06-15T00:55:40.102 INFO:teuthology.orchestra.run.smithi142.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.4 MB/s | 135 kB 00:00 2024-06-15T00:55:40.141 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-06-15T00:55:40.142 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-06-15T00:55:40.142 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-06-15T00:55:40.142 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-15T00:55:40.177 INFO:teuthology.orchestra.run.smithi142.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-06-15T00:55:40.236 INFO:teuthology.orchestra.run.smithi142.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 746 kB/s | 125 kB 00:00 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:40.419 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:40.520 INFO:teuthology.orchestra.run.smithi142.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 1.4 MB/s | 1.2 MB 00:00 2024-06-15T00:55:40.537 INFO:teuthology.orchestra.run.smithi142.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 116 kB/s | 35 kB 00:00 2024-06-15T00:55:40.709 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph 2024-06-15T00:55:40.753 INFO:teuthology.orchestra.run.smithi142.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 299 kB/s | 172 kB 00:00 2024-06-15T00:55:40.795 INFO:teuthology.orchestra.run.smithi142.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-06-15T00:55:40.837 INFO:teuthology.orchestra.run.smithi142.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 520 kB/s | 43 kB 00:00 2024-06-15T00:55:40.854 INFO:teuthology.orchestra.run.smithi142.stdout:(17/33): python3-pytz-2021.1-5.el9.noarch.rpm 153 kB/s | 51 kB 00:00 2024-06-15T00:55:40.904 INFO:teuthology.orchestra.run.smithi142.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-06-15T00:55:40.930 INFO:teuthology.orchestra.run.smithi142.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 3.9 MB/s | 358 kB 00:00 2024-06-15T00:55:40.955 INFO:teuthology.orchestra.run.smithi142.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-06-15T00:55:40.980 INFO:teuthology.orchestra.run.smithi142.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 242 kB/s | 18 kB 00:00 2024-06-15T00:55:41.005 INFO:teuthology.orchestra.run.smithi142.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-06-15T00:55:41.030 INFO:teuthology.orchestra.run.smithi142.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-06-15T00:55:41.056 INFO:teuthology.orchestra.run.smithi142.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-06-15T00:55:41.081 INFO:teuthology.orchestra.run.smithi142.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 924 kB/s | 46 kB 00:00 2024-06-15T00:55:41.106 INFO:teuthology.orchestra.run.smithi142.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-06-15T00:55:41.132 INFO:teuthology.orchestra.run.smithi142.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-06-15T00:55:41.157 INFO:teuthology.orchestra.run.smithi142.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-06-15T00:55:41.182 INFO:teuthology.orchestra.run.smithi142.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-06-15T00:55:41.207 INFO:teuthology.orchestra.run.smithi142.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-15T00:55:41.232 INFO:teuthology.orchestra.run.smithi142.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-06-15T00:55:41.258 INFO:teuthology.orchestra.run.smithi142.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-15T00:55:41.291 INFO:teuthology.orchestra.run.smithi142.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-06-15T00:55:41.294 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:35 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:41.296 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:41.296 INFO:teuthology.orchestra.run.smithi142.stdout:Total 11 MB/s | 30 MB 00:02 2024-06-15T00:55:41.411 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:41.421 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:41.423 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-15T00:55:41.424 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-15T00:55:41.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-15T00:55:41.426 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:Install 33 Packages 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 30 M 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 110 M 2024-06-15T00:55:41.427 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:55:41.449 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:41.449 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:41.832 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:41.832 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:42.010 INFO:teuthology.orchestra.run.smithi137.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 60 kB/s | 6.4 kB 00:00 2024-06-15T00:55:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.2 MB/s | 1.4 MB 00:00 2024-06-15T00:55:42.119 INFO:teuthology.orchestra.run.smithi137.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.8 MB/s | 2.1 MB 00:00 2024-06-15T00:55:42.144 INFO:teuthology.orchestra.run.smithi137.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-06-15T00:55:42.228 INFO:teuthology.orchestra.run.smithi137.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-06-15T00:55:42.470 INFO:teuthology.orchestra.run.smithi137.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 43 MB/s | 18 MB 00:00 2024-06-15T00:55:42.954 INFO:teuthology.orchestra.run.smithi137.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 233 kB/s | 106 kB 00:00 2024-06-15T00:55:42.987 INFO:teuthology.orchestra.run.smithi137.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 300 kB/s | 253 kB 00:00 2024-06-15T00:55:42.995 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:43.060 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-15T00:55:43.096 INFO:teuthology.orchestra.run.smithi137.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 954 kB/s | 135 kB 00:00 2024-06-15T00:55:43.102 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-15T00:55:43.134 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-15T00:55:43.146 INFO:teuthology.orchestra.run.smithi137.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 788 kB/s | 125 kB 00:00 2024-06-15T00:55:43.225 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-15T00:55:43.255 INFO:teuthology.orchestra.run.smithi137.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-06-15T00:55:43.288 INFO:teuthology.orchestra.run.smithi137.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-06-15T00:55:43.304 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-15T00:55:43.346 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-15T00:55:43.380 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-15T00:55:43.410 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-15T00:55:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-15T00:55:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-15T00:55:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-15T00:55:43.545 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-15T00:55:43.579 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-15T00:55:43.632 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-15T00:55:43.708 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-15T00:55:43.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-15T00:55:43.926 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-15T00:55:43.930 INFO:teuthology.orchestra.run.smithi137.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 51 kB/s | 35 kB 00:00 2024-06-15T00:55:43.978 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-15T00:55:43.989 INFO:teuthology.orchestra.run.smithi137.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 204 kB/s | 172 kB 00:00 2024-06-15T00:55:44.014 INFO:teuthology.orchestra.run.smithi137.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 70 kB/s | 51 kB 00:00 2024-06-15T00:55:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-15T00:55:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-15T00:55:44.098 INFO:teuthology.orchestra.run.smithi137.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 401 kB/s | 43 kB 00:00 2024-06-15T00:55:44.130 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-15T00:55:44.131 INFO:teuthology.orchestra.run.smithi137.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-06-15T00:55:44.156 INFO:teuthology.orchestra.run.smithi137.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-06-15T00:55:44.182 INFO:teuthology.orchestra.run.smithi137.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-06-15T00:55:44.207 INFO:teuthology.orchestra.run.smithi137.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-06-15T00:55:44.232 INFO:teuthology.orchestra.run.smithi137.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-06-15T00:55:44.282 INFO:teuthology.orchestra.run.smithi137.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-06-15T00:55:44.308 INFO:teuthology.orchestra.run.smithi137.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-06-15T00:55:44.322 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-15T00:55:44.333 INFO:teuthology.orchestra.run.smithi137.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-06-15T00:55:44.358 INFO:teuthology.orchestra.run.smithi137.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-06-15T00:55:44.378 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-15T00:55:44.383 INFO:teuthology.orchestra.run.smithi137.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-06-15T00:55:44.409 INFO:teuthology.orchestra.run.smithi137.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-06-15T00:55:44.434 INFO:teuthology.orchestra.run.smithi137.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-06-15T00:55:44.449 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-15T00:55:44.459 INFO:teuthology.orchestra.run.smithi137.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-06-15T00:55:44.493 INFO:teuthology.orchestra.run.smithi137.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-06-15T00:55:44.518 INFO:teuthology.orchestra.run.smithi137.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-06-15T00:55:44.536 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-15T00:55:44.543 INFO:teuthology.orchestra.run.smithi137.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-06-15T00:55:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-15T00:55:44.711 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-15T00:55:44.727 INFO:teuthology.orchestra.run.smithi137.stdout:(33/33): python3-toml-0.10.2-6.el9.noarch.rpm 52 kB/s | 42 kB 00:00 2024-06-15T00:55:44.734 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:44.735 INFO:teuthology.orchestra.run.smithi137.stdout:Total 9.1 MB/s | 30 MB 00:03 2024-06-15T00:55:44.853 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:55:44.872 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-06-15T00:55:44.890 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:55:44.891 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:55:44.905 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-15T00:55:44.930 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-15T00:55:44.930 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:44.930 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-15T00:55:44.930 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-15T00:55:44.930 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-15T00:55:44.931 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:45.266 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:55:45.267 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:55:46.441 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:55:46.498 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-15T00:55:46.504 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-15T00:55:46.529 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-15T00:55:46.529 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:46.530 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-15T00:55:46.530 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-15T00:55:46.530 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-15T00:55:46.530 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-15T00:55:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-15T00:55:46.661 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-15T00:55:46.747 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-15T00:55:46.797 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-15T00:55:46.841 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-15T00:55:46.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-15T00:55:46.905 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-15T00:55:46.939 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-15T00:55:46.974 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-15T00:55:46.979 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-15T00:55:46.998 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-15T00:55:46.999 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:46.999 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-15T00:55:46.999 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-15T00:55:46.999 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-15T00:55:47.000 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:47.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-15T00:55:47.057 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-15T00:55:47.104 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-15T00:55:47.176 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-15T00:55:47.199 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-15T00:55:47.223 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-15T00:55:47.223 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:47.223 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-15T00:55:47.223 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-15T00:55:47.223 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-15T00:55:47.224 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-15T00:55:47.313 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-15T00:55:47.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-15T00:55:47.450 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-15T00:55:47.487 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-15T00:55:47.544 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-15T00:55:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-15T00:55:47.794 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-15T00:55:47.849 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-15T00:55:47.919 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-15T00:55:48.006 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-15T00:55:48.040 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-15T00:55:48.163 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-15T00:55:48.337 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-06-15T00:55:48.375 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-15T00:55:48.401 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-06-15T00:55:48.401 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:48.401 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-15T00:55:48.401 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-15T00:55:48.401 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-15T00:55:48.402 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-06-15T00:55:49.115 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-15T00:55:49.116 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-15T00:55:49.117 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-15T00:55:49.118 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-15T00:55:49.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-15T00:55:49.913 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-15T00:55:49.914 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:49.915 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-15T00:55:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-06-15T00:55:49.973 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:49.973 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-15T00:55:49.973 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-15T00:55:49.973 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-15T00:55:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:50.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-base 2024-06-15T00:55:50.435 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-15T00:55:50.460 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-06-15T00:55:50.460 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:50.460 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-15T00:55:50.461 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-15T00:55:50.461 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-15T00:55:50.461 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:50.672 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-15T00:55:50.696 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:50.761 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-15T00:55:50.820 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:48 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:50.904 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:55:50.945 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:50.946 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:55:50.947 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:51.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install cephadm 2024-06-15T00:55:51.585 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:49 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:51.711 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:51.712 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:51.712 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 221 k 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 213 k 2024-06-15T00:55:51.713 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:51.780 INFO:teuthology.orchestra.run.smithi142.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.2 MB/s | 221 kB 00:00 2024-06-15T00:55:51.781 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:51.781 INFO:teuthology.orchestra.run.smithi142.stdout:Total 3.2 MB/s | 221 kB 00:00 2024-06-15T00:55:51.781 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:51.786 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:51.786 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:51.788 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:51.789 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:51.856 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:52.275 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:52.366 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:52.394 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-06-15T00:55:52.394 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-06-15T00:55:52.394 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-06-15T00:55:52.394 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-15T00:55:52.395 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-15T00:55:52.396 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-15T00:55:52.397 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-15T00:55:52.706 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:52.940 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:52.941 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:52.941 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:52.941 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-06-15T00:55:52.941 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:52.941 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:53.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-immutable-object-cache 2024-06-15T00:55:53.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-15T00:55:53.262 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:53.262 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:55:53.262 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-15T00:55:53.263 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-15T00:55:53.264 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:53.265 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:53.667 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-base 2024-06-15T00:55:53.748 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:51 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:53.876 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:53.877 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:53.878 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:55:53.878 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:53.878 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 142 k 2024-06-15T00:55:53.878 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 395 k 2024-06-15T00:55:53.878 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:53.956 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.8 MB/s | 142 kB 00:00 2024-06-15T00:55:53.957 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:53.957 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.7 MB/s | 142 kB 00:00 2024-06-15T00:55:53.957 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:53.965 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:53.965 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:54.022 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:54.022 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:54.159 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:54.247 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:48 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:54.255 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:54.280 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:54.280 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:54.280 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-15T00:55:54.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:54.331 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:55:54.373 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:54.374 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:55:54.374 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:54.439 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install cephadm 2024-06-15T00:55:55.011 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:49 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:55.137 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:55.138 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:55.138 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:55:55.138 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:55.138 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 221 k 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 213 k 2024-06-15T00:55:55.139 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:55:55.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:55.157 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:55.157 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:55.158 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:55.158 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:55.158 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:55.213 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.9 MB/s | 221 kB 00:00 2024-06-15T00:55:55.213 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:55.214 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-06-15T00:55:55.214 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:55:55.219 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:55:55.219 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:55:55.221 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:55:55.222 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:55:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:55:55.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr 2024-06-15T00:55:55.733 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:55.825 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:55.998 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:53 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:56.083 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:55:56.125 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:56.126 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:55:56.126 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:56.170 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:56.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-dashboard 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:56.417 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:56.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-immutable-object-cache 2024-06-15T00:55:56.800 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:54 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:55:56.928 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:55:56.929 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:56.929 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:55:56.929 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:56.929 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:55:56.930 INFO:teuthology.orchestra.run.smithi142.stdout:Install 7 Packages 2024-06-15T00:55:56.931 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:56.931 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.1 M 2024-06-15T00:55:56.931 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 8.5 M 2024-06-15T00:55:56.931 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:55:57.206 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:51 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:57.308 INFO:teuthology.orchestra.run.smithi142.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 197 kB/s | 23 kB 00:00 2024-06-15T00:55:57.333 INFO:teuthology.orchestra.run.smithi142.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 102 kB/s | 15 kB 00:00 2024-06-15T00:55:57.335 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:55:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:57.337 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 142 k 2024-06-15T00:55:57.337 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 395 k 2024-06-15T00:55:57.337 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:55:57.410 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-06-15T00:55:57.410 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:57.411 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-06-15T00:55:57.411 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:55:57.417 INFO:teuthology.orchestra.run.smithi142.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-06-15T00:55:57.418 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:55:57.418 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:55:57.467 INFO:teuthology.orchestra.run.smithi142.stdout:(4/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 6.4 MB/s | 1.8 MB 00:00 2024-06-15T00:55:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:55:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:55:57.492 INFO:teuthology.orchestra.run.smithi142.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 422 kB/s | 31 kB 00:00 2024-06-15T00:55:57.542 INFO:teuthology.orchestra.run.smithi142.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 196 kB/s | 41 kB 00:00 2024-06-15T00:55:57.576 INFO:teuthology.orchestra.run.smithi142.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.7 MB/s | 188 kB 00:00 2024-06-15T00:55:57.580 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:55:57.580 INFO:teuthology.orchestra.run.smithi142.stdout:Total 3.2 MB/s | 2.1 MB 00:00 2024-06-15T00:55:57.580 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:55:57.588 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:55:57.588 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:55:57.614 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:55:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:55:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:55:57.678 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:57.703 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:57.703 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:55:57.703 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-15T00:55:57.703 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:57.995 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:55:58.069 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-15T00:55:58.103 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-15T00:55:58.149 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-15T00:55:58.182 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-15T00:55:58.210 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-06-15T00:55:58.447 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-06-15T00:55:58.471 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-15T00:55:58.585 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:55:58.585 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:58.585 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:55:58.585 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-06-15T00:55:58.586 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:55:58.586 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:58.783 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-15T00:55:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-15T00:55:59.009 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-15T00:55:59.283 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:55:59.284 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:55:59.367 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:53 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:55:59.457 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:55:59.471 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-15T00:55:59.501 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:55:59.502 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:55:59.502 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:55:59.564 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-dashboard 2024-06-15T00:56:00.044 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:58 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:00.142 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:54 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:00.171 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:00.172 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-15T00:56:00.173 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout:Install 12 Packages 2024-06-15T00:56:00.174 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:00.175 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 38 M 2024-06-15T00:56:00.175 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 209 M 2024-06-15T00:56:00.175 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:00.271 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-06-15T00:56:00.272 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout:Install 7 Packages 2024-06-15T00:56:00.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:00.274 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 2.1 M 2024-06-15T00:56:00.274 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 8.5 M 2024-06-15T00:56:00.274 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:00.491 INFO:teuthology.orchestra.run.smithi142.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 44 MB/s | 7.4 MB 00:00 2024-06-15T00:56:00.599 INFO:teuthology.orchestra.run.smithi142.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-06-15T00:56:00.625 INFO:teuthology.orchestra.run.smithi142.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 647 kB/s | 195 kB 00:00 2024-06-15T00:56:00.650 INFO:teuthology.orchestra.run.smithi142.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-06-15T00:56:00.675 INFO:teuthology.orchestra.run.smithi142.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 362 kB/s | 18 kB 00:00 2024-06-15T00:56:00.700 INFO:teuthology.orchestra.run.smithi142.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-15T00:56:00.751 INFO:teuthology.orchestra.run.smithi142.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-15T00:56:00.864 INFO:teuthology.orchestra.run.smithi137.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 217 kB/s | 23 kB 00:00 2024-06-15T00:56:00.889 INFO:teuthology.orchestra.run.smithi137.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 111 kB/s | 15 kB 00:00 2024-06-15T00:56:00.965 INFO:teuthology.orchestra.run.smithi137.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 90 kB/s | 9.0 kB 00:00 2024-06-15T00:56:00.998 INFO:teuthology.orchestra.run.smithi137.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 997 kB/s | 31 kB 00:00 2024-06-15T00:56:01.024 INFO:teuthology.orchestra.run.smithi137.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 305 kB/s | 41 kB 00:00 2024-06-15T00:56:01.049 INFO:teuthology.orchestra.run.smithi137.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-06-15T00:56:01.083 INFO:teuthology.orchestra.run.smithi137.stdout:(7/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 5.4 MB/s | 1.8 MB 00:00 2024-06-15T00:56:01.086 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:01.086 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.5 MB/s | 2.1 MB 00:00 2024-06-15T00:56:01.086 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:01.094 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:01.095 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:01.146 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:01.147 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:01.401 INFO:teuthology.orchestra.run.smithi142.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.7 MB/s | 3.0 MB 00:00 2024-06-15T00:56:01.493 INFO:teuthology.orchestra.run.smithi142.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-06-15T00:56:01.504 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:01.550 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-15T00:56:01.584 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-15T00:56:01.639 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-15T00:56:01.679 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-15T00:56:01.708 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-06-15T00:56:01.943 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-06-15T00:56:01.986 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-15T00:56:02.235 INFO:teuthology.orchestra.run.smithi142.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.4 MB/s | 5.1 MB 00:01 2024-06-15T00:56:02.368 INFO:teuthology.orchestra.run.smithi142.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 2.8 MB/s | 4.7 MB 00:01 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-15T00:56:02.479 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-15T00:56:02.741 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:02.742 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:02.935 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-15T00:56:03.502 INFO:teuthology.orchestra.run.smithi142.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.1 MB/s | 16 MB 00:02 2024-06-15T00:56:03.502 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:03.503 INFO:teuthology.orchestra.run.smithi142.stdout:Total 11 MB/s | 38 MB 00:03 2024-06-15T00:56:03.509 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:57 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:03.637 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:03.638 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:03.638 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:56:03.638 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:03.638 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:03.638 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:03.639 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout:Install 12 Packages 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 38 M 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 209 M 2024-06-15T00:56:03.640 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:03.858 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:03.883 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:03.883 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:04.181 INFO:teuthology.orchestra.run.smithi137.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 50 MB/s | 7.4 MB 00:00 2024-06-15T00:56:04.306 INFO:teuthology.orchestra.run.smithi137.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-06-15T00:56:04.331 INFO:teuthology.orchestra.run.smithi137.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 653 kB/s | 195 kB 00:00 2024-06-15T00:56:04.404 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:04.404 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:04.457 INFO:teuthology.orchestra.run.smithi137.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-06-15T00:56:04.507 INFO:teuthology.orchestra.run.smithi137.stdout:(5/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-06-15T00:56:04.557 INFO:teuthology.orchestra.run.smithi137.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-06-15T00:56:04.682 INFO:teuthology.orchestra.run.smithi137.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-06-15T00:56:04.999 INFO:teuthology.orchestra.run.smithi137.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.3 MB/s | 3.0 MB 00:00 2024-06-15T00:56:05.066 INFO:teuthology.orchestra.run.smithi137.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-06-15T00:56:05.091 INFO:teuthology.orchestra.run.smithi137.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-06-15T00:56:05.166 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:05.175 INFO:teuthology.orchestra.run.smithi137.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 7.1 MB/s | 4.7 MB 00:00 2024-06-15T00:56:05.241 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-15T00:56:05.291 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-15T00:56:05.390 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-15T00:56:05.426 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-15T00:56:05.603 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-15T00:56:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-15T00:56:05.731 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-15T00:56:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-06-15T00:56:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:Total 17 MB/s | 38 MB 00:02 2024-06-15T00:56:06.212 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:06.237 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:06.237 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:06.375 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-15T00:56:06.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-15T00:56:06.731 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:06.732 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:07.492 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:07.585 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-15T00:56:07.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-15T00:56:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-15T00:56:07.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-15T00:56:07.774 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-15T00:56:07.951 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-15T00:56:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-15T00:56:08.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-15T00:56:08.723 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-15T00:56:08.768 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-15T00:56:08.799 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-15T00:56:08.853 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-15T00:56:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-15T00:56:11.130 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-15T00:56:11.164 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-15T00:56:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-15T00:56:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-15T00:56:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-15T00:56:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-15T00:56:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-15T00:56:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-15T00:56:11.707 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-15T00:56:11.707 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-15T00:56:11.708 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:11.709 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:12.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-rook 2024-06-15T00:56:12.721 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:10 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:12.845 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:12.846 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-15T00:56:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-15T00:56:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:12.849 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-06-15T00:56:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:12.850 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.2 M 2024-06-15T00:56:12.851 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 26 M 2024-06-15T00:56:12.851 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:13.140 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 771 kB/s | 50 kB 00:00 2024-06-15T00:56:13.232 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 163 kB/s | 26 kB 00:00 2024-06-15T00:56:13.257 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 105 kB/s | 19 kB 00:00 2024-06-15T00:56:13.399 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 952 kB/s | 159 kB 00:00 2024-06-15T00:56:13.416 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 806 kB/s | 222 kB 00:00 2024-06-15T00:56:13.467 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 800 kB/s | 54 kB 00:00 2024-06-15T00:56:13.492 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-06-15T00:56:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 256 kB/s | 32 kB 00:00 2024-06-15T00:56:13.592 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 121 kB/s | 15 kB 00:00 2024-06-15T00:56:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-06-15T00:56:13.640 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-15T00:56:13.641 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-15T00:56:13.643 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): python3-google-auth-2.29.0-2.el9.noarc 1.6 MB/s | 205 kB 00:00 2024-06-15T00:56:13.668 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-06-15T00:56:13.702 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-06-15T00:56:13.704 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:13.704 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-06-15T00:56:13.767 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:13.782 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:13.782 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:13.868 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:13.868 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-15T00:56:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:14.067 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-06-15T00:56:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-15T00:56:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-15T00:56:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-15T00:56:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-15T00:56:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-15T00:56:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-15T00:56:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:14.071 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:14.378 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:14.440 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-rook 2024-06-15T00:56:14.556 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-15T00:56:14.616 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-15T00:56:14.672 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-15T00:56:14.790 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-15T00:56:14.790 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-15T00:56:14.844 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-15T00:56:14.953 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-15T00:56:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-15T00:56:15.023 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:09 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:15.148 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:15.149 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:15.149 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:56:15.149 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-15T00:56:15.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:15.151 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 2.2 M 2024-06-15T00:56:15.152 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 26 M 2024-06-15T00:56:15.152 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:15.720 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-15T00:56:15.747 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 755 kB/s | 50 kB 00:00 2024-06-15T00:56:15.774 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-15T00:56:15.806 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 19 kB 00:00 2024-06-15T00:56:15.817 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-15T00:56:15.831 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 171 kB/s | 26 kB 00:00 2024-06-15T00:56:15.864 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-15T00:56:15.886 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-15T00:56:15.947 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-06-15T00:56:15.972 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 987 kB/s | 222 kB 00:00 2024-06-15T00:56:16.023 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 707 kB/s | 54 kB 00:00 2024-06-15T00:56:16.048 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-06-15T00:56:16.098 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 256 kB/s | 32 kB 00:00 2024-06-15T00:56:16.148 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 123 kB/s | 15 kB 00:00 2024-06-15T00:56:16.181 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2024-06-15T00:56:16.207 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-06-15T00:56:16.232 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): python3-google-auth-2.29.0-2.el9.noarc 1.3 MB/s | 205 kB 00:00 2024-06-15T00:56:16.282 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.5 MB/s | 1.0 MB 00:00 2024-06-15T00:56:16.284 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:16.285 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-06-15T00:56:16.347 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:16.362 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:16.362 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:16.447 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:16.447 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-15T00:56:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-15T00:56:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-15T00:56:16.949 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:17.010 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-15T00:56:17.010 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-15T00:56:17.011 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:17.012 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:17.112 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-15T00:56:17.167 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-15T00:56:17.222 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-15T00:56:17.267 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-15T00:56:17.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-cephadm 2024-06-15T00:56:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-15T00:56:17.412 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-15T00:56:17.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-15T00:56:17.586 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-15T00:56:17.866 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:15 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:17.992 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:17.993 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:17.993 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-06-15T00:56:17.993 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:17.993 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:17.993 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:17.994 INFO:teuthology.orchestra.run.smithi142.stdout:Install 6 Packages 2024-06-15T00:56:17.995 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:17.995 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 7.0 M 2024-06-15T00:56:17.995 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 33 M 2024-06-15T00:56:17.995 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-15T00:56:18.357 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-15T00:56:18.391 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-15T00:56:18.439 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-15T00:56:18.461 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-15T00:56:18.776 INFO:teuthology.orchestra.run.smithi142.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-06-15T00:56:18.843 INFO:teuthology.orchestra.run.smithi142.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-06-15T00:56:18.918 INFO:teuthology.orchestra.run.smithi142.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.1 MB/s | 58 kB 00:00 2024-06-15T00:56:18.944 INFO:teuthology.orchestra.run.smithi142.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-06-15T00:56:19.273 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-06-15T00:56:19.273 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-06-15T00:56:19.273 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-15T00:56:19.274 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-15T00:56:19.560 INFO:teuthology.orchestra.run.smithi142.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 291 kB/s | 248 kB 00:00 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-15T00:56:19.600 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:19.601 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:19.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-cephadm 2024-06-15T00:56:20.069 INFO:teuthology.orchestra.run.smithi142.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-06-15T00:56:20.073 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:20.074 INFO:teuthology.orchestra.run.smithi142.stdout:Total 3.4 MB/s | 7.0 MB 00:02 2024-06-15T00:56:20.147 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:20.159 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:20.159 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:20.256 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:20.256 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:14 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:20.558 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:20.558 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:20.559 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 7.0 M 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 33 M 2024-06-15T00:56:20.560 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:20.582 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:20.757 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-15T00:56:20.812 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-15T00:56:21.251 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-06-15T00:56:21.317 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-06-15T00:56:21.337 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-15T00:56:21.343 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-06-15T00:56:21.360 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2024-06-15T00:56:21.410 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 764 kB/s | 51 kB 00:00 2024-06-15T00:56:21.439 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-15T00:56:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-15T00:56:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-15T00:56:21.543 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-06-15T00:56:21.546 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:21.546 INFO:teuthology.orchestra.run.smithi137.stdout:Total 7.1 MB/s | 7.0 MB 00:00 2024-06-15T00:56:21.616 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:21.627 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:21.627 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:21.724 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:21.725 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-15T00:56:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-15T00:56:22.339 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-15T00:56:22.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-06-15T00:56:22.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-15T00:56:22.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-15T00:56:22.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-15T00:56:22.340 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-15T00:56:22.606 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-15T00:56:22.607 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-15T00:56:22.607 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:22.607 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-15T00:56:22.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-fuse 2024-06-15T00:56:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-15T00:56:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-15T00:56:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-15T00:56:23.431 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:21 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:23.556 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-15T00:56:23.557 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout:Install 2 Packages 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 920 k 2024-06-15T00:56:23.558 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 2.7 M 2024-06-15T00:56:23.559 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-06-15T00:56:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-06-15T00:56:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-15T00:56:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-15T00:56:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-15T00:56:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-15T00:56:23.762 INFO:teuthology.orchestra.run.smithi142.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-06-15T00:56:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-15T00:56:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:23.970 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:23.971 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:24.104 INFO:teuthology.orchestra.run.smithi142.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 190 kB/s | 80 kB 00:00 2024-06-15T00:56:24.104 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:24.104 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.6 MB/s | 920 kB 00:00 2024-06-15T00:56:24.116 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:24.139 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:24.139 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:24.200 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-fuse 2024-06-15T00:56:24.217 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:24.217 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:24.409 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:24.551 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-15T00:56:24.627 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-15T00:56:24.772 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:18 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:24.898 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-06-15T00:56:24.899 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:24.900 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 920 k 2024-06-15T00:56:24.900 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 2.7 M 2024-06-15T00:56:24.900 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:25.037 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-06-15T00:56:25.282 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-15T00:56:25.282 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-06-15T00:56:25.445 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-06-15T00:56:25.447 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:25.447 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.6 MB/s | 920 kB 00:00 2024-06-15T00:56:25.459 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:25.481 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:25.481 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:25.558 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:25.558 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:25.579 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:25.756 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:25.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados-devel 2024-06-15T00:56:25.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-15T00:56:25.961 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-15T00:56:26.419 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:24 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:26.546 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 127 k 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 456 k 2024-06-15T00:56:26.547 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:26.576 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-06-15T00:56:26.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-06-15T00:56:26.608 INFO:teuthology.orchestra.run.smithi142.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.0 MB/s | 127 kB 00:00 2024-06-15T00:56:26.609 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:26.609 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.0 MB/s | 127 kB 00:00 2024-06-15T00:56:26.609 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:26.614 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:26.615 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:26.676 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:26.677 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:26.827 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:26.828 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:26.939 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:27.067 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados-devel 2024-06-15T00:56:27.370 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:27.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:27.624 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:27.624 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:27.624 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:27.624 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:27.624 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:27.656 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:21 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:27.789 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:27.790 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:27.790 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:27.790 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:27.790 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:27.790 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 127 k 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 456 k 2024-06-15T00:56:27.791 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:27.850 INFO:teuthology.orchestra.run.smithi137.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 127 kB 00:00 2024-06-15T00:56:27.851 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:27.851 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.0 MB/s | 127 kB 00:00 2024-06-15T00:56:27.851 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:27.856 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:27.857 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:27.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs2 2024-06-15T00:56:27.919 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:27.920 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:28.173 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:28.464 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:26 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:28.553 INFO:teuthology.orchestra.run.smithi142.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:28.564 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:28.596 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:28.597 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:28.597 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:28.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs-devel 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:28.820 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:29.053 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs2 2024-06-15T00:56:29.269 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:27 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:29.394 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-06-15T00:56:29.395 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 31 k 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 139 k 2024-06-15T00:56:29.396 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:29.462 INFO:teuthology.orchestra.run.smithi142.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 469 kB/s | 31 kB 00:00 2024-06-15T00:56:29.462 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:29.463 INFO:teuthology.orchestra.run.smithi142.stdout:Total 460 kB/s | 31 kB 00:00 2024-06-15T00:56:29.463 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:29.466 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:29.466 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:29.480 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:29.481 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:29.557 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:29.641 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:23 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:29.667 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:29.728 INFO:teuthology.orchestra.run.smithi137.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:29.770 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:29.771 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:29.771 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:29.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs-devel 2024-06-15T00:56:30.033 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:30.271 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:30.465 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:24 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:30.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados2 2024-06-15T00:56:30.590 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:30.590 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 31 k 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 139 k 2024-06-15T00:56:30.591 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:30.654 INFO:teuthology.orchestra.run.smithi137.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 494 kB/s | 31 kB 00:00 2024-06-15T00:56:30.655 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:30.655 INFO:teuthology.orchestra.run.smithi137.stdout:Total 485 kB/s | 31 kB 00:00 2024-06-15T00:56:30.655 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:30.658 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:30.658 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:30.673 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:30.674 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:30.755 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:30.860 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:31.041 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:29 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:31.124 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:31.166 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:31.167 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:31.167 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:31.215 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:31.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librbd1 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:31.451 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:31.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados2 2024-06-15T00:56:31.815 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:29 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:31.899 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:31.941 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:31.942 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:31.943 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:32.014 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rados 2024-06-15T00:56:32.247 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:26 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:32.330 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:32.372 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:32.373 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:32.373 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:32.437 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librbd1 2024-06-15T00:56:32.589 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:30 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:32.673 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:32.714 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:32.715 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:32.715 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:32.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rgw 2024-06-15T00:56:33.014 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:27 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:33.098 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:33.139 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:33.140 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:33.140 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:33.199 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rados 2024-06-15T00:56:33.359 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:31 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:33.443 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:33.485 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:33.486 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:33.486 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:33.544 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-cephfs 2024-06-15T00:56:33.778 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:27 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:33.861 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:33.903 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:33.904 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:33.904 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:33.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rgw 2024-06-15T00:56:34.128 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:32 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:34.212 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:34.255 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:34.256 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:34.256 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:34.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rbd 2024-06-15T00:56:34.553 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:28 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:34.637 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:34.679 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:34.680 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:34.680 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:34.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-cephfs 2024-06-15T00:56:34.898 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:32 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:34.982 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:35.024 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:35.025 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-06-15T00:56:35.025 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:35.103 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-fuse 2024-06-15T00:56:35.342 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:29 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:35.427 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:35.468 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:35.469 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:35.470 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:35.535 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rbd 2024-06-15T00:56:35.678 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:33 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:35.804 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-06-15T00:56:35.805 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 86 k 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 226 k 2024-06-15T00:56:35.806 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:35.890 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.0 MB/s | 86 kB 00:00 2024-06-15T00:56:35.891 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:35.891 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.0 MB/s | 86 kB 00:00 2024-06-15T00:56:35.891 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:35.898 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:35.898 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:35.961 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:35.961 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:36.105 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:36.116 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:30 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:36.200 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-06-15T00:56:36.214 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:36.242 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:36.243 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-06-15T00:56:36.243 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:36.297 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-fuse 2024-06-15T00:56:36.607 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:36.869 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:36.872 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:30 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:36.998 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:36.999 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:36.999 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:36.999 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:36.999 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:36.999 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 86 k 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 226 k 2024-06-15T00:56:37.000 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:37.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-mirror 2024-06-15T00:56:37.065 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-06-15T00:56:37.066 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:37.066 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-06-15T00:56:37.066 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:37.073 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:37.073 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:37.137 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:37.137 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:37.396 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:37.657 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:35 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:37.771 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:37.791 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:37.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:37.793 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:56:37.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:37.793 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 3.0 M 2024-06-15T00:56:37.793 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 12 M 2024-06-15T00:56:37.793 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:37.893 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 30 MB/s | 3.0 MB 00:00 2024-06-15T00:56:37.894 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:37.894 INFO:teuthology.orchestra.run.smithi142.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-06-15T00:56:37.894 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:37.903 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:37.903 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:37.982 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:37.982 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:38.026 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:38.241 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-mirror 2024-06-15T00:56:38.400 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:38.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-15T00:56:38.486 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:38.833 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:32 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:38.961 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:38.963 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 3.0 M 2024-06-15T00:56:38.964 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 12 M 2024-06-15T00:56:38.964 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:39.078 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-06-15T00:56:39.078 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:39.079 INFO:teuthology.orchestra.run.smithi137.stdout:Total 26 MB/s | 3.0 MB 00:00 2024-06-15T00:56:39.079 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:39.166 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:39.166 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:39.577 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:39.603 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:39.628 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:39.653 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:39.653 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-15T00:56:39.653 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-15T00:56:39.653 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-15T00:56:39.654 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-15T00:56:39.654 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:39.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-nbd 2024-06-15T00:56:40.419 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:38 ago on Sat 15 Jun 2024 12:55:02 AM UTC. 2024-06-15T00:56:40.544 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-06-15T00:56:40.544 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 171 k 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 494 k 2024-06-15T00:56:40.545 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-06-15T00:56:40.613 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-06-15T00:56:40.614 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:40.614 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.4 MB/s | 171 kB 00:00 2024-06-15T00:56:40.614 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-06-15T00:56:40.622 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-06-15T00:56:40.622 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-06-15T00:56:40.668 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:40.669 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:40.669 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:40.669 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:40.669 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:40.669 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:40.685 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-06-15T00:56:40.685 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-06-15T00:56:40.841 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-06-15T00:56:40.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-nbd 2024-06-15T00:56:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:41.351 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:41.473 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:35 ago on Sat 15 Jun 2024 12:55:06 AM UTC. 2024-06-15T00:56:41.599 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-06-15T00:56:41.599 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 171 k 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 494 k 2024-06-15T00:56:41.600 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-06-15T00:56:41.609 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:41.610 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:41.610 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-06-15T00:56:41.610 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:41.610 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:56:41.610 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-06-15T00:56:41.664 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.6 MB/s | 171 kB 00:00 2024-06-15T00:56:41.665 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:56:41.665 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.6 MB/s | 171 kB 00:00 2024-06-15T00:56:41.665 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-06-15T00:56:41.673 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-06-15T00:56:41.673 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-06-15T00:56:41.736 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-06-15T00:56:41.736 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-06-15T00:56:41.856 DEBUG:teuthology.parallel:result is None 2024-06-15T00:56:41.905 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-06-15T00:56:42.011 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:42.395 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:56:42.628 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-06-15T00:56:42.907 DEBUG:teuthology.parallel:result is None 2024-06-15T00:56:42.907 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:56:42.907 INFO:teuthology.packaging:ref: None 2024-06-15T00:56:42.907 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:56:42.907 INFO:teuthology.packaging:branch: None 2024-06-15T00:56:42.907 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:56:42.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-15T00:56:43.149 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-15T00:56:43.170 INFO:teuthology.orchestra.run.smithi137.stdout:18.2.1-0.el9 2024-06-15T00:56:43.171 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-06-15T00:56:43.171 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-06-15T00:56:43.173 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-06-15T00:56:43.173 INFO:teuthology.packaging:ref: None 2024-06-15T00:56:43.173 INFO:teuthology.packaging:tag: v18.2.1 2024-06-15T00:56:43.173 INFO:teuthology.packaging:branch: None 2024-06-15T00:56:43.173 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:56:43.173 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-06-15T00:56:43.341 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-15T00:56:43.366 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.1-0.el9 2024-06-15T00:56:43.367 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-06-15T00:56:43.367 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-06-15T00:56:43.369 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-15T00:56:43.369 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:56:43.369 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-15T00:56:43.395 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:56:43.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-15T00:56:43.435 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-15T00:56:43.435 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:56:43.435 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-06-15T00:56:43.460 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-15T00:56:43.525 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:56:43.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-06-15T00:56:43.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-15T00:56:43.620 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-15T00:56:43.620 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:56:43.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-15T00:56:43.651 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-15T00:56:43.716 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:56:43.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-15T00:56:43.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-15T00:56:43.808 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-15T00:56:43.808 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:56:43.808 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-06-15T00:56:43.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-15T00:56:43.896 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:56:43.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-06-15T00:56:43.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-15T00:56:43.987 INFO:teuthology.run_tasks:Running task print... 2024-06-15T00:56:43.995 INFO:teuthology.task.print:**** done install task... 2024-06-15T00:56:43.995 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-15T00:56:44.081 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'} 2024-06-15T00:56:44.081 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-06-15T00:56:44.081 INFO:tasks.cephadm:Cluster fsid is 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:56:44.081 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-15T00:56:44.082 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-06-15T00:56:44.082 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi137': '172.21.15.137', 'mon.smithi142': '172.21.15.142'} 2024-06-15T00:56:44.082 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-15T00:56:44.082 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-06-15T00:56:44.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-06-15T00:56:44.132 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-15T00:56:44.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:56:44.361 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/80077/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1443-g6f4b0956', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1443.g6f4b0956'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2024-06-08-0902-reef/6f4b09560ed399c3f139402a02f389c2fc8bb2cc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-06-08 17:22:24.233074', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri5-testing-2024-06-08-0902-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-yuri5-testing-2024-06-08-0902-reef/6f4b09560ed399c3f139402a02f389c2fc8bb2cc/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-06-15T00:56:44.504 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-yuri5-testing-2024-06-08-0902-reef, sha1 6f4b09560ed399c3f139402a02f389c2fc8bb2cc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T00:56:44.505 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-yuri5-testing-2024-06-08-0902-reef/6f4b09560ed399c3f139402a02f389c2fc8bb2cc/centos/9/x86_64/flavors/default/cephadm 2024-06-15T00:56:44.505 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-yuri5-testing-2024-06-08-0902-reef/6f4b09560ed399c3f139402a02f389c2fc8bb2cc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.088 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Jun 15 00:56 /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.088 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-yuri5-testing-2024-06-08-0902-reef/6f4b09560ed399c3f139402a02f389c2fc8bb2cc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.653 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Jun 15 00:56 /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.653 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.673 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-15T00:56:45.702 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-06-15T00:56:45.702 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-06-15T00:56:45.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-06-15T00:56:46.565 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-15T00:56:46.637 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-15T00:57:03.242 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-06-15T00:57:03.243 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T00:57:03.414 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-06-15T00:57:03.415 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-06-15T00:57:03.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-06-15T00:57:03.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-06-15T00:57:03.502 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-06-15T00:57:03.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-06-15T00:57:03.567 INFO:tasks.cephadm:Writing seed config... 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-15T00:57:03.568 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-06-15T00:57:03.569 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-15T00:57:03.570 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-15T00:57:03.571 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:57:03.571 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-15T00:57:03.586 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 225ca7a2-2ab2-11ef-bc9d-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-15T00:57:03.587 DEBUG:teuthology.orchestra.run.smithi137:mon.smithi137> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service 2024-06-15T00:57:03.628 INFO:tasks.cephadm:Bootstrapping... 2024-06-15T00:57:03.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.137 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-15T00:57:03.782 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-06-15T00:57:03.783 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '225ca7a2-2ab2-11ef-bc9d-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.137', '--skip-admin-label'] 2024-06-15T00:57:03.808 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5.1.0 2024-06-15T00:57:03.808 INFO:teuthology.orchestra.run.smithi137.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-15T00:57:03.808 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying podman|docker is present... 2024-06-15T00:57:03.832 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5.1.0 2024-06-15T00:57:03.833 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying lvm2 is present... 2024-06-15T00:57:03.833 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying time synchronization is in place... 2024-06-15T00:57:03.840 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-15T00:57:03.840 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-15T00:57:03.846 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-15T00:57:03.846 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:57:03.853 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-06-15T00:57:03.859 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-06-15T00:57:03.859 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-06-15T00:57:03.860 INFO:teuthology.orchestra.run.smithi137.stdout:Repeating the final host check... 2024-06-15T00:57:03.883 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5.1.0 2024-06-15T00:57:03.883 INFO:teuthology.orchestra.run.smithi137.stdout:podman (/bin/podman) version 5.1.0 is present 2024-06-15T00:57:03.883 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl is present 2024-06-15T00:57:03.883 INFO:teuthology.orchestra.run.smithi137.stdout:lvcreate is present 2024-06-15T00:57:03.889 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-15T00:57:03.890 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-15T00:57:03.896 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-15T00:57:03.896 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:57:03.902 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-06-15T00:57:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-06-15T00:57:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-06-15T00:57:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:Host looks OK 2024-06-15T00:57:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:Cluster fsid: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:Acquiring lock 139828880617776 on /run/cephadm/225ca7a2-2ab2-11ef-bc9d-c7b262605968.lock 2024-06-15T00:57:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:Lock 139828880617776 acquired on /run/cephadm/225ca7a2-2ab2-11ef-bc9d-c7b262605968.lock 2024-06-15T00:57:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 3300 ... 2024-06-15T00:57:03.911 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 6789 ... 2024-06-15T00:57:03.911 INFO:teuthology.orchestra.run.smithi137.stdout:Base mon IP(s) is [172.21.15.137:3300, 172.21.15.137:6789], mon addrv is [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-06-15T00:57:03.913 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.137 metric 100 2024-06-15T00:57:03.913 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.137 metric 100 2024-06-15T00:57:03.916 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-15T00:57:03.916 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-15T00:57:03.916 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1780sec hoplimit 64 pref medium 2024-06-15T00:57:03.918 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-15T00:57:03.919 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-06-15T00:57:03.920 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-06-15T00:57:03.920 INFO:teuthology.orchestra.run.smithi137.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-15T00:57:03.920 INFO:teuthology.orchestra.run.smithi137.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-15T00:57:03.920 INFO:teuthology.orchestra.run.smithi137.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Getting image source signatures 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-06-15T00:57:04.787 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-06-15T00:57:04.788 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-15T00:57:05.716 INFO:teuthology.orchestra.run.smithi137.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-06-15T00:57:05.716 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-06-15T00:57:05.716 INFO:teuthology.orchestra.run.smithi137.stdout:Extracting ceph user uid/gid from container image... 2024-06-15T00:57:06.571 INFO:teuthology.orchestra.run.smithi137.stdout:stat: stdout 167 167 2024-06-15T00:57:06.571 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial keys... 2024-06-15T00:57:07.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQDi5mxmod13MhAA/O0+1QGXpuKWcq+d/IektQ== 2024-06-15T00:57:08.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQDj5mxmSLpfLRAAI8eRAnpwqhIjhJhqrZG9Rg== 2024-06-15T00:57:09.254 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQDk5mxmVT86JRAA5PWaH3gz48kStLTnd4wpvg== 2024-06-15T00:57:09.255 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial monmap... 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool for smithi137 [v2:172.21.15.137:3300,v1:172.21.15.137:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:setting min_mon_release = pacific 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: set fsid to 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:57:10.161 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mon... 2024-06-15T00:57:11.091 INFO:teuthology.orchestra.run.smithi137.stdout:create mon.smithi137 on 2024-06-15T00:57:11.298 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-15T00:57:11.437 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-15T00:57:11.616 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target. 2024-06-15T00:57:11.616 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target. 2024-06-15T00:57:11.827 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137 2024-06-15T00:57:11.827 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service: Unit ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service not loaded. 2024-06-15T00:57:11.988 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target.wants/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service → /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@.service. 2024-06-15T00:57:12.473 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-15T00:57:12.474 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-06-15T00:57:12.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:12 smithi137 systemd[1]: Started Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T00:57:12.480 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-15T00:57:12.480 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:57:12.480 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-06-15T00:57:12.481 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-06-15T00:57:12.481 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon to start... 2024-06-15T00:57:12.481 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon... 2024-06-15T00:57:13.549 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: mon.smithi137 is new leader, mons smithi137 in quorum (ranks 0) 2024-06-15T00:57:13.549 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: monmap e1: 1 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T00:57:13.549 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: fsmap 2024-06-15T00:57:13.549 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: osdmap e1: 0 total, 0 up, 0 in 2024-06-15T00:57:13.550 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: mgrmap e1: no daemons active 2024-06-15T00:57:13.550 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:13 smithi137 ceph-mon[28984]: from='client.? 172.21.15.137:0/2619745347' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-15T00:57:13.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout cluster: 2024-06-15T00:57:13.557 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout id: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:13.557 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout services: 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi137 (age 0.520117s) 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout data: 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pgs: 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:13.558 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd662501700 1 Processor -- start 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd662501700 1 -- start start 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd662501700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd662501700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c106d20 con 0x7fd65c106330 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd65bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd65bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32974/0 (socket says 172.21.15.137:32974) 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.886+0000 7fd65bfff700 1 -- 172.21.15.137:0/1267438469 learned_addr learned my addr 172.21.15.137:0/1267438469 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd65bfff700 1 -- 172.21.15.137:0/1267438469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c107530 con 0x7fd65c106330 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd65bfff700 1 --2- 172.21.15.137:0/1267438469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098a0 tx=0x7fd64c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1c4381a0ba9dc server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd65affd700 1 -- 172.21.15.137:0/1267438469 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd64c004180 con 0x7fd65c106330 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd65affd700 1 -- 172.21.15.137:0/1267438469 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c0042e0 con 0x7fd65c106330 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd65affd700 1 -- 172.21.15.137:0/1267438469 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd64c0036a0 con 0x7fd65c106330 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd662501700 1 -- 172.21.15.137:0/1267438469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 msgr2=0x7fd65c106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:13.559 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd662501700 1 --2- 172.21.15.137:0/1267438469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098a0 tx=0x7fd64c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd662501700 1 -- 172.21.15.137:0/1267438469 shutdown_connections 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd662501700 1 --2- 172.21.15.137:0/1267438469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c106750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.887+0000 7fd662501700 1 -- 172.21.15.137:0/1267438469 >> 172.21.15.137:0/1267438469 conn(0x7fd65c101910 msgr2=0x7fd65c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.888+0000 7fd662501700 1 -- 172.21.15.137:0/1267438469 shutdown_connections 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.888+0000 7fd662501700 1 -- 172.21.15.137:0/1267438469 wait complete. 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.888+0000 7fd662501700 1 Processor -- start 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.888+0000 7fd662501700 1 -- start start 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd662501700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd662501700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c199d70 con 0x7fd65c106330 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd65bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd65bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32984/0 (socket says 172.21.15.137:32984) 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd65bfff700 1 -- 172.21.15.137:0/2619745347 learned_addr learned my addr 172.21.15.137:0/2619745347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:13.560 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd65bfff700 1 -- 172.21.15.137:0/2619745347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c009550 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd65bfff700 1 --2- 172.21.15.137:0/2619745347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd64c004000 tx=0x7fd64c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd64c003ce0 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c020070 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c199f70 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd64c011630 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c19a390 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd64c022030 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.889+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd64c025d00 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.890+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c193110 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.892+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd64c025480 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.962+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd65c02cc70 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.963+0000 7fd6597fa700 1 -- 172.21.15.137:0/2619745347 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fd64c011c20 con 0x7fd65c106330 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 msgr2=0x7fd65c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:13.561 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 --2- 172.21.15.137:0/2619745347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd64c004000 tx=0x7fd64c004750 comp rx=0 tx=0).stop 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 shutdown_connections 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 --2- 172.21.15.137:0/2619745347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd65c106330 0x7fd65c199830 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 >> 172.21.15.137:0/2619745347 conn(0x7fd65c101910 msgr2=0x7fd65c18dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 shutdown_connections 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:12.964+0000 7fd662501700 1 -- 172.21.15.137:0/2619745347 wait complete. 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:mon is available 2024-06-15T00:57:13.562 INFO:teuthology.orchestra.run.smithi137.stdout:Assimilating anything we can from ceph.conf... 2024-06-15T00:57:14.580 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:14 smithi137 ceph-mon[28984]: from='client.? 172.21.15.137:0/900528061' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-15T00:57:14.581 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:14 smithi137 ceph-mon[28984]: from='client.? 172.21.15.137:0/900528061' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-15T00:57:14.590 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-15T00:57:14.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f442dada700 1 Processor -- start 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f442dada700 1 -- start start 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f442dada700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f442dada700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428106d20 con 0x7f4428106330 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f44277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f44277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33000/0 (socket says 172.21.15.137:33000) 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f44277fe700 1 -- 172.21.15.137:0/2372683101 learned_addr learned my addr 172.21.15.137:0/2372683101 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.950+0000 7f44277fe700 1 -- 172.21.15.137:0/2372683101 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428107530 con 0x7f4428106330 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f44277fe700 1 --2- 172.21.15.137:0/2372683101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f4418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730abbc03f2f8565 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f44267fc700 1 -- 172.21.15.137:0/2372683101 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4418004180 con 0x7f4428106330 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f44267fc700 1 -- 172.21.15.137:0/2372683101 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180042e0 con 0x7f4428106330 2024-06-15T00:57:14.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f44267fc700 1 -- 172.21.15.137:0/2372683101 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44180036a0 con 0x7f4428106330 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 -- 172.21.15.137:0/2372683101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 msgr2=0x7f4428106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 --2- 172.21.15.137:0/2372683101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f4418009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 -- 172.21.15.137:0/2372683101 shutdown_connections 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 --2- 172.21.15.137:0/2372683101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428106750 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 -- 172.21.15.137:0/2372683101 >> 172.21.15.137:0/2372683101 conn(0x7f4428101910 msgr2=0x7f4428103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 -- 172.21.15.137:0/2372683101 shutdown_connections 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.951+0000 7f442dada700 1 -- 172.21.15.137:0/2372683101 wait complete. 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f442dada700 1 Processor -- start 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f442dada700 1 -- start start 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f442dada700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f442dada700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428199d70 con 0x7f4428106330 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f44277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.952+0000 7f44277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33008/0 (socket says 172.21.15.137:33008) 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f44277fe700 1 -- 172.21.15.137:0/900528061 learned_addr learned my addr 172.21.15.137:0/900528061 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f44277fe700 1 -- 172.21.15.137:0/900528061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009550 con 0x7f4428106330 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f44277fe700 1 --2- 172.21.15.137:0/900528061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4418004000 tx=0x7f4418004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4418003ce0 con 0x7f4428106330 2024-06-15T00:57:14.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418020070 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428199f70 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4418011630 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442819a390 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4418022030 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.953+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4418025d00 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.954+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4428193110 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:13.956+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4418025480 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.022+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f442802cef0 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.025+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 7 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f4418011c20 con 0x7f4428106330 2024-06-15T00:57:14.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.025+0000 7f4424ff9700 1 -- 172.21.15.137:0/900528061 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f4418038590 con 0x7f4428106330 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 msgr2=0x7f4428199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 --2- 172.21.15.137:0/900528061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4418004000 tx=0x7f4418004750 comp rx=0 tx=0).stop 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 shutdown_connections 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 --2- 172.21.15.137:0/900528061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4428106330 0x7f4428199830 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 >> 172.21.15.137:0/900528061 conn(0x7f4428101910 msgr2=0x7f442818dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 shutdown_connections 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.026+0000 7f442dada700 1 -- 172.21.15.137:0/900528061 wait complete. 2024-06-15T00:57:14.596 INFO:teuthology.orchestra.run.smithi137.stdout:Generating new minimal ceph.conf... 2024-06-15T00:57:15.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:15 smithi137 ceph-mon[28984]: from='client.? 172.21.15.137:0/1108411594' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d4063c700 1 Processor -- start 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d4063c700 1 -- start start 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d4063c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d4063c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38105c20 con 0x7f7d38105230 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d3e3d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d3e3d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33010/0 (socket says 172.21.15.137:33010) 2024-06-15T00:57:15.635 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.989+0000 7f7d3e3d8700 1 -- 172.21.15.137:0/269715823 learned_addr learned my addr 172.21.15.137:0/269715823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d3e3d8700 1 -- 172.21.15.137:0/269715823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38106430 con 0x7f7d38105230 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d3e3d8700 1 --2- 172.21.15.137:0/269715823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b33ac5cae61845 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d3d3d6700 1 -- 172.21.15.137:0/269715823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d34004180 con 0x7f7d38105230 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d3d3d6700 1 -- 172.21.15.137:0/269715823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f7d340042e0 con 0x7f7d38105230 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d3d3d6700 1 -- 172.21.15.137:0/269715823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d340119d0 con 0x7f7d38105230 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d4063c700 1 -- 172.21.15.137:0/269715823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 msgr2=0x7f7d38105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d4063c700 1 --2- 172.21.15.137:0/269715823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d4063c700 1 -- 172.21.15.137:0/269715823 shutdown_connections 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d4063c700 1 --2- 172.21.15.137:0/269715823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38105650 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.990+0000 7f7d4063c700 1 -- 172.21.15.137:0/269715823 >> 172.21.15.137:0/269715823 conn(0x7f7d381007d0 msgr2=0x7f7d38102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 -- 172.21.15.137:0/269715823 shutdown_connections 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 -- 172.21.15.137:0/269715823 wait complete. 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 Processor -- start 2024-06-15T00:57:15.636 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 -- start start 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d4063c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3819a510 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d3e3d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d3e3d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33020/0 (socket says 172.21.15.137:33020) 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.991+0000 7f7d3e3d8700 1 -- 172.21.15.137:0/1108411594 learned_addr learned my addr 172.21.15.137:0/1108411594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d3e3d8700 1 -- 172.21.15.137:0/1108411594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009550 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d3e3d8700 1 --2- 172.21.15.137:0/1108411594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006af0 tx=0x7f7d34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d34003b40 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f7d3401a880 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3819a710 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d34019890 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3819ab30 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7d3402d440 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.992+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7d34024800 con 0x7f7d38105230 2024-06-15T00:57:15.637 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.993+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d380623c0 con 0x7f7d38105230 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:14.995+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7d3401adf0 con 0x7f7d38105230 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.062+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7d38109630 con 0x7f7d38105230 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.062+0000 7f7d2f7fe700 1 -- 172.21.15.137:0/1108411594 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f7d3401aaa0 con 0x7f7d38105230 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 msgr2=0x7f7d38199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 --2- 172.21.15.137:0/1108411594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006af0 tx=0x7f7d34011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 shutdown_connections 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 --2- 172.21.15.137:0/1108411594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d38105230 0x7f7d38199fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 >> 172.21.15.137:0/1108411594 conn(0x7f7d381007d0 msgr2=0x7f7d380797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 shutdown_connections 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:15.063+0000 7f7d4063c700 1 -- 172.21.15.137:0/1108411594 wait complete. 2024-06-15T00:57:15.638 INFO:teuthology.orchestra.run.smithi137.stdout:Restarting the monitor... 2024-06-15T00:57:15.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:15 smithi137 systemd[1]: Stopping Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T00:57:16.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:15 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[28980]: 2024-06-15T00:57:15.881+0000 7f07d3f35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi137 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T00:57:16.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:15 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[28980]: 2024-06-15T00:57:15.881+0000 7f07d3f35700 -1 mon.smithi137@0(leader) e1 *** Got Signal Terminated *** 2024-06-15T00:57:16.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 podman[29226]: 2024-06-15 00:57:16.055178215 +0000 UTC m=+0.258125819 container died ce0a645897a96caf9b7495a0d948d877ba68ca7c3be0437618949c55ea550cdd (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1) 2024-06-15T00:57:16.665 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 podman[29226]: 2024-06-15 00:57:16.406036188 +0000 UTC m=+0.608983789 container cleanup ce0a645897a96caf9b7495a0d948d877ba68ca7c3be0437618949c55ea550cdd (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-15T00:57:16.665 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 bash[29226]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137 2024-06-15T00:57:16.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 podman[29238]: 2024-06-15 00:57:16.665293369 +0000 UTC m=+0.612482715 container remove ce0a645897a96caf9b7495a0d948d877ba68ca7c3be0437618949c55ea550cdd (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1) 2024-06-15T00:57:16.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service: Deactivated successfully. 2024-06-15T00:57:16.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 systemd[1]: Stopped Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T00:57:16.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:16 smithi137 systemd[1]: Starting Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 podman[29335]: 2024-06-15 00:57:16.952373263 +0000 UTC m=+0.017185507 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 podman[29335]: 2024-06-15 00:57:17.058038381 +0000 UTC m=+0.122850658 container create 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.29.1) 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 podman[29335]: 2024-06-15 00:57:17.260765259 +0000 UTC m=+0.325577508 container init 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 podman[29335]: 2024-06-15 00:57:17.264142014 +0000 UTC m=+0.328954250 container start 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: set uid:gid to 167:167 (ceph:ceph) 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: pidfile_write: ignore empty --pid-file 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: load: jerasure load: lrc 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: RocksDB version: 7.9.2 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Git sha 0 2024-06-15T00:57:17.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Compile date 2023-12-11 22:07:34 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: DB SUMMARY 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: DB Session ID: OA79ASCRJOPEJ23VDTJQ 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: CURRENT file: CURRENT 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: IDENTITY file: IDENTITY 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi137/store.db dir, Total Num: 1, files: 000008.sst 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi137/store.db: 000009.log size: 96990 ; 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.error_if_exists: 0 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.create_if_missing: 0 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.paranoid_checks: 1 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-15T00:57:17.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.env: 0x55a2d85f0720 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.fs: PosixFileSystem 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.info_log: 0x55a2da2772e0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_file_opening_threads: 16 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.statistics: (nil) 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.use_fsync: 0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_log_file_size: 0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.keep_log_file_num: 1000 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.recycle_log_file_num: 0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_fallocate: 1 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_mmap_reads: 0 2024-06-15T00:57:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_mmap_writes: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.use_direct_reads: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.create_missing_column_families: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.db_log_dir: 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.wal_dir: 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.advise_random_on_open: 1 2024-06-15T00:57:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.db_write_buffer_size: 0 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.write_buffer_manager: 0x55a2d94e8140 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.rate_limiter: (nil) 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.wal_recovery_mode: 2 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enable_thread_tracking: 0 2024-06-15T00:57:17.348 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enable_pipelined_write: 0 2024-06-15T00:57:17.349 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.unordered_write: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.row_cache: None 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.wal_filter: None 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_ingest_behind: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.two_write_queues: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.manual_wal_flush: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.wal_compression: 0 2024-06-15T00:57:17.350 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.atomic_flush: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.log_readahead_size: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.best_efforts_recovery: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.allow_data_in_errors: 0 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.db_host_id: __hostname__ 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-15T00:57:17.351 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_background_jobs: 2 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_background_compactions: -1 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_subcompactions: 1 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_total_wal_size: 0 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_open_files: -1 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bytes_per_sync: 0 2024-06-15T00:57:17.352 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_readahead_size: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_background_flushes: -1 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Compression algorithms supported: 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kZSTD supported: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kXpressCompression supported: 0 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kLZ4HCCompression supported: 1 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kZlibCompression supported: 1 2024-06-15T00:57:17.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kSnappyCompression supported: 1 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kLZ4Compression supported: 1 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: kBZip2Compression supported: 0 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi137/store.db/MANIFEST-000010 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.merge_operator: 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_filter: None 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_filter_factory: None 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.sst_partitioner_factory: None 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-15T00:57:17.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a2da2773e0) 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_type: 0 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: data_block_index_type: 0 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_shortening: 1 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: checksum: 4 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: no_block_cache: 0 2024-06-15T00:57:17.355 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache: 0x55a2d95891f0 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_options: 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: capacity : 536870912 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: num_shard_bits : 4 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: strict_capacity_limit : 0 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_compressed: (nil) 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: persistent_cache: (nil) 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_size: 4096 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_size_deviation: 10 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_restart_interval: 16 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_block_restart_interval: 1 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: metadata_block_size: 4096 2024-06-15T00:57:17.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: partition_filters: 0 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: use_delta_encoding: 1 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: filter_policy: bloomfilter 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: whole_key_filtering: 1 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: verify_compression: 0 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: format_version: 5 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: enable_index_compression: 1 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_align: 0 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: max_auto_readahead_size: 262144 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: prepopulate_block_cache: 0 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: initial_auto_readahead_size: 8192 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-06-15T00:57:17.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.write_buffer_size: 33554432 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_write_buffer_number: 2 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression: NoCompression 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression: Disabled 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.prefix_extractor: nullptr 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.num_levels: 7 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-15T00:57:17.358 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.level: 32767 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.strategy: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.enabled: false 2024-06-15T00:57:17.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.target_file_size_base: 67108864 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-15T00:57:17.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.arena_block_size: 1048576 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.disable_auto_compactions: 0 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-15T00:57:17.361 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.table_properties_collectors: 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.inplace_update_support: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.bloom_locality: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.max_successive_merges: 0 2024-06-15T00:57:17.362 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.paranoid_file_checks: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.force_consistency_checks: 1 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.report_bg_io_stats: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.ttl: 2592000 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enable_blob_files: false 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.min_blob_size: 0 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_file_size: 268435456 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-15T00:57:17.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.blob_file_starting_level: 0 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi137/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 420d8cc3-0575-4f23-8f0f-236050d5091b 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413037293213, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413037294160, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92437, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 308, "table_properties": {"data_size": 90454, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 83193, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 267, "num_filter_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718413037, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "420d8cc3-0575-4f23-8f0f-236050d5091b", "db_session_id": "OA79ASCRJOPEJ23VDTJQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413037294246, "job": 1, "event": "recovery_finished"} 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi137/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-15T00:57:17.364 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a2d9626000 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: DB pointer 0x55a2d9612000 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** DB Stats ** 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** Compaction Stats [default] ** 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T00:57:17.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: L0 2/0 92.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Sum 2/0 92.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** Compaction Stats [default] ** 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T00:57:17.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval compaction: 0.00 GB write, 29.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Block cache BinnedLRUCache@0x55a2d95891f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: starting mon.smithi137 rank 0 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi137 fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???) e1 preinit fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:17.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).mds e1 new map 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).mds e1 print_map 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: e1 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: legacy client fscid: -1 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: No filesystems configured 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expand map: {default=false} 2024-06-15T00:57:17.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta from 'false' to 'false' 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expanded map: {default=false} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expand map: {default=info} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta from 'info' to 'info' 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expanded map: {default=info} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expand map: {default=daemon} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta from 'daemon' to 'daemon' 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expanded map: {default=daemon} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expand map: {default=debug} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta from 'debug' to 'debug' 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: expand_channel_meta expanded map: {default=debug} 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 bash[29335]: 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 2024-06-15T00:57:17.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 systemd[1]: Started Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T00:57:17.370 INFO:teuthology.orchestra.run.smithi137.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-15T00:57:17.638 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mon.smithi137 is new leader, mons smithi137 in quorum (ranks 0) 2024-06-15T00:57:17.638 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: monmap e1: 1 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T00:57:17.638 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: fsmap 2024-06-15T00:57:17.638 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: osdmap e1: 0 total, 0 up, 0 in 2024-06-15T00:57:17.638 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:17 smithi137 ceph-mon[29354]: mgrmap e1: no daemons active 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408f31f700 1 Processor -- start 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408f31f700 1 -- start start 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408f31f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408f31f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4088079da0 con 0x7f4088106de0 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408d0bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408d0bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60836/0 (socket says 172.21.15.137:60836) 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.741+0000 7f408d0bb700 1 -- 172.21.15.137:0/3547832806 learned_addr learned my addr 172.21.15.137:0/3547832806 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.742+0000 7f408d0bb700 1 -- 172.21.15.137:0/3547832806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088109710 con 0x7f4088106de0 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.742+0000 7f408d0bb700 1 --2- 172.21.15.137:0/3547832806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40840098a0 tx=0x7f4084009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfee365de0b0a33c server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.742+0000 7f407ffff700 1 -- 172.21.15.137:0/3547832806 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4084004180 con 0x7f4088106de0 2024-06-15T00:57:18.332 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.742+0000 7f407ffff700 1 -- 172.21.15.137:0/3547832806 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f40840042e0 con 0x7f4088106de0 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.742+0000 7f407ffff700 1 -- 172.21.15.137:0/3547832806 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40840119d0 con 0x7f4088106de0 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- 172.21.15.137:0/3547832806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 msgr2=0x7f40881091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 --2- 172.21.15.137:0/3547832806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40840098a0 tx=0x7f4084009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- 172.21.15.137:0/3547832806 shutdown_connections 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 --2- 172.21.15.137:0/3547832806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f40881091d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- 172.21.15.137:0/3547832806 >> 172.21.15.137:0/3547832806 conn(0x7f40881007d0 msgr2=0x7f4088102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- 172.21.15.137:0/3547832806 shutdown_connections 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- 172.21.15.137:0/3547832806 wait complete. 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 Processor -- start 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.743+0000 7f408f31f700 1 -- start start 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408f31f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408f31f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4088106c80 con 0x7f4088106de0 2024-06-15T00:57:18.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408d0bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408d0bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60850/0 (socket says 172.21.15.137:60850) 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408d0bb700 1 -- 172.21.15.137:0/4065754138 learned_addr learned my addr 172.21.15.137:0/4065754138 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408d0bb700 1 -- 172.21.15.137:0/4065754138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084009550 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408d0bb700 1 --2- 172.21.15.137:0/4065754138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4084011980 tx=0x7f4084011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4084003b40 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4088104d90 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f408401a880 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4084019910 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.744+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40881051b0 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.745+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f408402d440 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.745+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4084024810 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.745+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408810a400 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.748+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f408401adf0 con 0x7f4088106de0 2024-06-15T00:57:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.813+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4088105810 con 0x7f4088106de0 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.818+0000 7f407e7fc700 1 -- 172.21.15.137:0/4065754138 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f408401a9f0 con 0x7f4088106de0 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 msgr2=0x7f4088106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 --2- 172.21.15.137:0/4065754138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4084011980 tx=0x7f4084011a60 comp rx=0 tx=0).stop 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 shutdown_connections 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 --2- 172.21.15.137:0/4065754138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088106de0 0x7f4088106740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 >> 172.21.15.137:0/4065754138 conn(0x7f40881007d0 msgr2=0x7f4088101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 shutdown_connections 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:17.819+0000 7f408f31f700 1 -- 172.21.15.137:0/4065754138 wait complete. 2024-06-15T00:57:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-15T00:57:18.357 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-15T00:57:18.357 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mgr... 2024-06-15T00:57:18.358 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-15T00:57:18.358 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-15T00:57:18.358 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:8443 ... 2024-06-15T00:57:18.613 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mgr.smithi137.nbzbyc 2024-06-15T00:57:18.613 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mgr.smithi137.nbzbyc.service: Unit ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mgr.smithi137.nbzbyc.service not loaded. 2024-06-15T00:57:18.763 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968.target.wants/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mgr.smithi137.nbzbyc.service → /etc/systemd/system/ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@.service. 2024-06-15T00:57:18.863 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:18 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/4065754138' entity='client.admin' 2024-06-15T00:57:19.328 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-15T00:57:19.329 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-06-15T00:57:19.337 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-15T00:57:19.337 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:57:19.337 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-06-15T00:57:19.337 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-06-15T00:57:19.348 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-15T00:57:19.349 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr to start... 2024-06-15T00:57:19.358 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr... 2024-06-15T00:57:20.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:19 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1647418305' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "225ca7a2-2ab2-11ef-bc9d-c7b262605968", 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-15T00:57:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "smithi137" 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-15T00:57:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-15T00:57:20.417 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-15T00:57:20.424 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-15T00:57:20.424 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-15T00:57:20.424 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-15T00:57:20.424 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.424 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-15T00:57:20.425 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-15T00:57:20.426 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-06-15T00:57:12.445315+0000", 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.809+0000 7efe70eec700 1 Processor -- start 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe70eec700 1 -- start start 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe70eec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe70eec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c106d20 con 0x7efe6c106330 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60864/0 (socket says 172.21.15.137:60864) 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.810+0000 7efe6a59c700 1 -- 172.21.15.137:0/4212724638 learned_addr learned my addr 172.21.15.137:0/4212724638 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.811+0000 7efe6a59c700 1 -- 172.21.15.137:0/4212724638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c107530 con 0x7efe6c106330 2024-06-15T00:57:20.427 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.811+0000 7efe6a59c700 1 --2- 172.21.15.137:0/4212724638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098a0 tx=0x7efe5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f3169306d52c77 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.811+0000 7efe6959a700 1 -- 172.21.15.137:0/4212724638 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5c004180 con 0x7efe6c106330 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.811+0000 7efe6959a700 1 -- 172.21.15.137:0/4212724638 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7efe5c0042e0 con 0x7efe6c106330 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.811+0000 7efe6959a700 1 -- 172.21.15.137:0/4212724638 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5c0119d0 con 0x7efe6c106330 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 -- 172.21.15.137:0/4212724638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 msgr2=0x7efe6c106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 --2- 172.21.15.137:0/4212724638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098a0 tx=0x7efe5c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 -- 172.21.15.137:0/4212724638 shutdown_connections 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 --2- 172.21.15.137:0/4212724638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c106750 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 -- 172.21.15.137:0/4212724638 >> 172.21.15.137:0/4212724638 conn(0x7efe6c1018d0 msgr2=0x7efe6c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 -- 172.21.15.137:0/4212724638 shutdown_connections 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.812+0000 7efe70eec700 1 -- 172.21.15.137:0/4212724638 wait complete. 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe70eec700 1 Processor -- start 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe70eec700 1 -- start start 2024-06-15T00:57:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe70eec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe70eec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19a510 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe6a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60866/0 (socket says 172.21.15.137:60866) 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe6a59c700 1 -- 172.21.15.137:0/1647418305 learned_addr learned my addr 172.21.15.137:0/1647418305 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe6a59c700 1 -- 172.21.15.137:0/1647418305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c009550 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.813+0000 7efe6a59c700 1 --2- 172.21.15.137:0/1647418305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006af0 tx=0x7efe5c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5c024430 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7efe5c0249e0 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c19a710 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5c01a970 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c19ab30 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efe5c02d3f0 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.814+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efe5c0197e0 con 0x7efe6c106330 2024-06-15T00:57:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.815+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c0623c0 con 0x7efe6c106330 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.817+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7efe5c024b50 con 0x7efe6c106330 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.890+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efe6c10a730 con 0x7efe6c106330 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.891+0000 7efe5b7fe700 1 -- 172.21.15.137:0/1647418305 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7efe5c024cf0 con 0x7efe6c106330 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 msgr2=0x7efe6c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 --2- 172.21.15.137:0/1647418305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006af0 tx=0x7efe5c011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 shutdown_connections 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 --2- 172.21.15.137:0/1647418305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe6c106330 0x7efe6c199fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 >> 172.21.15.137:0/1647418305 conn(0x7efe6c1018d0 msgr2=0x7efe6c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 shutdown_connections 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:19.892+0000 7efe70eec700 1 -- 172.21.15.137:0/1647418305 wait complete. 2024-06-15T00:57:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (1/15)... 2024-06-15T00:57:23.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:23.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:23.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "225ca7a2-2ab2-11ef-bc9d-c7b262605968", 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "smithi137" 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:23.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-15T00:57:23.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-15T00:57:23.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-15T00:57:23.526 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-06-15T00:57:12.445315+0000", 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:23.527 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.863+0000 7fec635a1700 1 Processor -- start 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.863+0000 7fec635a1700 1 -- start start 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec635a1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec635a1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c07a7e0 con 0x7fec5c07bdb0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec6133d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec6133d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60876/0 (socket says 172.21.15.137:60876) 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec6133d700 1 -- 172.21.15.137:0/1232151193 learned_addr learned my addr 172.21.15.137:0/1232151193 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec6133d700 1 -- 172.21.15.137:0/1232151193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c07a920 con 0x7fec5c07bdb0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec6133d700 1 --2- 172.21.15.137:0/1232151193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec54009e80 tx=0x7fec5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b759e466ffdf8409 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec5bfff700 1 -- 172.21.15.137:0/1232151193 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec5400e070 con 0x7fec5c07bdb0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec5bfff700 1 -- 172.21.15.137:0/1232151193 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fec54003e70 con 0x7fec5c07bdb0 2024-06-15T00:57:23.528 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.864+0000 7fec5bfff700 1 -- 172.21.15.137:0/1232151193 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec540042e0 con 0x7fec5c07bdb0 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 -- 172.21.15.137:0/1232151193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 msgr2=0x7fec5c07a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 --2- 172.21.15.137:0/1232151193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec54009e80 tx=0x7fec5400b330 comp rx=0 tx=0).stop 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 -- 172.21.15.137:0/1232151193 shutdown_connections 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 --2- 172.21.15.137:0/1232151193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c07a210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 -- 172.21.15.137:0/1232151193 >> 172.21.15.137:0/1232151193 conn(0x7fec5c1018d0 msgr2=0x7fec5c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 -- 172.21.15.137:0/1232151193 shutdown_connections 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.865+0000 7fec635a1700 1 -- 172.21.15.137:0/1232151193 wait complete. 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec635a1700 1 Processor -- start 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec635a1700 1 -- start start 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec635a1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec635a1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19c610 con 0x7fec5c07bdb0 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec6133d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec6133d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60882/0 (socket says 172.21.15.137:60882) 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec6133d700 1 -- 172.21.15.137:0/2913899918 learned_addr learned my addr 172.21.15.137:0/2913899918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.866+0000 7fec6133d700 1 -- 172.21.15.137:0/2913899918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54009b30 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec6133d700 1 --2- 172.21.15.137:0/2913899918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec54009530 tx=0x7fec5400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec5400e040 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c19c810 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fec540047b0 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec5401b680 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c19cc90 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fec54019040 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.867+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fec5401a390 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.868+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c0623c0 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.872+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fec54029060 con 0x7fec5c07bdb0 2024-06-15T00:57:23.530 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.943+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fec5c19f670 con 0x7fec5c07bdb0 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.944+0000 7fec5a7fc700 1 -- 172.21.15.137:0/2913899918 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fec5401f070 con 0x7fec5c07bdb0 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 msgr2=0x7fec5c19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 --2- 172.21.15.137:0/2913899918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec54009530 tx=0x7fec5400bc30 comp rx=0 tx=0).stop 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 shutdown_connections 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 --2- 172.21.15.137:0/2913899918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec5c07bdb0 0x7fec5c19c0d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 >> 172.21.15.137:0/2913899918 conn(0x7fec5c1018d0 msgr2=0x7fec5c102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 shutdown_connections 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:22.945+0000 7fec635a1700 1 -- 172.21.15.137:0/2913899918 wait complete. 2024-06-15T00:57:23.531 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (2/15)... 2024-06-15T00:57:23.564 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:23 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2913899918' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: Activating manager daemon smithi137.nbzbyc 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: mgrmap e2: smithi137.nbzbyc(active, starting, since 0.00321669s) 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3572976105' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T00:57:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T00:57:26.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T00:57:26.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:26.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:26.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:26 smithi137 ceph-mon[29354]: from='mgr.14100 172.21.15.137:0/976280408' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:28.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:27 smithi137 ceph-mon[29354]: mgrmap e3: smithi137.nbzbyc(active, since 1.00721s) 2024-06-15T00:57:29.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:29 smithi137 ceph-mon[29354]: mgrmap e4: smithi137.nbzbyc(active, since 2s) 2024-06-15T00:57:31.522 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:31.522 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:31.522 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "225ca7a2-2ab2-11ef-bc9d-c7b262605968", 2024-06-15T00:57:31.522 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-15T00:57:31.523 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "smithi137" 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-15T00:57:31.524 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-15T00:57:31.525 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-15T00:57:31.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-15T00:57:31.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-06-15T00:57:12.445315+0000", 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:31.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.911+0000 7f202018b700 1 Processor -- start 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f202018b700 1 -- start start 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f202018b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f202018b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2018105c20 con 0x7f2018105230 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f201df27700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f201df27700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60884/0 (socket says 172.21.15.137:60884) 2024-06-15T00:57:31.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f201df27700 1 -- 172.21.15.137:0/3276791655 learned_addr learned my addr 172.21.15.137:0/3276791655 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f201df27700 1 -- 172.21.15.137:0/3276791655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018106430 con 0x7f2018105230 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.912+0000 7f201df27700 1 --2- 172.21.15.137:0/3276791655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20140098a0 tx=0x7f2014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c36ebeea29d99cf server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f201cf25700 1 -- 172.21.15.137:0/3276791655 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2014004180 con 0x7f2018105230 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f201cf25700 1 -- 172.21.15.137:0/3276791655 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f20140042e0 con 0x7f2018105230 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f201cf25700 1 -- 172.21.15.137:0/3276791655 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20140119d0 con 0x7f2018105230 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 -- 172.21.15.137:0/3276791655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 msgr2=0x7f2018105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 --2- 172.21.15.137:0/3276791655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20140098a0 tx=0x7f2014009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 -- 172.21.15.137:0/3276791655 shutdown_connections 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 --2- 172.21.15.137:0/3276791655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018105650 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 -- 172.21.15.137:0/3276791655 >> 172.21.15.137:0/3276791655 conn(0x7f20181007d0 msgr2=0x7f2018102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 -- 172.21.15.137:0/3276791655 shutdown_connections 2024-06-15T00:57:31.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.913+0000 7f202018b700 1 -- 172.21.15.137:0/3276791655 wait complete. 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f202018b700 1 Processor -- start 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f202018b700 1 -- start start 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f202018b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f202018b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201819cd60 con 0x7f2018105230 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f201df27700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f201df27700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60894/0 (socket says 172.21.15.137:60894) 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f201df27700 1 -- 172.21.15.137:0/3572976105 learned_addr learned my addr 172.21.15.137:0/3572976105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f201df27700 1 -- 172.21.15.137:0/3572976105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014009550 con 0x7f2018105230 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.914+0000 7f201df27700 1 --2- 172.21.15.137:0/3572976105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2014006af0 tx=0x7f2014011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:31.541 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2014003b40 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f201401a880 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2014019890 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201819cea0 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201819d260 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f201402d440 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.915+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2014024800 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.916+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20180623c0 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.918+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f201401adf0 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.989+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7f201401a9f0 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.990+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2018109630 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.991+0000 7f200effd700 1 -- 172.21.15.137:0/3572976105 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f201402b340 con 0x7f2018105230 2024-06-15T00:57:31.542 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 msgr2=0x7f2018079a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 --2- 172.21.15.137:0/3572976105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2014006af0 tx=0x7f2014011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 shutdown_connections 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 --2- 172.21.15.137:0/3572976105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2018105230 0x7f2018079a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 >> 172.21.15.137:0/3572976105 conn(0x7f20181007d0 msgr2=0x7f2018101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 shutdown_connections 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:25.992+0000 7f202018b700 1 -- 172.21.15.137:0/3572976105 wait complete. 2024-06-15T00:57:31.543 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (3/15)... 2024-06-15T00:57:37.545 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:37 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2789986671' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "225ca7a2-2ab2-11ef-bc9d-c7b262605968", 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-15T00:57:37.842 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "smithi137" 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-15T00:57:37.843 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-15T00:57:37.844 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-15T00:57:37.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-15T00:57:37.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-06-15T00:57:12.445315+0000", 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.991+0000 7f3ade8ef700 1 Processor -- start 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.991+0000 7f3ade8ef700 1 -- start start 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ade8ef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ade8ef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8105c20 con 0x7f3ad8105230 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42346/0 (socket says 172.21.15.137:42346) 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ad7fff700 1 -- 172.21.15.137:0/3380385178 learned_addr learned my addr 172.21.15.137:0/3380385178 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ad7fff700 1 -- 172.21.15.137:0/3380385178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad8106430 con 0x7f3ad8105230 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.992+0000 7f3ad7fff700 1 --2- 172.21.15.137:0/3380385178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80098a0 tx=0x7f3ac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3026d681cb01ccd server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ad77fe700 1 -- 172.21.15.137:0/3380385178 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac8004180 con 0x7f3ad8105230 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ad77fe700 1 -- 172.21.15.137:0/3380385178 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3ac80042e0 con 0x7f3ad8105230 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ad77fe700 1 -- 172.21.15.137:0/3380385178 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac80119d0 con 0x7f3ad8105230 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 -- 172.21.15.137:0/3380385178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 msgr2=0x7f3ad8105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:37.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/3380385178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80098a0 tx=0x7f3ac8009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 -- 172.21.15.137:0/3380385178 shutdown_connections 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/3380385178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8105650 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 -- 172.21.15.137:0/3380385178 >> 172.21.15.137:0/3380385178 conn(0x7f3ad81007d0 msgr2=0x7f3ad8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 -- 172.21.15.137:0/3380385178 shutdown_connections 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.993+0000 7f3ade8ef700 1 -- 172.21.15.137:0/3380385178 wait complete. 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ade8ef700 1 Processor -- start 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ade8ef700 1 -- start start 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ade8ef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ade8ef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8199470 con 0x7f3ad8105230 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ad7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42348/0 (socket says 172.21.15.137:42348) 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.994+0000 7f3ad7fff700 1 -- 172.21.15.137:0/2789986671 learned_addr learned my addr 172.21.15.137:0/2789986671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ad7fff700 1 -- 172.21.15.137:0/2789986671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8009550 con 0x7f3ad8105230 2024-06-15T00:57:37.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ad7fff700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006af0 tx=0x7f3ac8003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac8004110 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3ac8020070 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac801c6e0 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad8199670 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.995+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad8199a90 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.996+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f3ac8022030 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.996+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad80623c0 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.996+0000 7f3ad5ffb700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 0x7f3ac403a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.996+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3ac804ea20 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.996+0000 7f3acffff700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 0x7f3ac403a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:36.997+0000 7f3acffff700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 0x7f3ac403a750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006fd0 tx=0x7f3ac0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.001+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3ac80243d0 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.265+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3ad81096b0 con 0x7f3ad8105230 2024-06-15T00:57:37.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.265+0000 7f3ad5ffb700 1 -- 172.21.15.137:0/2789986671 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f3ac80243d0 con 0x7f3ad8105230 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 msgr2=0x7f3ac403a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 0x7f3ac403a750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006fd0 tx=0x7f3ac0006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 msgr2=0x7f3ad8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006af0 tx=0x7f3ac8003980 comp rx=0 tx=0).stop 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 shutdown_connections 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f3ac4038290 0x7f3ac403a750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 --2- 172.21.15.137:0/2789986671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ad8105230 0x7f3ad8198f30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 >> 172.21.15.137:0/2789986671 conn(0x7f3ad81007d0 msgr2=0x7f3ad8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 shutdown_connections 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:37.267+0000 7f3ade8ef700 1 -- 172.21.15.137:0/2789986671 wait complete. 2024-06-15T00:57:37.858 INFO:teuthology.orchestra.run.smithi137.stdout:mgr is available 2024-06-15T00:57:38.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:38 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2182552550' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-15T00:57:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97b1088700 1 Processor -- start 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97b1088700 1 -- start start 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97b1088700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97b1088700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac106d20 con 0x7f97ac106330 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42362/0 (socket says 172.21.15.137:42362) 2024-06-15T00:57:39.088 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97aad9d700 1 -- 172.21.15.137:0/3817218325 learned_addr learned my addr 172.21.15.137:0/3817218325 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97aad9d700 1 -- 172.21.15.137:0/3817218325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac107530 con 0x7f97ac106330 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.284+0000 7f97aad9d700 1 --2- 172.21.15.137:0/3817218325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f979c0098a0 tx=0x7f979c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518c8ec4ffb16e31 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97a9d9b700 1 -- 172.21.15.137:0/3817218325 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f979c004180 con 0x7f97ac106330 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97a9d9b700 1 -- 172.21.15.137:0/3817218325 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f979c0042e0 con 0x7f97ac106330 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97a9d9b700 1 -- 172.21.15.137:0/3817218325 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f979c0119d0 con 0x7f97ac106330 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 -- 172.21.15.137:0/3817218325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 msgr2=0x7f97ac106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 --2- 172.21.15.137:0/3817218325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f979c0098a0 tx=0x7f979c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 -- 172.21.15.137:0/3817218325 shutdown_connections 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 --2- 172.21.15.137:0/3817218325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac106750 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 -- 172.21.15.137:0/3817218325 >> 172.21.15.137:0/3817218325 conn(0x7f97ac101910 msgr2=0x7f97ac103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 -- 172.21.15.137:0/3817218325 shutdown_connections 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.285+0000 7f97b1088700 1 -- 172.21.15.137:0/3817218325 wait complete. 2024-06-15T00:57:39.089 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.286+0000 7f97b1088700 1 Processor -- start 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.286+0000 7f97b1088700 1 -- start start 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.286+0000 7f97b1088700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.286+0000 7f97b1088700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19a590 con 0x7f97ac106330 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.286+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42368/0 (socket says 172.21.15.137:42368) 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97aad9d700 1 -- 172.21.15.137:0/2182552550 learned_addr learned my addr 172.21.15.137:0/2182552550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97aad9d700 1 -- 172.21.15.137:0/2182552550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009550 con 0x7f97ac106330 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97aad9d700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f979c009130 tx=0x7f979c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f979c003900 con 0x7f97ac106330 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac19a790 con 0x7f97ac106330 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f979c003a60 con 0x7f97ac106330 2024-06-15T00:57:39.090 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f979c0246e0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.287+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac19abb0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.288+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f979c02d3f0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.288+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0623c0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.288+0000 7f9793fff700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 0x7f979403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.288+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f979c04c280 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.288+0000 7f97aa59c700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 0x7f979403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.289+0000 7f97aa59c700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 0x7f979403a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.293+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f979c02d6a0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.480+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f97ac10a760 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.481+0000 7f9793fff700 1 -- 172.21.15.137:0/2182552550 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f979c00b3e0 con 0x7f97ac106330 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.483+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 msgr2=0x7f979403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 0x7f979403a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:39.091 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 msgr2=0x7f97ac19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f979c009130 tx=0x7f979c011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 shutdown_connections 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f97940382d0 0x7f979403a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 --2- 172.21.15.137:0/2182552550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97ac106330 0x7f97ac19a050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 >> 172.21.15.137:0/2182552550 conn(0x7f97ac101910 msgr2=0x7f97ac102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 shutdown_connections 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:38.484+0000 7f97b1088700 1 -- 172.21.15.137:0/2182552550 wait complete. 2024-06-15T00:57:39.092 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling cephadm module... 2024-06-15T00:57:40.099 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:39 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3221586802' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.487+0000 7f0bce413700 1 Processor -- start 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.487+0000 7f0bce413700 1 -- start start 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bce413700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bce413700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8106d00 con 0x7f0bc8106310 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42378/0 (socket says 172.21.15.137:42378) 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc7fff700 1 -- 172.21.15.137:0/861826999 learned_addr learned my addr 172.21.15.137:0/861826999 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc7fff700 1 -- 172.21.15.137:0/861826999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc8107510 con 0x7f0bc8106310 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc7fff700 1 --2- 172.21.15.137:0/861826999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098a0 tx=0x7f0bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8695962df31948e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc77fe700 1 -- 172.21.15.137:0/861826999 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bb8004180 con 0x7f0bc8106310 2024-06-15T00:57:40.358 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc77fe700 1 -- 172.21.15.137:0/861826999 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f0bb80042e0 con 0x7f0bc8106310 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.488+0000 7f0bc77fe700 1 -- 172.21.15.137:0/861826999 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bb80119d0 con 0x7f0bc8106310 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 -- 172.21.15.137:0/861826999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 msgr2=0x7f0bc8106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 --2- 172.21.15.137:0/861826999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098a0 tx=0x7f0bb8009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 -- 172.21.15.137:0/861826999 shutdown_connections 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 --2- 172.21.15.137:0/861826999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc8106730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 -- 172.21.15.137:0/861826999 >> 172.21.15.137:0/861826999 conn(0x7f0bc81018d0 msgr2=0x7f0bc8103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 -- 172.21.15.137:0/861826999 shutdown_connections 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.489+0000 7f0bce413700 1 -- 172.21.15.137:0/861826999 wait complete. 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bce413700 1 Processor -- start 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bce413700 1 -- start start 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bce413700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bce413700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc819a550 con 0x7f0bc8106310 2024-06-15T00:57:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42382/0 (socket says 172.21.15.137:42382) 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc7fff700 1 -- 172.21.15.137:0/3221586802 learned_addr learned my addr 172.21.15.137:0/3221586802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc7fff700 1 -- 172.21.15.137:0/3221586802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009550 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc7fff700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8006af0 tx=0x7f0bb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bb8024430 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f0bb80249e0 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc819a750 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.490+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bb801a970 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.491+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc819ab70 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.491+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f0bb801abe0 con 0x7f0bc8106310 2024-06-15T00:57:40.360 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.491+0000 7f0bc5ffb700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 0x7f0bb403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.491+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc80623c0 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.491+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0bb804c4d0 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.492+0000 7f0bbffff700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 0x7f0bb403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.492+0000 7f0bbffff700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 0x7f0bb403a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006fd0 tx=0x7f0bb0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.496+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0bb80193c0 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.740+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f0bc819dc60 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.793+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0bb802d5d0 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.794+0000 7f0bc5ffb700 1 -- 172.21.15.137:0/3221586802 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f0bb804bef0 con 0x7f0bc8106310 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 msgr2=0x7f0bb403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 0x7f0bb403a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006fd0 tx=0x7f0bb0006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 msgr2=0x7f0bc819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8006af0 tx=0x7f0bb8011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:40.361 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 shutdown_connections 2024-06-15T00:57:40.362 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7f0bb4038320 0x7f0bb403a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:40.362 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 --2- 172.21.15.137:0/3221586802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc8106310 0x7f0bc819a010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:40.362 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 >> 172.21.15.137:0/3221586802 conn(0x7f0bc81018d0 msgr2=0x7f0bc8102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:40.362 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 shutdown_connections 2024-06-15T00:57:40.362 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:39.797+0000 7f0bce413700 1 -- 172.21.15.137:0/3221586802 wait complete. 2024-06-15T00:57:40.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:40 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3221586802' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-15T00:57:40.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:40 smithi137 ceph-mon[29354]: mgrmap e5: smithi137.nbzbyc(active, since 13s) 2024-06-15T00:57:41.683 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:41.683 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-15T00:57:41.683 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-15T00:57:41.683 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "smithi137.nbzbyc", 2024-06-15T00:57:41.683 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.769+0000 7fe261dfa700 1 Processor -- start 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.769+0000 7fe261dfa700 1 -- start start 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.769+0000 7fe261dfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.769+0000 7fe261dfa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c07a7e0 con 0x7fe25c07bdb0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42404/0 (socket says 172.21.15.137:42404) 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25b7fe700 1 -- 172.21.15.137:0/1029865470 learned_addr learned my addr 172.21.15.137:0/1029865470 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25b7fe700 1 -- 172.21.15.137:0/1029865470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c07a920 con 0x7fe25c07bdb0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25b7fe700 1 --2- 172.21.15.137:0/1029865470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f717bcb699116 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25a7fc700 1 -- 172.21.15.137:0/1029865470 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe250004180 con 0x7fe25c07bdb0 2024-06-15T00:57:41.684 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25a7fc700 1 -- 172.21.15.137:0/1029865470 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fe2500042e0 con 0x7fe25c07bdb0 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.770+0000 7fe25a7fc700 1 -- 172.21.15.137:0/1029865470 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2500119d0 con 0x7fe25c07bdb0 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 -- 172.21.15.137:0/1029865470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 msgr2=0x7fe25c07a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 --2- 172.21.15.137:0/1029865470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 -- 172.21.15.137:0/1029865470 shutdown_connections 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 --2- 172.21.15.137:0/1029865470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c07a210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 -- 172.21.15.137:0/1029865470 >> 172.21.15.137:0/1029865470 conn(0x7fe25c1018d0 msgr2=0x7fe25c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 -- 172.21.15.137:0/1029865470 shutdown_connections 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 -- 172.21.15.137:0/1029865470 wait complete. 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.771+0000 7fe261dfa700 1 Processor -- start 2024-06-15T00:57:41.685 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe261dfa700 1 -- start start 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe261dfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe261dfa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19c730 con 0x7fe25c07bdb0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42406/0 (socket says 172.21.15.137:42406) 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe25b7fe700 1 -- 172.21.15.137:0/2024667979 learned_addr learned my addr 172.21.15.137:0/2024667979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe25b7fe700 1 -- 172.21.15.137:0/2024667979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250009550 con 0x7fe25c07bdb0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe25b7fe700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe250011980 tx=0x7fe250011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.772+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe250003b40 con 0x7fe25c07bdb0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fe25001a880 con 0x7fe25c07bdb0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c19c930 con 0x7fe25c07bdb0 2024-06-15T00:57:41.686 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe250019910 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c19cd50 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe250019bd0 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.773+0000 7fe258ff9700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 0x7fe24403a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.774+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe25004c700 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.774+0000 7fe25affd700 1 -- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 msgr2=0x7fe24403a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.774+0000 7fe25affd700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 0x7fe24403a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.774+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c10c630 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.779+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe25002d440 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.975+0000 7fe25affd700 1 -- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 msgr2=0x7fe24403a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:40.975+0000 7fe25affd700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 0x7fe24403a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.068+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe25c19fd30 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.070+0000 7fe258ff9700 1 -- 172.21.15.137:0/2024667979 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe2500172b0 con 0x7fe25c07bdb0 2024-06-15T00:57:41.687 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.074+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 msgr2=0x7fe24403a7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.074+0000 7fe261dfa700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 0x7fe24403a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.074+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 msgr2=0x7fe25c19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.074+0000 7fe261dfa700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe250011980 tx=0x7fe250011a60 comp rx=0 tx=0).stop 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.077+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 shutdown_connections 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.077+0000 7fe261dfa700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fe2440382e0 0x7fe24403a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.077+0000 7fe261dfa700 1 --2- 172.21.15.137:0/2024667979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe25c07bdb0 0x7fe25c19c1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.077+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 >> 172.21.15.137:0/2024667979 conn(0x7fe25c1018d0 msgr2=0x7fe25c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.079+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 shutdown_connections 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:41.079+0000 7fe261dfa700 1 -- 172.21.15.137:0/2024667979 wait complete. 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-06-15T00:57:41.688 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 5... 2024-06-15T00:57:42.046 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:41 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2024667979' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-15T00:57:46.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: Active manager daemon smithi137.nbzbyc restarted 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: Activating manager daemon smithi137.nbzbyc 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: osdmap e2: 0 total, 0 up, 0 in 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: mgrmap e6: smithi137.nbzbyc(active, starting, since 0.00330153s) 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T00:57:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T00:57:46.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:46 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T00:57:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:47 smithi137 ceph-mon[29354]: Found migration_current of "None". Setting to last migration. 2024-06-15T00:57:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:47 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:47 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:47 smithi137 ceph-mon[29354]: mgrmap e7: smithi137.nbzbyc(active, since 1.00552s) 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd232f31700 1 Processor -- start 2024-06-15T00:57:48.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd232f31700 1 -- start start 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd232f31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd232f31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0744f0 con 0x7fd22c106de0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd230ccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd230ccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42416/0 (socket says 172.21.15.137:42416) 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd230ccd700 1 -- 172.21.15.137:0/2285956707 learned_addr learned my addr 172.21.15.137:0/2285956707 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.044+0000 7fd230ccd700 1 -- 172.21.15.137:0/2285956707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c109710 con 0x7fd22c106de0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.045+0000 7fd230ccd700 1 --2- 172.21.15.137:0/2285956707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2200098a0 tx=0x7fd220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315132afce696cc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.045+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2285956707 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd220004180 con 0x7fd22c106de0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.045+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2285956707 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd2200042e0 con 0x7fd22c106de0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.045+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2285956707 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2200119d0 con 0x7fd22c106de0 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- 172.21.15.137:0/2285956707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 msgr2=0x7fd22c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 --2- 172.21.15.137:0/2285956707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2200098a0 tx=0x7fd220009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- 172.21.15.137:0/2285956707 shutdown_connections 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 --2- 172.21.15.137:0/2285956707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c1091d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- 172.21.15.137:0/2285956707 >> 172.21.15.137:0/2285956707 conn(0x7fd22c1007f0 msgr2=0x7fd22c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- 172.21.15.137:0/2285956707 shutdown_connections 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- 172.21.15.137:0/2285956707 wait complete. 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 Processor -- start 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.046+0000 7fd232f31700 1 -- start start 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd232f31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd232f31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c19c750 con 0x7fd22c106de0 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd230ccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd230ccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42432/0 (socket says 172.21.15.137:42432) 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd230ccd700 1 -- 172.21.15.137:0/1776817046 learned_addr learned my addr 172.21.15.137:0/1776817046 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd230ccd700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009550 con 0x7fd22c106de0 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd230ccd700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd220009130 tx=0x7fd220011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:48.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd220003a00 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd22001a880 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.047+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2200197b0 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.048+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c19c950 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.048+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c19cd70 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd22001a9f0 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd229ffb700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd21c03aef0 con 0x7fd21c038320 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd22004c5e0 con 0x7fd22c106de0 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd22bfff700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.049+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.250+0000 7fd22bfff700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.250+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-15T00:57:48.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.651+0000 7fd22bfff700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:42.651+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:43.452+0000 7fd22bfff700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:43.452+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:45.053+0000 7fd22bfff700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/3423141192 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:45.053+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:46.427+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7fd22001a430 con 0x7fd22c106de0 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:46.428+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 msgr2=0x7fd21c03a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:46.428+0000 7fd229ffb700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.429+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fd22004bae0 con 0x7fd22c106de0 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.429+0000 7fd229ffb700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 0x7fd21c03e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.429+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd21c03aef0 con 0x7fd21c03bd20 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.430+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 0x7fd21c03e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.430+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 0x7fd21c03e110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd218003a10 tx=0x7fd2180092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:48.051 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.431+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd21c03aef0 con 0x7fd21c03bd20 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.437+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd22c19d550 con 0x7fd21c03bd20 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.437+0000 7fd229ffb700 1 -- 172.21.15.137:0/1776817046 <== mgr.14120 v2:172.21.15.137:6800/4018707027 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd22c19d550 con 0x7fd21c03bd20 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 msgr2=0x7fd21c03e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 0x7fd21c03e110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd218003a10 tx=0x7fd2180092b0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 msgr2=0x7fd22c19c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd220009130 tx=0x7fd220011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 shutdown_connections 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd21c03bd20 0x7fd21c03e110 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c106de0 0x7fd22c19c210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 --2- 172.21.15.137:0/1776817046 >> [v2:172.21.15.137:6800/3423141192,v1:172.21.15.137:6801/3423141192] conn(0x7fd21c038320 0x7fd21c03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 >> 172.21.15.137:0/1776817046 conn(0x7fd22c1007f0 msgr2=0x7fd22c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 shutdown_connections 2024-06-15T00:57:48.052 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:47.438+0000 7fd232f31700 1 -- 172.21.15.137:0/1776817046 wait complete. 2024-06-15T00:57:48.053 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 5 is available 2024-06-15T00:57:48.053 INFO:teuthology.orchestra.run.smithi137.stdout:Setting orchestrator backend to cephadm... 2024-06-15T00:57:48.934 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-15T00:57:48.934 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-15T00:57:48.934 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: [15/Jun/2024:00:57:47] ENGINE Bus STARTING 2024-06-15T00:57:48.934 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: [15/Jun/2024:00:57:47] ENGINE Serving on http://172.21.15.137:8765 2024-06-15T00:57:48.934 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: [15/Jun/2024:00:57:47] ENGINE Serving on https://172.21.15.137:7150 2024-06-15T00:57:48.935 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: [15/Jun/2024:00:57:47] ENGINE Bus STARTED 2024-06-15T00:57:48.935 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:48 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:49.177 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.474+0000 7f46e0356700 1 Processor -- start 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.474+0000 7f46e0356700 1 -- start start 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.474+0000 7f46e0356700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.474+0000 7f46e0356700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8105c20 con 0x7f46d8105230 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46de0f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46de0f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60314/0 (socket says 172.21.15.137:60314) 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46de0f2700 1 -- 172.21.15.137:0/1610600280 learned_addr learned my addr 172.21.15.137:0/1610600280 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46de0f2700 1 -- 172.21.15.137:0/1610600280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8106430 con 0x7f46d8105230 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46de0f2700 1 --2- 172.21.15.137:0/1610600280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46d40098a0 tx=0x7f46d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6984c83d72db0ed server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:49.178 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46dd0f0700 1 -- 172.21.15.137:0/1610600280 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d4004180 con 0x7f46d8105230 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46dd0f0700 1 -- 172.21.15.137:0/1610600280 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f46d40042e0 con 0x7f46d8105230 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.475+0000 7f46dd0f0700 1 -- 172.21.15.137:0/1610600280 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d40119d0 con 0x7f46d8105230 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- 172.21.15.137:0/1610600280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 msgr2=0x7f46d8105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 --2- 172.21.15.137:0/1610600280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46d40098a0 tx=0x7f46d4009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- 172.21.15.137:0/1610600280 shutdown_connections 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 --2- 172.21.15.137:0/1610600280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d8105650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- 172.21.15.137:0/1610600280 >> 172.21.15.137:0/1610600280 conn(0x7f46d81007f0 msgr2=0x7f46d8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- 172.21.15.137:0/1610600280 shutdown_connections 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- 172.21.15.137:0/1610600280 wait complete. 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 Processor -- start 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.476+0000 7f46e0356700 1 -- start start 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46e0356700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46e0356700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d819a5a0 con 0x7f46d8105230 2024-06-15T00:57:49.179 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46de0f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46de0f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60322/0 (socket says 172.21.15.137:60322) 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46de0f2700 1 -- 172.21.15.137:0/493909995 learned_addr learned my addr 172.21.15.137:0/493909995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46de0f2700 1 -- 172.21.15.137:0/493909995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4009550 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46de0f2700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009130 tx=0x7f46d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d4003a00 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f46d401a880 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d819a7a0 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46d4019760 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.477+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d819abc0 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.478+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f46d40199f0 con 0x7f46d8105230 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.478+0000 7f46cf7fe700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 0x7f46c403a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:49.180 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.478+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f46d404c470 con 0x7f46d8105230 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.478+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d80623c0 con 0x7f46d8105230 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.478+0000 7f46dd8f1700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 0x7f46c403a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.479+0000 7f46dd8f1700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 0x7f46c403a6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006fd0 tx=0x7f46c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.483+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f46d402c3a0 con 0x7f46d8105230 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.629+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f46d4016d40 con 0x7f46d8105230 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.686+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d8102ae0 con 0x7f46c4038210 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.695+0000 7f46cf7fe700 1 -- 172.21.15.137:0/493909995 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f46d8102ae0 con 0x7f46c4038210 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.697+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 msgr2=0x7f46c403a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.697+0000 7f46e0356700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 0x7f46c403a6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006fd0 tx=0x7f46c8006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.697+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 msgr2=0x7f46d819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.697+0000 7f46e0356700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009130 tx=0x7f46d4011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:49.181 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 shutdown_connections 2024-06-15T00:57:49.182 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f46c4038210 0x7f46c403a6d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:49.182 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 --2- 172.21.15.137:0/493909995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d8105230 0x7f46d819a060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:49.182 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 >> 172.21.15.137:0/493909995 conn(0x7f46d81007f0 msgr2=0x7f46d8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:49.182 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 shutdown_connections 2024-06-15T00:57:49.182 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:48.698+0000 7f46e0356700 1 -- 172.21.15.137:0/493909995 wait complete. 2024-06-15T00:57:49.841 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:49 smithi137 ceph-mon[29354]: mgrmap e8: smithi137.nbzbyc(active, since 2s) 2024-06-15T00:57:49.842 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:49 smithi137 ceph-mon[29354]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:49.842 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:49 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:49.842 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:49 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:50.398 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.606+0000 7f0d99295700 1 Processor -- start 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.606+0000 7f0d99295700 1 -- start start 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.606+0000 7f0d99295700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.606+0000 7f0d99295700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94106d00 con 0x7f0d94106310 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.606+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60328/0 (socket says 172.21.15.137:60328) 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d92ffd700 1 -- 172.21.15.137:0/834462068 learned_addr learned my addr 172.21.15.137:0/834462068 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d92ffd700 1 -- 172.21.15.137:0/834462068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d94107510 con 0x7f0d94106310 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d92ffd700 1 --2- 172.21.15.137:0/834462068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d880098a0 tx=0x7f0d88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d5ec3135b4a244 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d91ffb700 1 -- 172.21.15.137:0/834462068 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d88004180 con 0x7f0d94106310 2024-06-15T00:57:50.399 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d91ffb700 1 -- 172.21.15.137:0/834462068 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f0d880042e0 con 0x7f0d94106310 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.607+0000 7f0d91ffb700 1 -- 172.21.15.137:0/834462068 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d880119d0 con 0x7f0d94106310 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 -- 172.21.15.137:0/834462068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 msgr2=0x7f0d94106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 --2- 172.21.15.137:0/834462068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d880098a0 tx=0x7f0d88009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 -- 172.21.15.137:0/834462068 shutdown_connections 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 --2- 172.21.15.137:0/834462068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d94106730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 -- 172.21.15.137:0/834462068 >> 172.21.15.137:0/834462068 conn(0x7f0d941018d0 msgr2=0x7f0d94103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 -- 172.21.15.137:0/834462068 shutdown_connections 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.608+0000 7f0d99295700 1 -- 172.21.15.137:0/834462068 wait complete. 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d99295700 1 Processor -- start 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d99295700 1 -- start start 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d99295700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:50.400 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d99295700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419a550 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60342/0 (socket says 172.21.15.137:60342) 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.609+0000 7f0d92ffd700 1 -- 172.21.15.137:0/369795004 learned_addr learned my addr 172.21.15.137:0/369795004 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d92ffd700 1 -- 172.21.15.137:0/369795004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009550 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d92ffd700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d88006af0 tx=0x7f0d88011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d88024430 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f0d880249e0 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9419a750 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d8801a970 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.610+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9419ab70 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.611+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0d8801ac30 con 0x7f0d94106310 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.611+0000 7f0d7bfff700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 0x7f0d7c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:50.401 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.611+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d940623c0 con 0x7f0d94106310 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.611+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d8804c3b0 con 0x7f0d94106310 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.611+0000 7f0d927fc700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 0x7f0d7c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.612+0000 7f0d927fc700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 0x7f0d7c03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006fd0 tx=0x7f0d84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.616+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0d880193c0 con 0x7f0d94106310 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.840+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d9410a710 con 0x7f0d7c038320 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.841+0000 7f0d7bfff700 1 -- 172.21.15.137:0/369795004 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0d9410a710 con 0x7f0d7c038320 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 msgr2=0x7f0d7c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 0x7f0d7c03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006fd0 tx=0x7f0d84006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 msgr2=0x7f0d9419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:50.402 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d88006af0 tx=0x7f0d88011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 shutdown_connections 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f0d7c038320 0x7f0d7c03a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 --2- 172.21.15.137:0/369795004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94106310 0x7f0d9419a010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.843+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 >> 172.21.15.137:0/369795004 conn(0x7f0d941018d0 msgr2=0x7f0d94102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.844+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 shutdown_connections 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:49.844+0000 7f0d99295700 1 -- 172.21.15.137:0/369795004 wait complete. 2024-06-15T00:57:50.403 INFO:teuthology.orchestra.run.smithi137.stdout:Generating ssh key... 2024-06-15T00:57:50.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:50 smithi137 ceph-mon[29354]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f922d07e700 1 Processor -- start 2024-06-15T00:57:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f922d07e700 1 -- start start 2024-06-15T00:57:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f922d07e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f922d07e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228079d80 con 0x7f9228104bf0 2024-06-15T00:57:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f9226d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f9226d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60352/0 (socket says 172.21.15.137:60352) 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f9226d9d700 1 -- 172.21.15.137:0/485836116 learned_addr learned my addr 172.21.15.137:0/485836116 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.869+0000 7f9226d9d700 1 -- 172.21.15.137:0/485836116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228105550 con 0x7f9228104bf0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f9226d9d700 1 --2- 172.21.15.137:0/485836116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92180098a0 tx=0x7f9218009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af81ae131792cbb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f9225d9b700 1 -- 172.21.15.137:0/485836116 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9218004180 con 0x7f9228104bf0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f9225d9b700 1 -- 172.21.15.137:0/485836116 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f92180042e0 con 0x7f9228104bf0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f9225d9b700 1 -- 172.21.15.137:0/485836116 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92180119d0 con 0x7f9228104bf0 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 -- 172.21.15.137:0/485836116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 msgr2=0x7f9228105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 --2- 172.21.15.137:0/485836116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92180098a0 tx=0x7f9218009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 -- 172.21.15.137:0/485836116 shutdown_connections 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 --2- 172.21.15.137:0/485836116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f9228105010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 -- 172.21.15.137:0/485836116 >> 172.21.15.137:0/485836116 conn(0x7f92281007d0 msgr2=0x7f9228102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 -- 172.21.15.137:0/485836116 shutdown_connections 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.870+0000 7f922d07e700 1 -- 172.21.15.137:0/485836116 wait complete. 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.871+0000 7f922d07e700 1 Processor -- start 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.871+0000 7f922d07e700 1 -- start start 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f922d07e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f922d07e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819c6e0 con 0x7f9228104bf0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f9226d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f9226d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60368/0 (socket says 172.21.15.137:60368) 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f9226d9d700 1 -- 172.21.15.137:0/2209128922 learned_addr learned my addr 172.21.15.137:0/2209128922 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f9226d9d700 1 -- 172.21.15.137:0/2209128922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218009550 con 0x7f9228104bf0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f9226d9d700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9218006af0 tx=0x7f9218011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9218024430 con 0x7f9228104bf0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f92180249e0 con 0x7f9228104bf0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922819c8e0 con 0x7f9228104bf0 2024-06-15T00:57:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f921801aa00 con 0x7f9228104bf0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.872+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922819cd00 con 0x7f9228104bf0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.873+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f921801ac70 con 0x7f9228104bf0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.873+0000 7f920ffff700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 0x7f921003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.873+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f921804d450 con 0x7f9228104bf0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.873+0000 7f922659c700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 0x7f921003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.873+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228195a70 con 0x7f9228104bf0 2024-06-15T00:57:51.826 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.874+0000 7f922659c700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 0x7f921003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:50.878+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f921802d3f0 con 0x7f9228104bf0 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.068+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f92280611d0 con 0x7f9210038370 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.261+0000 7f920ffff700 1 -- 172.21.15.137:0/2209128922 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f92280611d0 con 0x7f9210038370 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 msgr2=0x7f921003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 0x7f921003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 msgr2=0x7f922819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9218006af0 tx=0x7f9218011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 shutdown_connections 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f9210038370 0x7f921003a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 --2- 172.21.15.137:0/2209128922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9228104bf0 0x7f922819c1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 >> 172.21.15.137:0/2209128922 conn(0x7f92281007d0 msgr2=0x7f92281907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:51.827 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 shutdown_connections 2024-06-15T00:57:51.828 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:51.263+0000 7f922d07e700 1 -- 172.21.15.137:0/2209128922 wait complete. 2024-06-15T00:57:52.476 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:52 smithi137 ceph-mon[29354]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:52 smithi137 ceph-mon[29354]: Generating ssh key... 2024-06-15T00:57:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:52 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:52 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHUDL1iv5GE4Y7PW4fdZAxRH193NO5v32YaFlrjZmhdH9CRcIAujQV57CIXWh8sm++a0iKjKuOcQaadj3DiwB4HYFl27JwtPKygvOaOMjWwG+CRQovkXTMKBagam5exrY/POjshF4PIAmCa69kjciFbxAEL3touqEmx1TT0i0kIhwhZPx3de2zaM6Xtl2avR1Ty1wZ+yCC2qTGnkQzja0wjK+z09yQ1oUFU0C5vJ7ifxHb+RzU9vjDB26Faf5CEYYEgeCM0vVNm/VzQr1c9nrmOODGHigaSoi0uBaCnT7dypgmrrM5RPLuOxu8X77F8u8OwpniAqmMbQM7EYZWqdLt71mqWfdKzod1r5KjzEMeRM4jZUMCNteYlQQAKYt/Sp8kTO5fH/34usJqUIwPYgxbdWud/xrCkZP4ncHq/x3ZoAef1LwdUhzmZA7+BK1AK1XkNS5hpK8dXhadlWo2lyiVScOcLtklOL/8JKbGQsbiYWBuzxyDG8w2Cu6NEZzIVE= ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.275+0000 7f31932ee700 1 Processor -- start 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.275+0000 7f31932ee700 1 -- start start 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f31932ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f31932ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c105000 con 0x7f318c104610 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f319108a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f319108a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60384/0 (socket says 172.21.15.137:60384) 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f319108a700 1 -- 172.21.15.137:0/2873169296 learned_addr learned my addr 172.21.15.137:0/2873169296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f319108a700 1 -- 172.21.15.137:0/2873169296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c1075a0 con 0x7f318c104610 2024-06-15T00:57:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f319108a700 1 --2- 172.21.15.137:0/2873169296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f317c0098a0 tx=0x7f317c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832c979abc7a3441 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f3183fff700 1 -- 172.21.15.137:0/2873169296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f317c004180 con 0x7f318c104610 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f3183fff700 1 -- 172.21.15.137:0/2873169296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f317c0042e0 con 0x7f318c104610 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.276+0000 7f3183fff700 1 -- 172.21.15.137:0/2873169296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f317c0119d0 con 0x7f318c104610 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 -- 172.21.15.137:0/2873169296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 msgr2=0x7f318c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 --2- 172.21.15.137:0/2873169296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f317c0098a0 tx=0x7f317c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 -- 172.21.15.137:0/2873169296 shutdown_connections 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 --2- 172.21.15.137:0/2873169296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c104a30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 -- 172.21.15.137:0/2873169296 >> 172.21.15.137:0/2873169296 conn(0x7f318c0fff40 msgr2=0x7f318c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 -- 172.21.15.137:0/2873169296 shutdown_connections 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.277+0000 7f31932ee700 1 -- 172.21.15.137:0/2873169296 wait complete. 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f31932ee700 1 Processor -- start 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f31932ee700 1 -- start start 2024-06-15T00:57:53.013 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f31932ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f31932ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c19a510 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f319108a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f319108a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60394/0 (socket says 172.21.15.137:60394) 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f319108a700 1 -- 172.21.15.137:0/538553851 learned_addr learned my addr 172.21.15.137:0/538553851 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f319108a700 1 -- 172.21.15.137:0/538553851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009550 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f319108a700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f317c006af0 tx=0x7f317c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.278+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f317c003b40 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.279+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f317c01a880 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.279+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c19a710 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.279+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f317c019890 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.279+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c19ab30 con 0x7f318c104610 2024-06-15T00:57:53.014 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f317c019ac0 con 0x7f318c104610 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c0623c0 con 0x7f318c104610 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f31827fc700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 0x7f317803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f317c04c7f0 con 0x7f318c104610 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f3190889700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 0x7f317803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.280+0000 7f3190889700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 0x7f317803a7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3188006fd0 tx=0x7f3188006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.285+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f317c02b340 con 0x7f318c104610 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.476+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f318c101460 con 0x7f3178038330 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.476+0000 7f31827fc700 1 -- 172.21.15.137:0/538553851 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f318c101460 con 0x7f3178038330 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 msgr2=0x7f317803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 0x7f317803a7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3188006fd0 tx=0x7f3188006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 msgr2=0x7f318c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:53.015 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f317c006af0 tx=0x7f317c011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 shutdown_connections 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3178038330 0x7f317803a7f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 --2- 172.21.15.137:0/538553851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318c104610 0x7f318c199fd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.478+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 >> 172.21.15.137:0/538553851 conn(0x7f318c0fff40 msgr2=0x7f318c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.479+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 shutdown_connections 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:52.479+0000 7f31932ee700 1 -- 172.21.15.137:0/538553851 wait complete. 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:Adding key to root@localhost authorized_keys... 2024-06-15T00:57:53.016 INFO:teuthology.orchestra.run.smithi137.stdout:Adding host smithi137... 2024-06-15T00:57:53.599 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:53 smithi137 ceph-mon[29354]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:54.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:54 smithi137 ceph-mon[29354]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:55.582 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:55 smithi137 ceph-mon[29354]: Deploying cephadm binary to smithi137 2024-06-15T00:57:56.535 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Added host 'smithi137' with addr '172.21.15.137' 2024-06-15T00:57:56.535 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaf17ed700 1 Processor -- start 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaf17ed700 1 -- start start 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaf17ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaf17ed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0744b0 con 0x7fdaec104c10 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60396/0 (socket says 172.21.15.137:60396) 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.399+0000 7fdaeaffd700 1 -- 172.21.15.137:0/1424607722 learned_addr learned my addr 172.21.15.137:0/1424607722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaeaffd700 1 -- 172.21.15.137:0/1424607722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec105570 con 0x7fdaec104c10 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaeaffd700 1 --2- 172.21.15.137:0/1424607722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0098a0 tx=0x7fdadc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28977cfdc0c086b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdae9ffb700 1 -- 172.21.15.137:0/1424607722 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdadc004180 con 0x7fdaec104c10 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdae9ffb700 1 -- 172.21.15.137:0/1424607722 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fdadc0042e0 con 0x7fdaec104c10 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdae9ffb700 1 -- 172.21.15.137:0/1424607722 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdadc0119d0 con 0x7fdaec104c10 2024-06-15T00:57:56.536 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 -- 172.21.15.137:0/1424607722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 msgr2=0x7fdaec105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/1424607722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0098a0 tx=0x7fdadc009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 -- 172.21.15.137:0/1424607722 shutdown_connections 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/1424607722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec105030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 -- 172.21.15.137:0/1424607722 >> 172.21.15.137:0/1424607722 conn(0x7fdaec1007f0 msgr2=0x7fdaec102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 -- 172.21.15.137:0/1424607722 shutdown_connections 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.400+0000 7fdaf17ed700 1 -- 172.21.15.137:0/1424607722 wait complete. 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaf17ed700 1 Processor -- start 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaf17ed700 1 -- start start 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaf17ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaf17ed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec19c760 con 0x7fdaec104c10 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60410/0 (socket says 172.21.15.137:60410) 2024-06-15T00:57:56.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.401+0000 7fdaeaffd700 1 -- 172.21.15.137:0/2587433936 learned_addr learned my addr 172.21.15.137:0/2587433936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdaeaffd700 1 -- 172.21.15.137:0/2587433936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc009550 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdaeaffd700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009130 tx=0x7fdadc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdadc003a00 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fdadc01a880 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec19c960 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdadc0197b0 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.402+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec19cd80 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fdadc019a70 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec195b00 con 0x7fdaec104c10 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdad3fff700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 0x7fdad403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdadc04c680 con 0x7fdaec104c10 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdaea7fc700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 0x7fdad403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.403+0000 7fdaea7fc700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 0x7fdad403a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.408+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdadc02b750 con 0x7fdaec104c10 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:53.598+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}) v1 -- 0x7fdaec0611d0 con 0x7fdad40382d0 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.976+0000 7fdad3fff700 1 -- 172.21.15.137:0/2587433936 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdaec0611d0 con 0x7fdad40382d0 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 msgr2=0x7fdad403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 0x7fdad403a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 msgr2=0x7fdaec19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009130 tx=0x7fdadc011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 shutdown_connections 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fdad40382d0 0x7fdad403a790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 --2- 172.21.15.137:0/2587433936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec104c10 0x7fdaec19c220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:56.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 >> 172.21.15.137:0/2587433936 conn(0x7fdaec1007f0 msgr2=0x7fdaec1907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:56.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 shutdown_connections 2024-06-15T00:57:56.540 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:55.979+0000 7fdaf17ed700 1 -- 172.21.15.137:0/2587433936 wait complete. 2024-06-15T00:57:56.540 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying mon service with default placement... 2024-06-15T00:57:57.089 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:56 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:57.089 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:56 smithi137 ceph-mon[29354]: Added host smithi137 2024-06-15T00:57:57.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:56 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:57:58.607 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:58 smithi137 ceph-mon[29354]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:57:58.607 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:58 smithi137 ceph-mon[29354]: Saving service mon spec with placement count:5 2024-06-15T00:57:58.607 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:58 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:58.607 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:58 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:58.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378f3b0700 1 Processor -- start 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378f3b0700 1 -- start start 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378f3b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378f3b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788104fd0 con 0x7f37881045e0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378d14c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378d14c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58782/0 (socket says 172.21.15.137:58782) 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.117+0000 7f378d14c700 1 -- 172.21.15.137:0/1903506550 learned_addr learned my addr 172.21.15.137:0/1903506550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.118+0000 7f378d14c700 1 -- 172.21.15.137:0/1903506550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788105110 con 0x7f37881045e0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.118+0000 7f378d14c700 1 --2- 172.21.15.137:0/1903506550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37840098a0 tx=0x7f3784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02484a89a735413 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.118+0000 7f377ffff700 1 -- 172.21.15.137:0/1903506550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3784004180 con 0x7f37881045e0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.118+0000 7f377ffff700 1 -- 172.21.15.137:0/1903506550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f37840042e0 con 0x7f37881045e0 2024-06-15T00:57:58.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.118+0000 7f377ffff700 1 -- 172.21.15.137:0/1903506550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37840119d0 con 0x7f37881045e0 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- 172.21.15.137:0/1903506550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 msgr2=0x7f3788104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 --2- 172.21.15.137:0/1903506550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37840098a0 tx=0x7f3784009bb0 comp rx=0 tx=0).stop 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- 172.21.15.137:0/1903506550 shutdown_connections 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 --2- 172.21.15.137:0/1903506550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788104a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- 172.21.15.137:0/1903506550 >> 172.21.15.137:0/1903506550 conn(0x7f37880fff40 msgr2=0x7f3788102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- 172.21.15.137:0/1903506550 shutdown_connections 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- 172.21.15.137:0/1903506550 wait complete. 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 Processor -- start 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- start start 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.119+0000 7f378f3b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378819a4d0 con 0x7f37881045e0 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378d14c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:58.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378d14c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58786/0 (socket says 172.21.15.137:58786) 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378d14c700 1 -- 172.21.15.137:0/102994618 learned_addr learned my addr 172.21.15.137:0/102994618 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378d14c700 1 -- 172.21.15.137:0/102994618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009550 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378d14c700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3784006af0 tx=0x7f3784011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3784024430 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f37840249e0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f378401a970 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378819a6d0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.120+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378819aaf0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.121+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f378401abe0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.121+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37880623c0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.121+0000 7f377e7fc700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 0x7f377403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.121+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f378404c3a0 con 0x7f37881045e0 2024-06-15T00:57:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.121+0000 7f378c94b700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 0x7f377403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.122+0000 7f378c94b700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 0x7f377403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3778006fd0 tx=0x7f3778006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.126+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3784024590 con 0x7f37881045e0 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.335+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f378819d950 con 0x7f3774038370 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.339+0000 7f377e7fc700 1 -- 172.21.15.137:0/102994618 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f378819d950 con 0x7f3774038370 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 msgr2=0x7f377403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 0x7f377403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3778006fd0 tx=0x7f3778006e40 comp rx=0 tx=0).stop 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 msgr2=0x7f3788199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3784006af0 tx=0x7f3784011a30 comp rx=0 tx=0).stop 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 shutdown_connections 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f3774038370 0x7f377403a830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 --2- 172.21.15.137:0/102994618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37881045e0 0x7f3788199f90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.341+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 >> 172.21.15.137:0/102994618 conn(0x7f37880fff40 msgr2=0x7f3788101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:57:58.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.342+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 shutdown_connections 2024-06-15T00:57:58.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:57.342+0000 7f378f3b0700 1 -- 172.21.15.137:0/102994618 wait complete. 2024-06-15T00:57:58.618 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying mgr service with default placement... 2024-06-15T00:57:59.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:59 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:57:59.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:57:59 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:00.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-15T00:58:00.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.046+0000 7fe85e7e7700 1 Processor -- start 2024-06-15T00:58:00.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.046+0000 7fe85e7e7700 1 -- start start 2024-06-15T00:58:00.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.046+0000 7fe85e7e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:00.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.046+0000 7fe85e7e7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858106110 con 0x7fe858105720 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe857fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe857fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58788/0 (socket says 172.21.15.137:58788) 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe857fff700 1 -- 172.21.15.137:0/3629032311 learned_addr learned my addr 172.21.15.137:0/3629032311 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe857fff700 1 -- 172.21.15.137:0/3629032311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858106440 con 0x7fe858105720 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe857fff700 1 --2- 172.21.15.137:0/3629032311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098a0 tx=0x7fe848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28584314d3d78e1b server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe856ffd700 1 -- 172.21.15.137:0/3629032311 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848004180 con 0x7fe858105720 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe856ffd700 1 -- 172.21.15.137:0/3629032311 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe8480042e0 con 0x7fe858105720 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.047+0000 7fe856ffd700 1 -- 172.21.15.137:0/3629032311 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848011a30 con 0x7fe858105720 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.048+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3629032311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 msgr2=0x7fe858105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.048+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3629032311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098a0 tx=0x7fe848009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.048+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3629032311 shutdown_connections 2024-06-15T00:58:00.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.048+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3629032311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858105b40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.048+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3629032311 >> 172.21.15.137:0/3629032311 conn(0x7fe8580fff40 msgr2=0x7fe858102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.049+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3629032311 shutdown_connections 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.049+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3629032311 wait complete. 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.049+0000 7fe85e7e7700 1 Processor -- start 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.049+0000 7fe85e7e7700 1 -- start start 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe85e7e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:00.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe85e7e7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85819cd20 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe857fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe857fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58804/0 (socket says 172.21.15.137:58804) 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe857fff700 1 -- 172.21.15.137:0/3407884195 learned_addr learned my addr 172.21.15.137:0/3407884195 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe857fff700 1 -- 172.21.15.137:0/3407884195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848009550 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe857fff700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe848006af0 tx=0x7fe848011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848003bc0 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe84801a8c0 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe848019860 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85819ce60 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.050+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85819d220 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.051+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe848019b20 con 0x7fe858105720 2024-06-15T00:58:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.052+0000 7fe8557fa700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 0x7fe84003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.052+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe84804ca70 con 0x7fe858105720 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.052+0000 7fe8577fe700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 0x7fe84003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.052+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580623c0 con 0x7fe858105720 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.052+0000 7fe8577fe700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 0x7fe84003a7a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.057+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe8480243f0 con 0x7fe858105720 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.270+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fe858102210 con 0x7fe8400382e0 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.275+0000 7fe8557fa700 1 -- 172.21.15.137:0/3407884195 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe858102210 con 0x7fe8400382e0 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 msgr2=0x7fe84003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 0x7fe84003a7a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 msgr2=0x7fe858104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe848006af0 tx=0x7fe848011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 shutdown_connections 2024-06-15T00:58:00.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe8400382e0 0x7fe84003a7a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:00.378 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 --2- 172.21.15.137:0/3407884195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe858105720 0x7fe858104de0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:00.378 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 >> 172.21.15.137:0/3407884195 conn(0x7fe8580fff40 msgr2=0x7fe8581015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:00.378 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 shutdown_connections 2024-06-15T00:58:00.378 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:57:59.277+0000 7fe85e7e7700 1 -- 172.21.15.137:0/3407884195 wait complete. 2024-06-15T00:58:00.378 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying crash service with default placement... 2024-06-15T00:58:00.819 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:00 smithi137 ceph-mon[29354]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:00.820 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:00 smithi137 ceph-mon[29354]: Saving service mgr spec with placement count:2 2024-06-15T00:58:00.820 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:00 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.819+0000 7fb526412700 1 Processor -- start 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.819+0000 7fb526412700 1 -- start start 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.819+0000 7fb526412700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.819+0000 7fb526412700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520079da0 con 0x7fb520106de0 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58812/0 (socket says 172.21.15.137:58812) 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51ffff700 1 -- 172.21.15.137:0/3962303229 learned_addr learned my addr 172.21.15.137:0/3962303229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:01.639 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51ffff700 1 -- 172.21.15.137:0/3962303229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520109710 con 0x7fb520106de0 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51ffff700 1 --2- 172.21.15.137:0/3962303229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098a0 tx=0x7fb510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fe4633616f715c server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51effd700 1 -- 172.21.15.137:0/3962303229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb510004180 con 0x7fb520106de0 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51effd700 1 -- 172.21.15.137:0/3962303229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb5100042e0 con 0x7fb520106de0 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.820+0000 7fb51effd700 1 -- 172.21.15.137:0/3962303229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb510011a30 con 0x7fb520106de0 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 -- 172.21.15.137:0/3962303229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 msgr2=0x7fb5201091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 --2- 172.21.15.137:0/3962303229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098a0 tx=0x7fb510009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 -- 172.21.15.137:0/3962303229 shutdown_connections 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 --2- 172.21.15.137:0/3962303229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb5201091d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 -- 172.21.15.137:0/3962303229 >> 172.21.15.137:0/3962303229 conn(0x7fb5201007d0 msgr2=0x7fb520102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 -- 172.21.15.137:0/3962303229 shutdown_connections 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.821+0000 7fb526412700 1 -- 172.21.15.137:0/3962303229 wait complete. 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.822+0000 7fb526412700 1 Processor -- start 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.822+0000 7fb526412700 1 -- start start 2024-06-15T00:58:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.822+0000 7fb526412700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.822+0000 7fb526412700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52019a560 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58820/0 (socket says 172.21.15.137:58820) 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51ffff700 1 -- 172.21.15.137:0/240816991 learned_addr learned my addr 172.21.15.137:0/240816991 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51ffff700 1 -- 172.21.15.137:0/240816991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510009550 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51ffff700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb510011a00 tx=0x7fb510011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb510003bc0 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb5100248c0 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52019a760 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb51001a850 con 0x7fb520106de0 2024-06-15T00:58:01.641 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.823+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52019ab80 con 0x7fb520106de0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.824+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb51001ab40 con 0x7fb520106de0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.824+0000 7fb51d7fa700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 0x7fb50803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.824+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52010a440 con 0x7fb520106de0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.824+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb51004c5f0 con 0x7fb520106de0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.824+0000 7fb51f7fe700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 0x7fb50803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.825+0000 7fb51f7fe700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 0x7fb50803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb514006fd0 tx=0x7fb514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:00.829+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb51001adf0 con 0x7fb520106de0 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.041+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fb52019d9e0 con 0x7fb508038370 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.048+0000 7fb51d7fa700 1 -- 172.21.15.137:0/240816991 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fb52019d9e0 con 0x7fb508038370 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 msgr2=0x7fb50803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 0x7fb50803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb514006fd0 tx=0x7fb514006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 msgr2=0x7fb52019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb510011a00 tx=0x7fb510011ae0 comp rx=0 tx=0).stop 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 shutdown_connections 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb508038370 0x7fb50803a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 --2- 172.21.15.137:0/240816991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb520106de0 0x7fb52019a020 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.050+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 >> 172.21.15.137:0/240816991 conn(0x7fb5201007d0 msgr2=0x7fb520101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.051+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 shutdown_connections 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:01.051+0000 7fb526412700 1 -- 172.21.15.137:0/240816991 wait complete. 2024-06-15T00:58:01.643 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying ceph-exporter service with default placement... 2024-06-15T00:58:02.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:02 smithi137 ceph-mon[29354]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:02.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:02 smithi137 ceph-mon[29354]: Saving service crash spec with placement * 2024-06-15T00:58:02.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:02 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:02.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:02 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:03.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:03 smithi137 ceph-mon[29354]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:03.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:03 smithi137 ceph-mon[29354]: Saving service ceph-exporter spec with placement * 2024-06-15T00:58:03.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:03 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1ae648a700 1 Processor -- start 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1ae648a700 1 -- start start 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1ae648a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1ae648a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0106d20 con 0x7f1ae0106330 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1adffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:03.349 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1adffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58824/0 (socket says 172.21.15.137:58824) 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.045+0000 7f1adffff700 1 -- 172.21.15.137:0/3814527688 learned_addr learned my addr 172.21.15.137:0/3814527688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1adffff700 1 -- 172.21.15.137:0/3814527688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0107530 con 0x7f1ae0106330 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1adffff700 1 --2- 172.21.15.137:0/3814527688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098a0 tx=0x7f1ad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191db27074e5816f server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1adeffd700 1 -- 172.21.15.137:0/3814527688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad0004180 con 0x7f1ae0106330 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1adeffd700 1 -- 172.21.15.137:0/3814527688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1ad00042e0 con 0x7f1ae0106330 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1adeffd700 1 -- 172.21.15.137:0/3814527688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad0011a30 con 0x7f1ae0106330 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1ae648a700 1 -- 172.21.15.137:0/3814527688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 msgr2=0x7f1ae0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.046+0000 7f1ae648a700 1 --2- 172.21.15.137:0/3814527688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098a0 tx=0x7f1ad0009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 -- 172.21.15.137:0/3814527688 shutdown_connections 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 --2- 172.21.15.137:0/3814527688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae0106750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:03.350 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 -- 172.21.15.137:0/3814527688 >> 172.21.15.137:0/3814527688 conn(0x7f1ae0101910 msgr2=0x7f1ae0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 -- 172.21.15.137:0/3814527688 shutdown_connections 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 -- 172.21.15.137:0/3814527688 wait complete. 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 Processor -- start 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.047+0000 7f1ae648a700 1 -- start start 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1ae648a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1ae648a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019a5d0 con 0x7f1ae0106330 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1adffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1adffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58838/0 (socket says 172.21.15.137:58838) 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1adffff700 1 -- 172.21.15.137:0/910270925 learned_addr learned my addr 172.21.15.137:0/910270925 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1adffff700 1 -- 172.21.15.137:0/910270925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009550 con 0x7f1ae0106330 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.048+0000 7f1adffff700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0006af0 tx=0x7f1ad0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.050+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad0003a80 con 0x7f1ae0106330 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.050+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1ad0003be0 con 0x7f1ae0106330 2024-06-15T00:58:03.351 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.050+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae019a7d0 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.050+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad0019770 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.050+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae019abf0 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1ad0019a30 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1add7fa700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 0x7f1ac803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ad004d4c0 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1adf7fe700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 0x7f1ac803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae0194110 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.051+0000 7f1adf7fe700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 0x7f1ac803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006fd0 tx=0x7f1ad4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.058+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1ad002c3a0 con 0x7f1ae0106330 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.291+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae010a8a0 con 0x7f1ac80382e0 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.295+0000 7f1add7fa700 1 -- 172.21.15.137:0/910270925 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1ae010a8a0 con 0x7f1ac80382e0 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.297+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 msgr2=0x7f1ac803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.297+0000 7f1ae648a700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 0x7f1ac803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006fd0 tx=0x7f1ad4006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.297+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 msgr2=0x7f1ae019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.297+0000 7f1ae648a700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0006af0 tx=0x7f1ad0011b20 comp rx=0 tx=0).stop 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 shutdown_connections 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1ac80382e0 0x7f1ac803a7a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 --2- 172.21.15.137:0/910270925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae0106330 0x7f1ae019a090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 >> 172.21.15.137:0/910270925 conn(0x7f1ae0101910 msgr2=0x7f1ae0103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 shutdown_connections 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:02.298+0000 7f1ae648a700 1 -- 172.21.15.137:0/910270925 wait complete. 2024-06-15T00:58:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying prometheus service with default placement... 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.772+0000 7f1dd6443700 1 Processor -- start 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.772+0000 7f1dd6443700 1 -- start start 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dd6443700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dd6443700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd0106d20 con 0x7f1dd0106330 2024-06-15T00:58:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58854/0 (socket says 172.21.15.137:58854) 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dcffff700 1 -- 172.21.15.137:0/2062930674 learned_addr learned my addr 172.21.15.137:0/2062930674 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dcffff700 1 -- 172.21.15.137:0/2062930674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0107530 con 0x7f1dd0106330 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dcffff700 1 --2- 172.21.15.137:0/2062930674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db80098a0 tx=0x7f1db8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8ed7cc39bb2748 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dceffd700 1 -- 172.21.15.137:0/2062930674 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db8004180 con 0x7f1dd0106330 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dceffd700 1 -- 172.21.15.137:0/2062930674 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1db80042e0 con 0x7f1dd0106330 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.773+0000 7f1dceffd700 1 -- 172.21.15.137:0/2062930674 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db8011a30 con 0x7f1dd0106330 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 -- 172.21.15.137:0/2062930674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 msgr2=0x7f1dd0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 --2- 172.21.15.137:0/2062930674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db80098a0 tx=0x7f1db8009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 -- 172.21.15.137:0/2062930674 shutdown_connections 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 --2- 172.21.15.137:0/2062930674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd0106750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 -- 172.21.15.137:0/2062930674 >> 172.21.15.137:0/2062930674 conn(0x7f1dd0101910 msgr2=0x7f1dd0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 -- 172.21.15.137:0/2062930674 shutdown_connections 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.774+0000 7f1dd6443700 1 -- 172.21.15.137:0/2062930674 wait complete. 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dd6443700 1 Processor -- start 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dd6443700 1 -- start start 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dd6443700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dd6443700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019a5d0 con 0x7f1dd0106330 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58862/0 (socket says 172.21.15.137:58862) 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dcffff700 1 -- 172.21.15.137:0/557717013 learned_addr learned my addr 172.21.15.137:0/557717013 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dcffff700 1 -- 172.21.15.137:0/557717013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db8009550 con 0x7f1dd0106330 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.775+0000 7f1dcffff700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.776+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db8003b40 con 0x7f1dd0106330 2024-06-15T00:58:04.651 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.776+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1db80248d0 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.776+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd019a7d0 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.776+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db801a820 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.776+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd019abf0 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1db8024440 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd0194110 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dcd7fa700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 0x7f1dbc03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1db804c330 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dcf7fe700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 0x7f1dbc03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.777+0000 7f1dcf7fe700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 0x7f1dbc03a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.782+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1db80172b0 con 0x7f1dd0106330 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.987+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd00611d0 con 0x7f1dbc038370 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.991+0000 7f1dcd7fa700 1 -- 172.21.15.137:0/557717013 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f1dd00611d0 con 0x7f1dbc038370 2024-06-15T00:58:04.652 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 msgr2=0x7f1dbc03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 0x7f1dbc03a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 msgr2=0x7f1dd019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db8011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 shutdown_connections 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f1dbc038370 0x7f1dbc03a830 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 --2- 172.21.15.137:0/557717013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0106330 0x7f1dd019a090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 >> 172.21.15.137:0/557717013 conn(0x7f1dd0101910 msgr2=0x7f1dd0103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 shutdown_connections 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:03.993+0000 7f1dd6443700 1 -- 172.21.15.137:0/557717013 wait complete. 2024-06-15T00:58:04.653 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying grafana service with default placement... 2024-06-15T00:58:05.069 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:04 smithi137 ceph-mon[29354]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:05.069 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:04 smithi137 ceph-mon[29354]: Saving service prometheus spec with placement count:1 2024-06-15T00:58:05.069 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:04 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb9951a1700 1 Processor -- start 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb9951a1700 1 -- start start 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb9951a1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb9951a1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990105000 con 0x7fb990104610 2024-06-15T00:58:05.856 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58876/0 (socket says 172.21.15.137:58876) 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.070+0000 7fb98ed9d700 1 -- 172.21.15.137:0/2833688370 learned_addr learned my addr 172.21.15.137:0/2833688370 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb98ed9d700 1 -- 172.21.15.137:0/2833688370 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9901075a0 con 0x7fb990104610 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb98ed9d700 1 --2- 172.21.15.137:0/2833688370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb9880098a0 tx=0x7fb988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad398f66871ae1b server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb98dd9b700 1 -- 172.21.15.137:0/2833688370 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb988004180 con 0x7fb990104610 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb98dd9b700 1 -- 172.21.15.137:0/2833688370 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb9880042e0 con 0x7fb990104610 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb98dd9b700 1 -- 172.21.15.137:0/2833688370 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb988011a30 con 0x7fb990104610 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb9951a1700 1 -- 172.21.15.137:0/2833688370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 msgr2=0x7fb990104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb9951a1700 1 --2- 172.21.15.137:0/2833688370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb9880098a0 tx=0x7fb988009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb9951a1700 1 -- 172.21.15.137:0/2833688370 shutdown_connections 2024-06-15T00:58:05.857 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb9951a1700 1 --2- 172.21.15.137:0/2833688370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb990104a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.071+0000 7fb9951a1700 1 -- 172.21.15.137:0/2833688370 >> 172.21.15.137:0/2833688370 conn(0x7fb9900fff40 msgr2=0x7fb990102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.072+0000 7fb9951a1700 1 -- 172.21.15.137:0/2833688370 shutdown_connections 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.072+0000 7fb9951a1700 1 -- 172.21.15.137:0/2833688370 wait complete. 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.072+0000 7fb9951a1700 1 Processor -- start 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.072+0000 7fb9951a1700 1 -- start start 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb9951a1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb9951a1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99019a590 con 0x7fb990104610 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb98ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58892/0 (socket says 172.21.15.137:58892) 2024-06-15T00:58:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb98ed9d700 1 -- 172.21.15.137:0/1127281325 learned_addr learned my addr 172.21.15.137:0/1127281325 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb98ed9d700 1 -- 172.21.15.137:0/1127281325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb988009550 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb98ed9d700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb988006af0 tx=0x7fb988011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb988003bc0 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb98801a8c0 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99019a790 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb988019820 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.073+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99019abb0 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.074+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb988019aa0 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.074+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9900623c0 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.074+0000 7fb987fff700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 0x7fb97c03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.075+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb98804c570 con 0x7fb990104610 2024-06-15T00:58:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.075+0000 7fb98e59c700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 0x7fb97c03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.075+0000 7fb98e59c700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 0x7fb97c03a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.079+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb988018760 con 0x7fb990104610 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.279+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fb990102250 con 0x7fb97c0382e0 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.284+0000 7fb987fff700 1 -- 172.21.15.137:0/1127281325 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb990102250 con 0x7fb97c0382e0 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 msgr2=0x7fb97c03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 0x7fb97c03a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 msgr2=0x7fb99019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb988006af0 tx=0x7fb988011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 shutdown_connections 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fb97c0382e0 0x7fb97c03a7a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:05.860 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 --2- 172.21.15.137:0/1127281325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb990104610 0x7fb99019a050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:05.861 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 >> 172.21.15.137:0/1127281325 conn(0x7fb9900fff40 msgr2=0x7fb9901015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:05.861 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 shutdown_connections 2024-06-15T00:58:05.861 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:05.286+0000 7fb9951a1700 1 -- 172.21.15.137:0/1127281325 wait complete. 2024-06-15T00:58:05.861 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying node-exporter service with default placement... 2024-06-15T00:58:06.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:06 smithi137 ceph-mon[29354]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:06.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:06 smithi137 ceph-mon[29354]: Saving service grafana spec with placement count:1 2024-06-15T00:58:06.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:06 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:07.045 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-06-15T00:58:07.045 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.258+0000 7fd676df6700 1 Processor -- start 2024-06-15T00:58:07.045 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd676df6700 1 -- start start 2024-06-15T00:58:07.045 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd676df6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd676df6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670079d80 con 0x7fd670106e00 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd674b92700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd674b92700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60802/0 (socket says 172.21.15.137:60802) 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd674b92700 1 -- 172.21.15.137:0/3485860019 learned_addr learned my addr 172.21.15.137:0/3485860019 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd674b92700 1 -- 172.21.15.137:0/3485860019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670109730 con 0x7fd670106e00 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.259+0000 7fd674b92700 1 --2- 172.21.15.137:0/3485860019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6640098a0 tx=0x7fd664009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ac32b7b5bea160 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd66f7fe700 1 -- 172.21.15.137:0/3485860019 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd664004180 con 0x7fd670106e00 2024-06-15T00:58:07.046 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd66f7fe700 1 -- 172.21.15.137:0/3485860019 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd6640042e0 con 0x7fd670106e00 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd66f7fe700 1 -- 172.21.15.137:0/3485860019 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd664011a30 con 0x7fd670106e00 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 -- 172.21.15.137:0/3485860019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 msgr2=0x7fd6701091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 --2- 172.21.15.137:0/3485860019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6640098a0 tx=0x7fd664009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 -- 172.21.15.137:0/3485860019 shutdown_connections 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 --2- 172.21.15.137:0/3485860019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd6701091f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 -- 172.21.15.137:0/3485860019 >> 172.21.15.137:0/3485860019 conn(0x7fd6701007d0 msgr2=0x7fd670102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 -- 172.21.15.137:0/3485860019 shutdown_connections 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.260+0000 7fd676df6700 1 -- 172.21.15.137:0/3485860019 wait complete. 2024-06-15T00:58:07.047 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd676df6700 1 Processor -- start 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd676df6700 1 -- start start 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd676df6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd676df6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67019a550 con 0x7fd670106e00 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd674b92700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd674b92700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60810/0 (socket says 172.21.15.137:60810) 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd674b92700 1 -- 172.21.15.137:0/3160282259 learned_addr learned my addr 172.21.15.137:0/3160282259 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.261+0000 7fd674b92700 1 -- 172.21.15.137:0/3160282259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009550 con 0x7fd670106e00 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd674b92700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd664006af0 tx=0x7fd664011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd664003bc0 con 0x7fd670106e00 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd66401a8c0 con 0x7fd670106e00 2024-06-15T00:58:07.048 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67019a750 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd664019860 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.262+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67019ab70 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.263+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd664019b20 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.263+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6700623c0 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.263+0000 7fd66dffb700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 0x7fd66003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.263+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd66404c8f0 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.263+0000 7fd66ffff700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 0x7fd66003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.264+0000 7fd66ffff700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 0x7fd66003a7a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.268+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd66402b750 con 0x7fd670106e00 2024-06-15T00:58:07.049 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.468+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd670102b10 con 0x7fd6600382e0 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.473+0000 7fd66dffb700 1 -- 172.21.15.137:0/3160282259 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd670102b10 con 0x7fd6600382e0 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 msgr2=0x7fd66003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 0x7fd66003a7a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 msgr2=0x7fd67019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd664006af0 tx=0x7fd664011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 shutdown_connections 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fd6600382e0 0x7fd66003a7a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 --2- 172.21.15.137:0/3160282259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670106e00 0x7fd67019a010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 >> 172.21.15.137:0/3160282259 conn(0x7fd6701007d0 msgr2=0x7fd670101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 shutdown_connections 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:06.475+0000 7fd676df6700 1 -- 172.21.15.137:0/3160282259 wait complete. 2024-06-15T00:58:07.050 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying alertmanager service with default placement... 2024-06-15T00:58:07.672 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:07 smithi137 ceph-mon[29354]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:07.673 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:07 smithi137 ceph-mon[29354]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:07.673 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:07 smithi137 ceph-mon[29354]: Saving service node-exporter spec with placement * 2024-06-15T00:58:07.673 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:07 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe6380b2700 1 Processor -- start 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe6380b2700 1 -- start start 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe6380b2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe6380b2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630079de0 con 0x7fe630106da0 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe635e4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe635e4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60812/0 (socket says 172.21.15.137:60812) 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.445+0000 7fe635e4e700 1 -- 172.21.15.137:0/2785911596 learned_addr learned my addr 172.21.15.137:0/2785911596 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:08.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.446+0000 7fe635e4e700 1 -- 172.21.15.137:0/2785911596 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6301096d0 con 0x7fe630106da0 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.446+0000 7fe635e4e700 1 --2- 172.21.15.137:0/2785911596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6200098a0 tx=0x7fe620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce770344954bfc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.446+0000 7fe634e4c700 1 -- 172.21.15.137:0/2785911596 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe620004180 con 0x7fe630106da0 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.446+0000 7fe634e4c700 1 -- 172.21.15.137:0/2785911596 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe6200042e0 con 0x7fe630106da0 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.446+0000 7fe634e4c700 1 -- 172.21.15.137:0/2785911596 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe620011a30 con 0x7fe630106da0 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- 172.21.15.137:0/2785911596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 msgr2=0x7fe630109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 --2- 172.21.15.137:0/2785911596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6200098a0 tx=0x7fe620009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- 172.21.15.137:0/2785911596 shutdown_connections 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 --2- 172.21.15.137:0/2785911596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630109190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- 172.21.15.137:0/2785911596 >> 172.21.15.137:0/2785911596 conn(0x7fe6301007d0 msgr2=0x7fe630102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- 172.21.15.137:0/2785911596 shutdown_connections 2024-06-15T00:58:08.160 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- 172.21.15.137:0/2785911596 wait complete. 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 Processor -- start 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- start start 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.447+0000 7fe6380b2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63019a530 con 0x7fe630106da0 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe635e4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe635e4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60816/0 (socket says 172.21.15.137:60816) 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe635e4e700 1 -- 172.21.15.137:0/171880196 learned_addr learned my addr 172.21.15.137:0/171880196 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe635e4e700 1 -- 172.21.15.137:0/171880196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620009550 con 0x7fe630106da0 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe635e4e700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe620011a00 tx=0x7fe620011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe620003bc0 con 0x7fe630106da0 2024-06-15T00:58:08.161 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe6200248c0 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe62001a850 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63019a730 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.448+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63019ab50 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.449+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe62001aac0 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.449+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63010a460 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.449+0000 7fe626ffd700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 0x7fe61c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.449+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe62004c610 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.449+0000 7fe63564d700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 0x7fe61c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.450+0000 7fe63564d700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 0x7fe61c03a830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.454+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe620024a30 con 0x7fe630106da0 2024-06-15T00:58:08.162 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.654+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fe63019d9b0 con 0x7fe61c038370 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.659+0000 7fe626ffd700 1 -- 172.21.15.137:0/171880196 <== mgr.14120 v2:172.21.15.137:6800/4018707027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fe63019d9b0 con 0x7fe61c038370 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 msgr2=0x7fe61c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 0x7fe61c03a830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 msgr2=0x7fe630199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe620011a00 tx=0x7fe620011ae0 comp rx=0 tx=0).stop 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 shutdown_connections 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7fe61c038370 0x7fe61c03a830 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 --2- 172.21.15.137:0/171880196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe630106da0 0x7fe630199ff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 >> 172.21.15.137:0/171880196 conn(0x7fe6301007d0 msgr2=0x7fe630101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 shutdown_connections 2024-06-15T00:58:08.163 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:07.661+0000 7fe6380b2700 1 -- 172.21.15.137:0/171880196 wait complete. 2024-06-15T00:58:08.830 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:08 smithi137 ceph-mon[29354]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:08.830 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:08 smithi137 ceph-mon[29354]: Saving service alertmanager spec with placement count:1 2024-06-15T00:58:08.830 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:08 smithi137 ceph-mon[29354]: from='mgr.14120 172.21.15.137:0/2951563140' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:09.439 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.631+0000 7f7f41246700 1 Processor -- start 2024-06-15T00:58:09.439 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.631+0000 7f7f41246700 1 -- start start 2024-06-15T00:58:09.439 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f41246700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:09.439 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f41246700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c07a800 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.439 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f3ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f3ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60832/0 (socket says 172.21.15.137:60832) 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f3ad9d700 1 -- 172.21.15.137:0/3989447722 learned_addr learned my addr 172.21.15.137:0/3989447722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f3ad9d700 1 -- 172.21.15.137:0/3989447722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c07a940 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f3ad9d700 1 --2- 172.21.15.137:0/3989447722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098a0 tx=0x7f7f2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53787286c3a25fe server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f39d9b700 1 -- 172.21.15.137:0/3989447722 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f2c004180 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f39d9b700 1 -- 172.21.15.137:0/3989447722 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7f2c0042e0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.632+0000 7f7f39d9b700 1 -- 172.21.15.137:0/3989447722 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f2c011a30 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 -- 172.21.15.137:0/3989447722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 msgr2=0x7f7f3c07a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 --2- 172.21.15.137:0/3989447722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098a0 tx=0x7f7f2c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 -- 172.21.15.137:0/3989447722 shutdown_connections 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 --2- 172.21.15.137:0/3989447722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c07a230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 -- 172.21.15.137:0/3989447722 >> 172.21.15.137:0/3989447722 conn(0x7f7f3c101910 msgr2=0x7f7f3c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 -- 172.21.15.137:0/3989447722 shutdown_connections 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.633+0000 7f7f41246700 1 -- 172.21.15.137:0/3989447722 wait complete. 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f41246700 1 Processor -- start 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f41246700 1 -- start start 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f41246700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f41246700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19c800 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f3ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f3ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60848/0 (socket says 172.21.15.137:60848) 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f3ad9d700 1 -- 172.21.15.137:0/2828684302 learned_addr learned my addr 172.21.15.137:0/2828684302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f3ad9d700 1 -- 172.21.15.137:0/2828684302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009550 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.634+0000 7f7f3ad9d700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c000c00 tx=0x7f7f2c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.635+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f2c003b40 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.635+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7f2c0248d0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.635+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c19ca00 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.441 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.635+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f2c01a820 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.635+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c19ce20 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7f2c01aae0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f23fff700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 0x7f7f2403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c196310 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7f2c04c3b0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f3a59c700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 0x7f7f2403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.636+0000 7f7f3a59c700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 0x7f7f2403a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f30006fd0 tx=0x7f7f30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.641+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f2c0233a0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.442 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.829+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7f3c0623c0 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.833+0000 7f7f23fff700 1 -- 172.21.15.137:0/2828684302 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7f2c02c350 con 0x7f7f3c07bdd0 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 msgr2=0x7f7f2403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 0x7f7f2403a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f30006fd0 tx=0x7f7f30006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 msgr2=0x7f7f3c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c000c00 tx=0x7f7f2c011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 shutdown_connections 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f7f24038320 0x7f7f2403a7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 --2- 172.21.15.137:0/2828684302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f3c07bdd0 0x7f7f3c19c2c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.835+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 >> 172.21.15.137:0/2828684302 conn(0x7f7f3c101910 msgr2=0x7f7f3c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.836+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 shutdown_connections 2024-06-15T00:58:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:08.836+0000 7f7f41246700 1 -- 172.21.15.137:0/2828684302 wait complete. 2024-06-15T00:58:10.042 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:09 smithi137 ceph-mon[29354]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:10.042 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:09 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2828684302' entity='client.admin' 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.845+0000 7f59b312b700 1 Processor -- start 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.845+0000 7f59b312b700 1 -- start start 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b312b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b312b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac106d20 con 0x7f59ac106330 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b0ec7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b0ec7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60862/0 (socket says 172.21.15.137:60862) 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b0ec7700 1 -- 172.21.15.137:0/1018939022 learned_addr learned my addr 172.21.15.137:0/1018939022 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b0ec7700 1 -- 172.21.15.137:0/1018939022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac107530 con 0x7f59ac106330 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59b0ec7700 1 --2- 172.21.15.137:0/1018939022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aa1b65dd3276d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:10.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59ab7fe700 1 -- 172.21.15.137:0/1018939022 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59a4004180 con 0x7f59ac106330 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59ab7fe700 1 -- 172.21.15.137:0/1018939022 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f59a40042e0 con 0x7f59ac106330 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.846+0000 7f59ab7fe700 1 -- 172.21.15.137:0/1018939022 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59a4011a30 con 0x7f59ac106330 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 -- 172.21.15.137:0/1018939022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 msgr2=0x7f59ac106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 --2- 172.21.15.137:0/1018939022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 -- 172.21.15.137:0/1018939022 shutdown_connections 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 --2- 172.21.15.137:0/1018939022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac106750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 -- 172.21.15.137:0/1018939022 >> 172.21.15.137:0/1018939022 conn(0x7f59ac101910 msgr2=0x7f59ac103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 -- 172.21.15.137:0/1018939022 shutdown_connections 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.847+0000 7f59b312b700 1 -- 172.21.15.137:0/1018939022 wait complete. 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b312b700 1 Processor -- start 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b312b700 1 -- start start 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b312b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b312b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac19a600 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b0ec7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b0ec7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60868/0 (socket says 172.21.15.137:60868) 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b0ec7700 1 -- 172.21.15.137:0/3085778402 learned_addr learned my addr 172.21.15.137:0/3085778402 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b0ec7700 1 -- 172.21.15.137:0/3085778402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4009550 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b0ec7700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59a4003b40 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f59a40248d0 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59a401a820 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.848+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac19a800 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.849+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac19ac20 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.849+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f59a4024a40 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.850+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac194110 con 0x7f59ac106330 2024-06-15T00:58:10.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.850+0000 7f59a9ffb700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 0x7f599803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.850+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f59a404c2e0 con 0x7f59ac106330 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.850+0000 7f59abfff700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 0x7f599803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.850+0000 7f59abfff700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 0x7f599803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f599c00ad30 tx=0x7f599c0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:09.854+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f59a4024e20 con 0x7f59ac106330 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.041+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f59ac0619a0 con 0x7f59ac106330 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.045+0000 7f59a9ffb700 1 -- 172.21.15.137:0/3085778402 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f59a4018350 con 0x7f59ac106330 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 msgr2=0x7f599803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 0x7f599803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f599c00ad30 tx=0x7f599c0093f0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 msgr2=0x7f59ac19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59a4000c00 tx=0x7f59a4011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 shutdown_connections 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f5998038370 0x7f599803a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 --2- 172.21.15.137:0/3085778402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ac106330 0x7f59ac19a0c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 >> 172.21.15.137:0/3085778402 conn(0x7f59ac101910 msgr2=0x7f59ac103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:10.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 shutdown_connections 2024-06-15T00:58:10.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:10.047+0000 7f59b312b700 1 -- 172.21.15.137:0/3085778402 wait complete. 2024-06-15T00:58:10.609 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling the dashboard module... 2024-06-15T00:58:11.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:11 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3085778402' entity='client.admin' 2024-06-15T00:58:11.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:11 smithi137 ceph-mon[29354]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:12.336 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:12 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1283733077' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f648b3f4700 1 Processor -- start 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f648b3f4700 1 -- start start 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f648b3f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f648b3f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648407a820 con 0x7f648407bdf0 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f6489190700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:12.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f6489190700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60878/0 (socket says 172.21.15.137:60878) 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.023+0000 7f6489190700 1 -- 172.21.15.137:0/60810890 learned_addr learned my addr 172.21.15.137:0/60810890 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f6489190700 1 -- 172.21.15.137:0/60810890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648407a960 con 0x7f648407bdf0 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f6489190700 1 --2- 172.21.15.137:0/60810890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64800098a0 tx=0x7f6480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbd7981f9f61777 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f647bfff700 1 -- 172.21.15.137:0/60810890 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6480004180 con 0x7f648407bdf0 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f647bfff700 1 -- 172.21.15.137:0/60810890 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f64800042e0 con 0x7f648407bdf0 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f647bfff700 1 -- 172.21.15.137:0/60810890 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6480011a30 con 0x7f648407bdf0 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f648b3f4700 1 -- 172.21.15.137:0/60810890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 msgr2=0x7f648407a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f648b3f4700 1 --2- 172.21.15.137:0/60810890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64800098a0 tx=0x7f6480009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f648b3f4700 1 -- 172.21.15.137:0/60810890 shutdown_connections 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f648b3f4700 1 --2- 172.21.15.137:0/60810890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648407a250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.024+0000 7f648b3f4700 1 -- 172.21.15.137:0/60810890 >> 172.21.15.137:0/60810890 conn(0x7f6484101910 msgr2=0x7f6484103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 -- 172.21.15.137:0/60810890 shutdown_connections 2024-06-15T00:58:12.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 -- 172.21.15.137:0/60810890 wait complete. 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 Processor -- start 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 -- start start 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.025+0000 7f648b3f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648419c7d0 con 0x7f648407bdf0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f6489190700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f6489190700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60886/0 (socket says 172.21.15.137:60886) 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f6489190700 1 -- 172.21.15.137:0/1283733077 learned_addr learned my addr 172.21.15.137:0/1283733077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f6489190700 1 -- 172.21.15.137:0/1283733077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6480009550 con 0x7f648407bdf0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f6489190700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6480006af0 tx=0x7f6480011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6480003a80 con 0x7f648407bdf0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6480003be0 con 0x7f648407bdf0 2024-06-15T00:58:12.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648419c9d0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6480019770 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.026+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648419cdf0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.027+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f64800199e0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.027+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648410c670 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.027+0000 7f647a7fc700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 0x7f647003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.027+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f648004c4d0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.027+0000 7f648898f700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 0x7f647003a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.028+0000 7f648898f700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 0x7f647003a7f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6474006fd0 tx=0x7f6474006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.032+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f648001acd0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:11.278+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f648410c830 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.050+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f648000b3e0 con 0x7f648407bdf0 2024-06-15T00:58:12.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.052+0000 7f647a7fc700 1 -- 172.21.15.137:0/1283733077 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f6480019c90 con 0x7f648407bdf0 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 msgr2=0x7f647003a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 0x7f647003a7f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6474006fd0 tx=0x7f6474006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 msgr2=0x7f648419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6480006af0 tx=0x7f6480011b20 comp rx=0 tx=0).stop 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 shutdown_connections 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f6470038330 0x7f647003a7f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 --2- 172.21.15.137:0/1283733077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f648407bdf0 0x7f648419c290 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 >> 172.21.15.137:0/1283733077 conn(0x7f6484101910 msgr2=0x7f6484103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.054+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 shutdown_connections 2024-06-15T00:58:12.623 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:12.055+0000 7f648b3f4700 1 -- 172.21.15.137:0/1283733077 wait complete. 2024-06-15T00:58:13.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:13 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1283733077' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-15T00:58:13.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:13 smithi137 ceph-mon[29354]: mgrmap e9: smithi137.nbzbyc(active, since 25s) 2024-06-15T00:58:13.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:58:13.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "smithi137.nbzbyc", 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed925d700 1 Processor -- start 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed925d700 1 -- start start 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed925d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed925d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4079da0 con 0x7f2ed4106de0 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60916/0 (socket says 172.21.15.137:60916) 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.056+0000 7f2ed2d9d700 1 -- 172.21.15.137:0/3075190937 learned_addr learned my addr 172.21.15.137:0/3075190937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.057+0000 7f2ed2d9d700 1 -- 172.21.15.137:0/3075190937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed4109710 con 0x7f2ed4106de0 2024-06-15T00:58:13.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.057+0000 7f2ed2d9d700 1 --2- 172.21.15.137:0/3075190937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40098a0 tx=0x7f2ec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d7a2c226a2130d server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.057+0000 7f2ed1d9b700 1 -- 172.21.15.137:0/3075190937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ec4004180 con 0x7f2ed4106de0 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.057+0000 7f2ed1d9b700 1 -- 172.21.15.137:0/3075190937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2ec40042e0 con 0x7f2ed4106de0 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.057+0000 7f2ed1d9b700 1 -- 172.21.15.137:0/3075190937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ec4011a30 con 0x7f2ed4106de0 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 -- 172.21.15.137:0/3075190937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 msgr2=0x7f2ed41091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 --2- 172.21.15.137:0/3075190937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40098a0 tx=0x7f2ec4009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 -- 172.21.15.137:0/3075190937 shutdown_connections 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 --2- 172.21.15.137:0/3075190937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed41091d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 -- 172.21.15.137:0/3075190937 >> 172.21.15.137:0/3075190937 conn(0x7f2ed41007d0 msgr2=0x7f2ed4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 -- 172.21.15.137:0/3075190937 shutdown_connections 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.058+0000 7f2ed925d700 1 -- 172.21.15.137:0/3075190937 wait complete. 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed925d700 1 Processor -- start 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed925d700 1 -- start start 2024-06-15T00:58:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed925d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed925d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4198330 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60932/0 (socket says 172.21.15.137:60932) 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.059+0000 7f2ed2d9d700 1 -- 172.21.15.137:0/1909888581 learned_addr learned my addr 172.21.15.137:0/1909888581 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ed2d9d700 1 -- 172.21.15.137:0/1909888581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4009550 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ed2d9d700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4006af0 tx=0x7f2ec4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ec4003bc0 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2ec401a8c0 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed4198530 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ec4019860 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.060+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed4198950 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f2ec4019b20 con 0x7f2ed4106de0 2024-06-15T00:58:13.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed41921f0 con 0x7f2ed4106de0 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ebbfff700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 0x7f2ebc03ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2ec404d5e0 con 0x7f2ed4106de0 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ed259c700 1 -- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 msgr2=0x7f2ebc03ebf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.061+0000 7f2ed259c700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 0x7f2ebc03ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.066+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2ec402d440 con 0x7f2ed4106de0 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.262+0000 7f2ed259c700 1 -- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 msgr2=0x7f2ebc03ebf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.262+0000 7f2ed259c700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 0x7f2ebc03ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.351+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2ed402cc70 con 0x7f2ed4106de0 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.351+0000 7f2ebbfff700 1 -- 172.21.15.137:0/1909888581 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f2ec40172b0 con 0x7f2ed4106de0 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.353+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 msgr2=0x7f2ebc03ebf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.353+0000 7f2ed925d700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 0x7f2ebc03ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.353+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 msgr2=0x7f2ed4197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:13.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.353+0000 7f2ed925d700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4006af0 tx=0x7f2ec4011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 shutdown_connections 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f2ebc03c730 0x7f2ebc03ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 --2- 172.21.15.137:0/1909888581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed4106de0 0x7f2ed4197df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 >> 172.21.15.137:0/1909888581 conn(0x7f2ed41007d0 msgr2=0x7f2ed4101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 shutdown_connections 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:13.354+0000 7f2ed925d700 1 -- 172.21.15.137:0/1909888581 wait complete. 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-06-15T00:58:13.897 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 9... 2024-06-15T00:58:14.204 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:14 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1909888581' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: Active manager daemon smithi137.nbzbyc restarted 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: Activating manager daemon smithi137.nbzbyc 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: osdmap e3: 0 total, 0 up, 0 in 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: mgrmap e10: smithi137.nbzbyc(active, starting, since 0.00372211s) 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T00:58:18.767 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T00:58:18.768 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:18 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T00:58:20.428 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.305+0000 7f4b5899c700 1 Processor -- start 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.305+0000 7f4b5899c700 1 -- start start 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.305+0000 7f4b5899c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.305+0000 7f4b5899c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54106070 con 0x7f4b54105680 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:20.429 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60942/0 (socket says 172.21.15.137:60942) 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5259c700 1 -- 172.21.15.137:0/2367594728 learned_addr learned my addr 172.21.15.137:0/2367594728 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5259c700 1 -- 172.21.15.137:0/2367594728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b541063a0 con 0x7f4b54105680 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5259c700 1 --2- 172.21.15.137:0/2367594728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b480098a0 tx=0x7f4b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e47f16131511bcf server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5159a700 1 -- 172.21.15.137:0/2367594728 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b48004180 con 0x7f4b54105680 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5159a700 1 -- 172.21.15.137:0/2367594728 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4b480042e0 con 0x7f4b54105680 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.306+0000 7f4b5159a700 1 -- 172.21.15.137:0/2367594728 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b48011a30 con 0x7f4b54105680 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 -- 172.21.15.137:0/2367594728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 msgr2=0x7f4b54105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 --2- 172.21.15.137:0/2367594728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b480098a0 tx=0x7f4b48009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 -- 172.21.15.137:0/2367594728 shutdown_connections 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 --2- 172.21.15.137:0/2367594728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54105aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 -- 172.21.15.137:0/2367594728 >> 172.21.15.137:0/2367594728 conn(0x7f4b540ffec0 msgr2=0x7f4b541022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 -- 172.21.15.137:0/2367594728 shutdown_connections 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.307+0000 7f4b5899c700 1 -- 172.21.15.137:0/2367594728 wait complete. 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5899c700 1 Processor -- start 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5899c700 1 -- start start 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5899c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5899c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54199350 con 0x7f4b54105680 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60948/0 (socket says 172.21.15.137:60948) 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5259c700 1 -- 172.21.15.137:0/1501579722 learned_addr learned my addr 172.21.15.137:0/1501579722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5259c700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009550 con 0x7f4b54105680 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.308+0000 7f4b5259c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006af0 tx=0x7f4b48011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.309+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b48003bc0 con 0x7f4b54105680 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.309+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b54199550 con 0x7f4b54105680 2024-06-15T00:58:20.431 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.309+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b54199970 con 0x7f4b54105680 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.309+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4b4801a8c0 con 0x7f4b54105680 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.309+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b48019860 con 0x7f4b54105680 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f4b48019b20 con 0x7f4b54105680 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b437fe700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b3c03d110 con 0x7f4b3c03a540 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b4804c560 con 0x7f4b54105680 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b51d9b700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.310+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.511+0000 7f4b51d9b700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.511+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.912+0000 7f4b51d9b700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:14.912+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:15.713+0000 7f4b51d9b700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:15.713+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-15T00:58:20.432 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:17.314+0000 7f4b51d9b700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/4018707027 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:17.314+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:18.712+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f4b4802d610 con 0x7f4b54105680 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:18.712+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 msgr2=0x7f4b3c03ca00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:18.712+0000 7f4b437fe700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.713+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f4b48020240 con 0x7f4b54105680 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.713+0000 7f4b437fe700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 0x7f4b3c0403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.713+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b3c03d110 con 0x7f4b3c03dfb0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.714+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 0x7f4b3c0403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.715+0000 7f4b51d9b700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 0x7f4b3c0403a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b44003a10 tx=0x7f4b440092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.715+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4b3c03d110 con 0x7f4b3c03dfb0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.721+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4b34002630 con 0x7f4b3c03dfb0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b437fe700 1 -- 172.21.15.137:0/1501579722 <== mgr.14164 v2:172.21.15.137:6800/2929246681 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4b34002630 con 0x7f4b3c03dfb0 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 msgr2=0x7f4b3c0403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 0x7f4b3c0403a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b44003a10 tx=0x7f4b440092b0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.433 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 msgr2=0x7f4b54198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006af0 tx=0x7f4b48011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 shutdown_connections 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f4b3c03dfb0 0x7f4b3c0403a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b54105680 0x7f4b54198e10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 --2- 172.21.15.137:0/1501579722 >> [v2:172.21.15.137:6800/4018707027,v1:172.21.15.137:6801/4018707027] conn(0x7f4b3c03a540 0x7f4b3c03ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 >> 172.21.15.137:0/1501579722 conn(0x7f4b540ffec0 msgr2=0x7f4b54101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 shutdown_connections 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:19.722+0000 7f4b5899c700 1 -- 172.21.15.137:0/1501579722 wait complete. 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 9 is available 2024-06-15T00:58:20.434 INFO:teuthology.orchestra.run.smithi137.stdout:Generating a dashboard self-signed certificate... 2024-06-15T00:58:20.863 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: [15/Jun/2024:00:58:19] ENGINE Bus STARTING 2024-06-15T00:58:20.863 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: [15/Jun/2024:00:58:19] ENGINE Serving on http://172.21.15.137:8765 2024-06-15T00:58:20.863 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: [15/Jun/2024:00:58:19] ENGINE Serving on https://172.21.15.137:7150 2024-06-15T00:58:20.863 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: [15/Jun/2024:00:58:19] ENGINE Bus STARTED 2024-06-15T00:58:20.864 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-15T00:58:20.864 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: mgrmap e11: smithi137.nbzbyc(active, since 1.00628s) 2024-06-15T00:58:20.864 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-15T00:58:20.864 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:20 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:21.708 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-15T00:58:21.708 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb9eed700 1 Processor -- start 2024-06-15T00:58:21.708 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb9eed700 1 -- start start 2024-06-15T00:58:21.708 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb9eed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:21.708 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb9eed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40744f0 con 0x7fdeb4106de0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60236/0 (socket says 172.21.15.137:60236) 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.901+0000 7fdeb37fe700 1 -- 172.21.15.137:0/1155811477 learned_addr learned my addr 172.21.15.137:0/1155811477 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.902+0000 7fdeb37fe700 1 -- 172.21.15.137:0/1155811477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4109710 con 0x7fdeb4106de0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.902+0000 7fdeb37fe700 1 --2- 172.21.15.137:0/1155811477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdea40098a0 tx=0x7fdea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6196c1a05ae4bad server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.902+0000 7fdeb27fc700 1 -- 172.21.15.137:0/1155811477 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdea4004180 con 0x7fdeb4106de0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.902+0000 7fdeb27fc700 1 -- 172.21.15.137:0/1155811477 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdea40042e0 con 0x7fdeb4106de0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.902+0000 7fdeb27fc700 1 -- 172.21.15.137:0/1155811477 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdea4011a30 con 0x7fdeb4106de0 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 -- 172.21.15.137:0/1155811477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 msgr2=0x7fdeb41091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/1155811477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdea40098a0 tx=0x7fdea4009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 -- 172.21.15.137:0/1155811477 shutdown_connections 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/1155811477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb41091d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 -- 172.21.15.137:0/1155811477 >> 172.21.15.137:0/1155811477 conn(0x7fdeb41007f0 msgr2=0x7fdeb4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 -- 172.21.15.137:0/1155811477 shutdown_connections 2024-06-15T00:58:21.709 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.903+0000 7fdeb9eed700 1 -- 172.21.15.137:0/1155811477 wait complete. 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb9eed700 1 Processor -- start 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb9eed700 1 -- start start 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb9eed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb9eed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4106c80 con 0x7fdeb4106de0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60240/0 (socket says 172.21.15.137:60240) 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.904+0000 7fdeb37fe700 1 -- 172.21.15.137:0/2498905806 learned_addr learned my addr 172.21.15.137:0/2498905806 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb37fe700 1 -- 172.21.15.137:0/2498905806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4009550 con 0x7fdeb4106de0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb37fe700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdea4006af0 tx=0x7fdea4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdea4003a80 con 0x7fdeb4106de0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdea4003be0 con 0x7fdeb4106de0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb4104d90 con 0x7fdeb4106de0 2024-06-15T00:58:21.710 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdea4019770 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.905+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb41051b0 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fdea4019a00 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb0ff9700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 0x7fde9c03a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb40623c0 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdea404c490 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb2ffd700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 0x7fde9c03a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.906+0000 7fdeb2ffd700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 0x7fde9c03a720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdea8006fd0 tx=0x7fdea8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.911+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdea402c3a0 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:20.962+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fdea4016d40 con 0x7fdeb4106de0 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.124+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fdeb4105c90 con 0x7fde9c038260 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.211+0000 7fdeb0ff9700 1 -- 172.21.15.137:0/2498905806 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fdeb4105c90 con 0x7fde9c038260 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 msgr2=0x7fde9c03a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:21.711 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 0x7fde9c03a720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdea8006fd0 tx=0x7fdea8006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 msgr2=0x7fdeb4106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdea4006af0 tx=0x7fdea4011b20 comp rx=0 tx=0).stop 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 shutdown_connections 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fde9c038260 0x7fde9c03a720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 --2- 172.21.15.137:0/2498905806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb4106de0 0x7fdeb4106740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 >> 172.21.15.137:0/2498905806 conn(0x7fdeb41007f0 msgr2=0x7fdeb4101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 shutdown_connections 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:21.213+0000 7fdeb9eed700 1 -- 172.21.15.137:0/2498905806 wait complete. 2024-06-15T00:58:21.712 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial admin user... 2024-06-15T00:58:22.092 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:21 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:22.092 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:21 smithi137 ceph-mon[29354]: mgrmap e12: smithi137.nbzbyc(active, since 2s) 2024-06-15T00:58:22.092 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:21 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:22.092 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:21 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:22.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:22 smithi137 ceph-mon[29354]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:22.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:22 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:23.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$d2XuPDK1tRXN/xoSk945t.9iJO/ykzGWr8E8ck1jBEBOc7uk7MarW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718413102, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.091+0000 7ffbd89b6700 1 Processor -- start 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.091+0000 7ffbd89b6700 1 -- start start 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.091+0000 7ffbd89b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.091+0000 7ffbd89b6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40796b0 con 0x7ffbd407ac80 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60242/0 (socket says 172.21.15.137:60242) 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd259c700 1 -- 172.21.15.137:0/443041130 learned_addr learned my addr 172.21.15.137:0/443041130 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd259c700 1 -- 172.21.15.137:0/443041130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40797f0 con 0x7ffbd407ac80 2024-06-15T00:58:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd259c700 1 --2- 172.21.15.137:0/443041130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b378147c5504e server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd159a700 1 -- 172.21.15.137:0/443041130 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbc4004180 con 0x7ffbd407ac80 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd159a700 1 -- 172.21.15.137:0/443041130 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffbc40042e0 con 0x7ffbd407ac80 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.092+0000 7ffbd159a700 1 -- 172.21.15.137:0/443041130 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbc4011a30 con 0x7ffbd407ac80 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 -- 172.21.15.137:0/443041130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 msgr2=0x7ffbd40790e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/443041130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 -- 172.21.15.137:0/443041130 shutdown_connections 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/443041130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd40790e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 -- 172.21.15.137:0/443041130 >> 172.21.15.137:0/443041130 conn(0x7ffbd41018d0 msgr2=0x7ffbd4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 -- 172.21.15.137:0/443041130 shutdown_connections 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.093+0000 7ffbd89b6700 1 -- 172.21.15.137:0/443041130 wait complete. 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd89b6700 1 Processor -- start 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd89b6700 1 -- start start 2024-06-15T00:58:23.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd89b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd89b6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd419c720 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60252/0 (socket says 172.21.15.137:60252) 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.094+0000 7ffbd259c700 1 -- 172.21.15.137:0/1052405184 learned_addr learned my addr 172.21.15.137:0/1052405184 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbd259c700 1 -- 172.21.15.137:0/1052405184 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4009550 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbd259c700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006af0 tx=0x7ffbc4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbc4003bc0 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd419c920 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffbc401a8c0 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbc4019860 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.095+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd419cd40 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.096+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7ffbc4019b20 con 0x7ffbd407ac80 2024-06-15T00:58:23.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.096+0000 7ffbc37fe700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 0x7ffbbc03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.096+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbc404d760 con 0x7ffbd407ac80 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.096+0000 7ffbd1d9b700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 0x7ffbbc03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.096+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd4196300 con 0x7ffbd407ac80 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.097+0000 7ffbd1d9b700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 0x7ffbbc03a7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006fd0 tx=0x7ffbc8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.101+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffbc402d440 con 0x7ffbd407ac80 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.311+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ffbd40621e0 con 0x7ffbbc038330 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.535+0000 7ffbc37fe700 1 -- 172.21.15.137:0/1052405184 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ffbd40621e0 con 0x7ffbbc038330 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 msgr2=0x7ffbbc03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 0x7ffbbc03a7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006fd0 tx=0x7ffbc8006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 msgr2=0x7ffbd419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006af0 tx=0x7ffbc4011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:23.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 shutdown_connections 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffbbc038330 0x7ffbbc03a7f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 --2- 172.21.15.137:0/1052405184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbd407ac80 0x7ffbd419c1e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.537+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 >> 172.21.15.137:0/1052405184 conn(0x7ffbd41018d0 msgr2=0x7ffbd4103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.538+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 shutdown_connections 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:22.538+0000 7ffbd89b6700 1 -- 172.21.15.137:0/1052405184 wait complete. 2024-06-15T00:58:23.659 INFO:teuthology.orchestra.run.smithi137.stdout:Fetching dashboard port number... 2024-06-15T00:58:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:23 smithi137 ceph-mon[29354]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:24.960 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 8443 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88d08e4700 1 Processor -- start 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88d08e4700 1 -- start start 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88d08e4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88d08e4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1071f0 con 0x7f88cc106800 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60256/0 (socket says 172.21.15.137:60256) 2024-06-15T00:58:24.961 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.149+0000 7f88ca59c700 1 -- 172.21.15.137:0/3967460172 learned_addr learned my addr 172.21.15.137:0/3967460172 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88ca59c700 1 -- 172.21.15.137:0/3967460172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc105410 con 0x7f88cc106800 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88ca59c700 1 --2- 172.21.15.137:0/3967460172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88c00098a0 tx=0x7f88c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d57a26f7375bb22 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88c959a700 1 -- 172.21.15.137:0/3967460172 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88c0004180 con 0x7f88cc106800 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88c959a700 1 -- 172.21.15.137:0/3967460172 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f88c00042e0 con 0x7f88cc106800 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88c959a700 1 -- 172.21.15.137:0/3967460172 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88c0011a30 con 0x7f88cc106800 2024-06-15T00:58:24.962 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 -- 172.21.15.137:0/3967460172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 msgr2=0x7f88cc106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3967460172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88c00098a0 tx=0x7f88c0009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 -- 172.21.15.137:0/3967460172 shutdown_connections 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3967460172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc106c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 -- 172.21.15.137:0/3967460172 >> 172.21.15.137:0/3967460172 conn(0x7f88cc0fff40 msgr2=0x7f88cc102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.150+0000 7f88d08e4700 1 -- 172.21.15.137:0/3967460172 shutdown_connections 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.151+0000 7f88d08e4700 1 -- 172.21.15.137:0/3967460172 wait complete. 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.151+0000 7f88d08e4700 1 Processor -- start 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.151+0000 7f88d08e4700 1 -- start start 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.151+0000 7f88d08e4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.151+0000 7f88d08e4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc198330 con 0x7f88cc106800 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60262/0 (socket says 172.21.15.137:60262) 2024-06-15T00:58:24.963 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88ca59c700 1 -- 172.21.15.137:0/3253822921 learned_addr learned my addr 172.21.15.137:0/3253822921 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88ca59c700 1 -- 172.21.15.137:0/3253822921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0009550 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88ca59c700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88c0006af0 tx=0x7f88c0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88c0003bc0 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc198530 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f88c001a8c0 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88c0019820 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.152+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc198950 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.153+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f88c001a430 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.153+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc191ab0 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.153+0000 7f88bb7fe700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 0x7f88b403c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.153+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f88c004c830 con 0x7f88cc106800 2024-06-15T00:58:24.964 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.153+0000 7f88c9d9b700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 0x7f88b403c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.154+0000 7f88c9d9b700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 0x7f88b403c9f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006fd0 tx=0x7f88bc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.158+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f88c0018530 con 0x7f88cc106800 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.348+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f88cc02d090 con 0x7f88cc106800 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.348+0000 7f88bb7fe700 1 -- 172.21.15.137:0/3253822921 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f88c0018350 con 0x7f88cc106800 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.350+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 msgr2=0x7f88b403c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.350+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 0x7f88b403c9f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006fd0 tx=0x7f88bc006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.350+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 msgr2=0x7f88cc197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.350+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88c0006af0 tx=0x7f88c0011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 shutdown_connections 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f88b403a530 0x7f88b403c9f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 --2- 172.21.15.137:0/3253822921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc106800 0x7f88cc197df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 >> 172.21.15.137:0/3253822921 conn(0x7f88cc0fff40 msgr2=0x7f88cc18ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 shutdown_connections 2024-06-15T00:58:24.966 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:24.351+0000 7f88d08e4700 1 -- 172.21.15.137:0/3253822921 wait complete. 2024-06-15T00:58:24.966 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-15T00:58:24.967 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-06-15T00:58:24.974 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-15T00:58:24.974 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-06-15T00:58:24.974 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-06-15T00:58:24.974 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph Dashboard is now available at: 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout: URL: https://smithi137.front.sepia.ceph.com:8443/ 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout: User: admin 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout: Password: prvcwj2g4b 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:24.976 INFO:teuthology.orchestra.run.smithi137.stdout:Saving cluster configuration to /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config directory 2024-06-15T00:58:24.977 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling autotune for osd_memory_target 2024-06-15T00:58:24.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:24 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3253822921' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-15T00:58:26.184 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.413+0000 7fa8c6959700 1 Processor -- start 2024-06-15T00:58:26.184 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.413+0000 7fa8c6959700 1 -- start start 2024-06-15T00:58:26.184 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8c6959700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:26.184 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8c6959700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0079da0 con 0x7fa8c0106de0 2024-06-15T00:58:26.184 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60274/0 (socket says 172.21.15.137:60274) 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bffff700 1 -- 172.21.15.137:0/2341993491 learned_addr learned my addr 172.21.15.137:0/2341993491 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bffff700 1 -- 172.21.15.137:0/2341993491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0109710 con 0x7fa8c0106de0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bffff700 1 --2- 172.21.15.137:0/2341993491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00098a0 tx=0x7fa8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88d1011680253a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bf7fe700 1 -- 172.21.15.137:0/2341993491 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b0004180 con 0x7fa8c0106de0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bf7fe700 1 -- 172.21.15.137:0/2341993491 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa8b00042e0 con 0x7fa8c0106de0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.414+0000 7fa8bf7fe700 1 -- 172.21.15.137:0/2341993491 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b0011a30 con 0x7fa8c0106de0 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 -- 172.21.15.137:0/2341993491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 msgr2=0x7fa8c01091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 --2- 172.21.15.137:0/2341993491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00098a0 tx=0x7fa8b0009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 -- 172.21.15.137:0/2341993491 shutdown_connections 2024-06-15T00:58:26.185 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 --2- 172.21.15.137:0/2341993491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c01091d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 -- 172.21.15.137:0/2341993491 >> 172.21.15.137:0/2341993491 conn(0x7fa8c01007d0 msgr2=0x7fa8c0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 -- 172.21.15.137:0/2341993491 shutdown_connections 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.415+0000 7fa8c6959700 1 -- 172.21.15.137:0/2341993491 wait complete. 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8c6959700 1 Processor -- start 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8c6959700 1 -- start start 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8c6959700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8c6959700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c019a560 con 0x7fa8c0106de0 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8bffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8bffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60276/0 (socket says 172.21.15.137:60276) 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8bffff700 1 -- 172.21.15.137:0/25009213 learned_addr learned my addr 172.21.15.137:0/25009213 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:26.186 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8bffff700 1 -- 172.21.15.137:0/25009213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b0009550 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.416+0000 7fa8bffff700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0011a00 tx=0x7fa8b0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b0003bc0 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa8b00248c0 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b001a850 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c019a760 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c019ab80 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.417+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa8b0024a30 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.418+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c00623c0 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.418+0000 7fa8bdffb700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 0x7fa8ac03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.418+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa8b004c830 con 0x7fa8c0106de0 2024-06-15T00:58:26.187 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.418+0000 7fa8b7fff700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 0x7fa8ac03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.418+0000 7fa8b7fff700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 0x7fa8ac03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.422+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa8b00189a0 con 0x7fa8c0106de0 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.611+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa8c019da00 con 0x7fa8c0106de0 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.611+0000 7fa8bdffb700 1 -- 172.21.15.137:0/25009213 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa8b0024ce0 con 0x7fa8c0106de0 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 msgr2=0x7fa8ac03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 0x7fa8ac03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 msgr2=0x7fa8c019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0011a00 tx=0x7fa8b0011ae0 comp rx=0 tx=0).stop 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 shutdown_connections 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fa8ac038460 0x7fa8ac03a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 --2- 172.21.15.137:0/25009213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8c0106de0 0x7fa8c019a020 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 >> 172.21.15.137:0/25009213 conn(0x7fa8c01007d0 msgr2=0x7fa8c0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:26.189 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 shutdown_connections 2024-06-15T00:58:26.189 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:25.613+0000 7fa8c6959700 1 -- 172.21.15.137:0/25009213 wait complete. 2024-06-15T00:58:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:25 smithi137 ceph-mon[29354]: mgrmap e13: smithi137.nbzbyc(active, since 6s) 2024-06-15T00:58:27.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f3920f6c700 1 Processor -- start 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f3920f6c700 1 -- start start 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f3920f6c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f3920f6c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c105000 con 0x7f391c104610 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f391a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f391a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47234/0 (socket says 172.21.15.137:47234) 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.643+0000 7f391a59c700 1 -- 172.21.15.137:0/2214148413 learned_addr learned my addr 172.21.15.137:0/2214148413 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f391a59c700 1 -- 172.21.15.137:0/2214148413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c1075a0 con 0x7f391c104610 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f391a59c700 1 --2- 172.21.15.137:0/2214148413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f390c0098a0 tx=0x7f390c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7e8647c74bc0c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f391959a700 1 -- 172.21.15.137:0/2214148413 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f390c004180 con 0x7f391c104610 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f391959a700 1 -- 172.21.15.137:0/2214148413 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f390c0042e0 con 0x7f391c104610 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f391959a700 1 -- 172.21.15.137:0/2214148413 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f390c011a30 con 0x7f391c104610 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f3920f6c700 1 -- 172.21.15.137:0/2214148413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 msgr2=0x7f391c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:27.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.644+0000 7f3920f6c700 1 --2- 172.21.15.137:0/2214148413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f390c0098a0 tx=0x7f390c009bb0 comp rx=0 tx=0).stop 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 -- 172.21.15.137:0/2214148413 shutdown_connections 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 --2- 172.21.15.137:0/2214148413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c104a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 -- 172.21.15.137:0/2214148413 >> 172.21.15.137:0/2214148413 conn(0x7f391c0fff40 msgr2=0x7f391c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 -- 172.21.15.137:0/2214148413 shutdown_connections 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 -- 172.21.15.137:0/2214148413 wait complete. 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 Processor -- start 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.645+0000 7f3920f6c700 1 -- start start 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f3920f6c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f3920f6c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19a5c0 con 0x7f391c104610 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f391a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f391a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47242/0 (socket says 172.21.15.137:47242) 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f391a59c700 1 -- 172.21.15.137:0/3189395575 learned_addr learned my addr 172.21.15.137:0/3189395575 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f391a59c700 1 -- 172.21.15.137:0/3189395575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009550 con 0x7f391c104610 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f391a59c700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:27.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f390c003bc0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f390c0248d0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f390c01a890 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c19a7c0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.646+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c19abe0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.647+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f390c024440 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.647+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c0623c0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.647+0000 7f390b7fe700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 0x7f390403a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.647+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f390c04c710 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.648+0000 7f3919d9b700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 0x7f390403a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.648+0000 7f3919d9b700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 0x7f390403a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3910006fd0 tx=0x7f3910006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.652+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f390c024a40 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.948+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f391c19dce0 con 0x7f391c104610 2024-06-15T00:58:27.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.950+0000 7f390b7fe700 1 -- 172.21.15.137:0/3189395575 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f390c018b00 con 0x7f391c104610 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 msgr2=0x7f390403a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 0x7f390403a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3910006fd0 tx=0x7f3910006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 msgr2=0x7f391c19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c011ab0 comp rx=0 tx=0).stop 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 shutdown_connections 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f3904038460 0x7f390403a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 --2- 172.21.15.137:0/3189395575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f391c104610 0x7f391c19a080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 >> 172.21.15.137:0/3189395575 conn(0x7f391c0fff40 msgr2=0x7f391c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 shutdown_connections 2024-06-15T00:58:27.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-06-15T00:58:26.952+0000 7f3920f6c700 1 -- 172.21.15.137:0/3189395575 wait complete. 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.619 INFO:teuthology.orchestra.run.smithi137.stdout:Or, if you are only running a single cluster on this host: 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.620 INFO:teuthology.orchestra.run.smithi137.stdout: ceph telemetry on 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout:For more information see: 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:27.621 INFO:teuthology.orchestra.run.smithi137.stdout:Bootstrap complete. 2024-06-15T00:58:27.645 INFO:tasks.cephadm:Fetching config... 2024-06-15T00:58:27.646 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:58:27.646 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-15T00:58:27.662 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-15T00:58:27.662 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:58:27.663 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-15T00:58:27.719 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-15T00:58:27.719 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:58:27.719 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/keyring of=/dev/stdout 2024-06-15T00:58:27.785 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-15T00:58:27.785 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T00:58:27.785 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-15T00:58:27.843 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-15T00:58:27.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHUDL1iv5GE4Y7PW4fdZAxRH193NO5v32YaFlrjZmhdH9CRcIAujQV57CIXWh8sm++a0iKjKuOcQaadj3DiwB4HYFl27JwtPKygvOaOMjWwG+CRQovkXTMKBagam5exrY/POjshF4PIAmCa69kjciFbxAEL3touqEmx1TT0i0kIhwhZPx3de2zaM6Xtl2avR1Ty1wZ+yCC2qTGnkQzja0wjK+z09yQ1oUFU0C5vJ7ifxHb+RzU9vjDB26Faf5CEYYEgeCM0vVNm/VzQr1c9nrmOODGHigaSoi0uBaCnT7dypgmrrM5RPLuOxu8X77F8u8OwpniAqmMbQM7EYZWqdLt71mqWfdKzod1r5KjzEMeRM4jZUMCNteYlQQAKYt/Sp8kTO5fH/34usJqUIwPYgxbdWud/xrCkZP4ncHq/x3ZoAef1LwdUhzmZA7+BK1AK1XkNS5hpK8dXhadlWo2lyiVScOcLtklOL/8JKbGQsbiYWBuzxyDG8w2Cu6NEZzIVE= ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-15T00:58:27.918 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHUDL1iv5GE4Y7PW4fdZAxRH193NO5v32YaFlrjZmhdH9CRcIAujQV57CIXWh8sm++a0iKjKuOcQaadj3DiwB4HYFl27JwtPKygvOaOMjWwG+CRQovkXTMKBagam5exrY/POjshF4PIAmCa69kjciFbxAEL3touqEmx1TT0i0kIhwhZPx3de2zaM6Xtl2avR1Ty1wZ+yCC2qTGnkQzja0wjK+z09yQ1oUFU0C5vJ7ifxHb+RzU9vjDB26Faf5CEYYEgeCM0vVNm/VzQr1c9nrmOODGHigaSoi0uBaCnT7dypgmrrM5RPLuOxu8X77F8u8OwpniAqmMbQM7EYZWqdLt71mqWfdKzod1r5KjzEMeRM4jZUMCNteYlQQAKYt/Sp8kTO5fH/34usJqUIwPYgxbdWud/xrCkZP4ncHq/x3ZoAef1LwdUhzmZA7+BK1AK1XkNS5hpK8dXhadlWo2lyiVScOcLtklOL/8JKbGQsbiYWBuzxyDG8w2Cu6NEZzIVE= ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:58:27.929 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHUDL1iv5GE4Y7PW4fdZAxRH193NO5v32YaFlrjZmhdH9CRcIAujQV57CIXWh8sm++a0iKjKuOcQaadj3DiwB4HYFl27JwtPKygvOaOMjWwG+CRQovkXTMKBagam5exrY/POjshF4PIAmCa69kjciFbxAEL3touqEmx1TT0i0kIhwhZPx3de2zaM6Xtl2avR1Ty1wZ+yCC2qTGnkQzja0wjK+z09yQ1oUFU0C5vJ7ifxHb+RzU9vjDB26Faf5CEYYEgeCM0vVNm/VzQr1c9nrmOODGHigaSoi0uBaCnT7dypgmrrM5RPLuOxu8X77F8u8OwpniAqmMbQM7EYZWqdLt71mqWfdKzod1r5KjzEMeRM4jZUMCNteYlQQAKYt/Sp8kTO5fH/34usJqUIwPYgxbdWud/xrCkZP4ncHq/x3ZoAef1LwdUhzmZA7+BK1AK1XkNS5hpK8dXhadlWo2lyiVScOcLtklOL/8JKbGQsbiYWBuzxyDG8w2Cu6NEZzIVE= ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-15T00:58:27.971 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGHUDL1iv5GE4Y7PW4fdZAxRH193NO5v32YaFlrjZmhdH9CRcIAujQV57CIXWh8sm++a0iKjKuOcQaadj3DiwB4HYFl27JwtPKygvOaOMjWwG+CRQovkXTMKBagam5exrY/POjshF4PIAmCa69kjciFbxAEL3touqEmx1TT0i0kIhwhZPx3de2zaM6Xtl2avR1Ty1wZ+yCC2qTGnkQzja0wjK+z09yQ1oUFU0C5vJ7ifxHb+RzU9vjDB26Faf5CEYYEgeCM0vVNm/VzQr1c9nrmOODGHigaSoi0uBaCnT7dypgmrrM5RPLuOxu8X77F8u8OwpniAqmMbQM7EYZWqdLt71mqWfdKzod1r5KjzEMeRM4jZUMCNteYlQQAKYt/Sp8kTO5fH/34usJqUIwPYgxbdWud/xrCkZP4ncHq/x3ZoAef1LwdUhzmZA7+BK1AK1XkNS5hpK8dXhadlWo2lyiVScOcLtklOL/8JKbGQsbiYWBuzxyDG8w2Cu6NEZzIVE= ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:58:27.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-15T00:58:28.201 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:27 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3189395575' entity='client.admin' 2024-06-15T00:58:28.290 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T00:58:30.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.772+0000 7f2d80463700 1 -- 172.21.15.137:0/913727845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 msgr2=0x7f2d780ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:30.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.772+0000 7f2d80463700 1 --2- 172.21.15.137:0/913727845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d780ffbc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009910 tx=0x7f2d6c009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:30.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 -- 172.21.15.137:0/913727845 shutdown_connections 2024-06-15T00:58:30.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 --2- 172.21.15.137:0/913727845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d780ffbc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 -- 172.21.15.137:0/913727845 >> 172.21.15.137:0/913727845 conn(0x7f2d780fad20 msgr2=0x7f2d780fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 -- 172.21.15.137:0/913727845 shutdown_connections 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 -- 172.21.15.137:0/913727845 wait complete. 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 Processor -- start 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.773+0000 7f2d80463700 1 -- start start 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d80463700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d80463700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78196440 con 0x7f2d780ff7a0 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d7e1ff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d7e1ff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47260/0 (socket says 172.21.15.137:47260) 2024-06-15T00:58:30.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d7e1ff700 1 -- 172.21.15.137:0/399789410 learned_addr learned my addr 172.21.15.137:0/399789410 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d7e1ff700 1 -- 172.21.15.137:0/399789410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0095f0 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d7e1ff700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006010 tx=0x7f2d6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d6c01d070 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2d6c00bdf0 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7819aa50 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d6c022e50 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.774+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7819adb0 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.775+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f2d6c00f4f0 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.775+0000 7f2d6b7fe700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 0x7f2d6403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.775+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d6c04c340 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.775+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7818e280 con 0x7f2d780ff7a0 2024-06-15T00:58:30.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.776+0000 7f2d7d9fe700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 0x7f2d6403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:30.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.776+0000 7f2d7d9fe700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 0x7f2d6403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:30.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.780+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2d6c027070 con 0x7f2d780ff7a0 2024-06-15T00:58:30.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.993+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2d7802cc70 con 0x7f2d780ff7a0 2024-06-15T00:58:30.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.997+0000 7f2d6b7fe700 1 -- 172.21.15.137:0/399789410 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2d6c034070 con 0x7f2d780ff7a0 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 msgr2=0x7f2d6403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 0x7f2d6403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 msgr2=0x7f2d78068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006010 tx=0x7f2d6c0057b0 comp rx=0 tx=0).stop 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 shutdown_connections 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2d64038370 0x7f2d6403a830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:31.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 --2- 172.21.15.137:0/399789410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d780ff7a0 0x7f2d78068b00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:31.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:30.999+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 >> 172.21.15.137:0/399789410 conn(0x7f2d780fad20 msgr2=0x7f2d780fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:31.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:31.000+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 shutdown_connections 2024-06-15T00:58:31.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:31.000+0000 7f2d80463700 1 -- 172.21.15.137:0/399789410 wait complete. 2024-06-15T00:58:31.190 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:58:31.191 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:30 smithi137 ceph-mon[29354]: Deploying daemon ceph-exporter.smithi137 on smithi137 2024-06-15T00:58:32.146 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:31 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/399789410' entity='client.admin' 2024-06-15T00:58:32.166 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-15T00:58:32.166 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-15T00:58:32.862 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T00:58:35.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 -- 172.21.15.137:0/3483315630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 msgr2=0x7fb41c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:35.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 --2- 172.21.15.137:0/3483315630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c100cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009910 tx=0x7fb40c009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 -- 172.21.15.137:0/3483315630 shutdown_connections 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 --2- 172.21.15.137:0/3483315630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c100cd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 -- 172.21.15.137:0/3483315630 >> 172.21.15.137:0/3483315630 conn(0x7fb41c0fbe30 msgr2=0x7fb41c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 -- 172.21.15.137:0/3483315630 shutdown_connections 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.155+0000 7fb421b90700 1 -- 172.21.15.137:0/3483315630 wait complete. 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.156+0000 7fb421b90700 1 Processor -- start 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.156+0000 7fb421b90700 1 -- start start 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.156+0000 7fb421b90700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.156+0000 7fb421b90700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c075070 con 0x7fb41c1008b0 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.156+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47284/0 (socket says 172.21.15.137:47284) 2024-06-15T00:58:35.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb41b7fe700 1 -- 172.21.15.137:0/1355981495 learned_addr learned my addr 172.21.15.137:0/1355981495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb41b7fe700 1 -- 172.21.15.137:0/1355981495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0095f0 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb41b7fe700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c01d070 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb40c00bdf0 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c073180 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c022e50 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.157+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c073620 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.158+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fb40c00f4b0 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.158+0000 7fb418ff9700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 0x7fb40403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.158+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb40c04d2f0 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.158+0000 7fb41affd700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 0x7fb40403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.158+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c0623c0 con 0x7fb41c1008b0 2024-06-15T00:58:35.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.159+0000 7fb41affd700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 0x7fb40403a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb410006fd0 tx=0x7fb410006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:35.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.167+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb40c027070 con 0x7fb41c1008b0 2024-06-15T00:58:35.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.374+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb41c073860 con 0x7fb404038370 2024-06-15T00:58:35.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.379+0000 7fb418ff9700 1 -- 172.21.15.137:0/1355981495 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb41c073860 con 0x7fb404038370 2024-06-15T00:58:35.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 msgr2=0x7fb40403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 0x7fb40403a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb410006fd0 tx=0x7fb410006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 msgr2=0x7fb41c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c0057b0 comp rx=0 tx=0).stop 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 shutdown_connections 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fb404038370 0x7fb40403a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 --2- 172.21.15.137:0/1355981495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41c1008b0 0x7fb41c074b30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 >> 172.21.15.137:0/1355981495 conn(0x7fb41c0fbe30 msgr2=0x7fb41c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 shutdown_connections 2024-06-15T00:58:35.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:35.381+0000 7fb421b90700 1 -- 172.21.15.137:0/1355981495 wait complete. 2024-06-15T00:58:35.924 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-06-15T00:58:35.924 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:58:35.924 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-06-15T00:58:35.944 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T00:58:35.944 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-15T00:58:36.003 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-06-15T00:58:36.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch host add smithi142 2024-06-15T00:58:36.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: Deploying daemon crash.smithi137 on smithi137 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:36 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:37.097 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.030+0000 7f2ee4d86700 1 -- 172.21.15.137:0/3480933147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 msgr2=0x7f2ee00ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.030+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/3480933147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee00ffbc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009910 tx=0x7f2ed0009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.030+0000 7f2ee4d86700 1 -- 172.21.15.137:0/3480933147 shutdown_connections 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.030+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/3480933147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee00ffbc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.030+0000 7f2ee4d86700 1 -- 172.21.15.137:0/3480933147 >> 172.21.15.137:0/3480933147 conn(0x7f2ee00fad20 msgr2=0x7f2ee00fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.031+0000 7f2ee4d86700 1 -- 172.21.15.137:0/3480933147 shutdown_connections 2024-06-15T00:58:39.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.031+0000 7f2ee4d86700 1 -- 172.21.15.137:0/3480933147 wait complete. 2024-06-15T00:58:39.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.031+0000 7f2ee4d86700 1 Processor -- start 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.031+0000 7f2ee4d86700 1 -- start start 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ee4d86700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ee4d86700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0195de0 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59812/0 (socket says 172.21.15.137:59812) 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ede59c700 1 -- 172.21.15.137:0/1646362797 learned_addr learned my addr 172.21.15.137:0/1646362797 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ede59c700 1 -- 172.21.15.137:0/1646362797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00095f0 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ede59c700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006010 tx=0x7f2ed00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ed001d070 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2ed000bdf0 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ed0022e50 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.032+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee0195fe0 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.033+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee0196480 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.033+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f2ed000f460 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.033+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee01054a0 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.034+0000 7f2ecf7fe700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 0x7f2ec803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.034+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ed004c330 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.034+0000 7f2eddd9b700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 0x7f2ec803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.034+0000 7f2eddd9b700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 0x7f2ec803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006fd0 tx=0x7f2ed4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:39.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.039+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2ed0022700 con 0x7f2ee00ff7a0 2024-06-15T00:58:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:39.247+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ee01992c0 con 0x7f2ec80383c0 2024-06-15T00:58:39.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:39.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:39 smithi137 ceph-mon[29354]: Deploying daemon node-exporter.smithi137 on smithi137 2024-06-15T00:58:41.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:40 smithi137 ceph-mon[29354]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:41.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:40 smithi137 ceph-mon[29354]: Deploying cephadm binary to smithi142 2024-06-15T00:58:41.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.728+0000 7f2ecf7fe700 1 -- 172.21.15.137:0/1646362797 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2ee01992c0 con 0x7f2ec80383c0 2024-06-15T00:58:41.730 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 msgr2=0x7f2ec803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 0x7f2ec803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006fd0 tx=0x7f2ed4006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 msgr2=0x7f2ee01958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006010 tx=0x7f2ed00057b0 comp rx=0 tx=0).stop 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 shutdown_connections 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2ec80383c0 0x7f2ec803a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 --2- 172.21.15.137:0/1646362797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee00ff7a0 0x7f2ee01958a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 >> 172.21.15.137:0/1646362797 conn(0x7f2ee00fad20 msgr2=0x7f2ee00fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:41.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 shutdown_connections 2024-06-15T00:58:41.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:41.730+0000 7f2ee4d86700 1 -- 172.21.15.137:0/1646362797 wait complete. 2024-06-15T00:58:41.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:41 smithi137 ceph-mon[29354]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:41.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:41 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:42.603 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch host ls --format=json 2024-06-15T00:58:42.840 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:42 smithi137 ceph-mon[29354]: Added host smithi142 2024-06-15T00:58:43.033 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T00:58:44.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:44.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:44.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:44.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:44.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:44.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: Deploying daemon alertmanager.smithi137 on smithi137 2024-06-15T00:58:44.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:44 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.475+0000 7f2f8c413700 1 -- 172.21.15.137:0/1405287772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 msgr2=0x7f2f840ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.475+0000 7f2f8c413700 1 --2- 172.21.15.137:0/1405287772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f840ffbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009910 tx=0x7f2f78009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- 172.21.15.137:0/1405287772 shutdown_connections 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 --2- 172.21.15.137:0/1405287772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f840ffbb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- 172.21.15.137:0/1405287772 >> 172.21.15.137:0/1405287772 conn(0x7f2f840fad30 msgr2=0x7f2f840fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- 172.21.15.137:0/1405287772 shutdown_connections 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- 172.21.15.137:0/1405287772 wait complete. 2024-06-15T00:58:44.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 Processor -- start 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- start start 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.476+0000 7f2f8c413700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8418cb70 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8a1af700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8a1af700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59824/0 (socket says 172.21.15.137:59824) 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8a1af700 1 -- 172.21.15.137:0/2015096649 learned_addr learned my addr 172.21.15.137:0/2015096649 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8a1af700 1 -- 172.21.15.137:0/2015096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780095f0 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8a1af700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2f78004f70 tx=0x7f2f78005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f7801d070 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2f7800bdf0 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8418ac80 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f78022e40 con 0x7f2f840ff790 2024-06-15T00:58:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.477+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8418b120 con 0x7f2f840ff790 2024-06-15T00:58:44.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.478+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f2f78022700 con 0x7f2f840ff790 2024-06-15T00:58:44.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.478+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84062430 con 0x7f2f840ff790 2024-06-15T00:58:44.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.478+0000 7f2f777fe700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 0x7f2f7003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:44.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.478+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f7804c300 con 0x7f2f840ff790 2024-06-15T00:58:44.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.478+0000 7f2f899ae700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 0x7f2f7003a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:44.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.479+0000 7f2f899ae700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 0x7f2f7003a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:44.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.483+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2f84062430 con 0x7f2f840ff790 2024-06-15T00:58:44.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.694+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f840623c0 con 0x7f2f70038410 2024-06-15T00:58:44.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.695+0000 7f2f777fe700 1 -- 172.21.15.137:0/2015096649 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f2f840623c0 con 0x7f2f70038410 2024-06-15T00:58:44.696 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T00:58:44.696 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-06-15T00:58:44.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 msgr2=0x7f2f7003a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:44.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 0x7f2f7003a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 msgr2=0x7f2f8418c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2f78004f70 tx=0x7f2f78005ee0 comp rx=0 tx=0).stop 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 shutdown_connections 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f2f70038410 0x7f2f7003a8d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 --2- 172.21.15.137:0/2015096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f840ff790 0x7f2f8418c630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 >> 172.21.15.137:0/2015096649 conn(0x7f2f840fad30 msgr2=0x7f2f840fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 shutdown_connections 2024-06-15T00:58:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:44.698+0000 7f2f8c413700 1 -- 172.21.15.137:0/2015096649 wait complete. 2024-06-15T00:58:45.564 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-15T00:58:45.564 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd crush tunables default 2024-06-15T00:58:45.768 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T00:58:46.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:46 smithi137 ceph-mon[29354]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-15T00:58:46.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:46 smithi137 ceph-mon[29354]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 -- 172.21.15.137:0/4015840838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 msgr2=0x7f888c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 --2- 172.21.15.137:0/4015840838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c0ffbc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8880009910 tx=0x7f8880009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 -- 172.21.15.137:0/4015840838 shutdown_connections 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 --2- 172.21.15.137:0/4015840838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c0ffbc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 -- 172.21.15.137:0/4015840838 >> 172.21.15.137:0/4015840838 conn(0x7f888c0fad20 msgr2=0x7f888c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 -- 172.21.15.137:0/4015840838 shutdown_connections 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.062+0000 7f8892a8b700 1 -- 172.21.15.137:0/4015840838 wait complete. 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8892a8b700 1 Processor -- start 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8892a8b700 1 -- start start 2024-06-15T00:58:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8892a8b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:48.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8892a8b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c195e20 con 0x7f888c0ff7a0 2024-06-15T00:58:48.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8890827700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:48.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8890827700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56616/0 (socket says 172.21.15.137:56616) 2024-06-15T00:58:48.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.063+0000 7f8890827700 1 -- 172.21.15.137:0/2702696111 learned_addr learned my addr 172.21.15.137:0/2702696111 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T00:58:48.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8890827700 1 -- 172.21.15.137:0/2702696111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800095f0 con 0x7f888c0ff7a0 2024-06-15T00:58:48.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8890827700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8880004f70 tx=0x7f8880005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:48.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f888001d070 con 0x7f888c0ff7a0 2024-06-15T00:58:48.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f888000bdf0 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c196020 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c1964c0 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.064+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8880022e40 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.065+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f888000f460 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.065+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c18f290 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.065+0000 7f8889ffb700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 0x7f887c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.066+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f888004c4f0 con 0x7f888c0ff7a0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.066+0000 7f888bfff700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 0x7f887c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:48.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.066+0000 7f888bfff700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 0x7f887c03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:48.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.075+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f888000b960 con 0x7f888c0ff7a0 2024-06-15T00:58:48.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:48.289+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f888c02cc70 con 0x7f888c0ff7a0 2024-06-15T00:58:48.363 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:48 smithi137 ceph-mon[29354]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:49.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.084+0000 7f8889ffb700 1 -- 172.21.15.137:0/2702696111 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8880027020 con 0x7f888c0ff7a0 2024-06-15T00:58:49.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 msgr2=0x7f887c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:49.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 0x7f887c03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 msgr2=0x7f888c1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8880004f70 tx=0x7f8880005ee0 comp rx=0 tx=0).stop 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 shutdown_connections 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f887c038410 0x7f887c03a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 --2- 172.21.15.137:0/2702696111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f888c0ff7a0 0x7f888c1958e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 >> 172.21.15.137:0/2702696111 conn(0x7f888c0fad20 msgr2=0x7f888c0690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:49.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 shutdown_connections 2024-06-15T00:58:49.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T00:58:49.086+0000 7f8892a8b700 1 -- 172.21.15.137:0/2702696111 wait complete. 2024-06-15T00:58:49.089 INFO:teuthology.orchestra.run.smithi137.stderr:adjusted tunables profile to default 2024-06-15T00:58:49.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2702696111' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-15T00:58:50.179 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:50 smithi137 ceph-mon[29354]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:50.179 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:50 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2702696111' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-15T00:58:50.180 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:50 smithi137 ceph-mon[29354]: osdmap e4: 0 total, 0 up, 0 in 2024-06-15T00:58:50.205 INFO:tasks.cephadm:Adding mon.smithi137 on smithi137 2024-06-15T00:58:50.205 INFO:tasks.cephadm:Adding mon.smithi142 on smithi142 2024-06-15T00:58:50.205 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch apply mon '2;smithi137:172.21.15.137=smithi137;smithi142:172.21.15.142=smithi142' 2024-06-15T00:58:50.406 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:50.452 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:51.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 -- 172.21.15.142:0/1280603306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 msgr2=0x7f0410073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:51.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 --2- 172.21.15.142:0/1280603306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f0410073220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0400009910 tx=0x7f0400009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:51.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 -- 172.21.15.142:0/1280603306 shutdown_connections 2024-06-15T00:58:51.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 --2- 172.21.15.142:0/1280603306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f0410073220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:51.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 -- 172.21.15.142:0/1280603306 >> 172.21.15.142:0/1280603306 conn(0x7f04100fbe30 msgr2=0x7f04100fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 -- 172.21.15.142:0/1280603306 shutdown_connections 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.904+0000 7f041564d700 1 -- 172.21.15.142:0/1280603306 wait complete. 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f041564d700 1 Processor -- start 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f041564d700 1 -- start start 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f041564d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f041564d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041019c420 con 0x7f0410074dc0 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f040effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f040effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:33738/0 (socket says 172.21.15.142:33738) 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.905+0000 7f040effd700 1 -- 172.21.15.142:0/2936337643 learned_addr learned my addr 172.21.15.142:0/2936337643 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f040effd700 1 -- 172.21.15.142:0/2936337643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000095f0 con 0x7f0410074dc0 2024-06-15T00:58:51.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f040effd700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f04000057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:51.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f040001d070 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041019c620 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f040000bdf0 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0400022e50 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.906+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041019cac0 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.907+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f040000b960 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.907+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04101959b0 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.907+0000 7f03f7fff700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 0x7f03f803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.907+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f040004c2c0 con 0x7f0410074dc0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.907+0000 7f040e7fc700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 0x7f03f803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:51.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.908+0000 7f040e7fc700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 0x7f03f803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0404006fd0 tx=0x7f0404006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:51.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:51.912+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f040000f460 con 0x7f0410074dc0 2024-06-15T00:58:52.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.110+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 --> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi137:172.21.15.137=smithi137;smithi142:172.21.15.142=smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7f04100611d0 con 0x7f03f80383c0 2024-06-15T00:58:52.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.116+0000 7f03f7fff700 1 -- 172.21.15.142:0/2936337643 <== mgr.14164 v2:172.21.15.137:6800/2929246681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f04100611d0 con 0x7f03f80383c0 2024-06-15T00:58:52.117 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 msgr2=0x7f03f803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 0x7f03f803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0404006fd0 tx=0x7f0404006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 msgr2=0x7f041019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f04000057b0 comp rx=0 tx=0).stop 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 shutdown_connections 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f03f80383c0 0x7f03f803a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 --2- 172.21.15.142:0/2936337643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0410074dc0 0x7f041019bee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:52.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.118+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 >> 172.21.15.142:0/2936337643 conn(0x7f04100fbe30 msgr2=0x7f04100fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:52.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.119+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 shutdown_connections 2024-06-15T00:58:52.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:52.119+0000 7f041564d700 1 -- 172.21.15.142:0/2936337643 wait complete. 2024-06-15T00:58:52.482 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:52 smithi137 ceph-mon[29354]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:52.754 DEBUG:teuthology.orchestra.run.smithi142:mon.smithi142> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service 2024-06-15T00:58:52.756 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:58:52.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:58:53.155 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:53.202 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:53.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:53 smithi137 ceph-mon[29354]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi137:172.21.15.137=smithi137;smithi142:172.21.15.142=smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T00:58:53.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:53 smithi137 ceph-mon[29354]: Saving service mon spec with placement smithi137:172.21.15.137=smithi137;smithi142:172.21.15.142=smithi142;count:2 2024-06-15T00:58:53.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:53 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.482 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-15T00:58:54.482 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:54 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:58:54.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 -- 172.21.15.142:0/310506422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 msgr2=0x7f3110103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:54.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 --2- 172.21.15.142:0/310506422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110103720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3100009910 tx=0x7f3100009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:54.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 -- 172.21.15.142:0/310506422 shutdown_connections 2024-06-15T00:58:54.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 --2- 172.21.15.142:0/310506422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110103720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:54.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 -- 172.21.15.142:0/310506422 >> 172.21.15.142:0/310506422 conn(0x7f31100fad20 msgr2=0x7f31100fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:54.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 -- 172.21.15.142:0/310506422 shutdown_connections 2024-06-15T00:58:54.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 -- 172.21.15.142:0/310506422 wait complete. 2024-06-15T00:58:54.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.572+0000 7f3117f1a700 1 Processor -- start 2024-06-15T00:58:54.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3117f1a700 1 -- start start 2024-06-15T00:58:54.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3117f1a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:54.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3117f1a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101011d0 con 0x7f3110101330 2024-06-15T00:58:54.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3115cb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:54.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3115cb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:33752/0 (socket says 172.21.15.142:33752) 2024-06-15T00:58:54.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3115cb6700 1 -- 172.21.15.142:0/512151356 learned_addr learned my addr 172.21.15.142:0/512151356 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3115cb6700 1 -- 172.21.15.142:0/512151356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000095f0 con 0x7f3110101330 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.573+0000 7f3115cb6700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3100004f40 tx=0x7f31000057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.574+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f310001d070 con 0x7f3110101330 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.574+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f310000bdf0 con 0x7f3110101330 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.574+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3100022e30 con 0x7f3110101330 2024-06-15T00:58:54.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.574+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31100ff2e0 con 0x7f3110101330 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.574+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31100ff780 con 0x7f3110101330 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.575+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f310000b960 con 0x7f3110101330 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.575+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31100623c0 con 0x7f3110101330 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.575+0000 7f3106ffd700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 0x7f30fc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.575+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f310004c340 con 0x7f3110101330 2024-06-15T00:58:54.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.575+0000 7f31154b5700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 0x7f30fc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:54.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.576+0000 7f31154b5700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 0x7f30fc03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f310c006fd0 tx=0x7f310c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:54.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.580+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f310000d340 con 0x7f3110101330 2024-06-15T00:58:54.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.841+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f311018ed60 con 0x7f3110101330 2024-06-15T00:58:54.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.841+0000 7f3106ffd700 1 -- 172.21.15.142:0/512151356 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3100027030 con 0x7f3110101330 2024-06-15T00:58:54.842 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:58:54.842 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 msgr2=0x7f30fc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 0x7f30fc03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f310c006fd0 tx=0x7f310c006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 msgr2=0x7f3110100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3100004f40 tx=0x7f31000057b0 comp rx=0 tx=0).stop 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 shutdown_connections 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f30fc0383c0 0x7f30fc03a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 --2- 172.21.15.142:0/512151356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3110101330 0x7f3110100c90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:54.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.843+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 >> 172.21.15.142:0/512151356 conn(0x7f31100fad20 msgr2=0x7f31100fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:54.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.844+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 shutdown_connections 2024-06-15T00:58:54.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:54.844+0000 7f3117f1a700 1 -- 172.21.15.142:0/512151356 wait complete. 2024-06-15T00:58:54.845 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:58:55.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:55 smithi137 ceph-mon[29354]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-15T00:58:55.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:55 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-15T00:58:55.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:55 smithi137 ceph-mon[29354]: Deploying daemon grafana.smithi137 on smithi137 2024-06-15T00:58:55.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:55 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/512151356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:58:56.372 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:58:56.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:58:56.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:56 smithi137 ceph-mon[29354]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:56.575 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:56.621 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:58:57.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:57 smithi137 ceph-mon[29354]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:58:58.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 -- 172.21.15.142:0/2743307733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 msgr2=0x7f99dc101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:58.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 --2- 172.21.15.142:0/2743307733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc101150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009910 tx=0x7f99cc009c20 comp rx=0 tx=0).stop 2024-06-15T00:58:58.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 -- 172.21.15.142:0/2743307733 shutdown_connections 2024-06-15T00:58:58.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 --2- 172.21.15.142:0/2743307733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc101150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:58.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 -- 172.21.15.142:0/2743307733 >> 172.21.15.142:0/2743307733 conn(0x7f99dc0fa4a0 msgr2=0x7f99dc0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 -- 172.21.15.142:0/2743307733 shutdown_connections 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 -- 172.21.15.142:0/2743307733 wait complete. 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.099+0000 7f99e16f4700 1 Processor -- start 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99e16f4700 1 -- start start 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99e16f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99e16f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc198610 con 0x7f99dc100d30 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99daffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:58.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99daffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:33784/0 (socket says 172.21.15.142:33784) 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99daffd700 1 -- 172.21.15.142:0/299598360 learned_addr learned my addr 172.21.15.142:0/299598360 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99daffd700 1 -- 172.21.15.142:0/299598360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0095f0 con 0x7f99dc100d30 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.100+0000 7f99daffd700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99cc004f70 tx=0x7f99cc005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99cc01d070 con 0x7f99dc100d30 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc198750 con 0x7f99dc100d30 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f99cc00bdf0 con 0x7f99dc100d30 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99cc022e40 con 0x7f99dc100d30 2024-06-15T00:58:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc198bf0 con 0x7f99dc100d30 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f99cc00b960 con 0x7f99dc100d30 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.101+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc18e280 con 0x7f99dc100d30 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.102+0000 7f99c3fff700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 0x7f99c403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.102+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99cc04c420 con 0x7f99dc100d30 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.102+0000 7f99da7fc700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 0x7f99c403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:58:58.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.102+0000 7f99da7fc700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 0x7f99c403a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006fd0 tx=0x7f99d0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:58:58.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.106+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f99cc02ab50 con 0x7f99dc100d30 2024-06-15T00:58:58.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.371+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f99dc02cc70 con 0x7f99dc100d30 2024-06-15T00:58:58.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.371+0000 7f99c3fff700 1 -- 172.21.15.142:0/299598360 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f99cc027020 con 0x7f99dc100d30 2024-06-15T00:58:58.372 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:58:58.372 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:58:58.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 msgr2=0x7f99c403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:58.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 0x7f99c403a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006fd0 tx=0x7f99d0006e40 comp rx=0 tx=0).stop 2024-06-15T00:58:58.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 msgr2=0x7f99dc0ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99cc004f70 tx=0x7f99cc005ee0 comp rx=0 tx=0).stop 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 shutdown_connections 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f99c40383c0 0x7f99c403a880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 --2- 172.21.15.142:0/299598360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99dc100d30 0x7f99dc0ff4e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 >> 172.21.15.142:0/299598360 conn(0x7f99dc0fa4a0 msgr2=0x7f99dc0fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 shutdown_connections 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:58:58.374+0000 7f99e16f4700 1 -- 172.21.15.142:0/299598360 wait complete. 2024-06-15T00:58:58.375 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:58:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:58 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/299598360' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:58:59.969 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:58:59.969 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:00.163 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:00.210 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:59 smithi137 ceph-mon[29354]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:58:59 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 -- 172.21.15.142:0/2312979062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 msgr2=0x7eff3c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 --2- 172.21.15.142:0/2312979062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0ffbb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b0a0 tx=0x7eff3400b3b0 comp rx=0 tx=0).stop 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 -- 172.21.15.142:0/2312979062 shutdown_connections 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 --2- 172.21.15.142:0/2312979062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0ffbb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 -- 172.21.15.142:0/2312979062 >> 172.21.15.142:0/2312979062 conn(0x7eff3c0fad30 msgr2=0x7eff3c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 -- 172.21.15.142:0/2312979062 shutdown_connections 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 -- 172.21.15.142:0/2312979062 wait complete. 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.593+0000 7eff4174d700 1 Processor -- start 2024-06-15T00:59:01.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff4174d700 1 -- start start 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff4174d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff4174d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c198640 con 0x7eff3c0ff790 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff3affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff3affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:44094/0 (socket says 172.21.15.142:44094) 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff3affd700 1 -- 172.21.15.142:0/415518458 learned_addr learned my addr 172.21.15.142:0/415518458 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff3affd700 1 -- 172.21.15.142:0/415518458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009d00 con 0x7eff3c0ff790 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff3affd700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff340062a0 tx=0x7eff3400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff3400e040 con 0x7eff3c0ff790 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7eff34012ad0 con 0x7eff3c0ff790 2024-06-15T00:59:01.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c198780 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.594+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff3401c860 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.595+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c198c20 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.595+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7eff34012640 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.595+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c18e270 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.596+0000 7eff23fff700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 0x7eff2403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.596+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff3404c540 con 0x7eff3c0ff790 2024-06-15T00:59:01.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.596+0000 7eff3a7fc700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 0x7eff2403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:01.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.596+0000 7eff3a7fc700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 0x7eff2403a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff30006fd0 tx=0x7eff30006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:01.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.600+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff3401aa00 con 0x7eff3c0ff790 2024-06-15T00:59:01.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.862+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff3c02cc70 con 0x7eff3c0ff790 2024-06-15T00:59:01.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.862+0000 7eff23fff700 1 -- 172.21.15.142:0/415518458 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7eff3401a3b0 con 0x7eff3c0ff790 2024-06-15T00:59:01.864 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:01.864 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:01.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 msgr2=0x7eff2403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:01.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 0x7eff2403a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff30006fd0 tx=0x7eff30006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:01.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 msgr2=0x7eff3c0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff340062a0 tx=0x7eff3400bd40 comp rx=0 tx=0).stop 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 shutdown_connections 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7eff24038410 0x7eff2403a8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 --2- 172.21.15.142:0/415518458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff3c0ff790 0x7eff3c0693f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.864+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 >> 172.21.15.142:0/415518458 conn(0x7eff3c0fad30 msgr2=0x7eff3c0fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.865+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 shutdown_connections 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:01.865+0000 7eff4174d700 1 -- 172.21.15.142:0/415518458 wait complete. 2024-06-15T00:59:01.867 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:02.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:01 smithi137 ceph-mon[29354]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:03.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:02 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/415518458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:03.429 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:03.430 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:03.630 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:03.677 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:03.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:03 smithi137 ceph-mon[29354]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:05.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.152+0000 7fefba035700 1 -- 172.21.15.142:0/1685336874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 msgr2=0x7fefb4103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:05.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.152+0000 7fefba035700 1 --2- 172.21.15.142:0/1685336874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4103720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009910 tx=0x7fefa4009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:05.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.152+0000 7fefba035700 1 -- 172.21.15.142:0/1685336874 shutdown_connections 2024-06-15T00:59:05.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.152+0000 7fefba035700 1 --2- 172.21.15.142:0/1685336874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4103720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:05.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.152+0000 7fefba035700 1 -- 172.21.15.142:0/1685336874 >> 172.21.15.142:0/1685336874 conn(0x7fefb40fad20 msgr2=0x7fefb40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:05.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.153+0000 7fefba035700 1 -- 172.21.15.142:0/1685336874 shutdown_connections 2024-06-15T00:59:05.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.153+0000 7fefba035700 1 -- 172.21.15.142:0/1685336874 wait complete. 2024-06-15T00:59:05.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.153+0000 7fefba035700 1 Processor -- start 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.153+0000 7fefba035700 1 -- start start 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefba035700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefba035700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4195db0 con 0x7fefb4101330 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:44118/0 (socket says 172.21.15.142:44118) 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb37fe700 1 -- 172.21.15.142:0/755614220 learned_addr learned my addr 172.21.15.142:0/755614220 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb37fe700 1 -- 172.21.15.142:0/755614220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40095f0 con 0x7fefb4101330 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb37fe700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefa4004f40 tx=0x7fefa40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefa401d070 con 0x7fefb4101330 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb4195fb0 con 0x7fefb4101330 2024-06-15T00:59:05.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fefa400bdf0 con 0x7fefb4101330 2024-06-15T00:59:05.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.154+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefa4022e30 con 0x7fefb4101330 2024-06-15T00:59:05.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.155+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb4196450 con 0x7fefb4101330 2024-06-15T00:59:05.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.156+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fefa400b960 con 0x7fefb4101330 2024-06-15T00:59:05.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.155+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb418f380 con 0x7fefb4101330 2024-06-15T00:59:05.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.156+0000 7fefb0ff9700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 0x7fef9c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:05.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.156+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fefa404c340 con 0x7fefb4101330 2024-06-15T00:59:05.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.156+0000 7fefb2ffd700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 0x7fef9c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:05.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.156+0000 7fefb2ffd700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 0x7fef9c03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006fd0 tx=0x7fefa8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:05.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.165+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fefa400d340 con 0x7fefb4101330 2024-06-15T00:59:05.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.426+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fefb402cc70 con 0x7fefb4101330 2024-06-15T00:59:05.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.426+0000 7fefb0ff9700 1 -- 172.21.15.142:0/755614220 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fefa4027030 con 0x7fefb4101330 2024-06-15T00:59:05.428 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:05.428 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:05.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.428+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 msgr2=0x7fef9c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:05.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.428+0000 7fefba035700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 0x7fef9c03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006fd0 tx=0x7fefa8006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:05.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.428+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 msgr2=0x7fefb4195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:05.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.428+0000 7fefba035700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefa4004f40 tx=0x7fefa40057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 shutdown_connections 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fef9c0383c0 0x7fef9c03a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 --2- 172.21.15.142:0/755614220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb4101330 0x7fefb4195870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 >> 172.21.15.142:0/755614220 conn(0x7fefb40fad20 msgr2=0x7fefb40fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 shutdown_connections 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:05.429+0000 7fefba035700 1 -- 172.21.15.142:0/755614220 wait complete. 2024-06-15T00:59:05.431 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:06.038 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:05 smithi137 ceph-mon[29354]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:06.038 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:05 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/755614220' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:07.016 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:07.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:07.219 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:07.265 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:08.070 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:07 smithi137 ceph-mon[29354]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:08.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 -- 172.21.15.142:0/4292351125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 msgr2=0x7fc534100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 --2- 172.21.15.142:0/4292351125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc534100cb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc530009910 tx=0x7fc530009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 -- 172.21.15.142:0/4292351125 shutdown_connections 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 --2- 172.21.15.142:0/4292351125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc534100cb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 -- 172.21.15.142:0/4292351125 >> 172.21.15.142:0/4292351125 conn(0x7fc5340fbe30 msgr2=0x7fc5340fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 -- 172.21.15.142:0/4292351125 shutdown_connections 2024-06-15T00:59:08.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.693+0000 7fc53b968700 1 -- 172.21.15.142:0/4292351125 wait complete. 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc53b968700 1 Processor -- start 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc53b968700 1 -- start start 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc53b968700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc53b968700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534195e00 con 0x7fc534100890 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc539704700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc539704700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:44132/0 (socket says 172.21.15.142:44132) 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.694+0000 7fc539704700 1 -- 172.21.15.142:0/3152892113 learned_addr learned my addr 172.21.15.142:0/3152892113 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:08.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc539704700 1 -- 172.21.15.142:0/3152892113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300095f0 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc539704700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc5300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc53001d070 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc53000bdf0 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc534196000 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc530022e50 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.695+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341964a0 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.696+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fc53000b960 con 0x7fc534100890 2024-06-15T00:59:08.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.696+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53418f370 con 0x7fc534100890 2024-06-15T00:59:08.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.696+0000 7fc52a7fc700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 0x7fc52003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:08.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.696+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc53004c380 con 0x7fc534100890 2024-06-15T00:59:08.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.696+0000 7fc538f03700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 0x7fc52003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:08.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.697+0000 7fc538f03700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 0x7fc52003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc524006fd0 tx=0x7fc524006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:08.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.701+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc53000f460 con 0x7fc534100890 2024-06-15T00:59:08.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.963+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc53402cc70 con 0x7fc534100890 2024-06-15T00:59:08.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.964+0000 7fc52a7fc700 1 -- 172.21.15.142:0/3152892113 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc530027020 con 0x7fc534100890 2024-06-15T00:59:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:08.965 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:08.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 msgr2=0x7fc52003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:08.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 0x7fc52003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc524006fd0 tx=0x7fc524006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:08.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 msgr2=0x7fc5341958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:08.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc5300057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 shutdown_connections 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc5200383c0 0x7fc52003a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 --2- 172.21.15.142:0/3152892113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534100890 0x7fc5341958c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 >> 172.21.15.142:0/3152892113 conn(0x7fc5340fbe30 msgr2=0x7fc5340fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 shutdown_connections 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:08.966+0000 7fc53b968700 1 -- 172.21.15.142:0/3152892113 wait complete. 2024-06-15T00:59:08.968 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:10.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:10.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/3152892113' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:09 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:10.659 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:10.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:10.860 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:10.907 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:10.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:10 smithi137 ceph-mon[29354]: Deploying daemon prometheus.smithi137 on smithi137 2024-06-15T00:59:11.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:11 smithi137 ceph-mon[29354]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 -- 172.21.15.142:0/569821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 msgr2=0x7f54700ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 --2- 172.21.15.142:0/569821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f54700ffbc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5464009910 tx=0x7f5464009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 -- 172.21.15.142:0/569821859 shutdown_connections 2024-06-15T00:59:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 --2- 172.21.15.142:0/569821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f54700ffbc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 -- 172.21.15.142:0/569821859 >> 172.21.15.142:0/569821859 conn(0x7f54700fad20 msgr2=0x7f54700fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.254+0000 7f5474f79700 1 -- 172.21.15.142:0/569821859 shutdown_connections 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.255+0000 7f5474f79700 1 -- 172.21.15.142:0/569821859 wait complete. 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.255+0000 7f5474f79700 1 Processor -- start 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.255+0000 7f5474f79700 1 -- start start 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.255+0000 7f5474f79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.255+0000 7f5474f79700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470194d40 con 0x7f54700ff7a0 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f546e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f546e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:59316/0 (socket says 172.21.15.142:59316) 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f546e59c700 1 -- 172.21.15.142:0/1160926675 learned_addr learned my addr 172.21.15.142:0/1160926675 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:12.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f546e59c700 1 -- 172.21.15.142:0/1160926675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640095f0 con 0x7f54700ff7a0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f546e59c700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f54640057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f546401d070 con 0x7f54700ff7a0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5470194f40 con 0x7f54700ff7a0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f546400bdf0 con 0x7f54700ff7a0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5464022e50 con 0x7f54700ff7a0 2024-06-15T00:59:12.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.256+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54701953e0 con 0x7f54700ff7a0 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.257+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f546400f4b0 con 0x7f54700ff7a0 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.257+0000 7f545f7fe700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 0x7f545803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.257+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547018e280 con 0x7f54700ff7a0 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.257+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f546404c400 con 0x7f54700ff7a0 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.257+0000 7f546dd9b700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 0x7f545803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.258+0000 7f546dd9b700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 0x7f545803a830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5460006fd0 tx=0x7f5460006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:12.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.266+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5464017440 con 0x7f54700ff7a0 2024-06-15T00:59:12.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.529+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f547002cc70 con 0x7f54700ff7a0 2024-06-15T00:59:12.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.530+0000 7f545f7fe700 1 -- 172.21.15.142:0/1160926675 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5464027030 con 0x7f54700ff7a0 2024-06-15T00:59:12.530 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:12.531 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:12.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 msgr2=0x7f545803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:12.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 0x7f545803a830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5460006fd0 tx=0x7f5460006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:12.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 msgr2=0x7f5470194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:12.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f54640057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:12.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 shutdown_connections 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f5458038370 0x7f545803a830 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 --2- 172.21.15.142:0/1160926675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54700ff7a0 0x7f5470194800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 >> 172.21.15.142:0/1160926675 conn(0x7f54700fad20 msgr2=0x7f54700fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 shutdown_connections 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:12.532+0000 7f5474f79700 1 -- 172.21.15.142:0/1160926675 wait complete. 2024-06-15T00:59:12.533 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:13.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:13 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/1160926675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:13.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:13 smithi137 ceph-mon[29354]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:14.107 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:14.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:14.309 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:14.356 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:15.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:14 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:15.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.774+0000 7fc20a7ce700 1 -- 172.21.15.142:0/1091362107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 msgr2=0x7fc2040ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:15.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.774+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/1091362107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2040ffbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009910 tx=0x7fc1f4009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:15.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.774+0000 7fc20a7ce700 1 -- 172.21.15.142:0/1091362107 shutdown_connections 2024-06-15T00:59:15.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.774+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/1091362107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2040ffbc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.774+0000 7fc20a7ce700 1 -- 172.21.15.142:0/1091362107 >> 172.21.15.142:0/1091362107 conn(0x7fc2040fad20 msgr2=0x7fc2040fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 -- 172.21.15.142:0/1091362107 shutdown_connections 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 -- 172.21.15.142:0/1091362107 wait complete. 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 Processor -- start 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 -- start start 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.775+0000 7fc20a7ce700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc204195e20 con 0x7fc2040ff7a0 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc203fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc203fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:59326/0 (socket says 172.21.15.142:59326) 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc203fff700 1 -- 172.21.15.142:0/2581163841 learned_addr learned my addr 172.21.15.142:0/2581163841 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc203fff700 1 -- 172.21.15.142:0/2581163841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f40095f0 con 0x7fc2040ff7a0 2024-06-15T00:59:15.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc203fff700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4004f70 tx=0x7fc1f4005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1f401d070 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc1f400bdf0 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1f4022e40 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc204196020 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.776+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2041964c0 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.777+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fc1f400f4b0 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.777+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20418f290 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.777+0000 7fc2017fa700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 0x7fc1ec03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.777+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1f404c590 con 0x7fc2040ff7a0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.777+0000 7fc2037fe700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 0x7fc1ec03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.778+0000 7fc2037fe700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 0x7fc1ec03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:15.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:15.782+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc1f4020ba0 con 0x7fc2040ff7a0 2024-06-15T00:59:16.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.047+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc20402cc70 con 0x7fc2040ff7a0 2024-06-15T00:59:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.048+0000 7fc2017fa700 1 -- 172.21.15.142:0/2581163841 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc1f4027030 con 0x7fc2040ff7a0 2024-06-15T00:59:16.048 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:16.049 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:16.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 msgr2=0x7fc1ec03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 0x7fc1ec03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 msgr2=0x7fc2041958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4004f70 tx=0x7fc1f4005ee0 comp rx=0 tx=0).stop 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 shutdown_connections 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fc1ec0383c0 0x7fc1ec03a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 --2- 172.21.15.142:0/2581163841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2040ff7a0 0x7fc2041958e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 >> 172.21.15.142:0/2581163841 conn(0x7fc2040fad20 msgr2=0x7fc204069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 shutdown_connections 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:16.050+0000 7fc20a7ce700 1 -- 172.21.15.142:0/2581163841 wait complete. 2024-06-15T00:59:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:16.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:15 smithi137 ceph-mon[29354]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:17.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:16 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2581163841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:17.592 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:17.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:17.839 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:18.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:17 smithi137 ceph-mon[29354]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- 172.21.15.142:0/3061391782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 msgr2=0x7f58f00ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 --2- 172.21.15.142:0/3061391782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00ffbb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009910 tx=0x7f58ec009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- 172.21.15.142:0/3061391782 shutdown_connections 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 --2- 172.21.15.142:0/3061391782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00ffbb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- 172.21.15.142:0/3061391782 >> 172.21.15.142:0/3061391782 conn(0x7f58f00fad30 msgr2=0x7f58f00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:19.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- 172.21.15.142:0/3061391782 shutdown_connections 2024-06-15T00:59:19.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- 172.21.15.142:0/3061391782 wait complete. 2024-06-15T00:59:19.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 Processor -- start 2024-06-15T00:59:19.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.360+0000 7f58f79e8700 1 -- start start 2024-06-15T00:59:19.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f79e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:19.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f79e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00fd210 con 0x7f58f00ff790 2024-06-15T00:59:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f5784700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f5784700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:59352/0 (socket says 172.21.15.142:59352) 2024-06-15T00:59:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f5784700 1 -- 172.21.15.142:0/1055871288 learned_addr learned my addr 172.21.15.142:0/1055871288 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f5784700 1 -- 172.21.15.142:0/1055871288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0095f0 con 0x7f58f00ff790 2024-06-15T00:59:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f5784700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58ec01d070 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00fb320 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f58ec00bdf0 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.361+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58ec022e50 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.362+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f00fb7c0 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.362+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f00623c0 con 0x7f58f00ff790 2024-06-15T00:59:19.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.362+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f58ec022700 con 0x7f58f00ff790 2024-06-15T00:59:19.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.363+0000 7f58e27fc700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 0x7f58dc03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:19.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.363+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58ec04c330 con 0x7f58f00ff790 2024-06-15T00:59:19.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.363+0000 7f58f4f83700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 0x7f58dc03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:19.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.363+0000 7f58f4f83700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 0x7f58dc03a830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006fd0 tx=0x7f58e4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:19.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.368+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f58ec00b960 con 0x7f58f00ff790 2024-06-15T00:59:19.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.631+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58f018ec60 con 0x7f58f00ff790 2024-06-15T00:59:19.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.631+0000 7f58e27fc700 1 -- 172.21.15.142:0/1055871288 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f58ec027030 con 0x7f58f00ff790 2024-06-15T00:59:19.632 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:19.632 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.633+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 msgr2=0x7f58dc03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.633+0000 7f58f79e8700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 0x7f58dc03a830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006fd0 tx=0x7f58e4006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.633+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 msgr2=0x7f58f00fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.633+0000 7f58f79e8700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec0057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.633+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 shutdown_connections 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.634+0000 7f58f79e8700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f58dc038370 0x7f58dc03a830 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:19.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.634+0000 7f58f79e8700 1 --2- 172.21.15.142:0/1055871288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58f00ff790 0x7f58f00fccd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:19.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.634+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 >> 172.21.15.142:0/1055871288 conn(0x7f58f00fad30 msgr2=0x7f58f0069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:19.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.634+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 shutdown_connections 2024-06-15T00:59:19.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:19.634+0000 7f58f79e8700 1 -- 172.21.15.142:0/1055871288 wait complete. 2024-06-15T00:59:19.635 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:19.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:19 smithi137 ceph-mon[29354]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:19.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:19 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/1055871288' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:21.230 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:21.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:21.432 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:21.479 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:21.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:21 smithi137 ceph-mon[29354]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:22.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 -- 172.21.15.142:0/452852095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 msgr2=0x7ffabc100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:22.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 --2- 172.21.15.142:0/452852095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc100cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009910 tx=0x7ffaac009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 -- 172.21.15.142:0/452852095 shutdown_connections 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 --2- 172.21.15.142:0/452852095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc100cd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 -- 172.21.15.142:0/452852095 >> 172.21.15.142:0/452852095 conn(0x7ffabc0fbe30 msgr2=0x7ffabc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 -- 172.21.15.142:0/452852095 shutdown_connections 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 -- 172.21.15.142:0/452852095 wait complete. 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.880+0000 7ffac2714700 1 Processor -- start 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffac2714700 1 -- start start 2024-06-15T00:59:22.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffac2714700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffac2714700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc195e00 con 0x7ffabc1008b0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:54090/0 (socket says 172.21.15.142:54090) 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffabbfff700 1 -- 172.21.15.142:0/3421193948 learned_addr learned my addr 172.21.15.142:0/3421193948 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.881+0000 7ffabbfff700 1 -- 172.21.15.142:0/3421193948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0095f0 con 0x7ffabc1008b0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffabbfff700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaac01d070 con 0x7ffabc1008b0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc196000 con 0x7ffabc1008b0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffaac00bdf0 con 0x7ffabc1008b0 2024-06-15T00:59:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaac022e50 con 0x7ffabc1008b0 2024-06-15T00:59:22.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.882+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1964a0 con 0x7ffabc1008b0 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7ffaac00b960 con 0x7ffabc1008b0 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc18f390 con 0x7ffabc1008b0 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffab97fa700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 0x7ffaa403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffaac04c360 con 0x7ffabc1008b0 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffabb7fe700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 0x7ffaa403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:22.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.883+0000 7ffabb7fe700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 0x7ffaa403a830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:22.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:22.888+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffaac00f4b0 con 0x7ffabc1008b0 2024-06-15T00:59:23.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.150+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffabc02cc70 con 0x7ffabc1008b0 2024-06-15T00:59:23.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.151+0000 7ffab97fa700 1 -- 172.21.15.142:0/3421193948 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ffaac027030 con 0x7ffabc1008b0 2024-06-15T00:59:23.151 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:23.151 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 msgr2=0x7ffaa403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 0x7ffaa403a830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 msgr2=0x7ffabc1958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac0057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 shutdown_connections 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7ffaa4038370 0x7ffaa403a830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 --2- 172.21.15.142:0/3421193948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc1008b0 0x7ffabc1958c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 >> 172.21.15.142:0/3421193948 conn(0x7ffabc0fbe30 msgr2=0x7ffabc0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 shutdown_connections 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:23.153+0000 7ffac2714700 1 -- 172.21.15.142:0/3421193948 wait complete. 2024-06-15T00:59:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:23.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:23 smithi137 ceph-mon[29354]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:23.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:23 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/3421193948' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:24.697 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:24.698 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:24.896 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:24.943 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:26.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:25 smithi137 ceph-mon[29354]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:25 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:25 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:25 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:26.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:25 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 -- 172.21.15.142:0/1149392613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 msgr2=0x7f39cc1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 --2- 172.21.15.142:0/1149392613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1022b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009910 tx=0x7f39bc009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 -- 172.21.15.142:0/1149392613 shutdown_connections 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 --2- 172.21.15.142:0/1149392613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1022b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 -- 172.21.15.142:0/1149392613 >> 172.21.15.142:0/1149392613 conn(0x7f39cc0fa4d0 msgr2=0x7f39cc0fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:26.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 -- 172.21.15.142:0/1149392613 shutdown_connections 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.307+0000 7f39d3830700 1 -- 172.21.15.142:0/1149392613 wait complete. 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d3830700 1 Processor -- start 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d3830700 1 -- start start 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d3830700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d3830700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc195e00 con 0x7f39cc101e90 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d15cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d15cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:54112/0 (socket says 172.21.15.142:54112) 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.308+0000 7f39d15cc700 1 -- 172.21.15.142:0/1828769047 learned_addr learned my addr 172.21.15.142:0/1828769047 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39d15cc700 1 -- 172.21.15.142:0/1828769047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0095f0 con 0x7f39cc101e90 2024-06-15T00:59:26.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39d15cc700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39bc004f70 tx=0x7f39bc005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39bc01d070 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f39bc00bdf0 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc196000 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39bc022e40 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.309+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc1964a0 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.310+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc18f390 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.310+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f39bc022700 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.310+0000 7f39c27fc700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.310+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f39bc04c4d0 con 0x7f39cc101e90 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.310+0000 7f39d0dcb700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.311+0000 7f39d0dcb700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39c8006fd0 tx=0x7f39c8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:26.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.315+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f39bc02ba00 con 0x7f39cc101e90 2024-06-15T00:59:26.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.450+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f39bc00b960 con 0x7f39cc101e90 2024-06-15T00:59:26.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39d0dcb700 1 -- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 msgr2=0x7f39b803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-06-15T00:59:26.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39d0dcb700 1 -- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 msgr2=0x7f39b803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-15T00:59:26.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39d0dcb700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39c8006fd0 tx=0x7f39c8006e40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-15T00:59:26.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39d0dcb700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39c8006fd0 tx=0x7f39c8006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:26.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 msgr2=0x7f39b803a880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-06-15T00:59:26.478 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.474+0000 7f39c27fc700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:26.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.591+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f39b40051a0 con 0x7f39cc101e90 2024-06-15T00:59:26.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.591+0000 7f39c27fc700 1 -- 172.21.15.142:0/1828769047 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f39bc027030 con 0x7f39cc101e90 2024-06-15T00:59:26.592 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:26.592 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 msgr2=0x7f39cc1958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39bc004f70 tx=0x7f39bc005ee0 comp rx=0 tx=0).stop 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 shutdown_connections 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7f39b80383c0 0x7f39b803a880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 --2- 172.21.15.142:0/1828769047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39cc101e90 0x7f39cc1958c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:26.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.593+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 >> 172.21.15.142:0/1828769047 conn(0x7f39cc0fa4d0 msgr2=0x7f39cc0fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:26.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.594+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 shutdown_connections 2024-06-15T00:59:26.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:26.594+0000 7f39d3830700 1 -- 172.21.15.142:0/1828769047 wait complete. 2024-06-15T00:59:26.595 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:27.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:27 smithi137 ceph-mon[29354]: from='mgr.14164 172.21.15.137:0/1485827682' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-15T00:59:27.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:27 smithi137 ceph-mon[29354]: mgrmap e14: smithi137.nbzbyc(active, since 67s) 2024-06-15T00:59:27.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:27 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/1828769047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:28.169 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:28.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:28.373 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:28.420 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.764+0000 7fed9742c700 1 -- 172.21.15.142:0/2748689747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 msgr2=0x7fed900fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.764+0000 7fed9742c700 1 --2- 172.21.15.142:0/2748689747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed900fef80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009910 tx=0x7fed8c009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.764+0000 7fed9742c700 1 -- 172.21.15.142:0/2748689747 shutdown_connections 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.764+0000 7fed9742c700 1 --2- 172.21.15.142:0/2748689747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed900fef80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.764+0000 7fed9742c700 1 -- 172.21.15.142:0/2748689747 >> 172.21.15.142:0/2748689747 conn(0x7fed900fa4a0 msgr2=0x7fed900fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 -- 172.21.15.142:0/2748689747 shutdown_connections 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 -- 172.21.15.142:0/2748689747 wait complete. 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 Processor -- start 2024-06-15T00:59:29.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 -- start start 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.765+0000 7fed9742c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed90195e20 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed951c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed951c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:54132/0 (socket says 172.21.15.142:54132) 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed951c8700 1 -- 172.21.15.142:0/3354454356 learned_addr learned my addr 172.21.15.142:0/3354454356 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed951c8700 1 -- 172.21.15.142:0/3354454356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0095f0 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed951c8700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed8c01d070 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fed8c00bdf0 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed90196020 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed8c022e50 con 0x7fed900feb60 2024-06-15T00:59:29.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.766+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed901964c0 con 0x7fed900feb60 2024-06-15T00:59:29.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.767+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7fed8c00f460 con 0x7fed900feb60 2024-06-15T00:59:29.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.767+0000 7fed867fc700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 0x7fed7c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:29.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.767+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fed8c04d200 con 0x7fed900feb60 2024-06-15T00:59:29.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.767+0000 7fed949c7700 1 -- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 msgr2=0x7fed7c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/2929246681 2024-06-15T00:59:29.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.767+0000 7fed949c7700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 0x7fed7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T00:59:29.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.768+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed74005320 con 0x7fed900feb60 2024-06-15T00:59:29.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.773+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fed8c027070 con 0x7fed900feb60 2024-06-15T00:59:29.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.968+0000 7fed949c7700 1 -- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 msgr2=0x7fed7c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/2929246681 2024-06-15T00:59:29.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:29.968+0000 7fed949c7700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 0x7fed7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-15T00:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.062+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed74005190 con 0x7fed900feb60 2024-06-15T00:59:30.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.062+0000 7fed867fc700 1 -- 172.21.15.142:0/3354454356 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fed8c034070 con 0x7fed900feb60 2024-06-15T00:59:30.063 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:30.063 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 msgr2=0x7fed7c03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 0x7fed7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 msgr2=0x7fed901958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c0057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 shutdown_connections 2024-06-15T00:59:30.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:6800/2929246681,v1:172.21.15.137:6801/2929246681] conn(0x7fed7c038410 0x7fed7c03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 --2- 172.21.15.142:0/3354454356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed900feb60 0x7fed901958e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 >> 172.21.15.142:0/3354454356 conn(0x7fed900fa4a0 msgr2=0x7fed900fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 shutdown_connections 2024-06-15T00:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:30.064+0000 7fed9742c700 1 -- 172.21.15.142:0/3354454356 wait complete. 2024-06-15T00:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:30.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:30 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/3354454356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:31.631 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:31.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:31.836 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:31.883 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.310+0000 7fa4c42fa700 1 -- 172.21.15.142:0/2300997916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 msgr2=0x7fa4bc100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.310+0000 7fa4c42fa700 1 --2- 172.21.15.142:0/2300997916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc100cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b0a0 tx=0x7fa4ac00b3b0 comp rx=0 tx=0).stop 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- 172.21.15.142:0/2300997916 shutdown_connections 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 --2- 172.21.15.142:0/2300997916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc100cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- 172.21.15.142:0/2300997916 >> 172.21.15.142:0/2300997916 conn(0x7fa4bc0fbe60 msgr2=0x7fa4bc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- 172.21.15.142:0/2300997916 shutdown_connections 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- 172.21.15.142:0/2300997916 wait complete. 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 Processor -- start 2024-06-15T00:59:33.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- start start 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c42fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc195e40 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c2096700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c2096700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:57052/0 (socket says 172.21.15.142:57052) 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.311+0000 7fa4c2096700 1 -- 172.21.15.142:0/762786566 learned_addr learned my addr 172.21.15.142:0/762786566 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4c2096700 1 -- 172.21.15.142:0/762786566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac009d00 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4c2096700 1 --2- 172.21.15.142:0/762786566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0062a0 tx=0x7fa4ac00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4ac00e040 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa4ac012ad0 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc196040 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4ac01c860 con 0x7fa4bc1008a0 2024-06-15T00:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.312+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc1964e0 con 0x7fa4bc1008a0 2024-06-15T00:59:33.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.313+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) v1 ==== 45077+0+0 (secure 0 0 0) 0x7fa4ac012640 con 0x7fa4bc1008a0 2024-06-15T00:59:33.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.313+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc18f380 con 0x7fa4bc1008a0 2024-06-15T00:59:33.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.313+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa4ac04c220 con 0x7fa4bc1008a0 2024-06-15T00:59:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.318+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa4ac01aa00 con 0x7fa4bc1008a0 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: Active manager daemon smithi137.nbzbyc restarted 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: Activating manager daemon smithi137.nbzbyc 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: osdmap e5: 0 total, 0 up, 0 in 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: mgrmap e15: smithi137.nbzbyc(active, starting, since 0.00386395s) 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T00:59:33.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T00:59:33.481 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T00:59:33.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.598+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa4bc02cc70 con 0x7fa4bc1008a0 2024-06-15T00:59:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.598+0000 7fa4bb7fe700 1 -- 172.21.15.142:0/762786566 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa4ac01a3b0 con 0x7fa4bc1008a0 2024-06-15T00:59:33.599 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:33.599 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.600+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 msgr2=0x7fa4bc195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.600+0000 7fa4c42fa700 1 --2- 172.21.15.142:0/762786566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0062a0 tx=0x7fa4ac00bd40 comp rx=0 tx=0).stop 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.601+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 shutdown_connections 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.601+0000 7fa4c42fa700 1 --2- 172.21.15.142:0/762786566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4bc1008a0 0x7fa4bc195900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.601+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 >> 172.21.15.142:0/762786566 conn(0x7fa4bc0fbe60 msgr2=0x7fa4bc0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.601+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 shutdown_connections 2024-06-15T00:59:33.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:33.601+0000 7fa4c42fa700 1 -- 172.21.15.142:0/762786566 wait complete. 2024-06-15T00:59:33.602 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:34.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:34 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/762786566' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:34.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:34.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:34 smithi137 ceph-mon[29354]: mgrmap e16: smithi137.nbzbyc(active, since 1.00562s) 2024-06-15T00:59:34.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:35.192 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:35.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:35.660 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:35 smithi137 ceph-mon[29354]: [15/Jun/2024:00:59:34] ENGINE Bus STARTING 2024-06-15T00:59:35.660 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:35 smithi137 ceph-mon[29354]: [15/Jun/2024:00:59:34] ENGINE Serving on http://172.21.15.137:8765 2024-06-15T00:59:35.660 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:35.970 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:36.358 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-06-15T00:59:36.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:36 smithi137 ceph-mon[29354]: [15/Jun/2024:00:59:34] ENGINE Serving on https://172.21.15.137:7150 2024-06-15T00:59:36.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:36 smithi137 ceph-mon[29354]: [15/Jun/2024:00:59:34] ENGINE Bus STARTED 2024-06-15T00:59:36.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:36 smithi137 ceph-mon[29354]: mgrmap e17: smithi137.nbzbyc(active, since 2s) 2024-06-15T00:59:38.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 -- 172.21.15.142:0/1132590012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 msgr2=0x7fae840ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:38.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/1132590012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae840ffbb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae80009910 tx=0x7fae80009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:38.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 -- 172.21.15.142:0/1132590012 shutdown_connections 2024-06-15T00:59:38.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/1132590012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae840ffbb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 -- 172.21.15.142:0/1132590012 >> 172.21.15.142:0/1132590012 conn(0x7fae840fad10 msgr2=0x7fae840fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 -- 172.21.15.142:0/1132590012 shutdown_connections 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.434+0000 7fae8bfb4700 1 -- 172.21.15.142:0/1132590012 wait complete. 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae8bfb4700 1 Processor -- start 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae8bfb4700 1 -- start start 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae8bfb4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae8bfb4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84195e30 con 0x7fae840ff790 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae89d50700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae89d50700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:57058/0 (socket says 172.21.15.142:57058) 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae89d50700 1 -- 172.21.15.142:0/2430282879 learned_addr learned my addr 172.21.15.142:0/2430282879 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:38.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae89d50700 1 -- 172.21.15.142:0/2430282879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae800095f0 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae89d50700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae80006010 tx=0x7fae800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.435+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae8001d070 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae84196030 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fae8000bdf0 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae80022e50 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae841964d0 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fae8000b960 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.436+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae84105490 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.437+0000 7fae7affd700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 0x7fae7003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.437+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fae8004c640 con 0x7fae840ff790 2024-06-15T00:59:38.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.437+0000 7fae8954f700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 0x7fae7003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:38.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.437+0000 7fae8954f700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 0x7fae7003a920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:38.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.441+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae8002acf0 con 0x7fae840ff790 2024-06-15T00:59:38.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.708+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fae841992b0 con 0x7fae840ff790 2024-06-15T00:59:38.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.709+0000 7fae7affd700 1 -- 172.21.15.142:0/2430282879 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fae80027020 con 0x7fae840ff790 2024-06-15T00:59:38.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:38.710 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 msgr2=0x7fae7003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 0x7fae7003a920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 msgr2=0x7fae841958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae80006010 tx=0x7fae800057b0 comp rx=0 tx=0).stop 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 shutdown_connections 2024-06-15T00:59:38.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fae70038460 0x7fae7003a920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:38.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 --2- 172.21.15.142:0/2430282879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae840ff790 0x7fae841958f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:38.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 >> 172.21.15.142:0/2430282879 conn(0x7fae840fad10 msgr2=0x7fae840fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:38.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.711+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 shutdown_connections 2024-06-15T00:59:38.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:38.712+0000 7fae8bfb4700 1 -- 172.21.15.142:0/2430282879 wait complete. 2024-06-15T00:59:38.713 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-06-15T00:59:39.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2430282879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:39.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:39.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:39.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:39.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:39.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-06-15T00:59:39.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:39.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T00:59:40.388 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:40.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:40.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:40.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:40.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T00:59:40.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-06-15T00:59:40.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:41.258 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:41.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:41 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T00:59:41.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:41 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T00:59:41.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:41 smithi137 ceph-mon[29354]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-06-15T00:59:43.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 -- 172.21.15.142:0/2287134156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 msgr2=0x7f5218105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:43.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 --2- 172.21.15.142:0/2287134156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f5218105b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f520c009910 tx=0x7f520c009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:43.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 -- 172.21.15.142:0/2287134156 shutdown_connections 2024-06-15T00:59:43.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 --2- 172.21.15.142:0/2287134156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f5218105b10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:43.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 -- 172.21.15.142:0/2287134156 >> 172.21.15.142:0/2287134156 conn(0x7f52180fb240 msgr2=0x7f52180fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:43.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 -- 172.21.15.142:0/2287134156 shutdown_connections 2024-06-15T00:59:43.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 -- 172.21.15.142:0/2287134156 wait complete. 2024-06-15T00:59:43.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.067+0000 7f521dab4700 1 Processor -- start 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f521dab4700 1 -- start start 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f521dab4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f521dab4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521819bf00 con 0x7f5218105730 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f52177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f52177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:44234/0 (socket says 172.21.15.142:44234) 2024-06-15T00:59:43.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.068+0000 7f52177fe700 1 -- 172.21.15.142:0/3775190853 learned_addr learned my addr 172.21.15.142:0/3775190853 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f52177fe700 1 -- 172.21.15.142:0/3775190853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c0095f0 con 0x7f5218105730 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f52177fe700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f520c004f40 tx=0x7f520c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f520c01d070 con 0x7f5218105730 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f520c009e60 con 0x7f5218105730 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521819c160 con 0x7f5218105730 2024-06-15T00:59:43.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5218195a40 con 0x7f5218105730 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.069+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f520c022ce0 con 0x7f5218105730 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.070+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f520c022470 con 0x7f5218105730 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.070+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52180623c0 con 0x7f5218105730 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.070+0000 7f5214ff9700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 0x7f520003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.070+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f520c04c420 con 0x7f5218105730 2024-06-15T00:59:43.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.070+0000 7f5216ffd700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 0x7f520003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:43.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.071+0000 7f5216ffd700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 0x7f520003a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f5208006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:43.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.075+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f520c02ba00 con 0x7f5218105730 2024-06-15T00:59:43.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.349+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5218196f50 con 0x7f5218105730 2024-06-15T00:59:43.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.349+0000 7f5214ff9700 1 -- 172.21.15.142:0/3775190853 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f520c027070 con 0x7f5218105730 2024-06-15T00:59:43.352 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:43.352 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:43.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 msgr2=0x7f520003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:43.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 0x7f520003a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f5208006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:43.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 msgr2=0x7f521819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:43.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f520c004f40 tx=0x7f520c005020 comp rx=0 tx=0).stop 2024-06-15T00:59:43.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 shutdown_connections 2024-06-15T00:59:43.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f52000384b0 0x7f520003a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:43.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 --2- 172.21.15.142:0/3775190853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5218105730 0x7f521819b9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:43.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 >> 172.21.15.142:0/3775190853 conn(0x7f52180fb240 msgr2=0x7f5218107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:43.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 shutdown_connections 2024-06-15T00:59:43.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:43.352+0000 7f521dab4700 1 -- 172.21.15.142:0/3775190853 wait complete. 2024-06-15T00:59:43.356 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:43.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:43 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/3775190853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:44.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:44.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:44 smithi137 ceph-mon[29354]: Deploying daemon crash.smithi142 on smithi142 2024-06-15T00:59:45.003 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:45.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:45.240 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:47.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 -- 172.21.15.142:0/85635792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 msgr2=0x7fb200105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:47.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 --2- 172.21.15.142:0/85635792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200105b10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009910 tx=0x7fb1f0009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 -- 172.21.15.142:0/85635792 shutdown_connections 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 --2- 172.21.15.142:0/85635792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200105b10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 -- 172.21.15.142:0/85635792 >> 172.21.15.142:0/85635792 conn(0x7fb2000fb240 msgr2=0x7fb2000fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 -- 172.21.15.142:0/85635792 shutdown_connections 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 -- 172.21.15.142:0/85635792 wait complete. 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.142+0000 7fb2056a3700 1 Processor -- start 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb2056a3700 1 -- start start 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb2056a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb2056a3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200196ff0 con 0x7fb200105730 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb1feffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb1feffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:44284/0 (socket says 172.21.15.142:44284) 2024-06-15T00:59:47.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb1feffd700 1 -- 172.21.15.142:0/821879927 learned_addr learned my addr 172.21.15.142:0/821879927 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.143+0000 7fb1feffd700 1 -- 172.21.15.142:0/821879927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00095f0 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb1feffd700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0004f40 tx=0x7fb1f0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f001d070 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb200197250 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb1f0005dc0 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2001976d0 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.144+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f0022d70 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.145+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2001039e0 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.145+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fb1f000b960 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.145+0000 7fb1e7fff700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 0x7fb1e803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.145+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb1f004c630 con 0x7fb200105730 2024-06-15T00:59:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.145+0000 7fb1fe7fc700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 0x7fb1e803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:47.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.146+0000 7fb1fe7fc700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 0x7fb1e803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006fd0 tx=0x7fb1f4006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:47.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.150+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb1f002bba0 con 0x7fb200105730 2024-06-15T00:59:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.435+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb200068f20 con 0x7fb200105730 2024-06-15T00:59:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.435+0000 7fb1e7fff700 1 -- 172.21.15.142:0/821879927 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb1f0027070 con 0x7fb200105730 2024-06-15T00:59:47.436 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:47.436 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 msgr2=0x7fb1e803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 0x7fb1e803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006fd0 tx=0x7fb1f4006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 msgr2=0x7fb200196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0004f40 tx=0x7fb1f0005020 comp rx=0 tx=0).stop 2024-06-15T00:59:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 shutdown_connections 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb1e80384b0 0x7fb1e803a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 --2- 172.21.15.142:0/821879927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb200105730 0x7fb200196ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 >> 172.21.15.142:0/821879927 conn(0x7fb2000fb240 msgr2=0x7fb200107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 shutdown_connections 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:47.438+0000 7fb2056a3700 1 -- 172.21.15.142:0/821879927 wait complete. 2024-06-15T00:59:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: Deploying daemon node-exporter.smithi142 on smithi142 2024-06-15T00:59:47.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:47 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/821879927' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:49.009 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:49.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:49.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:49.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T00:59:49.844 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:52.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2121067277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 msgr2=0x7f1b68105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:52.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2121067277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68105b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009910 tx=0x7f1b58009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:52.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2121067277 shutdown_connections 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2121067277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68105b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2121067277 >> 172.21.15.142:0/2121067277 conn(0x7f1b680fb250 msgr2=0x7f1b680fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2121067277 shutdown_connections 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2121067277 wait complete. 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.235+0000 7f1b6dce1700 1 Processor -- start 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b6dce1700 1 -- start start 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b6dce1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b6dce1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68196210 con 0x7f1b68105740 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:42058/0 (socket says 172.21.15.142:42058) 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b677fe700 1 -- 172.21.15.142:0/2030852818 learned_addr learned my addr 172.21.15.142:0/2030852818 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b677fe700 1 -- 172.21.15.142:0/2030852818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580095f0 con 0x7f1b68105740 2024-06-15T00:59:52.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b677fe700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b58004f40 tx=0x7f1b58005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:52.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.236+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b5801d070 con 0x7f1b68105740 2024-06-15T00:59:52.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1b5800bdf0 con 0x7f1b68105740 2024-06-15T00:59:52.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b58022dc0 con 0x7f1b68105740 2024-06-15T00:59:52.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b68196470 con 0x7f1b68105740 2024-06-15T00:59:52.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b681968f0 con 0x7f1b68105740 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b680623c0 con 0x7f1b68105740 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.237+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f1b5800b960 con 0x7f1b68105740 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.238+0000 7f1b64ff9700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 0x7f1b5003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.238+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1b5804c5a0 con 0x7f1b68105740 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.238+0000 7f1b66ffd700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 0x7f1b5003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:52.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.238+0000 7f1b66ffd700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 0x7f1b5003a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006fd0 tx=0x7f1b5c006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:52.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.243+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1b58022470 con 0x7f1b68105740 2024-06-15T00:59:52.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.524+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1b68199630 con 0x7f1b68105740 2024-06-15T00:59:52.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.524+0000 7f1b64ff9700 1 -- 172.21.15.142:0/2030852818 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f1b58027030 con 0x7f1b68105740 2024-06-15T00:59:52.525 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:52.525 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 msgr2=0x7f1b5003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 0x7f1b5003a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006fd0 tx=0x7f1b5c006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 msgr2=0x7f1b68195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b58004f40 tx=0x7f1b58005020 comp rx=0 tx=0).stop 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 shutdown_connections 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f1b500384b0 0x7f1b5003a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 --2- 172.21.15.142:0/2030852818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b68105740 0x7f1b68195cd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 >> 172.21.15.142:0/2030852818 conn(0x7f1b680fb250 msgr2=0x7f1b68068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:52.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 shutdown_connections 2024-06-15T00:59:52.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:52.526+0000 7f1b6dce1700 1 -- 172.21.15.142:0/2030852818 wait complete. 2024-06-15T00:59:52.528 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:52.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:52.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:52.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:52 smithi137 ceph-mon[29354]: Deploying daemon mgr.smithi142.dllryb on smithi142 2024-06-15T00:59:53.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:53 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2030852818' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:53.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:53 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:54.541 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T00:59:54.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T00:59:54.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:54 smithi137 ceph-mon[29354]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:55.001 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T00:59:56.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:56 smithi137 ceph-mon[29354]: Deploying daemon mon.smithi142 on smithi142 2024-06-15T00:59:58.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 -- 172.21.15.142:0/2871066327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 msgr2=0x7f44ec101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:58.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2871066327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec101e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009910 tx=0x7f44e8009c20 comp rx=0 tx=0).stop 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 -- 172.21.15.142:0/2871066327 shutdown_connections 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2871066327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec101e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 -- 172.21.15.142:0/2871066327 >> 172.21.15.142:0/2871066327 conn(0x7f44ec0fb220 msgr2=0x7f44ec0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 -- 172.21.15.142:0/2871066327 shutdown_connections 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.846+0000 7f44f3eed700 1 -- 172.21.15.142:0/2871066327 wait complete. 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.847+0000 7f44f3eed700 1 Processor -- start 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.847+0000 7f44f3eed700 1 -- start start 2024-06-15T00:59:58.629 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:57:09.594792Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.847+0000 7f44f3eed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.847+0000 7f44f3eed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec19bf70 con 0x7f44ec101a30 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f1c89700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f1c89700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.142:42104/0 (socket says 172.21.15.142:42104) 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f1c89700 1 -- 172.21.15.142:0/2123607412 learned_addr learned my addr 172.21.15.142:0/2123607412 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f1c89700 1 -- 172.21.15.142:0/2123607412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80095f0 con 0x7f44ec101a30 2024-06-15T00:59:58.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f1c89700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44e8004f40 tx=0x7f44e8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44e801d070 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f44e8009e60 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.848+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec19c170 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.849+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44e8022ce0 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.849+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec19c590 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.850+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f44e800f460 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.850+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec0623c0 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.850+0000 7f44e2ffd700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 0x7f44d803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.850+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f44e804c4a0 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.850+0000 7f44f1488700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 0x7f44d803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.851+0000 7f44f1488700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 0x7f44d803a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44dc006fd0 tx=0x7f44dc006e40 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:57.855+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f44e8020940 con 0x7f44ec101a30 2024-06-15T00:59:58.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.151+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f44ec196fc0 con 0x7f44ec101a30 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.152+0000 7f44e2ffd700 1 -- 172.21.15.142:0/2123607412 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f44e8027020 con 0x7f44ec101a30 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 msgr2=0x7f44d803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 0x7f44d803a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44dc006fd0 tx=0x7f44dc006e40 comp rx=0 tx=0).stop 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 msgr2=0x7f44ec19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44e8004f40 tx=0x7f44e8005020 comp rx=0 tx=0).stop 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 shutdown_connections 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f44d80384b0 0x7f44d803a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 --2- 172.21.15.142:0/2123607412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec101a30 0x7f44ec19ba30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 >> 172.21.15.142:0/2123607412 conn(0x7f44ec0fb220 msgr2=0x7f44ec105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 shutdown_connections 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T00:59:58.154+0000 7f44f3eed700 1 -- 172.21.15.142:0/2123607412 wait complete. 2024-06-15T00:59:58.632 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-06-15T00:59:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:58 smithi137 ceph-mon[29354]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:58 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:58.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 00:59:58 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2123607412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:59.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:58 smithi142 systemd[1]: Starting Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T00:59:59.660 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 podman[37351]: 2024-06-15 00:59:59.472671716 +0000 UTC m=+0.018038170 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3 2024-06-15T00:59:59.660 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 podman[37351]: 2024-06-15 00:59:59.597710222 +0000 UTC m=+0.143076666 container create 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-06-15T00:59:59.918 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 podman[37351]: 2024-06-15 00:59:59.826641293 +0000 UTC m=+0.372007737 container init 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_BRANCH=HEAD) 2024-06-15T00:59:59.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 podman[37351]: 2024-06-15 00:59:59.830266545 +0000 UTC m=+0.375632994 container start 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-06-15T00:59:59.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: set uid:gid to 167:167 (ceph:ceph) 2024-06-15T00:59:59.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-06-15T00:59:59.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: pidfile_write: ignore empty --pid-file 2024-06-15T00:59:59.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: load: jerasure load: lrc 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: RocksDB version: 7.9.2 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Git sha 0 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Compile date 2023-12-11 22:07:34 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: DB SUMMARY 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: DB Session ID: 6J00DSQR0KYWKPG4324A 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: CURRENT file: CURRENT 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: IDENTITY file: IDENTITY 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi142/store.db dir, Total Num: 0, files: 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi142/store.db: 000004.log size: 511 ; 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.error_if_exists: 0 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.create_if_missing: 0 2024-06-15T00:59:59.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.paranoid_checks: 1 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.env: 0x55858ecc0720 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.fs: PosixFileSystem 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.info_log: 0x5585921b32e0 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_file_opening_threads: 16 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.statistics: (nil) 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.use_fsync: 0 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_log_file_size: 0 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-15T00:59:59.951 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.keep_log_file_num: 1000 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.recycle_log_file_num: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_fallocate: 1 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_mmap_reads: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_mmap_writes: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.use_direct_reads: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.create_missing_column_families: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.db_log_dir: 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.wal_dir: 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-15T00:59:59.952 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.advise_random_on_open: 1 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.db_write_buffer_size: 0 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.write_buffer_manager: 0x558591424140 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-15T00:59:59.953 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.rate_limiter: (nil) 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.wal_recovery_mode: 2 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enable_thread_tracking: 0 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enable_pipelined_write: 0 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.unordered_write: 0 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.row_cache: None 2024-06-15T00:59:59.954 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.wal_filter: None 2024-06-15T00:59:59.955 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-15T00:59:59.956 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_ingest_behind: 0 2024-06-15T00:59:59.956 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.two_write_queues: 0 2024-06-15T00:59:59.956 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.manual_wal_flush: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.wal_compression: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.atomic_flush: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.log_readahead_size: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.best_efforts_recovery: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.allow_data_in_errors: 0 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.db_host_id: __hostname__ 2024-06-15T00:59:59.957 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_background_jobs: 2 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_background_compactions: -1 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_subcompactions: 1 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_total_wal_size: 0 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-15T00:59:59.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_open_files: -1 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bytes_per_sync: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_readahead_size: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_background_flushes: -1 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Compression algorithms supported: 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kZSTD supported: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kXpressCompression supported: 0 2024-06-15T00:59:59.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kLZ4HCCompression supported: 1 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kZlibCompression supported: 1 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kSnappyCompression supported: 1 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kLZ4Compression supported: 1 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: kBZip2Compression supported: 0 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000005 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.merge_operator: 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_filter: None 2024-06-15T00:59:59.960 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_filter_factory: None 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.sst_partitioner_factory: None 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5585921b33e0) 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_type: 0 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_index_type: 0 2024-06-15T00:59:59.961 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_shortening: 1 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: checksum: 4 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: no_block_cache: 0 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache: 0x5585914c5350 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_options: 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: capacity : 536870912 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_shard_bits : 4 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: strict_capacity_limit : 0 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_compressed: (nil) 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: persistent_cache: (nil) 2024-06-15T00:59:59.962 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size: 4096 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size_deviation: 10 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_restart_interval: 16 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_block_restart_interval: 1 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_block_size: 4096 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: partition_filters: 0 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: use_delta_encoding: 1 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: filter_policy: bloomfilter 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: whole_key_filtering: 1 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: verify_compression: 0 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: format_version: 5 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_index_compression: 1 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_align: 0 2024-06-15T00:59:59.963 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_auto_readahead_size: 262144 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: prepopulate_block_cache: 0 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: initial_auto_readahead_size: 8192 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.write_buffer_size: 33554432 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_write_buffer_number: 2 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression: NoCompression 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression: Disabled 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.prefix_extractor: nullptr 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.num_levels: 7 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-15T00:59:59.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-15T00:59:59.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.level: 32767 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.strategy: 0 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.enabled: false 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-15T00:59:59.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.target_file_size_base: 67108864 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-15T00:59:59.967 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.arena_block_size: 1048576 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.disable_auto_compactions: 0 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-15T00:59:59.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.table_properties_collectors: 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.inplace_update_support: 0 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-15T00:59:59.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.bloom_locality: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.max_successive_merges: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.paranoid_file_checks: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.force_consistency_checks: 1 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.report_bg_io_stats: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.ttl: 2592000 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enable_blob_files: false 2024-06-15T00:59:59.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.min_blob_size: 0 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_file_size: 268435456 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.blob_file_starting_level: 0 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eda2365a-5a11-455e-8015-8e14ca1e9b1f 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413199860856, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-15T00:59:59.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413199861304, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718413199, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eda2365a-5a11-455e-8015-8e14ca1e9b1f", "db_session_id": "6J00DSQR0KYWKPG4324A", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413199861393, "job": 1, "event": "recovery_finished"} 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi142/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558591562000 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: DB pointer 0x55859154e000 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** DB Stats ** 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T00:59:59.972 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T00:59:59.973 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T00:59:59.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache BinnedLRUCache@0x5585914c5350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142 does not exist in monmap, will attempt to join an existing cluster 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: starting mon.smithi142 rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi142 fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(???) e0 preinit fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).mds e1 new map 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).mds e1 print_map 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: e1 2024-06-15T00:59:59.975 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: legacy client fscid: -1 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: No filesystems configured 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:59:59.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: Deploying daemon mon.smithi142 on smithi142 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/2123607412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-06-15T00:59:59.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: mon.smithi142@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expand map: {default=false} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta from 'false' to 'false' 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expanded map: {default=false} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expand map: {default=info} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta from 'info' to 'info' 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expanded map: {default=info} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expand map: {default=daemon} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta from 'daemon' to 'daemon' 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expanded map: {default=daemon} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expand map: {default=debug} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta from 'debug' to 'debug' 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 ceph-mon[37379]: expand_channel_meta expanded map: {default=debug} 2024-06-15T00:59:59.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 bash[37351]: 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 2024-06-15T01:00:00.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 00:59:59 smithi142 systemd[1]: Started Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:00:00.384 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-06-15T01:00:00.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-15T01:00:00.603 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 -- 172.21.15.142:0/3112533836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd00035c0 msgr2=0x7f7cd0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 --2- 172.21.15.142:0/3112533836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd00035c0 0x7f7cd0005a50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40ff790 tx=0x7f7ce000feb0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 -- 172.21.15.142:0/3112533836 shutdown_connections 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 --2- 172.21.15.142:0/3112533836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd00035c0 0x7f7cd0005a50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 -- 172.21.15.142:0/3112533836 >> 172.21.15.142:0/3112533836 conn(0x7f7ce40fb390 msgr2=0x7f7ce40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 -- 172.21.15.142:0/3112533836 shutdown_connections 2024-06-15T01:00:05.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.095+0000 7f7cebe38700 1 -- 172.21.15.142:0/3112533836 wait complete. 2024-06-15T01:00:05.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 Processor -- start 2024-06-15T01:00:05.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 -- start start 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 0x7f7ce41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce419fc50 con 0x7f7ce4197120 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.096+0000 7f7cebe38700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce419fd90 con 0x7f7cd00035c0 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54762/0 (socket says 172.21.15.142:54762) 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 -- 172.21.15.142:0/2085853923 learned_addr learned my addr 172.21.15.142:0/2085853923 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce93d3700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 0x7f7ce41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 -- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 msgr2=0x7f7ce4196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 -- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 msgr2=0x7f7ce4196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce9bd4700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T01:00:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce93d3700 1 -- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 msgr2=0x7f7ce4196be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce93d3700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce93d3700 1 -- 172.21.15.142:0/2085853923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce000f900 con 0x7f7ce4197120 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7ce93d3700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 0x7f7ce41975a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b4d0 tx=0x7f7cd400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.097+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd4011800 con 0x7f7ce4197120 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.098+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce419fff0 con 0x7f7ce4197120 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.098+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7cd4011df0 con 0x7f7ce4197120 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.098+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce41a0510 con 0x7f7ce4197120 2024-06-15T01:00:05.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.098+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd400f2e0 con 0x7f7ce4197120 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.099+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce40623c0 con 0x7f7ce4197120 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.099+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f7cd4011960 con 0x7f7ce4197120 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.099+0000 7f7cdaffd700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 0x7f7cc806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.100+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7cd408a2a0 con 0x7f7ce4197120 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.100+0000 7f7ce9bd4700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 0x7f7cc806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:05.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.100+0000 7f7ce9bd4700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 0x7f7cc806ea20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005b40 tx=0x7f7ce0005a90 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:05.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.104+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7cd4055b00 con 0x7f7ce4197120 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: mon.smithi137 calling monitor election 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: mon.smithi142 calling monitor election 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: fsmap 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: osdmap e5: 0 total, 0 up, 0 in 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: mgrmap e17: smithi137.nbzbyc(active, since 31s) 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:00:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: overall HEALTH_OK 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.232 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:00:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: mon.smithi137 calling monitor election 2024-06-15T01:00:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: mon.smithi142 calling monitor election 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:00:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: fsmap 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: osdmap e5: 0 total, 0 up, 0 in 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: mgrmap e17: smithi137.nbzbyc(active, since 31s) 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: overall HEALTH_OK 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/348848512' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.324 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.325 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.325 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.325 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:05.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.383+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7ce4069370 con 0x7f7ce4197120 2024-06-15T01:00:05.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.384+0000 7f7cdaffd700 1 -- 172.21.15.142:0/2085853923 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f7cd4015420 con 0x7f7ce4197120 2024-06-15T01:00:05.385 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-06-15T01:00:05.385 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","modified":"2024-06-15T00:59:59.894273Z","created":"2024-06-15T00:57:09.594792Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi137","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 msgr2=0x7f7cc806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 0x7f7cc806ea20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005b40 tx=0x7f7ce0005a90 comp rx=0 tx=0).stop 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 msgr2=0x7f7ce41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 0x7f7ce41975a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b4d0 tx=0x7f7cd400b890 comp rx=0 tx=0).stop 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 shutdown_connections 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7cc806c560 0x7f7cc806ea20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd00035c0 0x7f7ce4196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 --2- 172.21.15.142:0/2085853923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ce4197120 0x7f7ce41975a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:05.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 >> 172.21.15.142:0/2085853923 conn(0x7f7ce40fb390 msgr2=0x7f7ce4101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:05.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 shutdown_connections 2024-06-15T01:00:05.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:00:05.386+0000 7f7cebe38700 1 -- 172.21.15.142:0/2085853923 wait complete. 2024-06-15T01:00:05.388 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-06-15T01:00:06.043 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-15T01:00:06.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph config generate-minimal-conf 2024-06-15T01:00:06.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: mgrmap e18: smithi137.nbzbyc(active, since 31s), standbys: smithi142.dllryb 2024-06-15T01:00:06.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:00:06.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:06.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:06.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2085853923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T01:00:06.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:05 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:06.259 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: mgrmap e18: smithi137.nbzbyc(active, since 31s), standbys: smithi142.dllryb 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/2085853923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-15T01:00:06.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:05 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:00:07.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 -- 172.21.15.137:0/495357877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 msgr2=0x7fd3b0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:07.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/495357877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b0101db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009910 tx=0x7fd3ac009c20 comp rx=0 tx=0).stop 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 -- 172.21.15.137:0/495357877 shutdown_connections 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/495357877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b0101db0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 -- 172.21.15.137:0/495357877 >> 172.21.15.137:0/495357877 conn(0x7fd3b00fb220 msgr2=0x7fd3b00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 -- 172.21.15.137:0/495357877 shutdown_connections 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.792+0000 7fd3b80c5700 1 -- 172.21.15.137:0/495357877 wait complete. 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 Processor -- start 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 -- start start 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b01a1f80 con 0x7fd3b01019d0 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b80c5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b01a20c0 con 0x7fd3b00ffb70 2024-06-15T01:00:07.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33010/0 (socket says 172.21.15.137:33010) 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 -- 172.21.15.137:0/2401312770 learned_addr learned my addr 172.21.15.137:0/2401312770 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5660700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5660700 1 -- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 msgr2=0x7fd3b00ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5660700 1 -- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 msgr2=0x7fd3b00ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5660700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5660700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 -- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 msgr2=0x7fd3b00ffff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.793+0000 7fd3b5e61700 1 -- 172.21.15.137:0/2401312770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0095f0 con 0x7fd3b01019d0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3b5e61700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac006010 tx=0x7fd3ac004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3ac01d070 con 0x7fd3b01019d0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd3ac005dc0 con 0x7fd3b01019d0 2024-06-15T01:00:07.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b01a2260 con 0x7fd3b01019d0 2024-06-15T01:00:07.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3ac00f460 con 0x7fd3b01019d0 2024-06-15T01:00:07.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.794+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b01a2700 con 0x7fd3b01019d0 2024-06-15T01:00:07.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.795+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b01080c0 con 0x7fd3b01019d0 2024-06-15T01:00:07.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.795+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fd3ac00bb30 con 0x7fd3b01019d0 2024-06-15T01:00:07.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.796+0000 7fd3a6ffd700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 0x7fd39c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:07.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.796+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd3ac08be10 con 0x7fd3b01019d0 2024-06-15T01:00:07.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.796+0000 7fd3b5660700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 0x7fd39c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:07.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.796+0000 7fd3b5660700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 0x7fd39c06ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009850 tx=0x7fd3a0008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:07.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:07.800+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3ac057780 con 0x7fd3b01019d0 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.001+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd3b0107aa0 con 0x7fd3b01019d0 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.001+0000 7fd3a6ffd700 1 -- 172.21.15.137:0/2401312770 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fd3ac08b5d0 con 0x7fd3b01019d0 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stdout:# minimal ceph.conf for 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:00:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: mon_host = [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-06-15T01:00:08.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.003+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 msgr2=0x7fd39c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:08.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.003+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 0x7fd39c06ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009850 tx=0x7fd3a0008040 comp rx=0 tx=0).stop 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.003+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 msgr2=0x7fd3b00ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.003+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac006010 tx=0x7fd3ac004d60 comp rx=0 tx=0).stop 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 shutdown_connections 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd39c06c5b0 0x7fd39c06ea70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3b01019d0 0x7fd3b00ff630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 --2- 172.21.15.137:0/2401312770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3b00ffb70 0x7fd3b00ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 >> 172.21.15.137:0/2401312770 conn(0x7fd3b00fb220 msgr2=0x7fd3b0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 shutdown_connections 2024-06-15T01:00:08.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:08.004+0000 7fd3b80c5700 1 -- 172.21.15.137:0/2401312770 wait complete. 2024-06-15T01:00:08.599 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:08.600 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.600 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.600 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:08.600 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:00:08.600 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2401312770' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.601 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.626 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2401312770' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:08.649 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-15T01:00:08.649 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:00:08.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-06-15T01:00:08.678 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:00:08.678 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:00:08.744 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T01:00:08.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-06-15T01:00:08.773 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T01:00:08.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:00:08.839 INFO:tasks.cephadm:Deploying OSDs... 2024-06-15T01:00:08.839 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:00:08.839 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-06-15T01:00:08.855 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-15T01:00:08.855 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-06-15T01:00:08.910 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-15T01:00:08.910 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:08.910 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-15T01:00:08.910 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:08.910 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:08.911 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-06-15 00:59:24.772203412 +0000 2024-06-15T01:00:08.911 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-06-15 00:54:02.382264262 +0000 2024-06-15T01:00:08.911 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-06-15 00:54:02.382264262 +0000 2024-06-15T01:00:08.911 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-06-15 00:54:02.382264262 +0000 2024-06-15T01:00:08.911 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-15T01:00:08.975 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-06-15T01:00:08.975 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-06-15T01:00:08.975 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000266785 s, 1.9 MB/s 2024-06-15T01:00:08.976 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-15T01:00:09.033 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:09.089 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-06-15 00:59:24.772203412 +0000 2024-06-15T01:00:09.090 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-06-15 00:54:02.743257723 +0000 2024-06-15T01:00:09.090 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-06-15 00:54:02.743257723 +0000 2024-06-15T01:00:09.090 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-06-15 00:54:02.743257723 +0000 2024-06-15T01:00:09.090 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-15T01:00:09.153 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-06-15T01:00:09.153 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-06-15T01:00:09.153 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00021513 s, 2.4 MB/s 2024-06-15T01:00:09.154 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-15T01:00:09.210 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-06-15 00:59:24.772203412 +0000 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-06-15 00:54:03.092251402 +0000 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-06-15 00:54:03.092251402 +0000 2024-06-15T01:00:09.266 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-06-15 00:54:03.092251402 +0000 2024-06-15T01:00:09.267 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-15T01:00:09.330 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-06-15T01:00:09.331 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-06-15T01:00:09.331 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000165086 s, 3.1 MB/s 2024-06-15T01:00:09.332 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-15T01:00:09.389 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-06-15T01:00:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-06-15 00:59:24.773203395 +0000 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-06-15 00:54:03.420245461 +0000 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-06-15 00:54:03.420245461 +0000 2024-06-15T01:00:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-06-15 00:54:03.420245461 +0000 2024-06-15T01:00:09.445 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-15T01:00:09.509 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-06-15T01:00:09.509 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-06-15T01:00:09.509 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000183907 s, 2.8 MB/s 2024-06-15T01:00:09.511 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-15T01:00:09.541 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:00:09.541 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Reconfiguring mon.smithi137 (unknown last config time)... 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:09.542 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:09 smithi137 ceph-mon[29354]: Reconfiguring daemon mon.smithi137 on smithi137 2024-06-15T01:00:09.558 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T01:00:09.558 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-06-15T01:00:09.577 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-15T01:00:09.577 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-06-15 00:59:58.933977773 +0000 2024-06-15T01:00:09.636 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-06-15 00:54:02.384891630 +0000 2024-06-15T01:00:09.637 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-06-15 00:54:02.384891630 +0000 2024-06-15T01:00:09.637 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-06-15 00:54:02.384891630 +0000 2024-06-15T01:00:09.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Reconfiguring mon.smithi137 (unknown last config time)... 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:00:09.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:09.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:09 smithi142 ceph-mon[37379]: Reconfiguring daemon mon.smithi137 on smithi137 2024-06-15T01:00:09.704 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-06-15T01:00:09.704 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-06-15T01:00:09.704 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000209333 s, 2.4 MB/s 2024-06-15T01:00:09.706 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-15T01:00:09.766 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-06-15 00:59:58.934977756 +0000 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-06-15 00:54:02.714885414 +0000 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-06-15 00:54:02.714885414 +0000 2024-06-15T01:00:09.825 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-06-15 00:54:02.714885414 +0000 2024-06-15T01:00:09.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-15T01:00:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-06-15T01:00:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-06-15T01:00:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000206847 s, 2.5 MB/s 2024-06-15T01:00:09.896 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-15T01:00:09.953 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-06-15T01:00:10.010 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-15T01:00:10.010 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:10.010 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 709 Links: 1 2024-06-15T01:00:10.010 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:10.011 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:10.011 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-06-15 00:59:58.935977738 +0000 2024-06-15T01:00:10.011 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-06-15 00:54:03.045879180 +0000 2024-06-15T01:00:10.011 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-06-15 00:54:03.045879180 +0000 2024-06-15T01:00:10.011 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-06-15 00:54:03.045879180 +0000 2024-06-15T01:00:10.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-15T01:00:10.076 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-06-15T01:00:10.076 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-06-15T01:00:10.076 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000197826 s, 2.6 MB/s 2024-06-15T01:00:10.077 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-15T01:00:10.134 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-06-15T01:00:10.190 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-15T01:00:10.190 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 728 Links: 1 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-06-15 00:59:58.935977738 +0000 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-06-15 00:54:03.393872626 +0000 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-06-15 00:54:03.393872626 +0000 2024-06-15T01:00:10.191 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-06-15 00:54:03.393872626 +0000 2024-06-15T01:00:10.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-15T01:00:10.255 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-06-15T01:00:10.255 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-06-15T01:00:10.255 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000256904 s, 2.0 MB/s 2024-06-15T01:00:10.257 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-15T01:00:10.314 INFO:tasks.cephadm:Deploying osd.0 on smithi137 with /dev/vg_nvme/lv_4... 2024-06-15T01:00:10.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-15T01:00:10.564 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:10.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:10.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:10.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:00:10.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:00:10.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:10.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:10.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:10.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:10.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:00:10.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:00:10.718 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:11.251 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:00:11.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: Reconfiguring mgr.smithi137.nbzbyc (unknown last config time)... 2024-06-15T01:00:11.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: Reconfiguring daemon mgr.smithi137.nbzbyc on smithi137 2024-06-15T01:00:11.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:11.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:11.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:00:11.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: Reconfiguring mgr.smithi137.nbzbyc (unknown last config time)... 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: Reconfiguring daemon mgr.smithi137.nbzbyc on smithi137 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:00:11.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:12.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:12 smithi137 ceph-mon[29354]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:12.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:12 smithi137 ceph-mon[29354]: Reconfiguring ceph-exporter.smithi137 (monmap changed)... 2024-06-15T01:00:12.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:12 smithi137 ceph-mon[29354]: Reconfiguring daemon ceph-exporter.smithi137 on smithi137 2024-06-15T01:00:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:12 smithi142 ceph-mon[37379]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:12 smithi142 ceph-mon[37379]: Reconfiguring ceph-exporter.smithi137 (monmap changed)... 2024-06-15T01:00:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:12 smithi142 ceph-mon[37379]: Reconfiguring daemon ceph-exporter.smithi137 on smithi137 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113567 s, 923 MB/s 2024-06-15T01:00:14.031 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: Reconfiguring crash.smithi137 (monmap changed)... 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:14.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:14 smithi137 ceph-mon[29354]: Reconfiguring daemon crash.smithi137 on smithi137 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: Reconfiguring crash.smithi137 (monmap changed)... 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:14 smithi142 ceph-mon[37379]: Reconfiguring daemon crash.smithi137 on smithi137 2024-06-15T01:00:14.869 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-06-15T01:00:15.418 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:00:16.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:16 smithi137 ceph-mon[29354]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:16.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:16 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:16.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:16 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:16.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:16 smithi137 ceph-mon[29354]: Reconfiguring alertmanager.smithi137 (dependencies changed)... 2024-06-15T01:00:16.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:16 smithi137 ceph-mon[29354]: Reconfiguring daemon alertmanager.smithi137 on smithi137 2024-06-15T01:00:16.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:16 smithi142 ceph-mon[37379]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:16.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:16 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:16.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:16 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:16.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:16 smithi142 ceph-mon[37379]: Reconfiguring alertmanager.smithi137 (dependencies changed)... 2024-06-15T01:00:16.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:16 smithi142 ceph-mon[37379]: Reconfiguring daemon alertmanager.smithi137 on smithi137 2024-06-15T01:00:17.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 -- 172.21.15.137:0/1503087795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39300ffb10 msgr2=0x7f39300fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 --2- 172.21.15.137:0/1503087795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39300ffb10 0x7f39300fff30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3924009910 tx=0x7f3924009c20 comp rx=0 tx=0).stop 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 -- 172.21.15.137:0/1503087795 shutdown_connections 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 --2- 172.21.15.137:0/1503087795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3930100d10 0x7f3930101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 --2- 172.21.15.137:0/1503087795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39300ffb10 0x7f39300fff30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 -- 172.21.15.137:0/1503087795 >> 172.21.15.137:0/1503087795 conn(0x7f39300fb0b0 msgr2=0x7f39300fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 -- 172.21.15.137:0/1503087795 shutdown_connections 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.566+0000 7f3936e76700 1 -- 172.21.15.137:0/1503087795 wait complete. 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 Processor -- start 2024-06-15T01:00:17.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 -- start start 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300ffb10 0x7f3930195230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930195d90 con 0x7f3930100d10 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3936e76700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930195ed0 con 0x7f39300ffb10 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f392ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f392ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37418/0 (socket says 172.21.15.137:37418) 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f392ffff700 1 -- 172.21.15.137:0/2827674431 learned_addr learned my addr 172.21.15.137:0/2827674431 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.567+0000 7f3934c12700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300ffb10 0x7f3930195230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392ffff700 1 -- 172.21.15.137:0/2827674431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300ffb10 msgr2=0x7f3930195230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392ffff700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300ffb10 0x7f3930195230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:17.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392ffff700 1 -- 172.21.15.137:0/2827674431 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240095f0 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392ffff700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f391c00d8d0 tx=0x7f391c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f391c009940 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f391c010460 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393019a980 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f391c00f5f0 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.568+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393019af50 con 0x7f3930100d10 2024-06-15T01:00:17.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.569+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930108680 con 0x7f3930100d10 2024-06-15T01:00:17.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.569+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f391c0105d0 con 0x7f3930100d10 2024-06-15T01:00:17.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.570+0000 7f392dffb700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 0x7f392006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:17.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.570+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f391c08a630 con 0x7f3930100d10 2024-06-15T01:00:17.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.570+0000 7f3934c12700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 0x7f392006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:17.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.570+0000 7f3934c12700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 0x7f392006e930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3924006010 tx=0x7f392400b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:17.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.574+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f391c059a00 con 0x7f3930100d10 2024-06-15T01:00:17.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:17.777+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3930105660 con 0x7f392006c470 2024-06-15T01:00:18.544 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:18 smithi137 ceph-mon[29354]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:18.544 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:00:18.544 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:00:18.544 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:18.544 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:18.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:18 smithi142 ceph-mon[37379]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:18.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:00:18.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:00:18.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:18.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:19.557 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:19 smithi137 ceph-mon[29354]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:00:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:19 smithi142 ceph-mon[37379]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:00:20.530 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:20 smithi137 ceph-mon[29354]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:20 smithi142 ceph-mon[37379]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:22.614 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:22 smithi137 ceph-mon[29354]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:22.614 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:22 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:22.615 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:22 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:22.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:22 smithi142 ceph-mon[37379]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:22.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:22 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:22.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:22 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:23.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: Reconfiguring grafana.smithi137 (dependencies changed)... 2024-06-15T01:00:23.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: Reconfiguring daemon grafana.smithi137 on smithi137 2024-06-15T01:00:23.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/195286348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69222b-cedc-4834-9d6d-0a1801c6a9bb"}]: dispatch 2024-06-15T01:00:23.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/195286348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a69222b-cedc-4834-9d6d-0a1801c6a9bb"}]': finished 2024-06-15T01:00:23.619 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: osdmap e6: 1 total, 0 up, 1 in 2024-06-15T01:00:23.619 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:23 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: Reconfiguring grafana.smithi137 (dependencies changed)... 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: Reconfiguring daemon grafana.smithi137 on smithi137 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/195286348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69222b-cedc-4834-9d6d-0a1801c6a9bb"}]: dispatch 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/195286348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a69222b-cedc-4834-9d6d-0a1801c6a9bb"}]': finished 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: osdmap e6: 1 total, 0 up, 1 in 2024-06-15T01:00:23.656 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:23 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:24.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:24 smithi137 ceph-mon[29354]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:24.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:24 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/457311084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:00:24.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:24 smithi142 ceph-mon[37379]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:24.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:24 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/457311084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:00:26.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:26 smithi137 ceph-mon[29354]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:26.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:26 smithi142 ceph-mon[37379]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:28.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:28 smithi137 ceph-mon[29354]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:28.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:28 smithi142 ceph-mon[37379]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:30.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:30 smithi137 ceph-mon[29354]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:30.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:30 smithi142 ceph-mon[37379]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:32.621 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:32 smithi137 ceph-mon[29354]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:32.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:32 smithi142 ceph-mon[37379]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:33.861 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:34.706 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-15T01:00:34.707 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:34.707 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:34 smithi137 ceph-mon[29354]: Deploying daemon osd.0 on smithi137 2024-06-15T01:00:35.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-15T01:00:35.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:35.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:34 smithi142 ceph-mon[37379]: Deploying daemon osd.0 on smithi137 2024-06-15T01:00:35.620 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:35 smithi137 ceph-mon[29354]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:36.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:35 smithi142 ceph-mon[37379]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:38.460 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:38 smithi137 ceph-mon[29354]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:38.460 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:38.460 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:38.460 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:00:38.460 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:38.639 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:38 smithi142 ceph-mon[37379]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:38.639 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:38 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:38.639 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:38 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:38.639 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:38 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:00:38.639 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:38 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:39.398 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:39 smithi137 ceph-mon[29354]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-06-15T01:00:39.398 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:39 smithi137 ceph-mon[29354]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-06-15T01:00:39.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:39 smithi142 ceph-mon[37379]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-06-15T01:00:39.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:39 smithi142 ceph-mon[37379]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-06-15T01:00:40.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:40 smithi142 ceph-mon[37379]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:40.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:40.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:40.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:00:40.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:40.646 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:40 smithi137 ceph-mon[29354]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:40.646 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:40.647 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:40.647 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:00:40.647 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: Reconfiguring crash.smithi142 (monmap changed)... 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: Reconfiguring daemon crash.smithi142 on smithi142 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:00:41.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: Reconfiguring crash.smithi142 (monmap changed)... 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: Reconfiguring daemon crash.smithi142 on smithi142 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:00:41.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:42.621 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:42 smithi137 ceph-mon[29354]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:42.621 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:42 smithi137 ceph-mon[29354]: Reconfiguring mgr.smithi142.dllryb (monmap changed)... 2024-06-15T01:00:42.622 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:42 smithi137 ceph-mon[29354]: Reconfiguring daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:00:42.663 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:42 smithi142 ceph-mon[37379]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:42.663 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:42 smithi142 ceph-mon[37379]: Reconfiguring mgr.smithi142.dllryb (monmap changed)... 2024-06-15T01:00:42.663 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:42 smithi142 ceph-mon[37379]: Reconfiguring daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: Reconfiguring mon.smithi142 (monmap changed)... 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: Reconfiguring daemon mon.smithi142 on smithi142 2024-06-15T01:00:43.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:43.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:43 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: Reconfiguring mon.smithi142 (monmap changed)... 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: Reconfiguring daemon mon.smithi142 on smithi142 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:43.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:43 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi137.front.sepia.ceph.com:9093"}]: dispatch 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi137.front.sepia.ceph.com:9093"}]: dispatch 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:00:45.365 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-06-15T01:00:45.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.367 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:45 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:45.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi137.front.sepia.ceph.com:9093"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi137.front.sepia.ceph.com:9093"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:00:45.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:00:45.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-06-15T01:00:45.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-06-15T01:00:45.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:45.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:45 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:46.349 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:46 smithi137 ceph-mon[29354]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:46.349 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:46 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:46.349 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:46 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:46.449 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 0 on host 'smithi137' 2024-06-15T01:00:46.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.447+0000 7f392dffb700 1 -- 172.21.15.137:0/2827674431 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3930105660 con 0x7f392006c470 2024-06-15T01:00:46.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.450+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 msgr2=0x7f392006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:46.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.450+0000 7f3936e76700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 0x7f392006e930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3924006010 tx=0x7f392400b540 comp rx=0 tx=0).stop 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.450+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 msgr2=0x7f3930195770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.450+0000 7f3936e76700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f391c00d8d0 tx=0x7f391c00dc90 comp rx=0 tx=0).stop 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 shutdown_connections 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f392006c470 0x7f392006e930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39300ffb10 0x7f3930195230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 --2- 172.21.15.137:0/2827674431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3930100d10 0x7f3930195770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 >> 172.21.15.137:0/2827674431 conn(0x7f39300fb0b0 msgr2=0x7f3930103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 shutdown_connections 2024-06-15T01:00:46.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:46.451+0000 7f3936e76700 1 -- 172.21.15.137:0/2827674431 wait complete. 2024-06-15T01:00:46.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:46 smithi142 ceph-mon[37379]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:46.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:46 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:46.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:46 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:47.057 DEBUG:teuthology.orchestra.run.smithi137:osd.0> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.0.service 2024-06-15T01:00:47.059 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_3... 2024-06-15T01:00:47.059 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-15T01:00:47.230 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:00:47 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:00:47.182+0000 7fd24e782640 -1 osd.0 0 log_to_monitors true 2024-06-15T01:00:47.655 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:47.655 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:47.655 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:47 smithi137 ceph-mon[29354]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-15T01:00:47.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:47.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:47.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:47 smithi142 ceph-mon[37379]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-15T01:00:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:00:48.730 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:00:48.415+0000 7fd244de8700 -1 osd.0 0 waiting for initial osdmap 2024-06-15T01:00:48.730 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:00:48.419+0000 7fd23e3d8700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: osdmap e7: 1 total, 0 up, 1 in 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:00:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: osdmap e7: 1 total, 0 up, 1 in 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:00:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:00:49.507 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:49 smithi137 ceph-mon[29354]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:00:49.508 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:49 smithi137 ceph-mon[29354]: osdmap e8: 1 total, 0 up, 1 in 2024-06-15T01:00:49.508 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:49.508 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:49.508 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:49 smithi142 ceph-mon[37379]: from='osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:00:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:49 smithi142 ceph-mon[37379]: osdmap e8: 1 total, 0 up, 1 in 2024-06-15T01:00:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:49 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:49 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:49 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:50.431 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:00:50.431 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:00:50.432 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:50.432 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] boot 2024-06-15T01:00:50.432 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: osdmap e9: 1 total, 1 up, 1 in 2024-06-15T01:00:50.432 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: osd.0 [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] boot 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: osdmap e9: 1 total, 1 up, 1 in 2024-06-15T01:00:50.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:00:51.678 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:51 smithi137 ceph-mon[29354]: osdmap e10: 1 total, 1 up, 1 in 2024-06-15T01:00:51.678 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:51.678 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:51.678 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-15T01:00:51.679 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-15T01:00:51.679 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-06-15T01:00:51.679 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-06-15T01:00:51.679 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115026 s, 912 MB/s 2024-06-15T01:00:51.679 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-06-15T01:00:51.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:51 smithi142 ceph-mon[37379]: osdmap e10: 1 total, 1 up, 1 in 2024-06-15T01:00:51.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:51.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:52.317 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-06-15T01:00:52.554 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:52 smithi137 ceph-mon[29354]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:52.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:52 smithi142 ceph-mon[37379]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:53.428 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:00:54.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:54 smithi137 ceph-mon[29354]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:54.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:54 smithi142 ceph-mon[37379]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 -- 172.21.15.137:0/1269209657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c101e00 msgr2=0x7faf3c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 --2- 172.21.15.137:0/1269209657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c101e00 0x7faf3c102260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf30009910 tx=0x7faf30009c20 comp rx=0 tx=0).stop 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 -- 172.21.15.137:0/1269209657 shutdown_connections 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 --2- 172.21.15.137:0/1269209657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c101e00 0x7faf3c102260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 --2- 172.21.15.137:0/1269209657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c100c00 0x7faf3c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 -- 172.21.15.137:0/1269209657 >> 172.21.15.137:0/1269209657 conn(0x7faf3c0fc180 msgr2=0x7faf3c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 -- 172.21.15.137:0/1269209657 shutdown_connections 2024-06-15T01:00:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.978+0000 7faf4091d700 1 -- 172.21.15.137:0/1269209657 wait complete. 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 Processor -- start 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 -- start start 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c101e00 0x7faf3c1967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c196df0 con 0x7faf3c100c00 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf4091d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c196f30 con 0x7faf3c101e00 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:55.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55180/0 (socket says 172.21.15.137:55180) 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 -- 172.21.15.137:0/498340974 learned_addr learned my addr 172.21.15.137:0/498340974 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf39d9b700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c101e00 0x7faf3c1967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 -- 172.21.15.137:0/498340974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c101e00 msgr2=0x7faf3c1967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c101e00 0x7faf3c1967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 -- 172.21.15.137:0/498340974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300095f0 con 0x7faf3c100c00 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.979+0000 7faf3a59c700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00d900 tx=0x7faf2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.980+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2c00b920 con 0x7faf3c100c00 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.980+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7faf2c010460 con 0x7faf3c100c00 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.980+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c19b9e0 con 0x7faf3c100c00 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.980+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2c00f5d0 con 0x7faf3c100c00 2024-06-15T01:00:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.980+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c19bf30 con 0x7faf3c100c00 2024-06-15T01:00:55.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.981+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c109620 con 0x7faf3c100c00 2024-06-15T01:00:55.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.981+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7faf2c0105d0 con 0x7faf3c100c00 2024-06-15T01:00:55.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.982+0000 7faf2b7fe700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 0x7faf2406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:00:55.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.982+0000 7faf39d9b700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 0x7faf2406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:00:55.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.982+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7faf2c08a4a0 con 0x7faf3c100c00 2024-06-15T01:00:55.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.982+0000 7faf39d9b700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 0x7faf2406e920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b5c0 tx=0x7faf30005ee0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:00:55.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:55.986+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faf2c059610 con 0x7faf3c100c00 2024-06-15T01:00:56.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:00:56.191+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faf3c106750 con 0x7faf2406c460 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:00:56.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:56.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:56.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:00:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:00:57.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:57 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi137 2024-06-15T01:00:57.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:57 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi137 to 11840M 2024-06-15T01:00:57.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:57 smithi142 ceph-mon[37379]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:00:57.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:57 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi137 2024-06-15T01:00:57.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:57 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi137 to 11840M 2024-06-15T01:00:57.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:57 smithi137 ceph-mon[29354]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:00:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:58 smithi137 ceph-mon[29354]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:58.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:58 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:58.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:00:58 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:58.732 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:58 smithi142 ceph-mon[37379]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:00:58.732 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:58 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:00:58.732 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:00:58 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:00 smithi137 ceph-mon[29354]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:00 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2082231285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6842e406-422f-463c-94cf-42233a80ccef"}]: dispatch 2024-06-15T01:01:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:00 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2082231285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6842e406-422f-463c-94cf-42233a80ccef"}]': finished 2024-06-15T01:01:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:00 smithi137 ceph-mon[29354]: osdmap e11: 2 total, 1 up, 2 in 2024-06-15T01:01:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:00 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:00.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:00 smithi142 ceph-mon[37379]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:00.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:00 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2082231285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6842e406-422f-463c-94cf-42233a80ccef"}]: dispatch 2024-06-15T01:01:00.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:00 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2082231285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6842e406-422f-463c-94cf-42233a80ccef"}]': finished 2024-06-15T01:01:00.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:00 smithi142 ceph-mon[37379]: osdmap e11: 2 total, 1 up, 2 in 2024-06-15T01:01:00.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:00 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:01.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:01 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/4227481640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:01.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:01.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:01.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:01.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:01.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:01 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/4227481640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:01.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:01.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:01.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:01.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:02.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:02 smithi142 ceph-mon[37379]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:02.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:02 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:02.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:02 smithi137 ceph-mon[29354]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:02.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:02 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:03 smithi142 ceph-mon[37379]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:03.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:03 smithi137 ceph-mon[29354]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:03.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:06.408 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:06 smithi137 ceph-mon[29354]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:06.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:06 smithi142 ceph-mon[37379]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:08.560 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:08 smithi137 ceph-mon[29354]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:08.561 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-15T01:01:08.561 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:08 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:08.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:08 smithi142 ceph-mon[37379]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:08.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-15T01:01:08.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:08 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:09.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:09 smithi142 ceph-mon[37379]: Deploying daemon osd.1 on smithi137 2024-06-15T01:01:09.583 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:09 smithi137 ceph-mon[29354]: Deploying daemon osd.1 on smithi137 2024-06-15T01:01:10.372 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:10 smithi137 ceph-mon[29354]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:10.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:10 smithi142 ceph-mon[37379]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:12.537 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:12 smithi137 ceph-mon[29354]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:12 smithi142 ceph-mon[37379]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:14.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:14 smithi142 ceph-mon[37379]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:14.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:14.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:14.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:14 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:14.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:14 smithi137 ceph-mon[29354]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:14.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:14.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:14.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:14 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:16.254 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:16.254 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:16.254 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:15 smithi137 ceph-mon[29354]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:16.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:16.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:16.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:15 smithi142 ceph-mon[37379]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:16.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.822+0000 7faf2b7fe700 1 -- 172.21.15.137:0/498340974 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faf3c106750 con 0x7faf2406c460 2024-06-15T01:01:16.824 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 msgr2=0x7faf2406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 0x7faf2406e920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b5c0 tx=0x7faf30005ee0 comp rx=0 tx=0).stop 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 msgr2=0x7faf3c196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00d900 tx=0x7faf2c00dc10 comp rx=0 tx=0).stop 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 shutdown_connections 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7faf2406c460 0x7faf2406e920 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf3c100c00 0x7faf3c196290 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:16.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 --2- 172.21.15.137:0/498340974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3c101e00 0x7faf3c1967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:16.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 >> 172.21.15.137:0/498340974 conn(0x7faf3c0fc180 msgr2=0x7faf3c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:01:16.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 shutdown_connections 2024-06-15T01:01:16.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:16.825+0000 7faf4091d700 1 -- 172.21.15.137:0/498340974 wait complete. 2024-06-15T01:01:17.630 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.1.service 2024-06-15T01:01:17.633 INFO:tasks.cephadm:Deploying osd.2 on smithi137 with /dev/vg_nvme/lv_2... 2024-06-15T01:01:17.633 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-15T01:01:17.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:17 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:17.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:17 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:17.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:17 smithi137 ceph-mon[29354]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-15T01:01:17.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:17 smithi137 ceph-mon[29354]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:18.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:17 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:18.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:17 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:18.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:17 smithi142 ceph-mon[37379]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-15T01:01:18.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:17 smithi142 ceph-mon[37379]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:18.234 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:01:18.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-mon[29354]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-15T01:01:18.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-mon[29354]: osdmap e12: 2 total, 1 up, 2 in 2024-06-15T01:01:18.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:18.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-mon[29354]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:01:18.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:18.980 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[56025]: 2024-06-15T01:01:18.807+0000 7f85c56a2700 -1 osd.1 0 waiting for initial osdmap 2024-06-15T01:01:18.981 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[56025]: 2024-06-15T01:01:18.812+0000 7f85bfc94700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:01:19.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:18 smithi142 ceph-mon[37379]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-15T01:01:19.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:18 smithi142 ceph-mon[37379]: osdmap e12: 2 total, 1 up, 2 in 2024-06-15T01:01:19.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:19.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:18 smithi142 ceph-mon[37379]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:01:19.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:20.064 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:19 smithi137 ceph-mon[29354]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:01:20.064 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:19 smithi137 ceph-mon[29354]: osdmap e13: 2 total, 1 up, 2 in 2024-06-15T01:01:20.064 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:19 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:20.064 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:19 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:20.065 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:19 smithi137 ceph-mon[29354]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:20.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:19 smithi142 ceph-mon[37379]: from='osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:01:20.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:19 smithi142 ceph-mon[37379]: osdmap e13: 2 total, 1 up, 2 in 2024-06-15T01:01:20.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:19 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:20.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:19 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:20.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:19 smithi142 ceph-mon[37379]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125232 s, 837 MB/s 2024-06-15T01:01:20.572 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-06-15T01:01:21.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:01:21.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:01:21.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:21.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] boot 2024-06-15T01:01:21.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: osdmap e14: 2 total, 2 up, 2 in 2024-06-15T01:01:21.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:21.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:01:21.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:01:21.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:21.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: osd.1 [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] boot 2024-06-15T01:01:21.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: osdmap e14: 2 total, 2 up, 2 in 2024-06-15T01:01:21.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:01:21.560 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-06-15T01:01:22.067 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:21 smithi137 ceph-mon[29354]: osdmap e15: 2 total, 2 up, 2 in 2024-06-15T01:01:22.067 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:21 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:22.067 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:21 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:22.067 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:21 smithi137 ceph-mon[29354]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:22.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:21 smithi142 ceph-mon[37379]: osdmap e15: 2 total, 2 up, 2 in 2024-06-15T01:01:22.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:21 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:22.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:21 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:22.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:21 smithi142 ceph-mon[37379]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:22.740 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:01:24.424 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:24 smithi137 ceph-mon[29354]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:24.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 -- 172.21.15.137:0/297756534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a00fee80 msgr2=0x7f63a00ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 --2- 172.21.15.137:0/297756534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a00fee80 0x7f63a00ff2a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6394009910 tx=0x7f6394009c20 comp rx=0 tx=0).stop 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 -- 172.21.15.137:0/297756534 shutdown_connections 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 --2- 172.21.15.137:0/297756534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a0101e60 0x7f63a01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 --2- 172.21.15.137:0/297756534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a00fee80 0x7f63a00ff2a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 -- 172.21.15.137:0/297756534 >> 172.21.15.137:0/297756534 conn(0x7f63a00fa7c0 msgr2=0x7f63a00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 -- 172.21.15.137:0/297756534 shutdown_connections 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.439+0000 7f63a61b4700 1 -- 172.21.15.137:0/297756534 wait complete. 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 Processor -- start 2024-06-15T01:01:24.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 -- start start 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a00fee80 0x7f63a01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0196e30 con 0x7f63a0101e60 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f63a61b4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0196f70 con 0x7f63a00fee80 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f639effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f639effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58882/0 (socket says 172.21.15.137:58882) 2024-06-15T01:01:24.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f639effd700 1 -- 172.21.15.137:0/114119527 learned_addr learned my addr 172.21.15.137:0/114119527 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.440+0000 7f639f7fe700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a00fee80 0x7f63a01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639effd700 1 -- 172.21.15.137:0/114119527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a00fee80 msgr2=0x7f63a01962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639effd700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a00fee80 0x7f63a01962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639effd700 1 -- 172.21.15.137:0/114119527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940095f0 con 0x7f63a0101e60 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639effd700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f639000d900 tx=0x7f639000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63900041f0 con 0x7f63a0101e60 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6390004cc0 con 0x7f63a0101e60 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390005660 con 0x7f63a0101e60 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a019ba20 con 0x7f63a0101e60 2024-06-15T01:01:24.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.441+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a019bf70 con 0x7f63a0101e60 2024-06-15T01:01:24.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.442+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a01095e0 con 0x7f63a0101e60 2024-06-15T01:01:24.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.443+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f6390004750 con 0x7f63a0101e60 2024-06-15T01:01:24.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.443+0000 7f639cff9700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 0x7f638806e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:24.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.443+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f639008a990 con 0x7f63a0101e60 2024-06-15T01:01:24.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.443+0000 7f639f7fe700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 0x7f638806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:24.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.444+0000 7f639f7fe700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 0x7f638806e930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6394006010 tx=0x7f639400b560 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:01:24.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.448+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6390059950 con 0x7f63a0101e60 2024-06-15T01:01:24.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:24.655+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a019c6c0 con 0x7f638806c470 2024-06-15T01:01:24.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:24 smithi142 ceph-mon[37379]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:25.453 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:25 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:01:25.453 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:25 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:01:25.454 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:25 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:25.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:25 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:01:25.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:25 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:01:25.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:25 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi137 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:26.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi137 to 5920M 2024-06-15T01:01:26.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:26.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:26.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:26.602 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:01:26.602 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi137 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:26.603 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi137 to 5920M 2024-06-15T01:01:26.604 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.604 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:26.604 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:26.604 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:26.604 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:28.686 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:28.686 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:28.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:28.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2927741554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb"}]: dispatch 2024-06-15T01:01:28.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2927741554' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb"}]': finished 2024-06-15T01:01:28.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: osdmap e16: 3 total, 2 up, 3 in 2024-06-15T01:01:28.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:28 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2927741554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb"}]: dispatch 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2927741554' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb"}]': finished 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: osdmap e16: 3 total, 2 up, 3 in 2024-06-15T01:01:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:28 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:29.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:29 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/665435116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:29.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:29 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/665435116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:30.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:30 smithi137 ceph-mon[29354]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:30.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:30 smithi142 ceph-mon[37379]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:32.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:32.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:32.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:31 smithi137 ceph-mon[29354]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:31 smithi142 ceph-mon[37379]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:33.655 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:33.712 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:34.564 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:34 smithi137 ceph-mon[29354]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:34.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:34 smithi142 ceph-mon[37379]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:36.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:36 smithi137 ceph-mon[29354]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:36.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:36 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-15T01:01:36.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:36 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:36.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:36 smithi142 ceph-mon[37379]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:36.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:36 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-15T01:01:36.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:36 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:37.607 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:37 smithi137 ceph-mon[29354]: Deploying daemon osd.2 on smithi137 2024-06-15T01:01:37.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:37 smithi142 ceph-mon[37379]: Deploying daemon osd.2 on smithi137 2024-06-15T01:01:38.718 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:38 smithi142 ceph-mon[37379]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:38 smithi137 ceph-mon[29354]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:40.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:40 smithi137 ceph-mon[29354]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:40.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:40 smithi142 ceph-mon[37379]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:41.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:41.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:41.627 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:41.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:41.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:41.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:42.707 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:42 smithi137 ceph-mon[29354]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:42.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:42 smithi142 ceph-mon[37379]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:44.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.284+0000 7f639cff9700 1 -- 172.21.15.137:0/114119527 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63a019c6c0 con 0x7f638806c470 2024-06-15T01:01:44.285 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 2 on host 'smithi137' 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 msgr2=0x7f638806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 0x7f638806e930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6394006010 tx=0x7f639400b560 comp rx=0 tx=0).stop 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 msgr2=0x7f63a0196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f639000d900 tx=0x7f639000dc10 comp rx=0 tx=0).stop 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 shutdown_connections 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f638806c470 0x7f638806e930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63a00fee80 0x7f63a01962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 --2- 172.21.15.137:0/114119527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63a0101e60 0x7f63a0196810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 >> 172.21.15.137:0/114119527 conn(0x7f63a00fa7c0 msgr2=0x7f63a0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:01:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 shutdown_connections 2024-06-15T01:01:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:01:44.286+0000 7f63a61b4700 1 -- 172.21.15.137:0/114119527 wait complete. 2024-06-15T01:01:44.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:44 smithi137 ceph-mon[29354]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:44.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:44.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:44 smithi142 ceph-mon[37379]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:44 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:44 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:45.329 DEBUG:teuthology.orchestra.run.smithi137:osd.2> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.2.service 2024-06-15T01:01:45.332 INFO:tasks.cephadm:Deploying osd.3 on smithi142 with /dev/vg_nvme/lv_4... 2024-06-15T01:01:45.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-15T01:01:45.548 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:01:45.641 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:45 smithi137 ceph-mon[29354]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-15T01:01:45.671 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:45 smithi142 ceph-mon[37379]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-15T01:01:46.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:46 smithi142 ceph-mon[37379]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:46.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:46 smithi142 ceph-mon[37379]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-15T01:01:46.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:46 smithi142 ceph-mon[37379]: osdmap e17: 3 total, 2 up, 3 in 2024-06-15T01:01:46.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:46 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:46.679 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:46 smithi142 ceph-mon[37379]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:01:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-mon[29354]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-mon[29354]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-15T01:01:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-mon[29354]: osdmap e17: 3 total, 2 up, 3 in 2024-06-15T01:01:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:46.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-mon[29354]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:01:46.730 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[63329]: 2024-06-15T01:01:46.389+0000 7fe303a1d700 -1 osd.2 0 waiting for initial osdmap 2024-06-15T01:01:46.730 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:46 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[63329]: 2024-06-15T01:01:46.396+0000 7fe2fc00b700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:01:47.431 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-15T01:01:47.458 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-15T01:01:47.459 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-06-15T01:01:47.459 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-06-15T01:01:47.459 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112856 s, 929 MB/s 2024-06-15T01:01:47.459 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-06-15T01:01:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:47 smithi137 ceph-mon[29354]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:47 smithi137 ceph-mon[29354]: osdmap e18: 3 total, 2 up, 3 in 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:47 smithi142 ceph-mon[37379]: from='osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:47 smithi142 ceph-mon[37379]: osdmap e18: 3 total, 2 up, 3 in 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:47.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:48.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-06-15T01:01:48.415 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:01:48.537 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] boot 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: osdmap e19: 3 total, 3 up, 3 in 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:48.538 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: osd.2 [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] boot 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: osdmap e19: 3 total, 3 up, 3 in 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:48.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:01:49.912 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:49 smithi142 ceph-mon[37379]: osdmap e20: 3 total, 3 up, 3 in 2024-06-15T01:01:49.912 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:49 smithi142 ceph-mon[37379]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:49.912 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:49 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-15T01:01:49.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:49 smithi137 ceph-mon[29354]: osdmap e20: 3 total, 3 up, 3 in 2024-06-15T01:01:49.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:49 smithi137 ceph-mon[29354]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:49.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:49 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-15T01:01:50.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 -- 172.21.15.142:0/1286786113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 msgr2=0x7fce68106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:50.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1286786113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce68106370 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009910 tx=0x7fce5c009c20 comp rx=0 tx=0).stop 2024-06-15T01:01:50.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 -- 172.21.15.142:0/1286786113 shutdown_connections 2024-06-15T01:01:50.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1286786113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce68106370 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:50.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1286786113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 -- 172.21.15.142:0/1286786113 >> 172.21.15.142:0/1286786113 conn(0x7fce680fb060 msgr2=0x7fce680fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 -- 172.21.15.142:0/1286786113 shutdown_connections 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.014+0000 7fce6cfff700 1 -- 172.21.15.142:0/1286786113 wait complete. 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 Processor -- start 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 -- start start 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68196df0 con 0x7fce68101650 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce6cfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68196f30 con 0x7fce68103f80 2024-06-15T01:01:50.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.015+0000 7fce65d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce6659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47990/0 (socket says 172.21.15.142:47990) 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 -- 172.21.15.142:0/1618889379 learned_addr learned my addr 172.21.15.142:0/1618889379 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 -- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 msgr2=0x7fce68196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 -- 172.21.15.142:0/1618889379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0095f0 con 0x7fce68103f80 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce6659c700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce65d9b700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bde0 tx=0x7fce5c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5c01d070 con 0x7fce68103f80 2024-06-15T01:01:50.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fce5c005dc0 con 0x7fce68103f80 2024-06-15T01:01:50.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6819b980 con 0x7fce68103f80 2024-06-15T01:01:50.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5c003b10 con 0x7fce68103f80 2024-06-15T01:01:50.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.016+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6819be90 con 0x7fce68103f80 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.017+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce681903f0 con 0x7fce68103f80 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.018+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fce5c003c70 con 0x7fce68103f80 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.018+0000 7fce577fe700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 0x7fce5006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.018+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3221+0+0 (secure 0 0 0) 0x7fce5c08c4b0 con 0x7fce68103f80 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.018+0000 7fce6659c700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 0x7fce5006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:01:50.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.019+0000 7fce6659c700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 0x7fce5006e920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce58011490 tx=0x7fce58009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:01:50.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.023+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce5c05b140 con 0x7fce68103f80 2024-06-15T01:01:50.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:50.216+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fce680611d0 con 0x7fce5006c460 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: osdmap e21: 3 total, 3 up, 3 in 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:01:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:50 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:50.923 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-15T01:01:50.923 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: osdmap e21: 3 total, 3 up, 3 in 2024-06-15T01:01:50.923 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-15T01:01:50.923 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:01:50.923 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:01:50.924 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:01:50.924 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:50 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:51.893 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:51 smithi137 sudo[67666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-15T01:01:51.893 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:51 smithi137 sudo[67666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:51.893 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:51 smithi137 sudo[67666]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:51.893 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:51 smithi137 sudo[67666]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: osdmap e22: 3 total, 3 up, 3 in 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi137 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi137 to 3946M 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.894 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:51 smithi137 ceph-mon[29354]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: osdmap e22: 3 total, 3 up, 3 in 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi137 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.895 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi137 to 3946M 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:01:51.896 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:51 smithi142 ceph-mon[37379]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:52.425 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-15T01:01:52.425 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:52.426 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67707]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:52.426 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67707]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:52.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67771]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-15T01:01:52.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67771]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:52.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67771]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:52.687 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67771]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67795]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67795]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67795]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67795]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67798]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-15T01:01:52.949 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67798]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:52.950 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67798]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:52.950 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:52 smithi137 sudo[67798]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: osdmap e23: 3 total, 3 up, 3 in 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/32814922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]: dispatch 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]: dispatch 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]': finished 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: osdmap e24: 4 total, 3 up, 4 in 2024-06-15T01:01:52.950 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:52 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:01:53.200 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-15T01:01:53.200 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:53.200 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67846]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:53.200 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67846]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: osdmap e23: 3 total, 3 up, 3 in 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/32814922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]: dispatch 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]: dispatch 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edcfe413-a142-4887-ac04-50c3ea15fb4f"}]': finished 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: osdmap e24: 4 total, 3 up, 4 in 2024-06-15T01:01:53.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:52 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:01:53.716 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67878]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-15T01:01:53.716 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67878]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:53.716 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67878]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:53.716 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67878]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:53.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-15T01:01:53.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:53.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:53.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:53 smithi137 sudo[67922]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:54.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:53 smithi142 sudo[41409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-15T01:01:54.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:53 smithi142 sudo[41409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:54.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:53 smithi142 sudo[41409]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:54.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:53 smithi142 sudo[41409]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:54.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:01:54.237+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fce5c057a80 con 0x7fce68103f80 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: pgmap v82: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2555298073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:01:54.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:54 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 sudo[41598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 sudo[41598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 sudo[41598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 sudo[41598]: pam_unix(sudo:session): session closed for user root 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: pgmap v82: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/2555298073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-15T01:01:54.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:01:54.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:01:54.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:01:54.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:54 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:01:55.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-15T01:01:55.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-15T01:01:55.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: mgrmap e19: smithi137.nbzbyc(active, since 2m), standbys: smithi142.dllryb 2024-06-15T01:01:55.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:55.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:55.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:55 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: mgrmap e19: smithi137.nbzbyc(active, since 2m), standbys: smithi142.dllryb 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:55.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:55 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:56.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:56 smithi142 ceph-mon[37379]: pgmap v83: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:56.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:56.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:56.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:56.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:56 smithi137 ceph-mon[29354]: pgmap v83: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:01:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:01:56.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:01:58.540 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:01:58 smithi142 ceph-mon[37379]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:01:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:01:58 smithi137 ceph-mon[29354]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:00 smithi137 ceph-mon[29354]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:00 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-15T01:02:00.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:00 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:00.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:00 smithi142 ceph-mon[37379]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:00.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:00 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-15T01:02:00.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:00 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:01.463 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:01 smithi142 ceph-mon[37379]: Deploying daemon osd.3 on smithi142 2024-06-15T01:02:01.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:01 smithi137 ceph-mon[29354]: Deploying daemon osd.3 on smithi142 2024-06-15T01:02:02.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:02 smithi137 ceph-mon[29354]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:02.732 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:02 smithi142 ceph-mon[37379]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:03.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:03.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:04.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:04 smithi142 ceph-mon[37379]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:04.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:04 smithi137 ceph-mon[29354]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:06.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:06 smithi137 ceph-mon[29354]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:06.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:06 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:06.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:06 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:06.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:06 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:06.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:06 smithi142 ceph-mon[37379]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:06.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:06 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:06.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:06 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:06.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:06 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:08.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:08 smithi137 ceph-mon[29354]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:08.744 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:08 smithi142 ceph-mon[37379]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:09.085 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 3 on host 'smithi142' 2024-06-15T01:02:09.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.084+0000 7fce577fe700 1 -- 172.21.15.142:0/1618889379 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fce680611d0 con 0x7fce5006c460 2024-06-15T01:02:09.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.086+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 msgr2=0x7fce5006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:09.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.086+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 0x7fce5006e920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce58011490 tx=0x7fce58009260 comp rx=0 tx=0).stop 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 msgr2=0x7fce681967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bde0 tx=0x7fce5c00be10 comp rx=0 tx=0).stop 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 shutdown_connections 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fce5006c460 0x7fce5006e920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce68101650 0x7fce68196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 --2- 172.21.15.142:0/1618889379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce68103f80 0x7fce681967d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 >> 172.21.15.142:0/1618889379 conn(0x7fce680fb060 msgr2=0x7fce680fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 shutdown_connections 2024-06-15T01:02:09.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:09.087+0000 7fce6cfff700 1 -- 172.21.15.142:0/1618889379 wait complete. 2024-06-15T01:02:09.681 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:09 smithi142 ceph-mon[37379]: from='osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:02:09.681 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:09 smithi142 ceph-mon[37379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:02:09.681 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:09 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:09.681 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:09 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:09.702 DEBUG:teuthology.orchestra.run.smithi142:osd.3> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.3.service 2024-06-15T01:02:09.705 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-06-15T01:02:09.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-15T01:02:09.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:09 smithi137 ceph-mon[29354]: from='osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:02:09.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:09 smithi137 ceph-mon[29354]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:02:09.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:09 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:09.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:09 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:09.959 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:02:10.541 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:10.541 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-15T01:02:10.541 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: osdmap e25: 4 total, 3 up, 4 in 2024-06-15T01:02:10.541 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:10.542 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:10.542 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:10.542 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:10.542 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:10.542 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[43427]: 2024-06-15T01:02:10.375+0000 7f91761fb700 -1 osd.3 0 waiting for initial osdmap 2024-06-15T01:02:10.542 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:02:10 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[43427]: 2024-06-15T01:02:10.381+0000 7f916e7e9700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: osdmap e25: 4 total, 3 up, 4 in 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:10.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:10 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:11.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:11 smithi142 ceph-mon[37379]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:02:11.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:11 smithi142 ceph-mon[37379]: osdmap e26: 4 total, 3 up, 4 in 2024-06-15T01:02:11.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:11.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:11 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:11.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:11 smithi137 ceph-mon[29354]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:02:11.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:11 smithi137 ceph-mon[29354]: osdmap e26: 4 total, 3 up, 4 in 2024-06-15T01:02:11.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:11.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:11 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132382 s, 792 MB/s 2024-06-15T01:02:12.413 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] boot 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: osdmap e27: 4 total, 4 up, 4 in 2024-06-15T01:02:12.716 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:12.717 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:12 smithi142 ceph-mon[37379]: osdmap e28: 4 total, 4 up, 4 in 2024-06-15T01:02:12.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:02:12.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:02:12.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-15T01:02:12.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:12.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: osd.3 [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] boot 2024-06-15T01:02:12.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: osdmap e27: 4 total, 4 up, 4 in 2024-06-15T01:02:12.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:02:12.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:12 smithi137 ceph-mon[29354]: osdmap e28: 4 total, 4 up, 4 in 2024-06-15T01:02:12.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-06-15T01:02:13.570 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:02:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:14 smithi142 ceph-mon[37379]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:14.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:14 smithi142 ceph-mon[37379]: osdmap e29: 4 total, 4 up, 4 in 2024-06-15T01:02:14.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:14 smithi137 ceph-mon[29354]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:14.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:14 smithi137 ceph-mon[29354]: osdmap e29: 4 total, 4 up, 4 in 2024-06-15T01:02:15.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: osdmap e30: 4 total, 4 up, 4 in 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:15.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:15 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: osdmap e30: 4 total, 4 up, 4 in 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:15.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:15.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:15.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:15 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 -- 172.21.15.142:0/1725816145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10103f80 msgr2=0x7f9b10106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:15.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 --2- 172.21.15.142:0/1725816145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10103f80 0x7f9b10106370 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009910 tx=0x7f9b04009c20 comp rx=0 tx=0).stop 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 -- 172.21.15.142:0/1725816145 shutdown_connections 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 --2- 172.21.15.142:0/1725816145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10103f80 0x7f9b10106370 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 --2- 172.21.15.142:0/1725816145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10101650 0x7f9b10103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 -- 172.21.15.142:0/1725816145 >> 172.21.15.142:0/1725816145 conn(0x7f9b100fb060 msgr2=0x7f9b100fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 -- 172.21.15.142:0/1725816145 shutdown_connections 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 -- 172.21.15.142:0/1725816145 wait complete. 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.777+0000 7f9b162fb700 1 Processor -- start 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b162fb700 1 -- start start 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b162fb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b162fb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10103f80 0x7f9b100ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b162fb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100ffb40 con 0x7f9b10103f80 2024-06-15T01:02:15.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b162fb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100ffc80 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52194/0 (socket says 172.21.15.142:52194) 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 -- 172.21.15.142:0/2153663958 learned_addr learned my addr 172.21.15.142:0/2153663958 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0f7fe700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10103f80 0x7f9b100ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 -- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10103f80 msgr2=0x7f9b100ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10103f80 0x7f9b100ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.778+0000 7f9b0ffff700 1 -- 172.21.15.142:0/2153663958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040095f0 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b0ffff700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d8d0 tx=0x7f9b0000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b000098a0 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b100fff60 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b00009e90 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0000f5f0 con 0x7f9b10101650 2024-06-15T01:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.779+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1019e2f0 con 0x7f9b10101650 2024-06-15T01:02:15.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.780+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b101903f0 con 0x7f9b10101650 2024-06-15T01:02:15.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.780+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9b00009a00 con 0x7f9b10101650 2024-06-15T01:02:15.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.781+0000 7f9b0d7fa700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 0x7f9af806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:15.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.781+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f9b0008b040 con 0x7f9b10101650 2024-06-15T01:02:15.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.781+0000 7f9b0f7fe700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 0x7f9af806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:15.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.781+0000 7f9b0f7fe700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 0x7f9af806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b04006010 tx=0x7f9b0400b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:02:15.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.785+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9b00059950 con 0x7f9b10101650 2024-06-15T01:02:15.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:15.984+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b100611d0 con 0x7f9af806c4e0 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi142 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi142 to 14912M 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: osdmap e31: 4 total, 4 up, 4 in 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:02:16.666 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:16 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:16.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi142 2024-06-15T01:02:16.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi142 to 14912M 2024-06-15T01:02:16.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:16.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: osdmap e31: 4 total, 4 up, 4 in 2024-06-15T01:02:16.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:02:16.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:02:16.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:16 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:17.605 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:17 smithi142 ceph-mon[37379]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:02:17.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:17 smithi137 ceph-mon[29354]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:02:18.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:18 smithi137 ceph-mon[29354]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-15T01:02:18.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:18.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:18 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/4115964511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]: dispatch 2024-06-15T01:02:18.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:18 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]: dispatch 2024-06-15T01:02:18.749 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:18 smithi142 ceph-mon[37379]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-15T01:02:18.749 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:18.749 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:18 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/4115964511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]: dispatch 2024-06-15T01:02:18.749 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:18 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]: dispatch 2024-06-15T01:02:19.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:19 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]': finished 2024-06-15T01:02:19.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:19 smithi137 ceph-mon[29354]: osdmap e32: 5 total, 4 up, 5 in 2024-06-15T01:02:19.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:19 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:19.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:19 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2888894113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:02:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:19 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fe44512-dd8e-468e-9a24-33bf9b288d27"}]': finished 2024-06-15T01:02:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:19 smithi142 ceph-mon[37379]: osdmap e32: 5 total, 4 up, 5 in 2024-06-15T01:02:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:19 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:19 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/2888894113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:02:20.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:20 smithi137 ceph-mon[29354]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-15T01:02:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:20 smithi142 ceph-mon[37379]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-06-15T01:02:22.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:22 smithi142 ceph-mon[37379]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-15T01:02:22.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:22 smithi137 ceph-mon[29354]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-06-15T01:02:24.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:24 smithi142 ceph-mon[37379]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-15T01:02:24.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:24 smithi137 ceph-mon[29354]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-15T01:02:26.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:26 smithi142 ceph-mon[37379]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-15T01:02:26.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-15T01:02:26.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:26.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:26 smithi137 ceph-mon[29354]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-15T01:02:26.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-15T01:02:26.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:27.708 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:27 smithi142 ceph-mon[37379]: Deploying daemon osd.4 on smithi142 2024-06-15T01:02:27.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:27 smithi137 ceph-mon[29354]: Deploying daemon osd.4 on smithi142 2024-06-15T01:02:28.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:28 smithi137 ceph-mon[29354]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:28.765 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:28 smithi142 ceph-mon[37379]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:30.525 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:30 smithi142 ceph-mon[37379]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:30.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:30 smithi137 ceph-mon[29354]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:32.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:32 smithi142 ceph-mon[37379]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:32 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:32 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:32.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:32 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:32.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:32 smithi137 ceph-mon[29354]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:32.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:32 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:32.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:32 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:32.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:32 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:33.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:33.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:34.463 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:34 smithi142 ceph-mon[37379]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:34.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:34 smithi137 ceph-mon[29354]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:34.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.798+0000 7f9b0d7fa700 1 -- 172.21.15.142:0/2153663958 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9b100611d0 con 0x7f9af806c4e0 2024-06-15T01:02:34.798 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-06-15T01:02:34.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 msgr2=0x7f9af806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:34.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 0x7f9af806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b04006010 tx=0x7f9b0400b540 comp rx=0 tx=0).stop 2024-06-15T01:02:34.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 msgr2=0x7f9b10100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d8d0 tx=0x7f9b0000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 shutdown_connections 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9af806c4e0 0x7f9af806e9a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10101650 0x7f9b10100fb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 --2- 172.21.15.142:0/2153663958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b10103f80 0x7f9b100ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 >> 172.21.15.142:0/2153663958 conn(0x7f9b100fb060 msgr2=0x7f9b100fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:02:34.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 shutdown_connections 2024-06-15T01:02:34.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:34.800+0000 7f9b162fb700 1 -- 172.21.15.142:0/2153663958 wait complete. 2024-06-15T01:02:35.409 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.4.service 2024-06-15T01:02:35.411 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_2... 2024-06-15T01:02:35.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-15T01:02:35.657 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:02:36.011 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.011 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.011 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:02:36.012 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:02:36.012 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:36.012 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.012 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:36.021 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:37.022 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-mon[37379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-15T01:02:37.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-mon[37379]: osdmap e33: 5 total, 4 up, 5 in 2024-06-15T01:02:37.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:37.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-mon[37379]: from='osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:37.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-mon[37379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:37.023 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[50119]: 2024-06-15T01:02:36.764+0000 7f95b24cb700 -1 osd.4 0 waiting for initial osdmap 2024-06-15T01:02:37.023 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:02:36 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[50119]: 2024-06-15T01:02:36.771+0000 7f95acabd700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:02:37.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:36 smithi137 ceph-mon[29354]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-15T01:02:37.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:36 smithi137 ceph-mon[29354]: osdmap e33: 5 total, 4 up, 5 in 2024-06-15T01:02:37.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:36 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:37.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:36 smithi137 ceph-mon[29354]: from='osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:37.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:36 smithi137 ceph-mon[29354]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:02:37.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:37 smithi142 ceph-mon[37379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:02:37.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:37 smithi142 ceph-mon[37379]: osdmap e34: 5 total, 4 up, 5 in 2024-06-15T01:02:37.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:37 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:37.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:37 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:37.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:37 smithi142 ceph-mon[37379]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:38.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:37 smithi137 ceph-mon[29354]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:02:38.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:37 smithi137 ceph-mon[29354]: osdmap e34: 5 total, 4 up, 5 in 2024-06-15T01:02:38.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:37 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:38.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:37 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:38.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:37 smithi137 ceph-mon[29354]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012224 s, 858 MB/s 2024-06-15T01:02:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-06-15T01:02:39.022 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:38 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:02:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:38 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:02:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:38 smithi142 ceph-mon[37379]: osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] boot 2024-06-15T01:02:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:38 smithi142 ceph-mon[37379]: osdmap e35: 5 total, 5 up, 5 in 2024-06-15T01:02:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:38 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:39.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-06-15T01:02:39.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:38 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:02:39.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:38 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:02:39.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:38 smithi137 ceph-mon[29354]: osd.4 [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] boot 2024-06-15T01:02:39.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:38 smithi137 ceph-mon[29354]: osdmap e35: 5 total, 5 up, 5 in 2024-06-15T01:02:39.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:38 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:02:39.773 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:02:39.927 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: osdmap e36: 5 total, 5 up, 5 in 2024-06-15T01:02:39.927 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:39.928 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:39 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: osdmap e36: 5 total, 5 up, 5 in 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:40.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:40.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:40.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:39 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:40.832 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi142 2024-06-15T01:02:40.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi142 to 7456M 2024-06-15T01:02:40.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:40.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:40.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:40.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:41.059 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi142 2024-06-15T01:02:41.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi142 to 7456M 2024-06-15T01:02:41.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:41.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:41.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:02:41.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:41.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 -- 172.21.15.142:0/242631226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 msgr2=0x7f7a9c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 --2- 172.21.15.142:0/242631226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c0ffee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009910 tx=0x7f7a8c009c20 comp rx=0 tx=0).stop 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 -- 172.21.15.142:0/242631226 shutdown_connections 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 --2- 172.21.15.142:0/242631226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 --2- 172.21.15.142:0/242631226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c0ffee0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 -- 172.21.15.142:0/242631226 >> 172.21.15.142:0/242631226 conn(0x7f7a9c0fb060 msgr2=0x7f7a9c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 -- 172.21.15.142:0/242631226 shutdown_connections 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.297+0000 7f7aa1063700 1 -- 172.21.15.142:0/242631226 wait complete. 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 Processor -- start 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 -- start start 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c19b6d0 con 0x7f7a9c100cc0 2024-06-15T01:02:41.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7aa1063700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c19b810 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7a9ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7a9ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56238/0 (socket says 172.21.15.142:56238) 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7a9ad9d700 1 -- 172.21.15.142:0/3279650693 learned_addr learned my addr 172.21.15.142:0/3279650693 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.298+0000 7f7a9a59c700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a9ad9d700 1 -- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 msgr2=0x7f7a9c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a9ad9d700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a9ad9d700 1 -- 172.21.15.142:0/3279650693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0095f0 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a9a59c700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a9ad9d700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00bda0 tx=0x7f7a8c00be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a8c01d070 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7a8c022470 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c19ba90 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a8c00f690 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.299+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c19c000 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.300+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c108560 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.300+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7a8c0229c0 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.301+0000 7f7a83fff700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 0x7f7a8406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:02:41.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.301+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f7a8c08ca10 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.301+0000 7f7a9a59c700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 0x7f7a8406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:02:41.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.302+0000 7f7a9a59c700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 0x7f7a8406e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a900099d0 tx=0x7f7a90008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:02:41.306 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.305+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a8c05b340 con 0x7f7a9c0ffac0 2024-06-15T01:02:41.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:41.503+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a9c105610 con 0x7f7a8406c4e0 2024-06-15T01:02:42.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:41 smithi142 ceph-mon[37379]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:42.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:41 smithi142 ceph-mon[37379]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:02:42.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:02:42.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:02:42.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:41 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:42.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:41 smithi137 ceph-mon[29354]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:42.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:41 smithi137 ceph-mon[29354]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:02:42.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-15T01:02:42.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-15T01:02:42.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:41 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/2884638495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]: dispatch 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]: dispatch 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]': finished 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: osdmap e37: 6 total, 5 up, 6 in 2024-06-15T01:02:44.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:44 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:02:44.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:44.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/2884638495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]: dispatch 2024-06-15T01:02:44.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]: dispatch 2024-06-15T01:02:44.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ec5d99-d399-439d-a909-709c9eff7f70"}]': finished 2024-06-15T01:02:44.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: osdmap e37: 6 total, 5 up, 6 in 2024-06-15T01:02:44.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:44 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:02:45.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:45 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/1846669672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:02:45.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:45 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/1846669672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-15T01:02:46.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:46 smithi137 ceph-mon[29354]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:46.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:46 smithi142 ceph-mon[37379]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:48.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:48 smithi142 ceph-mon[37379]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:48.572 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:48 smithi137 ceph-mon[29354]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:48.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:02:50.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:50 smithi137 ceph-mon[29354]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:50.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:50 smithi142 ceph-mon[37379]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:51.438 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-15T01:02:51.438 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:51.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-15T01:02:51.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:02:52.675 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:52 smithi142 ceph-mon[37379]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:52.675 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:52 smithi142 ceph-mon[37379]: Deploying daemon osd.5 on smithi142 2024-06-15T01:02:52.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:52 smithi137 ceph-mon[29354]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:52.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:52 smithi137 ceph-mon[29354]: Deploying daemon osd.5 on smithi142 2024-06-15T01:02:54.702 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:54 smithi142 ceph-mon[37379]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:54.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:54 smithi137 ceph-mon[29354]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:56.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:56 smithi137 ceph-mon[29354]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:56.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:56 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:56.646 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:56 smithi142 ceph-mon[37379]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:56.646 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:56.646 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:02:56.646 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:56 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:02:58.645 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:02:58 smithi142 ceph-mon[37379]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:58.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:02:58 smithi137 ceph-mon[29354]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:02:59.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.968+0000 7f7a83fff700 1 -- 172.21.15.142:0/3279650693 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7a9c105610 con 0x7f7a8406c4e0 2024-06-15T01:02:59.969 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 msgr2=0x7f7a8406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 0x7f7a8406e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a900099d0 tx=0x7f7a90008040 comp rx=0 tx=0).stop 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 msgr2=0x7f7a9c069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00bda0 tx=0x7f7a8c00be80 comp rx=0 tx=0).stop 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 shutdown_connections 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7a8406c4e0 0x7f7a8406e9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c0ffac0 0x7f7a9c069970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 --2- 172.21.15.142:0/3279650693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a9c100cc0 0x7f7a9c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:02:59.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 >> 172.21.15.142:0/3279650693 conn(0x7f7a9c0fb060 msgr2=0x7f7a9c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:02:59.976 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 shutdown_connections 2024-06-15T01:02:59.976 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:02:59.970+0000 7f7aa1063700 1 -- 172.21.15.142:0/3279650693 wait complete. 2024-06-15T01:03:00.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:00 smithi142 ceph-mon[37379]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:00.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:00 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:00.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:00 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:00.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:00 smithi142 ceph-mon[37379]: from='osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:03:00.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:00 smithi142 ceph-mon[37379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:03:00.511 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.5.service 2024-06-15T01:03:00.513 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-06-15T01:03:00.513 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd stat -f json 2024-06-15T01:03:00.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:00 smithi137 ceph-mon[29354]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:00.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:00 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:00.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:00 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:00.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:00 smithi137 ceph-mon[29354]: from='osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:03:00.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:00 smithi137 ceph-mon[29354]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:03:00.736 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:01.899 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: osdmap e38: 6 total, 5 up, 6 in 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:03:01.900 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-mon[37379]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:01.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:01.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:01.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-15T01:03:01.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: osdmap e38: 6 total, 5 up, 6 in 2024-06-15T01:03:01.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:01.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:03:01.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:03:01.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:01 smithi137 ceph-mon[29354]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:02.161 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[56598]: 2024-06-15T01:03:01.946+0000 7fb1d6c96700 -1 osd.5 0 waiting for initial osdmap 2024-06-15T01:03:02.161 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:03:01 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[56598]: 2024-06-15T01:03:01.954+0000 7fb1d328c700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:03:02.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 -- 172.21.15.137:0/4265912968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc740feed0 msgr2=0x7fbc740ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/4265912968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc740feed0 0x7fbc740ff2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009910 tx=0x7fbc64009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 -- 172.21.15.137:0/4265912968 shutdown_connections 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/4265912968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc74101eb0 0x7fbc74102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/4265912968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc740feed0 0x7fbc740ff2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 -- 172.21.15.137:0/4265912968 >> 172.21.15.137:0/4265912968 conn(0x7fbc740fa810 msgr2=0x7fbc740fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 -- 172.21.15.137:0/4265912968 shutdown_connections 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 -- 172.21.15.137:0/4265912968 wait complete. 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.322+0000 7fbc78a9a700 1 Processor -- start 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc78a9a700 1 -- start start 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc78a9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 0x7fbc74196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc78a9a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc78a9a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74196e10 con 0x7fbc74101eb0 2024-06-15T01:03:02.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc78a9a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74196f50 con 0x7fbc740feed0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41416/0 (socket says 172.21.15.137:41416) 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 -- 172.21.15.137:0/3429032938 learned_addr learned my addr 172.21.15.137:0/3429032938 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc7259c700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 0x7fbc74196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 -- 172.21.15.137:0/3429032938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 msgr2=0x7fbc74196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 0x7fbc74196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 -- 172.21.15.137:0/3429032938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640095f0 con 0x7fbc74101eb0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc7259c700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 0x7fbc74196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc71d9b700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c930 tx=0x7fbc6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.323+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc68007ad0 con 0x7fbc74101eb0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.324+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7419ba10 con 0x7fbc74101eb0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.324+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbc6800ce80 con 0x7fbc74101eb0 2024-06-15T01:03:02.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.324+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc68018630 con 0x7fbc74101eb0 2024-06-15T01:03:02.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.324+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7419bfb0 con 0x7fbc74101eb0 2024-06-15T01:03:02.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.325+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc74109630 con 0x7fbc74101eb0 2024-06-15T01:03:02.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.325+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbc68018790 con 0x7fbc74101eb0 2024-06-15T01:03:02.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.325+0000 7fbc637fe700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 0x7fbc5c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:02.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.325+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7fbc6808b3e0 con 0x7fbc74101eb0 2024-06-15T01:03:02.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.325+0000 7fbc7259c700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 0x7fbc5c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:02.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.326+0000 7fbc7259c700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 0x7fbc5c06e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc64006010 tx=0x7fbc6400b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:02.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.330+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc68059b10 con 0x7fbc74101eb0 2024-06-15T01:03:02.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.521+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbc74109820 con 0x7fbc74101eb0 2024-06-15T01:03:02.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.522+0000 7fbc637fe700 1 -- 172.21.15.137:0/3429032938 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fbc680596a0 con 0x7fbc74101eb0 2024-06-15T01:03:02.523 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 msgr2=0x7fbc5c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 0x7fbc5c06e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc64006010 tx=0x7fbc6400b540 comp rx=0 tx=0).stop 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 msgr2=0x7fbc74196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c930 tx=0x7fbc6800cc40 comp rx=0 tx=0).stop 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 shutdown_connections 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc5c06c4e0 0x7fbc5c06e9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc740feed0 0x7fbc74196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 --2- 172.21.15.137:0/3429032938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc74101eb0 0x7fbc74196880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:02.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 >> 172.21.15.137:0/3429032938 conn(0x7fbc740fa810 msgr2=0x7fbc741050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:02.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 shutdown_connections 2024-06-15T01:03:02.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:02.524+0000 7fbc78a9a700 1 -- 172.21.15.137:0/3429032938 wait complete. 2024-06-15T01:03:03.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:02 smithi142 ceph-mon[37379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:03:03.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:02 smithi142 ceph-mon[37379]: osdmap e39: 6 total, 5 up, 6 in 2024-06-15T01:03:03.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:02 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:03.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:02 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:03.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:02 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3429032938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-15T01:03:03.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:02 smithi137 ceph-mon[29354]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-06-15T01:03:03.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:02 smithi137 ceph-mon[29354]: osdmap e39: 6 total, 5 up, 6 in 2024-06-15T01:03:03.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:02 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:03.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:02 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:03.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:02 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3429032938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-15T01:03:03.306 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1718413357,"num_in_osds":6,"osd_in_since":1718413363,"num_remapped_pgs":0} 2024-06-15T01:03:04.051 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: purged_snaps scrub starts 2024-06-15T01:03:04.051 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: purged_snaps scrub ok 2024-06-15T01:03:04.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] boot 2024-06-15T01:03:04.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: osdmap e40: 6 total, 6 up, 6 in 2024-06-15T01:03:04.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:04.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:04.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:04.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: purged_snaps scrub starts 2024-06-15T01:03:04.261 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: purged_snaps scrub ok 2024-06-15T01:03:04.262 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: osd.5 [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] boot 2024-06-15T01:03:04.262 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: osdmap e40: 6 total, 6 up, 6 in 2024-06-15T01:03:04.262 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:03:04.262 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-15T01:03:04.262 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:04.307 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd stat -f json 2024-06-15T01:03:04.520 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: osdmap e41: 6 total, 6 up, 6 in 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: Detected new or changed devices on smithi142 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi142 to 4970M 2024-06-15T01:03:05.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:03:05.231 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:04 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: osdmap e41: 6 total, 6 up, 6 in 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi142 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi142 to 4970M 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:03:05.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:04 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:03:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 -- 172.21.15.137:0/3387697809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 msgr2=0x7f6e600ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 --2- 172.21.15.137:0/3387697809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e600ffed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b0a0 tx=0x7f6e5000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 -- 172.21.15.137:0/3387697809 shutdown_connections 2024-06-15T01:03:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 --2- 172.21.15.137:0/3387697809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 0x7f6e60101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 --2- 172.21.15.137:0/3387697809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e600ffed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 -- 172.21.15.137:0/3387697809 >> 172.21.15.137:0/3387697809 conn(0x7f6e600fb030 msgr2=0x7f6e600fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 -- 172.21.15.137:0/3387697809 shutdown_connections 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.177+0000 7f6e65824700 1 -- 172.21.15.137:0/3387697809 wait complete. 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 Processor -- start 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 -- start start 2024-06-15T01:03:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:06.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 0x7f6e60102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:06.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e601025d0 con 0x7f6e600ffab0 2024-06-15T01:03:06.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e65824700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e60102710 con 0x7f6e60100cb0 2024-06-15T01:03:06.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:06.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55714/0 (socket says 172.21.15.137:55714) 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 -- 172.21.15.137:0/2436907122 learned_addr learned my addr 172.21.15.137:0/2436907122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5e7fc700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 0x7f6e60102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 -- 172.21.15.137:0/2436907122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 msgr2=0x7f6e60102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 0x7f6e60102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 -- 172.21.15.137:0/2436907122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009d00 con 0x7f6e600ffab0 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.178+0000 7f6e5effd700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006130 tx=0x7f6e5000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.179+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5000e040 con 0x7f6e600ffab0 2024-06-15T01:03:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.179+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e60102990 con 0x7f6e600ffab0 2024-06-15T01:03:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.179+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6e50003de0 con 0x7f6e600ffab0 2024-06-15T01:03:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.179+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5001d660 con 0x7f6e600ffab0 2024-06-15T01:03:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.179+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e60102e80 con 0x7f6e600ffab0 2024-06-15T01:03:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.180+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e601084d0 con 0x7f6e600ffab0 2024-06-15T01:03:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.180+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6e50019070 con 0x7f6e600ffab0 2024-06-15T01:03:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.181+0000 7f6e64822700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 0x7f6e4c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.181+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6e5008ce60 con 0x7f6e600ffab0 2024-06-15T01:03:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.181+0000 7f6e5e7fc700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 0x7f6e4c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.181+0000 7f6e5e7fc700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 0x7f6e4c06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005fd0 tx=0x7f6e4800a560 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:06.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.185+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6e5005b4d0 con 0x7f6e600ffab0 2024-06-15T01:03:06.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:05 smithi137 ceph-mon[29354]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:06.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:05 smithi142 ceph-mon[37379]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:06.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.381+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6e6019e6d0 con 0x7f6e600ffab0 2024-06-15T01:03:06.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.381+0000 7f6e64822700 1 -- 172.21.15.137:0/2436907122 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f6e5005b060 con 0x7f6e600ffab0 2024-06-15T01:03:06.382 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:06.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 msgr2=0x7f6e4c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:06.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 0x7f6e4c06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005fd0 tx=0x7f6e4800a560 comp rx=0 tx=0).stop 2024-06-15T01:03:06.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 msgr2=0x7f6e60103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:06.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006130 tx=0x7f6e5000bdc0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 shutdown_connections 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f6e4c06c4e0 0x7f6e4c06e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e600ffab0 0x7f6e60103a40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 --2- 172.21.15.137:0/2436907122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e60100cb0 0x7f6e60102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 >> 172.21.15.137:0/2436907122 conn(0x7f6e600fb030 msgr2=0x7f6e60068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 shutdown_connections 2024-06-15T01:03:06.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:06.383+0000 7f6e65824700 1 -- 172.21.15.137:0/2436907122 wait complete. 2024-06-15T01:03:07.102 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1718413382,"num_in_osds":6,"osd_in_since":1718413363,"num_remapped_pgs":0} 2024-06-15T01:03:07.102 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-15T01:03:07.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:06 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2436907122' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-15T01:03:07.310 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:07.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:06 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2436907122' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-15T01:03:08.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:08 smithi137 ceph-mon[29354]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:08.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:08 smithi142 ceph-mon[37379]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:08.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 -- 172.21.15.137:0/3168855169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8100dc0 msgr2=0x7f82e81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:08.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/3168855169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8100dc0 0x7f82e81011a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82e000b0a0 tx=0x7f82e000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 -- 172.21.15.137:0/3168855169 shutdown_connections 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/3168855169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8101770 0x7f82e81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/3168855169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8100dc0 0x7f82e81011a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 -- 172.21.15.137:0/3168855169 >> 172.21.15.137:0/3168855169 conn(0x7f82e80fc650 msgr2=0x7f82e80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 -- 172.21.15.137:0/3168855169 shutdown_connections 2024-06-15T01:03:08.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.865+0000 7f82ec8a3700 1 -- 172.21.15.137:0/3168855169 wait complete. 2024-06-15T01:03:08.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 Processor -- start 2024-06-15T01:03:08.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 -- start start 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8100dc0 0x7f82e8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8197290 con 0x7f82e8101770 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82ec8a3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e819b020 con 0x7f82e8100dc0 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.866+0000 7f82e5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55736/0 (socket says 172.21.15.137:55736) 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 -- 172.21.15.137:0/4116294287 learned_addr learned my addr 172.21.15.137:0/4116294287 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e659c700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8100dc0 0x7f82e8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 -- 172.21.15.137:0/4116294287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8100dc0 msgr2=0x7f82e8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8100dc0 0x7f82e8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 -- 172.21.15.137:0/4116294287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0009d00 con 0x7f82e8101770 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82e5d9b700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82d000d900 tx=0x7f82d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d0009900 con 0x7f82e8101770 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f82d0010460 con 0x7f82e8101770 2024-06-15T01:03:08.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e819b300 con 0x7f82e8101770 2024-06-15T01:03:08.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d000f5f0 con 0x7f82e8101770 2024-06-15T01:03:08.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.867+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e819b850 con 0x7f82e8101770 2024-06-15T01:03:08.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.868+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8109160 con 0x7f82e8101770 2024-06-15T01:03:08.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.869+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f82d0009a60 con 0x7f82e8101770 2024-06-15T01:03:08.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.869+0000 7f82df7fe700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 0x7f82d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:08.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.869+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f82d008b060 con 0x7f82e8101770 2024-06-15T01:03:08.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.869+0000 7f82e659c700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 0x7f82d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:08.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.869+0000 7f82e659c700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 0x7f82d406e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f80 tx=0x7f82e00060c0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:08.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:08.874+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f82d0055b80 con 0x7f82e8101770 2024-06-15T01:03:09.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.065+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82e819bac0 con 0x7f82e8101770 2024-06-15T01:03:09.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.066+0000 7f82df7fe700 1 -- 172.21.15.137:0/4116294287 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11399 (secure 0 0 0) 0x7f82d00591a0 con 0x7f82e8101770 2024-06-15T01:03:09.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:09.067 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":41,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:03:03.946724+0000","last_up_change":"2024-06-15T01:03:02.945497+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6803","nonce":1195724024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6805","nonce":1195724024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6809","nonce":1195724024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6807","nonce":1195724024}]},"public_addr":"172.21.15.137:6803/1195724024","cluster_addr":"172.21.15.137:6805/1195724024","heartbeat_back_addr":"172.21.15.137:6809/1195724024","heartbeat_front_addr":"172.21.15.137:6807/1195724024","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6811","nonce":4277122690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6813","nonce":4277122690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6817","nonce":4277122690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6815","nonce":4277122690}]},"public_addr":"172.21.15.137:6811/4277122690","cluster_addr":"172.21.15.137:6813/4277122690","heartbeat_back_addr":"172.21.15.137:6817/4277122690","heartbeat_front_addr":"172.21.15.137:6815/4277122690","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6819","nonce":2443099131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6821","nonce":2443099131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6825","nonce":2443099131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6823","nonce":2443099131}]},"public_addr":"172.21.15.137:6819/2443099131","cluster_addr":"172.21.15.137:6821/2443099131","heartbeat_back_addr":"172.21.15.137:6825/2443099131","heartbeat_front_addr":"172.21.15.137:6823/2443099131","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6801","nonce":3385845597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6803","nonce":3385845597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6807","nonce":3385845597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6805","nonce":3385845597}]},"public_addr":"172.21.15.142:6801/3385845597","cluster_addr":"172.21.15.142:6803/3385845597","heartbeat_back_addr":"172.21.15.142:6807/3385845597","heartbeat_front_addr":"172.21.15.142:6805/3385845597","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6809","nonce":3568534083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6811","nonce":3568534083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6815","nonce":3568534083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6813","nonce":3568534083}]},"public_addr":"172.21.15.142:6809/3568534083","cluster_addr":"172.21.15.142:6811/3568534083","heartbeat_back_addr":"172.21.15.142:6815/3568534083","heartbeat_front_addr":"172.21.15.142:6813/3568534083","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6817","nonce":3801228766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6819","nonce":3801228766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6823","nonce":3801228766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6821","nonce":3801228766}]},"public_addr":"172.21.15.142:6817/3801228766","cluster_addr":"172.21.15.142:6819/3801228766","heartbeat_back_addr":"172.21.15.142:6823/3801228766","heartbeat_front_addr":"172.21.15.142:6821/3801228766","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 msgr2=0x7f82d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 0x7f82d406e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f80 tx=0x7f82e00060c0 comp rx=0 tx=0).stop 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 msgr2=0x7f82e8196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82d000d900 tx=0x7f82d000dc10 comp rx=0 tx=0).stop 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 shutdown_connections 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f82d406c4e0 0x7f82d406e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e8100dc0 0x7f82e8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 --2- 172.21.15.137:0/4116294287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82e8101770 0x7f82e8196bb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 >> 172.21.15.137:0/4116294287 conn(0x7f82e80fc650 msgr2=0x7f82e80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 shutdown_connections 2024-06-15T01:03:09.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:09.068+0000 7f82ec8a3700 1 -- 172.21.15.137:0/4116294287 wait complete. 2024-06-15T01:03:09.599 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-15T01:01:49.347551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-15T01:03:09.599 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-15T01:03:09.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:09 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/4116294287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:09.803 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:09.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:09 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/4116294287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:10.482 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:10 smithi137 ceph-mon[29354]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:10.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:10 smithi142 ceph-mon[37379]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:11.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- 172.21.15.137:0/3700308411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc100dc0 msgr2=0x7fe4fc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:11.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 --2- 172.21.15.137:0/3700308411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1011a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009910 tx=0x7fe4ec009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:11.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- 172.21.15.137:0/3700308411 shutdown_connections 2024-06-15T01:03:11.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 --2- 172.21.15.137:0/3700308411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc101770 0x7fe4fc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 --2- 172.21.15.137:0/3700308411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1011a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- 172.21.15.137:0/3700308411 >> 172.21.15.137:0/3700308411 conn(0x7fe4fc0fc650 msgr2=0x7fe4fc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:11.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- 172.21.15.137:0/3700308411 shutdown_connections 2024-06-15T01:03:11.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- 172.21.15.137:0/3700308411 wait complete. 2024-06-15T01:03:11.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 Processor -- start 2024-06-15T01:03:11.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.401+0000 7fe50407b700 1 -- start start 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc197270 con 0x7fe4fc101770 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc199fa0 con 0x7fe4fc100dc0 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55744/0 (socket says 172.21.15.137:55744) 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 -- 172.21.15.137:0/221581514 learned_addr learned my addr 172.21.15.137:0/221581514 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501e17700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 -- 172.21.15.137:0/221581514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 msgr2=0x7fe4fc1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 -- 172.21.15.137:0/221581514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0095f0 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501e17700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe501616700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800d8d0 tx=0x7fe4f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f8004a00 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe4f8005020 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f8009bd0 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc19a280 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.402+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc19a780 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.403+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc109160 con 0x7fe4fc101770 2024-06-15T01:03:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.404+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe4f8005190 con 0x7fe4fc101770 2024-06-15T01:03:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.404+0000 7fe4f2ffd700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 0x7fe4e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.404+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe4f808ae30 con 0x7fe4fc101770 2024-06-15T01:03:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.404+0000 7fe501e17700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 0x7fe4e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.405+0000 7fe501e17700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 0x7fe4e806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b5c0 tx=0x7fe4ec01a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:11.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.409+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe4f8055950 con 0x7fe4fc101770 2024-06-15T01:03:11.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.596+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4fc19aab0 con 0x7fe4fc101770 2024-06-15T01:03:11.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.596+0000 7fe4f2ffd700 1 -- 172.21.15.137:0/221581514 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4f8017020 con 0x7fe4fc101770 2024-06-15T01:03:11.597 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 msgr2=0x7fe4e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 0x7fe4e806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b5c0 tx=0x7fe4ec01a040 comp rx=0 tx=0).stop 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 msgr2=0x7fe4fc196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800d8d0 tx=0x7fe4f800dc90 comp rx=0 tx=0).stop 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 shutdown_connections 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe4e806c4e0 0x7fe4e806e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4fc100dc0 0x7fe4fc1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 --2- 172.21.15.137:0/221581514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc101770 0x7fe4fc196be0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:11.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 >> 172.21.15.137:0/221581514 conn(0x7fe4fc0fc650 msgr2=0x7fe4fc0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:11.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 shutdown_connections 2024-06-15T01:03:11.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:11.598+0000 7fe50407b700 1 -- 172.21.15.137:0/221581514 wait complete. 2024-06-15T01:03:12.182 INFO:tasks.cephadm:Setting up client nodes... 2024-06-15T01:03:12.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-15T01:03:12.392 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:12.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:12 smithi137 ceph-mon[29354]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:12.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:12 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/221581514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:03:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:12 smithi142 ceph-mon[37379]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:12.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:12 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/221581514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:03:13.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 -- 172.21.15.137:0/2774248353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 msgr2=0x7febcc103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 --2- 172.21.15.137:0/2774248353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc103ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7febc8009910 tx=0x7febc8009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 -- 172.21.15.137:0/2774248353 shutdown_connections 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 --2- 172.21.15.137:0/2774248353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 --2- 172.21.15.137:0/2774248353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc103ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 -- 172.21.15.137:0/2774248353 >> 172.21.15.137:0/2774248353 conn(0x7febcc0fb050 msgr2=0x7febcc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 -- 172.21.15.137:0/2774248353 shutdown_connections 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.910+0000 7febd330b700 1 -- 172.21.15.137:0/2774248353 wait complete. 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 Processor -- start 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 -- start start 2024-06-15T01:03:13.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc196da0 con 0x7febcc1016e0 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd330b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc196ee0 con 0x7febcc104010 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55768/0 (socket says 172.21.15.137:55768) 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 -- 172.21.15.137:0/2756349906 learned_addr learned my addr 172.21.15.137:0/2756349906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd08a6700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 -- 172.21.15.137:0/2756349906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 msgr2=0x7febcc196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 -- 172.21.15.137:0/2756349906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80095f0 con 0x7febcc1016e0 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd08a6700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:03:13.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.911+0000 7febd10a7700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:13.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.912+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc801d070 con 0x7febcc1016e0 2024-06-15T01:03:13.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.912+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7febc8022470 con 0x7febcc1016e0 2024-06-15T01:03:13.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.912+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc800f690 con 0x7febcc1016e0 2024-06-15T01:03:13.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.912+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc19b930 con 0x7febcc1016e0 2024-06-15T01:03:13.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.912+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc19bea0 con 0x7febcc1016e0 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.913+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc190400 con 0x7febcc1016e0 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.913+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7febc8022980 con 0x7febcc1016e0 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.914+0000 7febc27fc700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 0x7febb806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.914+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7febc808cd20 con 0x7febcc1016e0 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.914+0000 7febd08a6700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 0x7febb806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:13.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.914+0000 7febd08a6700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 0x7febb806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febbc005950 tx=0x7febbc005f50 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:13.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:13.918+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7febc805b3f0 con 0x7febcc1016e0 2024-06-15T01:03:14.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.188+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7febcc02cc70 con 0x7febcc1016e0 2024-06-15T01:03:14.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.192+0000 7febc27fc700 1 -- 172.21.15.137:0/2756349906 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7febc8027020 con 0x7febcc1016e0 2024-06-15T01:03:14.193 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-06-15T01:03:14.193 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBS6GxmsJpVCxAAPN/VZ09U2rkPk0Pll6UuUw== 2024-06-15T01:03:14.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.194+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 msgr2=0x7febb806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:14.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.194+0000 7febd330b700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 0x7febb806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febbc005950 tx=0x7febbc005f50 comp rx=0 tx=0).stop 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.194+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 msgr2=0x7febcc196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.194+0000 7febd330b700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc800be20 comp rx=0 tx=0).stop 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 shutdown_connections 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7febb806c4e0 0x7febb806e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febcc1016e0 0x7febcc196240 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 --2- 172.21.15.137:0/2756349906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febcc104010 0x7febcc196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 >> 172.21.15.137:0/2756349906 conn(0x7febcc0fb050 msgr2=0x7febcc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 shutdown_connections 2024-06-15T01:03:14.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:14.195+0000 7febd330b700 1 -- 172.21.15.137:0/2756349906 wait complete. 2024-06-15T01:03:14.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:14 smithi137 ceph-mon[29354]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:14.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:14 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2756349906' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:14.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:14 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2756349906' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-15T01:03:14.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:14 smithi142 ceph-mon[37379]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:14.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:14 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2756349906' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:14.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:14 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2756349906' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-15T01:03:14.834 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:03:14.834 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-15T01:03:14.835 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-15T01:03:14.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-15T01:03:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:03:16.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 -- 172.21.15.142:0/4120765587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ffab0 msgr2=0x7fd6a00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 --2- 172.21.15.142:0/4120765587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ffab0 0x7fd6a00ffed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009910 tx=0x7fd69c009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 -- 172.21.15.142:0/4120765587 shutdown_connections 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 --2- 172.21.15.142:0/4120765587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a0100cb0 0x7fd6a0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 --2- 172.21.15.142:0/4120765587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ffab0 0x7fd6a00ffed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 -- 172.21.15.142:0/4120765587 >> 172.21.15.142:0/4120765587 conn(0x7fd6a00fb030 msgr2=0x7fd6a00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 -- 172.21.15.142:0/4120765587 shutdown_connections 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.461+0000 7fd6a8563700 1 -- 172.21.15.142:0/4120765587 wait complete. 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 Processor -- start 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 -- start start 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00ffab0 0x7fd6a0068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a019b5a0 con 0x7fd6a00ffab0 2024-06-15T01:03:16.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a8563700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a019b6e0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45642/0 (socket says 172.21.15.142:45642) 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 -- 172.21.15.142:0/3696378547 learned_addr learned my addr 172.21.15.142:0/3696378547 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a62ff700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00ffab0 0x7fd6a0068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 -- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00ffab0 msgr2=0x7fd6a0068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00ffab0 0x7fd6a0068f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 -- 172.21.15.142:0/3696378547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0095f0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.462+0000 7fd6a5afe700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd69400c8d0 tx=0x7fd69400cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.463+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6940078d0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.463+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a019b9c0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.463+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd69400f450 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.463+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69400e5e0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.463+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a019bf90 con 0x7fd6a0100cb0 2024-06-15T01:03:16.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.464+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a01084d0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.464+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd69400f5c0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.464+0000 7fd6937fe700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 0x7fd68c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:16.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.465+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd69408b0f0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.465+0000 7fd6a62ff700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 0x7fd68c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:16.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.465+0000 7fd6a62ff700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 0x7fd68c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c0051b0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:16.469 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.469+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd6940596a0 con 0x7fd6a0100cb0 2024-06-15T01:03:16.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:16 smithi142 ceph-mon[37379]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:16.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:16 smithi137 ceph-mon[29354]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:16.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.740+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd6a0108690 con 0x7fd6a0100cb0 2024-06-15T01:03:16.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.745+0000 7fd6937fe700 1 -- 172.21.15.142:0/3696378547 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd694059230 con 0x7fd6a0100cb0 2024-06-15T01:03:16.746 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-06-15T01:03:16.746 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBU6GxmY8k/LBAA8+Jolj1EVGerBwRGl4ZvDw== 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 msgr2=0x7fd68c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 0x7fd68c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c0051b0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 msgr2=0x7fd6a019afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd69400c8d0 tx=0x7fd69400cbe0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 shutdown_connections 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd68c06c4e0 0x7fd68c06e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00ffab0 0x7fd6a0068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 --2- 172.21.15.142:0/3696378547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a0100cb0 0x7fd6a019afd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 >> 172.21.15.142:0/3696378547 conn(0x7fd6a00fb030 msgr2=0x7fd6a0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 shutdown_connections 2024-06-15T01:03:16.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-06-15T01:03:16.747+0000 7fd6a8563700 1 -- 172.21.15.142:0/3696378547 wait complete. 2024-06-15T01:03:17.368 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T01:03:17.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-15T01:03:17.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-15T01:03:17.406 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-15T01:03:17.406 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-15T01:03:17.406 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph mgr dump --format=json 2024-06-15T01:03:17.624 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:17.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:17 smithi137 ceph-mon[29354]: from='client.? 172.21.15.142:0/3696378547' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:17.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:17 smithi137 ceph-mon[29354]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:17.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:17 smithi137 ceph-mon[29354]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-15T01:03:17.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:17 smithi142 ceph-mon[37379]: from='client.? 172.21.15.142:0/3696378547' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:17.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:17 smithi142 ceph-mon[37379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-15T01:03:17.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:17 smithi142 ceph-mon[37379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-15T01:03:18.644 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:18 smithi137 ceph-mon[29354]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:18.644 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:18.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:18 smithi142 ceph-mon[37379]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:18.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 -- 172.21.15.137:0/1742792686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 msgr2=0x7f8158100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 --2- 172.21.15.137:0/1742792686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158100050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f814c009910 tx=0x7f814c009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 -- 172.21.15.137:0/1742792686 shutdown_connections 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 --2- 172.21.15.137:0/1742792686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158100050 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 --2- 172.21.15.137:0/1742792686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 0x7f8158105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 -- 172.21.15.137:0/1742792686 >> 172.21.15.137:0/1742792686 conn(0x7f81580755a0 msgr2=0x7f81580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 -- 172.21.15.137:0/1742792686 shutdown_connections 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.131+0000 7f815fac3700 1 -- 172.21.15.137:0/1742792686 wait complete. 2024-06-15T01:03:19.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 Processor -- start 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 -- start start 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 0x7f8158195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8158196160 con 0x7f81580ffbd0 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815fac3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8158199ef0 con 0x7f8158105a40 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815d85f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815d85f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32900/0 (socket says 172.21.15.137:32900) 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815d85f700 1 -- 172.21.15.137:0/523154432 learned_addr learned my addr 172.21.15.137:0/523154432 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.132+0000 7f815d05e700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 0x7f8158195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815d85f700 1 -- 172.21.15.137:0/523154432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 msgr2=0x7f8158195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815d85f700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 0x7f8158195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815d85f700 1 -- 172.21.15.137:0/523154432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0095f0 con 0x7f81580ffbd0 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815d85f700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f815000c8a0 tx=0x7f815000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:19.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81500078e0 con 0x7f81580ffbd0 2024-06-15T01:03:19.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f815000f450 con 0x7f81580ffbd0 2024-06-15T01:03:19.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815000e5e0 con 0x7f81580ffbd0 2024-06-15T01:03:19.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815819a1d0 con 0x7f81580ffbd0 2024-06-15T01:03:19.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.133+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815819a720 con 0x7f81580ffbd0 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.134+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158107fd0 con 0x7f81580ffbd0 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.135+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f815000f5c0 con 0x7f81580ffbd0 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.135+0000 7f8156ffd700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 0x7f814006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.135+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f815008b0f0 con 0x7f81580ffbd0 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.135+0000 7f815d05e700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 0x7f814006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:19.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.136+0000 7f815d05e700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 0x7f814006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8158196b60 tx=0x7f814c01a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.140+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8150055c10 con 0x7f81580ffbd0 2024-06-15T01:03:19.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.386+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f815804ea90 con 0x7f81580ffbd0 2024-06-15T01:03:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.389+0000 7f8156ffd700 1 -- 172.21.15.137:0/523154432 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173044 (secure 0 0 0) 0x7f8150059230 con 0x7f81580ffbd0 2024-06-15T01:03:19.392 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 msgr2=0x7f814006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 0x7f814006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8158196b60 tx=0x7f814c01a040 comp rx=0 tx=0).stop 2024-06-15T01:03:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 msgr2=0x7f8158195540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f815000c8a0 tx=0x7f815000cbb0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 shutdown_connections 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f814006c4e0 0x7f814006e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81580ffbd0 0x7f8158195540 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 --2- 172.21.15.137:0/523154432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8158105a40 0x7f8158195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 >> 172.21.15.137:0/523154432 conn(0x7f81580755a0 msgr2=0x7f81580fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 shutdown_connections 2024-06-15T01:03:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:19.392+0000 7f815fac3700 1 -- 172.21.15.137:0/523154432 wait complete. 2024-06-15T01:03:19.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:19 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/523154432' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-15T01:03:19.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:19 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/523154432' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-15T01:03:20.092 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":19,"active_gid":14219,"active_name":"smithi137.nbzbyc","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3216905676},{"type":"v1","addr":"172.21.15.137:6801","nonce":3216905676}]},"active_addr":"172.21.15.137:6801/3216905676","active_change":"2024-06-15T00:59:33.029511+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi142.dllryb","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.137:8443/","prometheus":"http://172.21.15.137:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":388348183}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":146013243}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":72877675}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":3277128014}]}]} 2024-06-15T01:03:20.096 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-15T01:03:20.096 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-15T01:03:20.096 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-15T01:03:20.305 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:20.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:20 smithi137 ceph-mon[29354]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:20 smithi142 ceph-mon[37379]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 -- 172.21.15.137:0/764289984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c1026f0 msgr2=0x7f094c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 --2- 172.21.15.137:0/764289984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c1026f0 0x7f094c10abe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0940009910 tx=0x7f0940009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 -- 172.21.15.137:0/764289984 shutdown_connections 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 --2- 172.21.15.137:0/764289984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c1026f0 0x7f094c10abe0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 --2- 172.21.15.137:0/764289984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c101dd0 0x7f094c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 -- 172.21.15.137:0/764289984 >> 172.21.15.137:0/764289984 conn(0x7f094c0fb540 msgr2=0x7f094c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 -- 172.21.15.137:0/764289984 shutdown_connections 2024-06-15T01:03:22.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.008+0000 7f0950bbf700 1 -- 172.21.15.137:0/764289984 wait complete. 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 Processor -- start 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 -- start start 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c1026f0 0x7f094c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c197270 con 0x7f094c101dd0 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f0950bbf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c199fa0 con 0x7f094c1026f0 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f094a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f094a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32922/0 (socket says 172.21.15.137:32922) 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.009+0000 7f094a59c700 1 -- 172.21.15.137:0/213773730 learned_addr learned my addr 172.21.15.137:0/213773730 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f0949d9b700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c1026f0 0x7f094c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f094a59c700 1 -- 172.21.15.137:0/213773730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c1026f0 msgr2=0x7f094c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f094a59c700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c1026f0 0x7f094c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f094a59c700 1 -- 172.21.15.137:0/213773730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400095f0 con 0x7f094c101dd0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f094a59c700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e920 tx=0x7f093c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c00c9d0 con 0x7f094c101dd0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c19a280 con 0x7f094c101dd0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f093c010430 con 0x7f094c101dd0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c018630 con 0x7f094c101dd0 2024-06-15T01:03:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.010+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c19a7a0 con 0x7f094c101dd0 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.011+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c1082e0 con 0x7f094c101dd0 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.012+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f093c0105a0 con 0x7f094c101dd0 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.012+0000 7f093b7fe700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 0x7f093406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.012+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f093c014070 con 0x7f094c101dd0 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.012+0000 7f0949d9b700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 0x7f093406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.013+0000 7f0949d9b700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 0x7f093406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0940009fd0 tx=0x7f0940005940 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:22.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.017+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f093c0520e0 con 0x7f094c101dd0 2024-06-15T01:03:22.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.216+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f094c04ea90 con 0x7f094c101dd0 2024-06-15T01:03:22.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.217+0000 7f093b7fe700 1 -- 172.21.15.137:0/213773730 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11399 (secure 0 0 0) 0x7f093c0562d0 con 0x7f094c101dd0 2024-06-15T01:03:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:22.218 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":41,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:03:03.946724+0000","last_up_change":"2024-06-15T01:03:02.945497+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6803","nonce":1195724024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6805","nonce":1195724024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6809","nonce":1195724024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6807","nonce":1195724024}]},"public_addr":"172.21.15.137:6803/1195724024","cluster_addr":"172.21.15.137:6805/1195724024","heartbeat_back_addr":"172.21.15.137:6809/1195724024","heartbeat_front_addr":"172.21.15.137:6807/1195724024","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6811","nonce":4277122690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6813","nonce":4277122690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6817","nonce":4277122690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6815","nonce":4277122690}]},"public_addr":"172.21.15.137:6811/4277122690","cluster_addr":"172.21.15.137:6813/4277122690","heartbeat_back_addr":"172.21.15.137:6817/4277122690","heartbeat_front_addr":"172.21.15.137:6815/4277122690","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6819","nonce":2443099131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6821","nonce":2443099131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6825","nonce":2443099131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6823","nonce":2443099131}]},"public_addr":"172.21.15.137:6819/2443099131","cluster_addr":"172.21.15.137:6821/2443099131","heartbeat_back_addr":"172.21.15.137:6825/2443099131","heartbeat_front_addr":"172.21.15.137:6823/2443099131","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6801","nonce":3385845597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6803","nonce":3385845597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6807","nonce":3385845597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6805","nonce":3385845597}]},"public_addr":"172.21.15.142:6801/3385845597","cluster_addr":"172.21.15.142:6803/3385845597","heartbeat_back_addr":"172.21.15.142:6807/3385845597","heartbeat_front_addr":"172.21.15.142:6805/3385845597","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6809","nonce":3568534083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6811","nonce":3568534083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6815","nonce":3568534083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6813","nonce":3568534083}]},"public_addr":"172.21.15.142:6809/3568534083","cluster_addr":"172.21.15.142:6811/3568534083","heartbeat_back_addr":"172.21.15.142:6815/3568534083","heartbeat_front_addr":"172.21.15.142:6813/3568534083","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6817","nonce":3801228766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6819","nonce":3801228766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6823","nonce":3801228766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6821","nonce":3801228766}]},"public_addr":"172.21.15.142:6817/3801228766","cluster_addr":"172.21.15.142:6819/3801228766","heartbeat_back_addr":"172.21.15.142:6823/3801228766","heartbeat_front_addr":"172.21.15.142:6821/3801228766","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:03:22.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.219+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 msgr2=0x7f093406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:22.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.219+0000 7f0950bbf700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 0x7f093406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0940009fd0 tx=0x7f0940005940 comp rx=0 tx=0).stop 2024-06-15T01:03:22.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.219+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 msgr2=0x7f094c1966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.219+0000 7f0950bbf700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e920 tx=0x7f093c00ec30 comp rx=0 tx=0).stop 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 shutdown_connections 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f093406c4e0 0x7f093406e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f094c101dd0 0x7f094c1966a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 --2- 172.21.15.137:0/213773730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f094c1026f0 0x7f094c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 >> 172.21.15.137:0/213773730 conn(0x7f094c0fb540 msgr2=0x7f094c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 shutdown_connections 2024-06-15T01:03:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:22.220+0000 7f0950bbf700 1 -- 172.21.15.137:0/213773730 wait complete. 2024-06-15T01:03:22.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:22 smithi137 ceph-mon[29354]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:22.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:22 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/213773730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:22.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:22 smithi142 ceph-mon[37379]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:22.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:22 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/213773730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:22.955 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-15T01:03:22.955 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-15T01:03:23.162 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:24.630 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:24 smithi137 ceph-mon[29354]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:24.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:24 smithi142 ceph-mon[37379]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 -- 172.21.15.137:0/1231260790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758100dc0 msgr2=0x7ff7581011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 --2- 172.21.15.137:0/1231260790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758100dc0 0x7ff7581011a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff748009910 tx=0x7ff748009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 -- 172.21.15.137:0/1231260790 shutdown_connections 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 --2- 172.21.15.137:0/1231260790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758101770 0x7ff7581057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 --2- 172.21.15.137:0/1231260790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758100dc0 0x7ff7581011a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 -- 172.21.15.137:0/1231260790 >> 172.21.15.137:0/1231260790 conn(0x7ff7580fc650 msgr2=0x7ff7580fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 -- 172.21.15.137:0/1231260790 shutdown_connections 2024-06-15T01:03:24.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.872+0000 7ff75d328700 1 -- 172.21.15.137:0/1231260790 wait complete. 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 Processor -- start 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 -- start start 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 0x7ff75819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75819d140 con 0x7ff758101770 2024-06-15T01:03:24.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff75d328700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581965f0 con 0x7ff758100dc0 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff74ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff74ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32942/0 (socket says 172.21.15.137:32942) 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.873+0000 7ff74ffff700 1 -- 172.21.15.137:0/763616490 learned_addr learned my addr 172.21.15.137:0/763616490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff756ffd700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 0x7ff75819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff74ffff700 1 -- 172.21.15.137:0/763616490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 msgr2=0x7ff75819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff74ffff700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 0x7ff75819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff74ffff700 1 -- 172.21.15.137:0/763616490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480095f0 con 0x7ff758101770 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff756ffd700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 0x7ff75819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff74ffff700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b480 tx=0x7ff74000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7400118c0 con 0x7ff758101770 2024-06-15T01:03:24.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff740010380 con 0x7ff758101770 2024-06-15T01:03:24.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7581968d0 con 0x7ff758101770 2024-06-15T01:03:24.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74000f510 con 0x7ff758101770 2024-06-15T01:03:24.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.874+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff758196e20 con 0x7ff758101770 2024-06-15T01:03:24.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.875+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758109160 con 0x7ff758101770 2024-06-15T01:03:24.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.875+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff7400104f0 con 0x7ff758101770 2024-06-15T01:03:24.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.876+0000 7ff754ff9700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 0x7ff74406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:24.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.876+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff74008b130 con 0x7ff758101770 2024-06-15T01:03:24.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.876+0000 7ff756ffd700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 0x7ff74406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:24.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.876+0000 7ff756ffd700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 0x7ff74406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b5c0 tx=0x7ff748005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:24.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:24.880+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff7400596e0 con 0x7ff758101770 2024-06-15T01:03:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.070+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff75804ea90 con 0x7ff758101770 2024-06-15T01:03:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.071+0000 7ff754ff9700 1 -- 172.21.15.137:0/763616490 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11399 (secure 0 0 0) 0x7ff7400107a0 con 0x7ff758101770 2024-06-15T01:03:25.072 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:25.072 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":41,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:03:03.946724+0000","last_up_change":"2024-06-15T01:03:02.945497+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6803","nonce":1195724024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6805","nonce":1195724024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6809","nonce":1195724024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6807","nonce":1195724024}]},"public_addr":"172.21.15.137:6803/1195724024","cluster_addr":"172.21.15.137:6805/1195724024","heartbeat_back_addr":"172.21.15.137:6809/1195724024","heartbeat_front_addr":"172.21.15.137:6807/1195724024","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6811","nonce":4277122690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6813","nonce":4277122690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6817","nonce":4277122690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6815","nonce":4277122690}]},"public_addr":"172.21.15.137:6811/4277122690","cluster_addr":"172.21.15.137:6813/4277122690","heartbeat_back_addr":"172.21.15.137:6817/4277122690","heartbeat_front_addr":"172.21.15.137:6815/4277122690","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6819","nonce":2443099131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6821","nonce":2443099131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6825","nonce":2443099131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6823","nonce":2443099131}]},"public_addr":"172.21.15.137:6819/2443099131","cluster_addr":"172.21.15.137:6821/2443099131","heartbeat_back_addr":"172.21.15.137:6825/2443099131","heartbeat_front_addr":"172.21.15.137:6823/2443099131","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6801","nonce":3385845597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6803","nonce":3385845597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6807","nonce":3385845597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6805","nonce":3385845597}]},"public_addr":"172.21.15.142:6801/3385845597","cluster_addr":"172.21.15.142:6803/3385845597","heartbeat_back_addr":"172.21.15.142:6807/3385845597","heartbeat_front_addr":"172.21.15.142:6805/3385845597","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6809","nonce":3568534083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6811","nonce":3568534083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6815","nonce":3568534083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6813","nonce":3568534083}]},"public_addr":"172.21.15.142:6809/3568534083","cluster_addr":"172.21.15.142:6811/3568534083","heartbeat_back_addr":"172.21.15.142:6815/3568534083","heartbeat_front_addr":"172.21.15.142:6813/3568534083","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6817","nonce":3801228766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6819","nonce":3801228766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6823","nonce":3801228766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6821","nonce":3801228766}]},"public_addr":"172.21.15.142:6817/3801228766","cluster_addr":"172.21.15.142:6819/3801228766","heartbeat_back_addr":"172.21.15.142:6823/3801228766","heartbeat_front_addr":"172.21.15.142:6821/3801228766","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:03:25.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 msgr2=0x7ff74406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:25.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 0x7ff74406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b5c0 tx=0x7ff748005960 comp rx=0 tx=0).stop 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 msgr2=0x7ff75819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b480 tx=0x7ff74000b840 comp rx=0 tx=0).stop 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 shutdown_connections 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff74406c4e0 0x7ff74406e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff758100dc0 0x7ff75819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 --2- 172.21.15.137:0/763616490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff758101770 0x7ff75819cab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 >> 172.21.15.137:0/763616490 conn(0x7ff7580fc650 msgr2=0x7ff7580fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 shutdown_connections 2024-06-15T01:03:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:25.073+0000 7ff75d328700 1 -- 172.21.15.137:0/763616490 wait complete. 2024-06-15T01:03:25.526 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:25 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/763616490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:25.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-15T01:03:25.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-15T01:03:25.786 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-15T01:03:25.786 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-15T01:03:25.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-15T01:03:25.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-15T01:03:25.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:25 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/763616490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:03:26.040 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:26.041 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:26.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:26 smithi137 ceph-mon[29354]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:26.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:26 smithi142 ceph-mon[37379]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:26.951 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:26.952 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:28.614 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:28 smithi137 ceph-mon[29354]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:28.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:28 smithi142 ceph-mon[37379]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:28.845 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:29.143 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:30.625 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:30 smithi137 ceph-mon[29354]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:30.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:30 smithi142 ceph-mon[37379]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:32.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:32 smithi137 ceph-mon[29354]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:32.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:32 smithi142 ceph-mon[37379]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3568283511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 msgr2=0x7f5e9c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3568283511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c1020e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009910 tx=0x7f5e8c009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3568283511 shutdown_connections 2024-06-15T01:03:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3568283511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3568283511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c1020e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3568283511 >> 172.21.15.137:0/3568283511 conn(0x7f5e9c0fb570 msgr2=0x7f5e9c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3568283511 shutdown_connections 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.954+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3568283511 wait complete. 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.955+0000 7f5ea27a9700 1 Processor -- start 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.955+0000 7f5ea27a9700 1 -- start start 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5ea27a9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5ea27a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5ea27a9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c197250 con 0x7f5e9c101d00 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5ea27a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c199f80 con 0x7f5e9c102620 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34058/0 (socket says 172.21.15.137:34058) 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 -- 172.21.15.137:0/3223735344 learned_addr learned my addr 172.21.15.137:0/3223735344 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9b7fe700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 -- 172.21.15.137:0/3223735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 msgr2=0x7f5e9c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0095f0 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9b7fe700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e9bfff700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8c01d070 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5e8c0044b0 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c19a200 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.956+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8c003a70 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.957+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c19a690 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.957+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e9c10df40 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.958+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5e8c003cb0 con 0x7f5e9c101d00 2024-06-15T01:03:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.958+0000 7f5e997fa700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 0x7f5e8806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.958+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5e8c08cb70 con 0x7f5e9c101d00 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5e9b7fe700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 0x7f5e8806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5e997fa700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 0x7f5e880745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e88074c90 con 0x7f5e880721c0 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e8c0912a0 con 0x7f5e9c101d00 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5ea0d46700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 0x7f5e880745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5e9b7fe700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 0x7f5e8806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c197ca0 tx=0x7f5e84009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.959+0000 7f5ea0d46700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 0x7f5e880745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.960+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== osd.1 v2:172.21.15.137:6810/4277122690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e88074c90 con 0x7f5e880721c0 2024-06-15T01:03:32.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 --> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e9c04ea90 con 0x7f5e880721c0 2024-06-15T01:03:32.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5e997fa700 1 -- 172.21.15.137:0/3223735344 <== osd.1 v2:172.21.15.137:6810/4277122690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e9c04ea90 con 0x7f5e880721c0 2024-06-15T01:03:32.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 msgr2=0x7f5e880745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:32.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 0x7f5e880745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 msgr2=0x7f5e8806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 0x7f5e8806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c197ca0 tx=0x7f5e84009260 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 msgr2=0x7f5e9c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.978+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c00be90 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 shutdown_connections 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6810/4277122690,v1:172.21.15.137:6811/4277122690] conn(0x7f5e880721c0 0x7f5e880745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f5e8806c4e0 0x7f5e8806e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e9c101d00 0x7f5e9c196680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 --2- 172.21.15.137:0/3223735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e9c102620 0x7f5e9c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 >> 172.21.15.137:0/3223735344 conn(0x7f5e9c0fb570 msgr2=0x7f5e9c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 shutdown_connections 2024-06-15T01:03:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:32.979+0000 7f5ea27a9700 1 -- 172.21.15.137:0/3223735344 wait complete. 2024-06-15T01:03:33.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:33.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- 172.21.15.137:0/4176626379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218100da0 msgr2=0x7f4218101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 --2- 172.21.15.137:0/4176626379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218100da0 0x7f4218101180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4208009910 tx=0x7f4208009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- 172.21.15.137:0/4176626379 shutdown_connections 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 --2- 172.21.15.137:0/4176626379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218101750 0x7f42181057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 --2- 172.21.15.137:0/4176626379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218100da0 0x7f4218101180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- 172.21.15.137:0/4176626379 >> 172.21.15.137:0/4176626379 conn(0x7f42180fc630 msgr2=0x7f42180fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- 172.21.15.137:0/4176626379 shutdown_connections 2024-06-15T01:03:34.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- 172.21.15.137:0/4176626379 wait complete. 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 Processor -- start 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.223+0000 7f421da15700 1 -- start start 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f421da15700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218100da0 0x7f4218196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f421da15700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f421da15700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218197260 con 0x7f4218101750 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f421da15700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218199f90 con 0x7f4218100da0 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34076/0 (socket says 172.21.15.137:34076) 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 -- 172.21.15.137:0/3753215974 learned_addr learned my addr 172.21.15.137:0/3753215974 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f4216ffd700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218100da0 0x7f4218196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 -- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218100da0 msgr2=0x7f4218196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218100da0 0x7f4218196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080095f0 con 0x7f4218101750 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.224+0000 7f420e7fc700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f420000c8f0 tx=0x7f420000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42000078a0 con 0x7f4218101750 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421819a270 con 0x7f4218101750 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4200007e90 con 0x7f4218101750 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4200018630 con 0x7f4218101750 2024-06-15T01:03:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421819a7c0 con 0x7f4218101750 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.225+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f421810df20 con 0x7f4218101750 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.226+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f420000f450 con 0x7f4218101750 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f421ca13700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 0x7f420406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f4216ffd700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 0x7f420406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f420008b980 con 0x7f4218101750 2024-06-15T01:03:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f421ca13700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 0x7f42040745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4204074c90 con 0x7f42040721c0 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f420008bd40 con 0x7f4218101750 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f4216ffd700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 0x7f420406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f420801a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f42177fe700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 0x7f42040745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.227+0000 7f42177fe700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 0x7f42040745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.228+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== osd.3 v2:172.21.15.142:6800/3385845597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4204074c90 con 0x7f42040721c0 2024-06-15T01:03:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.243+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 --> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f421804ea90 con 0x7f42040721c0 2024-06-15T01:03:34.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421ca13700 1 -- 172.21.15.137:0/3753215974 <== osd.3 v2:172.21.15.142:6800/3385845597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f421804ea90 con 0x7f42040721c0 2024-06-15T01:03:34.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 msgr2=0x7f42040745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:34.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 0x7f42040745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 msgr2=0x7f420406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 0x7f420406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f420801a040 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 msgr2=0x7f4218196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f420000c8f0 tx=0x7f420000cc00 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 shutdown_connections 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:6800/3385845597,v1:172.21.15.142:6801/3385845597] conn(0x7f42040721c0 0x7f42040745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f420406c4e0 0x7f420406e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4218100da0 0x7f4218196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 --2- 172.21.15.137:0/3753215974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218101750 0x7f4218196bd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 >> 172.21.15.137:0/3753215974 conn(0x7f42180fc630 msgr2=0x7f42180fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 shutdown_connections 2024-06-15T01:03:34.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:34.244+0000 7f421da15700 1 -- 172.21.15.137:0/3753215974 wait complete. 2024-06-15T01:03:34.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:34 smithi137 ceph-mon[29354]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:34.738 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:34 smithi142 ceph-mon[37379]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:35.133 INFO:teuthology.orchestra.run.smithi137.stdout:115964117012 2024-06-15T01:03:35.133 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-15T01:03:35.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- 172.21.15.137:0/1718119028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 msgr2=0x7fbab81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 --2- 172.21.15.137:0/1718119028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab81057e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009910 tx=0x7fbaac009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:35.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- 172.21.15.137:0/1718119028 shutdown_connections 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 --2- 172.21.15.137:0/1718119028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab81057e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 --2- 172.21.15.137:0/1718119028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- 172.21.15.137:0/1718119028 >> 172.21.15.137:0/1718119028 conn(0x7fbab80fc6b0 msgr2=0x7fbab80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- 172.21.15.137:0/1718119028 shutdown_connections 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- 172.21.15.137:0/1718119028 wait complete. 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 Processor -- start 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.344+0000 7fbabd856700 1 -- start start 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbabd856700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbabd856700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbabd856700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8073b10 con 0x7fbab8101790 2024-06-15T01:03:35.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbabd856700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8073c50 con 0x7fbab8100de0 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34090/0 (socket says 172.21.15.137:34090) 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 -- 172.21.15.137:0/831672062 learned_addr learned my addr 172.21.15.137:0/831672062 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab6ffd700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 -- 172.21.15.137:0/831672062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 msgr2=0x7fbab8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac0095f0 con 0x7fbab8101790 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab6ffd700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.345+0000 7fbab67fc700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0092e0 tx=0x7fbaac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.346+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac01d070 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.346+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbaac00f460 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.346+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab8073ed0 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.346+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac021600 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.346+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab81a2b70 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.347+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbab810df40 con 0x7fbab8101790 2024-06-15T01:03:35.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.347+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbaac003a60 con 0x7fbab8101790 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbabc854700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 0x7fbaa006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbaac08c7d0 con 0x7fbab8101790 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbab6ffd700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 0x7fbaa006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbabc854700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 0x7fbaa00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbaa0074c90 con 0x7fbaa00721c0 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaac0912a0 con 0x7fbab8101790 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbab77fe700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 0x7fbaa00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbab6ffd700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 0x7fbaa006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8005fd0 tx=0x7fbaa8005dc0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.348+0000 7fbab77fe700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 0x7fbaa00745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.349+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== osd.2 v2:172.21.15.137:6818/2443099131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbaa0074c90 con 0x7fbaa00721c0 2024-06-15T01:03:35.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.365+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 --> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbab804ea90 con 0x7fbaa00721c0 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabc854700 1 -- 172.21.15.137:0/831672062 <== osd.2 v2:172.21.15.137:6818/2443099131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbab804ea90 con 0x7fbaa00721c0 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 msgr2=0x7fbaa00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 0x7fbaa00745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 msgr2=0x7fbaa006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 0x7fbaa006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8005fd0 tx=0x7fbaa8005dc0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 msgr2=0x7fbab80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.366+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0092e0 tx=0x7fbaac00be20 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 shutdown_connections 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6818/2443099131,v1:172.21.15.137:6819/2443099131] conn(0x7fbaa00721c0 0x7fbaa00745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbaa006c4e0 0x7fbaa006e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab8100de0 0x7fbab8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 --2- 172.21.15.137:0/831672062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbab8101790 0x7fbab80735d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 >> 172.21.15.137:0/831672062 conn(0x7fbab80fc6b0 msgr2=0x7fbab80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:35.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 shutdown_connections 2024-06-15T01:03:35.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.367+0000 7fbabd856700 1 -- 172.21.15.137:0/831672062 wait complete. 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 -- 172.21.15.137:0/356252538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 msgr2=0x7fe83c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 --2- 172.21.15.137:0/356252538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1045c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009910 tx=0x7fe82c009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 -- 172.21.15.137:0/356252538 shutdown_connections 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 --2- 172.21.15.137:0/356252538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1045c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 --2- 172.21.15.137:0/356252538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 0x7fe83c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 -- 172.21.15.137:0/356252538 >> 172.21.15.137:0/356252538 conn(0x7fe83c074b90 msgr2=0x7fe83c074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 -- 172.21.15.137:0/356252538 shutdown_connections 2024-06-15T01:03:35.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.959+0000 7fe8427a0700 1 -- 172.21.15.137:0/356252538 wait complete. 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 Processor -- start 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 -- start start 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 0x7fe83c1954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1960d0 con 0x7fe83c100eb0 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe8427a0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c199e60 con 0x7fe83c100500 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe840f9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe840f9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34114/0 (socket says 172.21.15.137:34114) 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe840f9d700 1 -- 172.21.15.137:0/2581780215 learned_addr learned my addr 172.21.15.137:0/2581780215 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:35.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.960+0000 7fe84179e700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 0x7fe83c1954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe840f9d700 1 -- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 msgr2=0x7fe83c1954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe840f9d700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 0x7fe83c1954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe840f9d700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0095f0 con 0x7fe83c100eb0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe840f9d700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0092e0 tx=0x7fe82c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82c01d070 con 0x7fe83c100eb0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe82c00f460 con 0x7fe83c100eb0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c19a0e0 con 0x7fe83c100eb0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82c021620 con 0x7fe83c100eb0 2024-06-15T01:03:35.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.961+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c19a5d0 con 0x7fe83c100eb0 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.962+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe83c10cda0 con 0x7fe83c100eb0 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.962+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe82c00f5d0 con 0x7fe83c100eb0 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe8327fc700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 0x7fe82806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe84179e700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 0x7fe82806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe82c08c610 con 0x7fe83c100eb0 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe8327fc700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 0x7fe8280745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:35.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe828074c90 con 0x7fe8280721c0 2024-06-15T01:03:35.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.963+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe82c08ece0 con 0x7fe83c100eb0 2024-06-15T01:03:35.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.964+0000 7fe84179e700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 0x7fe82806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe838009a10 tx=0x7fe838009190 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.964+0000 7fe841f9f700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 0x7fe8280745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:35.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.964+0000 7fe841f9f700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 0x7fe8280745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:35.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.965+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== osd.5 v2:172.21.15.142:6816/3801228766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe828074c90 con 0x7fe8280721c0 2024-06-15T01:03:35.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.992+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 --> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe83c04f2e0 con 0x7fe8280721c0 2024-06-15T01:03:35.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.992+0000 7fe8327fc700 1 -- 172.21.15.137:0/2581780215 <== osd.5 v2:172.21.15.142:6816/3801228766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe83c04f2e0 con 0x7fe8280721c0 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 msgr2=0x7fe8280745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 0x7fe8280745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 msgr2=0x7fe82806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 0x7fe82806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe838009a10 tx=0x7fe838009190 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 msgr2=0x7fe83c1959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0092e0 tx=0x7fe82c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 shutdown_connections 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:6816/3801228766,v1:172.21.15.142:6817/3801228766] conn(0x7fe8280721c0 0x7fe8280745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe82806c4e0 0x7fe82806e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c100500 0x7fe83c1954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 --2- 172.21.15.137:0/2581780215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe83c100eb0 0x7fe83c1959f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:35.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 >> 172.21.15.137:0/2581780215 conn(0x7fe83c074b90 msgr2=0x7fe83c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:35.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 shutdown_connections 2024-06-15T01:03:35.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.993+0000 7fe8427a0700 1 -- 172.21.15.137:0/2581780215 wait complete. 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 -- 172.21.15.137:0/3944515384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 msgr2=0x7f5634105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 --2- 172.21.15.137:0/3944515384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634105e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5624009910 tx=0x7f5624009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 -- 172.21.15.137:0/3944515384 shutdown_connections 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 --2- 172.21.15.137:0/3944515384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 --2- 172.21.15.137:0/3944515384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634105e20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 -- 172.21.15.137:0/3944515384 >> 172.21.15.137:0/3944515384 conn(0x7f56340755a0 msgr2=0x7f56340759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 -- 172.21.15.137:0/3944515384 shutdown_connections 2024-06-15T01:03:36.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 -- 172.21.15.137:0/3944515384 wait complete. 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.193+0000 7f563b35f700 1 Processor -- start 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 -- start start 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634102090 con 0x7f5634105a40 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634102200 con 0x7f56340ffbd0 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:36.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36178/0 (socket says 172.21.15.137:36178) 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 -- 172.21.15.137:0/2282998574 learned_addr learned my addr 172.21.15.137:0/2282998574 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56390fb700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 -- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 msgr2=0x7f5634101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240095f0 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56390fb700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f56388fa700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f563000c930 tx=0x7f563000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5630007ad0 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5630007c30 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5630018630 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.194+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634102480 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.195+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56341a2bd0 con 0x7f5634105a40 2024-06-15T01:03:36.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.195+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f563410cd80 con 0x7f5634105a40 2024-06-15T01:03:36.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.196+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f563000f450 con 0x7f5634105a40 2024-06-15T01:03:36.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f562a7fc700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 0x7f562006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f563008b430 con 0x7f5634105a40 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f56390fb700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 0x7f562006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f562a7fc700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 0x7f56200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5620074c90 con 0x7f56200721c0 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f563008db00 con 0x7f5634105a40 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f56398fc700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 0x7f56200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.197+0000 7f56390fb700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 0x7f562006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f562400b5c0 tx=0x7f5624005960 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.198+0000 7f56398fc700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 0x7f56200745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:36.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.198+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== osd.4 v2:172.21.15.142:6808/3568534083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5620074c90 con 0x7f56200721c0 2024-06-15T01:03:36.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.214+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 --> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5634102480 con 0x7f56200721c0 2024-06-15T01:03:36.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.214+0000 7f562a7fc700 1 -- 172.21.15.137:0/2282998574 <== osd.4 v2:172.21.15.142:6808/3568534083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5634102480 con 0x7f56200721c0 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 msgr2=0x7f56200745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 0x7f56200745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 msgr2=0x7f562006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 0x7f562006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f562400b5c0 tx=0x7f5624005960 comp rx=0 tx=0).stop 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 msgr2=0x7f5634101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f563000c930 tx=0x7f563000ccf0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 shutdown_connections 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:6808/3568534083,v1:172.21.15.142:6809/3568534083] conn(0x7f56200721c0 0x7f56200745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f562006c4e0 0x7f562006e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56340ffbd0 0x7f5634101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 --2- 172.21.15.137:0/2282998574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5634105a40 0x7f5634101b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:36.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 >> 172.21.15.137:0/2282998574 conn(0x7f56340755a0 msgr2=0x7f5634068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:36.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 shutdown_connections 2024-06-15T01:03:36.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:36.215+0000 7f563b35f700 1 -- 172.21.15.137:0/2282998574 wait complete. 2024-06-15T01:03:36.701 INFO:teuthology.orchestra.run.smithi137.stdout:60129542174 2024-06-15T01:03:36.702 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-15T01:03:36.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:36 smithi137 ceph-mon[29354]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:36.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:36 smithi142 ceph-mon[37379]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:37.402 INFO:teuthology.orchestra.run.smithi137.stdout:81604378649 2024-06-15T01:03:37.402 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-15T01:03:37.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 -- 172.21.15.137:0/162652898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 msgr2=0x7fc030105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:37.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 --2- 172.21.15.137:0/162652898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030105e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc024009910 tx=0x7fc024009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 -- 172.21.15.137:0/162652898 shutdown_connections 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 --2- 172.21.15.137:0/162652898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc0300ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 --2- 172.21.15.137:0/162652898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030105e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 -- 172.21.15.137:0/162652898 >> 172.21.15.137:0/162652898 conn(0x7fc0300fb550 msgr2=0x7fc0300fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 -- 172.21.15.137:0/162652898 shutdown_connections 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.601+0000 7fc034aee700 1 -- 172.21.15.137:0/162652898 wait complete. 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 Processor -- start 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 -- start start 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc030197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030198220 con 0x7fc030105a40 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc034aee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030198390 con 0x7fc0300ffa40 2024-06-15T01:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc02dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc02dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34108/0 (socket says 172.21.15.137:34108) 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.602+0000 7fc02dd9b700 1 -- 172.21.15.137:0/4140219454 learned_addr learned my addr 172.21.15.137:0/4140219454 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02e59c700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc030197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02dd9b700 1 -- 172.21.15.137:0/4140219454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 msgr2=0x7fc030197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02dd9b700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc030197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02dd9b700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240095f0 con 0x7fc030105a40 2024-06-15T01:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02e59c700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc030197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc02dd9b700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b4b0 tx=0x7fc02000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc020011800 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301a0670 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc020011df0 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.603+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc020010e40 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.604+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301a0bc0 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.604+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc03010ce00 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.605+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc020011960 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc01f7fe700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 0x7fc01806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc02008b040 con 0x7fc030105a40 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc02e59c700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 0x7fc01806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc01f7fe700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 0x7fc0180745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc018074c90 con 0x7fc0180721c0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02008d710 con 0x7fc030105a40 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc02ed9d700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 0x7fc0180745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc02e59c700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 0x7fc01806e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc02401a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.606+0000 7fc02ed9d700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 0x7fc0180745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.607+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== osd.0 v2:172.21.15.137:6802/1195724024 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc018074c90 con 0x7fc0180721c0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.622+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 --> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc03004ea90 con 0x7fc0180721c0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc01f7fe700 1 -- 172.21.15.137:0/4140219454 <== osd.0 v2:172.21.15.137:6802/1195724024 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc03004ea90 con 0x7fc0180721c0 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 msgr2=0x7fc0180745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 0x7fc0180745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 msgr2=0x7fc01806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 0x7fc01806e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc02401a040 comp rx=0 tx=0).stop 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 msgr2=0x7fc030197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b4b0 tx=0x7fc02000b870 comp rx=0 tx=0).stop 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 shutdown_connections 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6802/1195724024,v1:172.21.15.137:6803/1195724024] conn(0x7fc0180721c0 0x7fc0180745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc01806c4e0 0x7fc01806e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0300ffa40 0x7fc030197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 --2- 172.21.15.137:0/4140219454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc030105a40 0x7fc030197b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 >> 172.21.15.137:0/4140219454 conn(0x7fc0300fb550 msgr2=0x7fc0300fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 shutdown_connections 2024-06-15T01:03:37.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:35.623+0000 7fc034aee700 1 -- 172.21.15.137:0/4140219454 wait complete. 2024-06-15T01:03:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:38654705701 2024-06-15T01:03:38.039 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-15T01:03:38.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:38 smithi137 ceph-mon[29354]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:38.803 INFO:teuthology.orchestra.run.smithi137.stdout:150323855374 2024-06-15T01:03:38.804 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-15T01:03:38.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:38 smithi142 ceph-mon[37379]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:39.172 INFO:teuthology.orchestra.run.smithi137.stdout:171798691849 2024-06-15T01:03:39.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-15T01:03:39.196 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:39.337 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:40.650 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:40.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:40 smithi137 ceph-mon[29354]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:40.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:40 smithi142 ceph-mon[37379]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:41.429 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:41.431 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:42.617 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:42 smithi137 ceph-mon[29354]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:42.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:42 smithi142 ceph-mon[37379]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 -- 172.21.15.137:0/117176375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 msgr2=0x7f668c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 --2- 172.21.15.137:0/117176375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c0ffec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6680009910 tx=0x7f6680009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 -- 172.21.15.137:0/117176375 shutdown_connections 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 --2- 172.21.15.137:0/117176375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c0ffec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 --2- 172.21.15.137:0/117176375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 0x7f668c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 -- 172.21.15.137:0/117176375 >> 172.21.15.137:0/117176375 conn(0x7f668c0fb570 msgr2=0x7f668c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 -- 172.21.15.137:0/117176375 shutdown_connections 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.111+0000 7f66918b0700 1 -- 172.21.15.137:0/117176375 wait complete. 2024-06-15T01:03:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 Processor -- start 2024-06-15T01:03:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 -- start start 2024-06-15T01:03:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 0x7f668c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c198110 con 0x7f668c0ffa40 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f66918b0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c198280 con 0x7f668c105a40 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f668a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 0x7f668c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f668affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f668affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36188/0 (socket says 172.21.15.137:36188) 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.112+0000 7f668affd700 1 -- 172.21.15.137:0/2307064005 learned_addr learned my addr 172.21.15.137:0/2307064005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f668affd700 1 -- 172.21.15.137:0/2307064005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 msgr2=0x7f668c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f668affd700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 0x7f668c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f668affd700 1 -- 172.21.15.137:0/2307064005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800095f0 con 0x7f668c0ffa40 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f668affd700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b4b0 tx=0x7f667c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667c011800 con 0x7f668c0ffa40 2024-06-15T01:03:44.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c1a0620 con 0x7f668c0ffa40 2024-06-15T01:03:44.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f667c011df0 con 0x7f668c0ffa40 2024-06-15T01:03:44.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667c010e50 con 0x7f668c0ffa40 2024-06-15T01:03:44.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.113+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c1a0b10 con 0x7f668c0ffa40 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.114+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c108060 con 0x7f668c0ffa40 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.115+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f667c010380 con 0x7f668c0ffa40 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.115+0000 7f66908ae700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 0x7f667406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.115+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f667c08b040 con 0x7f668c0ffa40 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.115+0000 7f668a7fc700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 0x7f667406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:44.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.115+0000 7f668a7fc700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 0x7f667406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f668c069570 tx=0x7f668001a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:44.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.120+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f667c055b60 con 0x7f668c0ffa40 2024-06-15T01:03:44.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.315+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f668c04ea90 con 0x7f668c0ffa40 2024-06-15T01:03:44.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.316+0000 7f66908ae700 1 -- 172.21.15.137:0/2307064005 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f667c059180 con 0x7f668c0ffa40 2024-06-15T01:03:44.317 INFO:teuthology.orchestra.run.smithi137.stdout:60129542176 2024-06-15T01:03:44.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 msgr2=0x7f667406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:44.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 0x7f667406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f668c069570 tx=0x7f668001a040 comp rx=0 tx=0).stop 2024-06-15T01:03:44.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 msgr2=0x7f668c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b4b0 tx=0x7f667c00b7c0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 shutdown_connections 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f667406c4e0 0x7f667406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f668c0ffa40 0x7f668c1975b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 --2- 172.21.15.137:0/2307064005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f668c105a40 0x7f668c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 >> 172.21.15.137:0/2307064005 conn(0x7f668c0fb570 msgr2=0x7f668c0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:44.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 shutdown_connections 2024-06-15T01:03:44.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:44.318+0000 7f66918b0700 1 -- 172.21.15.137:0/2307064005 wait complete. 2024-06-15T01:03:44.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:44 smithi137 ceph-mon[29354]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:44.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:44 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2307064005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-15T01:03:44.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:44 smithi142 ceph-mon[37379]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:44.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:44 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2307064005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-15T01:03:45.136 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-06-15T01:03:45.137 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 -- 172.21.15.137:0/3090407120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 msgr2=0x7f142c10ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 --2- 172.21.15.137:0/3090407120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c10ab70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f141c009940 tx=0x7f141c009c50 comp rx=0 tx=0).stop 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 -- 172.21.15.137:0/3090407120 shutdown_connections 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 --2- 172.21.15.137:0/3090407120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c10ab70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 --2- 172.21.15.137:0/3090407120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f142c101d60 0x7f142c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 -- 172.21.15.137:0/3090407120 >> 172.21.15.137:0/3090407120 conn(0x7f142c0fb570 msgr2=0x7f142c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 -- 172.21.15.137:0/3090407120 shutdown_connections 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.676+0000 7f1434579700 1 -- 172.21.15.137:0/3090407120 wait complete. 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 Processor -- start 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 -- start start 2024-06-15T01:03:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f142c101d60 0x7f142c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19d190 con 0x7f142c102680 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1434579700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c196640 con 0x7f142c101d60 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36198/0 (socket says 172.21.15.137:36198) 2024-06-15T01:03:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 -- 172.21.15.137:0/92234860 learned_addr learned my addr 172.21.15.137:0/92234860 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:45.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 -- 172.21.15.137:0/92234860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f142c101d60 msgr2=0x7f142c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:45.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f142c101d60 0x7f142c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 -- 172.21.15.137:0/92234860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c102680 2024-06-15T01:03:45.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.677+0000 7f1431b14700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f141c0059d0 tx=0x7f141c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.678+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c01d070 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.678+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f141c00f460 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.678+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c021640 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.678+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c1968c0 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.678+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c196db0 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.679+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c108270 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.679+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f141c00fa20 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.680+0000 7f14237fe700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 0x7f141806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.680+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f141c08c7b0 con 0x7f142c102680 2024-06-15T01:03:45.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.680+0000 7f1432315700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 0x7f141806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:45.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.680+0000 7f1432315700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 0x7f141806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14280097b0 tx=0x7f1428006d20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:45.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.684+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f141c05ad60 con 0x7f142c102680 2024-06-15T01:03:45.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.879+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f142c107c50 con 0x7f142c102680 2024-06-15T01:03:45.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.880+0000 7f14237fe700 1 -- 172.21.15.137:0/92234860 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f141c05a8f0 con 0x7f142c102680 2024-06-15T01:03:45.881 INFO:teuthology.orchestra.run.smithi137.stdout:115964117014 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 msgr2=0x7f141806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 0x7f141806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14280097b0 tx=0x7f1428006d20 comp rx=0 tx=0).stop 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 msgr2=0x7f142c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f141c0059d0 tx=0x7f141c0056a0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 shutdown_connections 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f141806c4e0 0x7f141806e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f142c101d60 0x7f142c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 --2- 172.21.15.137:0/92234860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f142c102680 0x7f142c19cb00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 >> 172.21.15.137:0/92234860 conn(0x7f142c0fb570 msgr2=0x7f142c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 shutdown_connections 2024-06-15T01:03:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:45.882+0000 7f1434579700 1 -- 172.21.15.137:0/92234860 wait complete. 2024-06-15T01:03:46.550 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:46 smithi137 ceph-mon[29354]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:46.550 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:46 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/92234860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-15T01:03:46.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117012 got 115964117014 for osd.3 2024-06-15T01:03:46.635 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:46.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:46 smithi142 ceph-mon[37379]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:46.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:46 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/92234860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-15T01:03:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 -- 172.21.15.137:0/4099097871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 msgr2=0x7f10bc1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 --2- 172.21.15.137:0/4099097871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc1011b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009910 tx=0x7f10ac009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 -- 172.21.15.137:0/4099097871 shutdown_connections 2024-06-15T01:03:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 --2- 172.21.15.137:0/4099097871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 --2- 172.21.15.137:0/4099097871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc1011b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.197+0000 7f10c3936700 1 -- 172.21.15.137:0/4099097871 >> 172.21.15.137:0/4099097871 conn(0x7f10bc0fc640 msgr2=0x7f10bc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 -- 172.21.15.137:0/4099097871 shutdown_connections 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 -- 172.21.15.137:0/4099097871 wait complete. 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 Processor -- start 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 -- start start 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc19d030 con 0x7f10bc100dd0 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.198+0000 7f10c3936700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc196570 con 0x7f10bc101780 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43482/0 (socket says 172.21.15.137:43482) 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 -- 172.21.15.137:0/2607627556 learned_addr learned my addr 172.21.15.137:0/2607627556 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c0ed1700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 -- 172.21.15.137:0/2607627556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 msgr2=0x7f10bc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 -- 172.21.15.137:0/2607627556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0095f0 con 0x7f10bc100dd0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c0ed1700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c16d2700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006010 tx=0x7f10ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10ac01d070 con 0x7f10bc100dd0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10ac00f460 con 0x7f10bc100dd0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc1967f0 con 0x7f10bc100dd0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10ac021640 con 0x7f10bc100dd0 2024-06-15T01:03:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.199+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc196ce0 con 0x7f10bc100dd0 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.200+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc109170 con 0x7f10bc100dd0 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.201+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f10ac00fa20 con 0x7f10bc100dd0 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.201+0000 7f10b27fc700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 0x7f10a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.201+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f10ac08c780 con 0x7f10bc100dd0 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.201+0000 7f10c0ed1700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 0x7f10a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.201+0000 7f10c0ed1700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 0x7f10a806e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005fd0 tx=0x7f10b8005e20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:48.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.206+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10ac05ad30 con 0x7f10bc100dd0 2024-06-15T01:03:48.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.403+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f10bc197b60 con 0x7f10bc100dd0 2024-06-15T01:03:48.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.403+0000 7f10b27fc700 1 -- 172.21.15.137:0/2607627556 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10ac05a8c0 con 0x7f10bc100dd0 2024-06-15T01:03:48.404 INFO:teuthology.orchestra.run.smithi137.stdout:150323855376 2024-06-15T01:03:48.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 msgr2=0x7f10a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 0x7f10a806e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005fd0 tx=0x7f10b8005e20 comp rx=0 tx=0).stop 2024-06-15T01:03:48.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 msgr2=0x7f10bc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006010 tx=0x7f10ac00be20 comp rx=0 tx=0).stop 2024-06-15T01:03:48.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 shutdown_connections 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f10a806c4e0 0x7f10a806e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc100dd0 0x7f10bc19c4f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 --2- 172.21.15.137:0/2607627556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10bc101780 0x7f10bc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 >> 172.21.15.137:0/2607627556 conn(0x7f10bc0fc640 msgr2=0x7f10bc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 shutdown_connections 2024-06-15T01:03:48.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.405+0000 7f10c3936700 1 -- 172.21.15.137:0/2607627556 wait complete. 2024-06-15T01:03:48.671 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:48 smithi137 ceph-mon[29354]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:48.671 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:48.671 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:48 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2607627556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-15T01:03:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:48 smithi142 ceph-mon[37379]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:03:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:48 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2607627556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.932+0000 7f7110472700 1 -- 172.21.15.137:0/4275146141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 msgr2=0x7f71081057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.932+0000 7f7110472700 1 --2- 172.21.15.137:0/4275146141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f71081057c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009910 tx=0x7f70f8009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 -- 172.21.15.137:0/4275146141 shutdown_connections 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 --2- 172.21.15.137:0/4275146141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f71081057c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 --2- 172.21.15.137:0/4275146141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 0x7f71081011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 -- 172.21.15.137:0/4275146141 >> 172.21.15.137:0/4275146141 conn(0x7f71080fc650 msgr2=0x7f71080fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 -- 172.21.15.137:0/4275146141 shutdown_connections 2024-06-15T01:03:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 -- 172.21.15.137:0/4275146141 wait complete. 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 Processor -- start 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.933+0000 7f7110472700 1 -- start start 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f7110472700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 0x7f7108196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f7110472700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f7110472700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081972e0 con 0x7f7108101770 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f7110472700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710819a010 con 0x7f7108100dc0 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43524/0 (socket says 172.21.15.137:43524) 2024-06-15T01:03:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 -- 172.21.15.137:0/73748850 learned_addr learned my addr 172.21.15.137:0/73748850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710e20e700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 0x7f7108196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 -- 172.21.15.137:0/73748850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 msgr2=0x7f7108196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 0x7f7108196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 -- 172.21.15.137:0/73748850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80095f0 con 0x7f7108101770 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f710da0d700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70f800bda0 tx=0x7f70f800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.934+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f801d070 con 0x7f7108101770 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.935+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f70f800f460 con 0x7f7108101770 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.935+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f8021640 con 0x7f7108101770 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.935+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710819a290 con 0x7f7108101770 2024-06-15T01:03:48.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.935+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710819a780 con 0x7f7108101770 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.936+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71081091c0 con 0x7f7108101770 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.936+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f70f800fa60 con 0x7f7108101770 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.937+0000 7f70ff7fe700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 0x7f70f406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.937+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f70f808c620 con 0x7f7108101770 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.937+0000 7f710e20e700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 0x7f70f406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.937+0000 7f710e20e700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 0x7f70f406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7104007be0 tx=0x7f71040073d0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.941+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f70f8057140 con 0x7f7108101770 2024-06-15T01:03:49.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- 172.21.15.137:0/1280813214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 msgr2=0x7f76381020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1280813214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76381020e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7634009910 tx=0x7f7634009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:49.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- 172.21.15.137:0/1280813214 shutdown_connections 2024-06-15T01:03:49.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1280813214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 0x7f763810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1280813214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76381020e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- 172.21.15.137:0/1280813214 >> 172.21.15.137:0/1280813214 conn(0x7f76380fb570 msgr2=0x7f76380fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- 172.21.15.137:0/1280813214 shutdown_connections 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- 172.21.15.137:0/1280813214 wait complete. 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 Processor -- start 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.856+0000 7f763fd0c700 1 -- start start 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 0x7f76380ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76381003e0 con 0x7f7638101d00 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638100520 con 0x7f7638102620 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43502/0 (socket says 172.21.15.137:43502) 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 -- 172.21.15.137:0/1937129499 learned_addr learned my addr 172.21.15.137:0/1937129499 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:49.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763d2a7700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 0x7f76380ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 -- 172.21.15.137:0/1937129499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 msgr2=0x7f76380ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 0x7f76380ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 -- 172.21.15.137:0/1937129499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340095f0 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763daa8700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f763400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763401d070 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f763400f460 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7634021640 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76381007a0 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.857+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76381a2cb0 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.858+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638108210 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.860+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f763400fa60 con 0x7f7638101d00 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.860+0000 7f762effd700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 0x7f762406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:49.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.860+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f763408c730 con 0x7f7638101d00 2024-06-15T01:03:49.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.860+0000 7f763d2a7700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 0x7f762406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:49.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.860+0000 7f763d2a7700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 0x7f762406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7638101150 tx=0x7f7628008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:49.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:48.864+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f76340572d0 con 0x7f7638101d00 2024-06-15T01:03:49.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.066+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f763802cc70 con 0x7f7638101d00 2024-06-15T01:03:49.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.067+0000 7f762effd700 1 -- 172.21.15.137:0/1937129499 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f763405a8f0 con 0x7f7638101d00 2024-06-15T01:03:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:171798691851 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 msgr2=0x7f762406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 0x7f762406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7638101150 tx=0x7f7628008040 comp rx=0 tx=0).stop 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 msgr2=0x7f76380ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f763400be20 comp rx=0 tx=0).stop 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 shutdown_connections 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f762406c4e0 0x7f762406e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7638101d00 0x7f76380ff960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 --2- 172.21.15.137:0/1937129499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7638102620 0x7f76380ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 >> 172.21.15.137:0/1937129499 conn(0x7f76380fb570 msgr2=0x7f7638105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:49.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 shutdown_connections 2024-06-15T01:03:49.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.069+0000 7f763fd0c700 1 -- 172.21.15.137:0/1937129499 wait complete. 2024-06-15T01:03:49.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.141+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7108108ba0 con 0x7f7108101770 2024-06-15T01:03:49.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.141+0000 7f70ff7fe700 1 -- 172.21.15.137:0/73748850 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70f805a760 con 0x7f7108101770 2024-06-15T01:03:49.142 INFO:teuthology.orchestra.run.smithi137.stdout:81604378651 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 msgr2=0x7f70f406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 0x7f70f406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7104007be0 tx=0x7f71040073d0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 msgr2=0x7f7108196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70f800bda0 tx=0x7f70f800bdd0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 shutdown_connections 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f70f406c4e0 0x7f70f406e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7108100dc0 0x7f7108196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 --2- 172.21.15.137:0/73748850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108101770 0x7f7108196c50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:49.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 >> 172.21.15.137:0/73748850 conn(0x7f71080fc650 msgr2=0x7f71080fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:49.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 shutdown_connections 2024-06-15T01:03:49.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:49.143+0000 7f7110472700 1 -- 172.21.15.137:0/73748850 wait complete. 2024-06-15T01:03:49.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1937129499' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-15T01:03:49.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/73748850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-15T01:03:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:49 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1937129499' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-15T01:03:49.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:49 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/73748850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-15T01:03:49.923 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-06-15T01:03:49.924 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:50.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855374 got 150323855376 for osd.4 2024-06-15T01:03:50.622 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:50.812 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:50 smithi137 ceph-mon[29354]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:50.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:50 smithi142 ceph-mon[37379]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:50.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- 172.21.15.137:0/1041135190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 msgr2=0x7fbca81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 --2- 172.21.15.137:0/1041135190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca81057c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009910 tx=0x7fbc90009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- 172.21.15.137:0/1041135190 shutdown_connections 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 --2- 172.21.15.137:0/1041135190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca81057c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 --2- 172.21.15.137:0/1041135190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 0x7fbca81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- 172.21.15.137:0/1041135190 >> 172.21.15.137:0/1041135190 conn(0x7fbca80fc650 msgr2=0x7fbca80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- 172.21.15.137:0/1041135190 shutdown_connections 2024-06-15T01:03:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- 172.21.15.137:0/1041135190 wait complete. 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 Processor -- start 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.838+0000 7fbcad394700 1 -- start start 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbcad394700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 0x7fbca819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbcad394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbcad394700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca819f240 con 0x7fbca8101770 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbcad394700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81986f0 con 0x7fbca8100dc0 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43530/0 (socket says 172.21.15.137:43530) 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 -- 172.21.15.137:0/380805754 learned_addr learned my addr 172.21.15.137:0/380805754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbca6ffd700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 0x7fbca819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 -- 172.21.15.137:0/380805754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 msgr2=0x7fbca819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:50.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 0x7fbca819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 -- 172.21.15.137:0/380805754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900095f0 con 0x7fbca8101770 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbc9ffff700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000bda0 tx=0x7fbc9000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9001d070 con 0x7fbca8101770 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbc9000f460 con 0x7fbca8101770 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8198970 con 0x7fbca8101770 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.839+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc90021640 con 0x7fbca8101770 2024-06-15T01:03:50.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.840+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8198e60 con 0x7fbca8101770 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.840+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8104f40 con 0x7fbca8101770 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.841+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbc9000fa60 con 0x7fbca8101770 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.841+0000 7fbca4ff9700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 0x7fbc9406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.841+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbc9008c600 con 0x7fbca8101770 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.841+0000 7fbca6ffd700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 0x7fbc9406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:50.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.842+0000 7fbca6ffd700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 0x7fbc9406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc98005950 tx=0x7fbc9800a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:50.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:50.846+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc9005ac30 con 0x7fbca8101770 2024-06-15T01:03:51.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.040+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbca804ea90 con 0x7fbca8101770 2024-06-15T01:03:51.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.040+0000 7fbca4ff9700 1 -- 172.21.15.137:0/380805754 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc9005a7c0 con 0x7fbca8101770 2024-06-15T01:03:51.041 INFO:teuthology.orchestra.run.smithi137.stdout:38654705703 2024-06-15T01:03:51.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.042+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 msgr2=0x7fbc9406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.042+0000 7fbcad394700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 0x7fbc9406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc98005950 tx=0x7fbc9800a300 comp rx=0 tx=0).stop 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.042+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 msgr2=0x7fbca819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.042+0000 7fbcad394700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000bda0 tx=0x7fbc9000bdd0 comp rx=0 tx=0).stop 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 shutdown_connections 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc9406c4e0 0x7fbc9406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbca8100dc0 0x7fbca819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 --2- 172.21.15.137:0/380805754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbca8101770 0x7fbca819ebb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 >> 172.21.15.137:0/380805754 conn(0x7fbca80fc650 msgr2=0x7fbca80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 shutdown_connections 2024-06-15T01:03:51.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:51.043+0000 7fbcad394700 1 -- 172.21.15.137:0/380805754 wait complete. 2024-06-15T01:03:51.363 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-06-15T01:03:51.363 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:51.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:51 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/380805754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-15T01:03:51.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:51 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/380805754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-15T01:03:52.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705703 for osd.0 2024-06-15T01:03:52.059 DEBUG:teuthology.parallel:result is None 2024-06-15T01:03:52.059 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-15T01:03:52.059 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-15T01:03:52.272 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:52.614 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:52 smithi137 ceph-mon[29354]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:52.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:52 smithi142 ceph-mon[37379]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 -- 172.21.15.137:0/2689072088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 msgr2=0x7f49c0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2689072088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c0105e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009910 tx=0x7f49b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 -- 172.21.15.137:0/2689072088 shutdown_connections 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2689072088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 0x7f49c00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2689072088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c0105e20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 -- 172.21.15.137:0/2689072088 >> 172.21.15.137:0/2689072088 conn(0x7f49c00fb550 msgr2=0x7f49c00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 -- 172.21.15.137:0/2689072088 shutdown_connections 2024-06-15T01:03:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.897+0000 7f49c49a3700 1 -- 172.21.15.137:0/2689072088 wait complete. 2024-06-15T01:03:53.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 Processor -- start 2024-06-15T01:03:53.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 -- start start 2024-06-15T01:03:53.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 0x7f49c019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:53.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019d120 con 0x7f49c0105a40 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49c49a3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01965d0 con 0x7f49c00ffa40 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43560/0 (socket says 172.21.15.137:43560) 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 -- 172.21.15.137:0/2511854702 learned_addr learned my addr 172.21.15.137:0/2511854702 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49be59c700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 0x7f49c019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 -- 172.21.15.137:0/2511854702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 msgr2=0x7f49c019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 0x7f49c019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 -- 172.21.15.137:0/2511854702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c0105a40 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49bdd9b700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b4b0 tx=0x7f49b400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.898+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49b4011800 con 0x7f49c0105a40 2024-06-15T01:03:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.899+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c01968b0 con 0x7f49c0105a40 2024-06-15T01:03:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.899+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f49b4011df0 con 0x7f49c0105a40 2024-06-15T01:03:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.899+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49b4010e50 con 0x7f49c0105a40 2024-06-15T01:03:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.899+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0196e00 con 0x7f49c0105a40 2024-06-15T01:03:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.900+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c0109050 con 0x7f49c0105a40 2024-06-15T01:03:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.900+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f49b4010380 con 0x7f49c0105a40 2024-06-15T01:03:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.900+0000 7f49af7fe700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 0x7f49a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.901+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f49b408b040 con 0x7f49c0105a40 2024-06-15T01:03:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.901+0000 7f49be59c700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 0x7f49a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.901+0000 7f49be59c700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 0x7f49a806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005f50 tx=0x7f49b001a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:53.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:53.905+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49b40595f0 con 0x7f49c0105a40 2024-06-15T01:03:54.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.088+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49c0197080 con 0x7f49a806c4e0 2024-06-15T01:03:54.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.089+0000 7f49af7fe700 1 -- 172.21.15.137:0/2511854702 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28138 (secure 0 0 0) 0x7f49c0197080 con 0x7f49a806c4e0 2024-06-15T01:03:54.091 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:54.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 msgr2=0x7f49a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 0x7f49a806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005f50 tx=0x7f49b001a040 comp rx=0 tx=0).stop 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 msgr2=0x7f49c019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b4b0 tx=0x7f49b400b7c0 comp rx=0 tx=0).stop 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 shutdown_connections 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49a806c4e0 0x7f49a806e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c00ffa40 0x7f49c019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 --2- 172.21.15.137:0/2511854702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49c0105a40 0x7f49c019ca90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 >> 172.21.15.137:0/2511854702 conn(0x7f49c00fb550 msgr2=0x7f49c0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 shutdown_connections 2024-06-15T01:03:54.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:54.092+0000 7f49c49a3700 1 -- 172.21.15.137:0/2511854702 wait complete. 2024-06-15T01:03:54.094 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-06-15T01:03:54.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:54 smithi137 ceph-mon[29354]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:54.671 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-06-15T01:03:53.076501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002089"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-06-15T01:03:02.947983+0000","last_change":"2024-06-15T01:02:37.765067+0000","last_active":"2024-06-15T01:03:02.947983+0000","last_peered":"2024-06-15T01:03:02.947983+0000","last_clean":"2024-06-15T01:03:02.947983+0000","last_became_active":"2024-06-15T01:02:37.764737+0000","last_became_peered":"2024-06-15T01:02:37.764737+0000","last_unstale":"2024-06-15T01:03:02.947983+0000","last_undegraded":"2024-06-15T01:03:02.947983+0000","last_fullsized":"2024-06-15T01:03:02.947983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-15T01:01:49.649299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-15T01:01:49.649299+0000","last_clean_scrub_stamp":"2024-06-15T01:01:49.649299+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:48:32.250823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.438},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":14.112,"5min":14.112,"15min":14.112},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46100000000000002}]},{"osd":2,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.495},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53500000000000003}]},{"osd":4,"last update":"Sat Jun 15 01:03:39 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.33200000000000002}]},{"osd":1,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Jun 15 01:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jun 15 01:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.22700000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34200000000000003},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.36299999999999999}]},{"osd":3,"last update":"Sat Jun 15 01:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Jun 15 01:03:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.314,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33100000000000002},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Jun 15 01:03:15 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":14.420999999999999,"5min":14.420999999999999,"15min":14.420999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.497}]},{"osd":4,"last update":"Sat Jun 15 01:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-15T01:03:54.672 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-15T01:03:54.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:54 smithi142 ceph-mon[37379]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:54.877 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:55.698 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:55 smithi137 ceph-mon[29354]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-15T01:03:55.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:55 smithi142 ceph-mon[37379]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-15T01:03:56.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 -- 172.21.15.137:0/3892246935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ff500 msgr2=0x7f682c0ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 --2- 172.21.15.137:0/3892246935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ff500 0x7f682c0ff8e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6828009910 tx=0x7f6828009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 -- 172.21.15.137:0/3892246935 shutdown_connections 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 --2- 172.21.15.137:0/3892246935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ffeb0 0x7f682c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 --2- 172.21.15.137:0/3892246935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ff500 0x7f682c0ff8e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 -- 172.21.15.137:0/3892246935 >> 172.21.15.137:0/3892246935 conn(0x7f682c074d80 msgr2=0x7f682c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 -- 172.21.15.137:0/3892246935 shutdown_connections 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.435+0000 7f68341ec700 1 -- 172.21.15.137:0/3892246935 wait complete. 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 Processor -- start 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 -- start start 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ff500 0x7f682c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:56.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c197280 con 0x7f682c0ffeb0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f68341ec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c199fb0 con 0x7f682c0ff500 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47462/0 (socket says 172.21.15.137:47462) 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 -- 172.21.15.137:0/3768541817 learned_addr learned my addr 172.21.15.137:0/3768541817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831f88700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ff500 0x7f682c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 -- 172.21.15.137:0/3768541817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ff500 msgr2=0x7f682c1966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ff500 0x7f682c1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 -- 172.21.15.137:0/3768541817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280095f0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.436+0000 7f6831787700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f681c00d900 tx=0x7f681c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.437+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681c009900 con 0x7f682c0ffeb0 2024-06-15T01:03:56.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.437+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c19a290 con 0x7f682c0ffeb0 2024-06-15T01:03:56.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.437+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f681c010460 con 0x7f682c0ffeb0 2024-06-15T01:03:56.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.437+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681c00f5f0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.437+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c19a7e0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.438+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c107820 con 0x7f682c0ffeb0 2024-06-15T01:03:56.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.438+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f681c0105d0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.438+0000 7f6822ffd700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 0x7f681806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:56.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.439+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f681c08b0b0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.439+0000 7f6831f88700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 0x7f681806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:56.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.439+0000 7f6831f88700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 0x7f681806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f682800b5c0 tx=0x7f6828005fd0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:56.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.443+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f681c055bd0 con 0x7f682c0ffeb0 2024-06-15T01:03:56.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.629+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f682c19aac0 con 0x7f681806c4e0 2024-06-15T01:03:56.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.630+0000 7f6822ffd700 1 -- 172.21.15.137:0/3768541817 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28167 (secure 0 0 0) 0x7f682c19aac0 con 0x7f681806c4e0 2024-06-15T01:03:56.631 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:56.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 msgr2=0x7f681806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:56.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 0x7f681806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f682800b5c0 tx=0x7f6828005fd0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 msgr2=0x7f682c196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:56.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f681c00d900 tx=0x7f681c00dcc0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 shutdown_connections 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f681806c4e0 0x7f681806e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f682c0ff500 0x7f682c1966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 --2- 172.21.15.137:0/3768541817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f682c0ffeb0 0x7f682c196bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 >> 172.21.15.137:0/3768541817 conn(0x7f682c074d80 msgr2=0x7f682c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 shutdown_connections 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:56.632+0000 7f68341ec700 1 -- 172.21.15.137:0/3768541817 wait complete. 2024-06-15T01:03:56.634 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-06-15T01:03:56.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:56 smithi142 ceph-mon[37379]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:56.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:56 smithi137 ceph-mon[29354]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-06-15T01:03:55.076879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002080"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-06-15T01:03:02.947983+0000","last_change":"2024-06-15T01:02:37.765067+0000","last_active":"2024-06-15T01:03:02.947983+0000","last_peered":"2024-06-15T01:03:02.947983+0000","last_clean":"2024-06-15T01:03:02.947983+0000","last_became_active":"2024-06-15T01:02:37.764737+0000","last_became_peered":"2024-06-15T01:02:37.764737+0000","last_unstale":"2024-06-15T01:03:02.947983+0000","last_undegraded":"2024-06-15T01:03:02.947983+0000","last_fullsized":"2024-06-15T01:03:02.947983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-15T01:01:49.649299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-15T01:01:49.649299+0000","last_clean_scrub_stamp":"2024-06-15T01:01:49.649299+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:48:32.250823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Jun 15 01:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.438},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":14.112,"5min":14.112,"15min":14.112},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46100000000000002}]},{"osd":2,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.495},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53500000000000003}]},{"osd":4,"last update":"Sat Jun 15 01:03:39 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.33200000000000002}]},{"osd":1,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Sat Jun 15 01:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Jun 15 01:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jun 15 01:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.22700000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34200000000000003},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:02:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"last":0.36299999999999999}]},{"osd":3,"last update":"Sat Jun 15 01:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Jun 15 01:03:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jun 15 01:03:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.314,"15min":0.309},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33100000000000002},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Sat Jun 15 01:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Jun 15 01:03:15 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":14.420999999999999,"5min":14.420999999999999,"15min":14.420999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.497}]},{"osd":4,"last update":"Sat Jun 15 01:03:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-15T01:03:57.355 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-15T01:03:57.355 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-15T01:03:57.355 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-15T01:03:57.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph health --format=json 2024-06-15T01:03:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:03:58.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:58 smithi137 ceph-mon[29354]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-15T01:03:58.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:58 smithi137 ceph-mon[29354]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:58.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:58 smithi142 ceph-mon[37379]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-15T01:03:58.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:58 smithi142 ceph-mon[37379]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 -- 172.21.15.137:0/855366271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c105a40 msgr2=0x7f799c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 --2- 172.21.15.137:0/855366271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c105a40 0x7f799c105e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7990009910 tx=0x7f7990009c20 comp rx=0 tx=0).stop 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 -- 172.21.15.137:0/855366271 shutdown_connections 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 --2- 172.21.15.137:0/855366271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c0ffbd0 0x7f799c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 --2- 172.21.15.137:0/855366271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c105a40 0x7f799c105e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 -- 172.21.15.137:0/855366271 >> 172.21.15.137:0/855366271 conn(0x7f799c0755a0 msgr2=0x7f799c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 -- 172.21.15.137:0/855366271 shutdown_connections 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.123+0000 7f79a0dad700 1 -- 172.21.15.137:0/855366271 wait complete. 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 Processor -- start 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 -- start start 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c105a40 0x7f799c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19d0d0 con 0x7f799c0ffbd0 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f79a0dad700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c196580 con 0x7f799c105a40 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f799a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f799a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47478/0 (socket says 172.21.15.137:47478) 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f799a59c700 1 -- 172.21.15.137:0/2229332382 learned_addr learned my addr 172.21.15.137:0/2229332382 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.124+0000 7f7999d9b700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c105a40 0x7f799c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f799a59c700 1 -- 172.21.15.137:0/2229332382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c105a40 msgr2=0x7f799c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f799a59c700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c105a40 0x7f799c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f799a59c700 1 -- 172.21.15.137:0/2229332382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900095f0 con 0x7f799c0ffbd0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f799a59c700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7990006010 tx=0x7f799000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799001d070 con 0x7f799c0ffbd0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f799000f460 con 0x7f799c0ffbd0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c196800 con 0x7f799c0ffbd0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7990021620 con 0x7f799c0ffbd0 2024-06-15T01:03:59.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.125+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c196cf0 con 0x7f799c0ffbd0 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.126+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c109050 con 0x7f799c0ffbd0 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.127+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f799000f5d0 con 0x7f799c0ffbd0 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.127+0000 7f798b7fe700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 0x7f798406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.127+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f799008c690 con 0x7f799c0ffbd0 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.127+0000 7f7999d9b700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 0x7f798406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:03:59.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.127+0000 7f7999d9b700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 0x7f798406e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f799c104e40 tx=0x7f798c009190 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:03:59.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.132+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f799005ac40 con 0x7f799c0ffbd0 2024-06-15T01:03:59.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.361+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f799c04ea90 con 0x7f799c0ffbd0 2024-06-15T01:03:59.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.361+0000 7f798b7fe700 1 -- 172.21.15.137:0/2229332382 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f799005a7d0 con 0x7f799c0ffbd0 2024-06-15T01:03:59.362 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:03:59.363 INFO:teuthology.orchestra.run.smithi137.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 msgr2=0x7f798406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 0x7f798406e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f799c104e40 tx=0x7f798c009190 comp rx=0 tx=0).stop 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 msgr2=0x7f799c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7990006010 tx=0x7f799000bd70 comp rx=0 tx=0).stop 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 shutdown_connections 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f798406c4e0 0x7f798406e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f799c0ffbd0 0x7f799c19c500 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 --2- 172.21.15.137:0/2229332382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f799c105a40 0x7f799c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.363+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 >> 172.21.15.137:0/2229332382 conn(0x7f799c0755a0 msgr2=0x7f799c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.364+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 shutdown_connections 2024-06-15T01:03:59.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:03:59.364+0000 7f79a0dad700 1 -- 172.21.15.137:0/2229332382 wait complete. 2024-06-15T01:03:59.691 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:03:59 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2229332382' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-15T01:03:59.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:03:59 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2229332382' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-15T01:03:59.906 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-15T01:03:59.907 INFO:tasks.cephadm:Setup complete, yielding 2024-06-15T01:03:59.907 INFO:teuthology.run_tasks:Running task print... 2024-06-15T01:03:59.917 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-06-15T01:03:59.917 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:03:59.928 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:03:59.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-06-15T01:04:00.134 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:00.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:00 smithi142 ceph-mon[37379]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:00.870 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:00 smithi137 ceph-mon[29354]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:01.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 -- 172.21.15.137:0/1767451876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 msgr2=0x7f43d4102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 --2- 172.21.15.137:0/1767451876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4102240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009910 tx=0x7f43c4009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 -- 172.21.15.137:0/1767451876 shutdown_connections 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 --2- 172.21.15.137:0/1767451876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4102240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 --2- 172.21.15.137:0/1767451876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 0x7f43d4101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 -- 172.21.15.137:0/1767451876 >> 172.21.15.137:0/1767451876 conn(0x7f43d40fc140 msgr2=0x7f43d40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 -- 172.21.15.137:0/1767451876 shutdown_connections 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.909+0000 7f43db72c700 1 -- 172.21.15.137:0/1767451876 wait complete. 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 Processor -- start 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 -- start start 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 0x7f43d41962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:01.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4196e30 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43db72c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4196f70 con 0x7f43d4100be0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43d8cc7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43d8cc7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47508/0 (socket says 172.21.15.137:47508) 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43d8cc7700 1 -- 172.21.15.137:0/805384255 learned_addr learned my addr 172.21.15.137:0/805384255 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.910+0000 7f43d94c8700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 0x7f43d41962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43d8cc7700 1 -- 172.21.15.137:0/805384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 msgr2=0x7f43d41962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43d8cc7700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 0x7f43d41962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43d8cc7700 1 -- 172.21.15.137:0/805384255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40095f0 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43d8cc7700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006010 tx=0x7f43c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43c401d070 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d419b9c0 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f43c400f460 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43c4021640 con 0x7f43d4101de0 2024-06-15T01:04:01.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.911+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d419bf30 con 0x7f43d4101de0 2024-06-15T01:04:01.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.912+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d4109600 con 0x7f43d4101de0 2024-06-15T01:04:01.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.912+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f43c400f9b0 con 0x7f43d4101de0 2024-06-15T01:04:01.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.913+0000 7f43ca7fc700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 0x7f43c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:01.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.913+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f43c408c730 con 0x7f43d4101de0 2024-06-15T01:04:01.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.913+0000 7f43d94c8700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 0x7f43c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:01.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.913+0000 7f43d94c8700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 0x7f43c006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43d4101c40 tx=0x7f43d000b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:01.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:01.917+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f43c405ace0 con 0x7f43d4101de0 2024-06-15T01:04:02.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.108+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f43d41097c0 con 0x7f43d4101de0 2024-06-15T01:04:02.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.112+0000 7f43ca7fc700 1 -- 172.21.15.137:0/805384255 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f43c405a870 con 0x7f43d4101de0 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 msgr2=0x7f43c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 0x7f43c006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43d4101c40 tx=0x7f43d000b410 comp rx=0 tx=0).stop 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 msgr2=0x7f43d4196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006010 tx=0x7f43c400bd70 comp rx=0 tx=0).stop 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 shutdown_connections 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f43c006c4e0 0x7f43c006e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d4100be0 0x7f43d41962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 --2- 172.21.15.137:0/805384255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101de0 0x7f43d4196810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 >> 172.21.15.137:0/805384255 conn(0x7f43d40fc140 msgr2=0x7f43d4105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:02.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 shutdown_connections 2024-06-15T01:04:02.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:02.115+0000 7f43db72c700 1 -- 172.21.15.137:0/805384255 wait complete. 2024-06-15T01:04:02.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:02 smithi137 ceph-mon[29354]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:02.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:02 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/805384255' entity='client.admin' 2024-06-15T01:04:02.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:02 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:02.762 INFO:teuthology.run_tasks:Running task print... 2024-06-15T01:04:02.773 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-06-15T01:04:02.774 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:02.782 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:02.782 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph orch status' 2024-06-15T01:04:02.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:02 smithi142 ceph-mon[37379]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:02.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:02 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/805384255' entity='client.admin' 2024-06-15T01:04:02.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:02 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:02.985 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:03.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:03.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:03.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:03.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:03.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:04.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 -- 172.21.15.137:0/1365104091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100cb0 msgr2=0x7f64cc101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:04.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1365104091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100cb0 0x7f64cc101110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009910 tx=0x7f64c0009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:04.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 -- 172.21.15.137:0/1365104091 shutdown_connections 2024-06-15T01:04:04.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1365104091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100cb0 0x7f64cc101110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1365104091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc0ffab0 0x7f64cc0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 -- 172.21.15.137:0/1365104091 >> 172.21.15.137:0/1365104091 conn(0x7f64cc0fb010 msgr2=0x7f64cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 -- 172.21.15.137:0/1365104091 shutdown_connections 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.508+0000 7f64d3b19700 1 -- 172.21.15.137:0/1365104091 wait complete. 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 Processor -- start 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 -- start start 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 0x7f64cc19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc19b5e0 con 0x7f64cc0ffab0 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d3b19700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc19b720 con 0x7f64cc100cb0 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47522/0 (socket says 172.21.15.137:47522) 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 -- 172.21.15.137:0/1619648576 learned_addr learned my addr 172.21.15.137:0/1619648576 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d10b4700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 0x7f64cc19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 -- 172.21.15.137:0/1619648576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 msgr2=0x7f64cc19b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 0x7f64cc19b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d18b5700 1 -- 172.21.15.137:0/1619648576 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00095f0 con 0x7f64cc0ffab0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.509+0000 7f64d10b4700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 0x7f64cc19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64d18b5700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c8f0 tx=0x7f64c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c8007a70 con 0x7f64cc0ffab0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f64c8007bd0 con 0x7f64cc0ffab0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc19ba00 con 0x7f64cc0ffab0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c8018630 con 0x7f64cc0ffab0 2024-06-15T01:04:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.510+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc19bfd0 con 0x7f64cc0ffab0 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.511+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc1084d0 con 0x7f64cc0ffab0 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.511+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f64c800f450 con 0x7f64cc0ffab0 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.512+0000 7f64beffd700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 0x7f64b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.512+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f64c808b370 con 0x7f64cc0ffab0 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.512+0000 7f64d10b4700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 0x7f64b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.512+0000 7f64d10b4700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 0x7f64b806e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f64cc069ed0 tx=0x7f64c0005920 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.516+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f64c8059a50 con 0x7f64cc0ffab0 2024-06-15T01:04:04.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.723+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64cc105600 con 0x7f64b806c4e0 2024-06-15T01:04:04.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:04 smithi137 ceph-mon[29354]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:04.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.724+0000 7f64beffd700 1 -- 172.21.15.137:0/1619648576 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f64cc105600 con 0x7f64b806c4e0 2024-06-15T01:04:04.725 INFO:teuthology.orchestra.run.smithi137.stdout:Backend: cephadm 2024-06-15T01:04:04.725 INFO:teuthology.orchestra.run.smithi137.stdout:Available: Yes 2024-06-15T01:04:04.725 INFO:teuthology.orchestra.run.smithi137.stdout:Paused: No 2024-06-15T01:04:04.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 msgr2=0x7f64b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:04.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 0x7f64b806e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f64cc069ed0 tx=0x7f64c0005920 comp rx=0 tx=0).stop 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 msgr2=0x7f64cc069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c8f0 tx=0x7f64c800ccb0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 shutdown_connections 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f64b806c4e0 0x7f64b806e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc0ffab0 0x7f64cc069950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 --2- 172.21.15.137:0/1619648576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64cc100cb0 0x7f64cc19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 >> 172.21.15.137:0/1619648576 conn(0x7f64cc0fb010 msgr2=0x7f64cc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 shutdown_connections 2024-06-15T01:04:04.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:04.727+0000 7f64d3b19700 1 -- 172.21.15.137:0/1619648576 wait complete. 2024-06-15T01:04:04.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:04 smithi142 ceph-mon[37379]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:05.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-15T01:04:05.577 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:06.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:06 smithi142 ceph-mon[37379]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:06.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:06 smithi142 ceph-mon[37379]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:06.898 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:06 smithi137 ceph-mon[29354]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:06.898 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:06 smithi137 ceph-mon[29354]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:07.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/402675869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 msgr2=0x7f2cd41063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:07.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/402675869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd41063c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009910 tx=0x7f2cbc009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:07.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/402675869 shutdown_connections 2024-06-15T01:04:07.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/402675869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd41063c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/402675869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 0x7f2cd4103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/402675869 >> 172.21.15.137:0/402675869 conn(0x7f2cd40fb0b0 msgr2=0x7f2cd40fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/402675869 shutdown_connections 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/402675869 wait complete. 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 Processor -- start 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.125+0000 7f2cd9fa2700 1 -- start start 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 0x7f2cd41962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4196db0 con 0x7f2cd4103fd0 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4196ef0 con 0x7f2cd41016a0 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59904/0 (socket says 172.21.15.137:59904) 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 -- 172.21.15.137:0/2105188259 learned_addr learned my addr 172.21.15.137:0/2105188259 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:07.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd37fe700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 0x7f2cd41962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 -- 172.21.15.137:0/2105188259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 msgr2=0x7f2cd41962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 0x7f2cd41962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 -- 172.21.15.137:0/2105188259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0095f0 con 0x7f2cd4103fd0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd2ffd700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b5c0 tx=0x7f2cbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc01d070 con 0x7f2cd4103fd0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2cbc00f460 con 0x7f2cd4103fd0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd419b950 con 0x7f2cd4103fd0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd419be90 con 0x7f2cd4103fd0 2024-06-15T01:04:07.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.126+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc021640 con 0x7f2cd4103fd0 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.127+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd4190440 con 0x7f2cd4103fd0 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.128+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2cbc00f9b0 con 0x7f2cd4103fd0 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.128+0000 7f2cd0ff9700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 0x7f2cc006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.128+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2cbc08c8c0 con 0x7f2cd4103fd0 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.128+0000 7f2cd37fe700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 0x7f2cc006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:07.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.129+0000 7f2cd37fe700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 0x7f2cc006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d260 tx=0x7f2cc4009040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:07.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.133+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2cbc05ae70 con 0x7f2cd4103fd0 2024-06-15T01:04:07.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.335+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cd40611d0 con 0x7f2cc006c4e0 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.344+0000 7f2cd0ff9700 1 -- 172.21.15.137:0/2105188259 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7f2cd40611d0 con 0x7f2cc006c4e0 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (3m) 2m ago 5m 26.8M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (5m) 2m ago 5m 7088k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (4m) 66s ago 4m 6941k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (5m) 2m ago 5m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (4m) 66s ago 4m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (3m) 2m ago 4m 78.3M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:9283,8765,8443 running (6m) 2m ago 6m 479M - 18.2.1 5be31c24972a 61486e562eb9 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (4m) 66s ago 4m 442M - 18.2.1 5be31c24972a e531298bfce9 2024-06-15T01:04:07.345 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (6m) 2m ago 6m 43.8M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (4m) 66s ago 4m 41.4M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (5m) 2m ago 5m 16.2M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (4m) 66s ago 4m 15.9M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (3m) 2m ago 3m 38.7M 3946M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (2m) 2m ago 2m 40.3M 3946M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (2m) 2m ago 2m 37.6M 3946M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (2m) 66s ago 2m 42.0M 4970M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (96s) 66s ago 96s 39.6M 4970M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (71s) 66s ago 71s 10.6M 4970M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:04:07.346 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (3m) 2m ago 4m 36.9M - 2.43.0 a07b618ecd1d 904ef44cd0a5 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 msgr2=0x7f2cc006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 0x7f2cc006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d260 tx=0x7f2cc4009040 comp rx=0 tx=0).stop 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 msgr2=0x7f2cd4196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00b5c0 tx=0x7f2cbc00be20 comp rx=0 tx=0).stop 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 shutdown_connections 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f2cc006c4e0 0x7f2cc006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd41016a0 0x7f2cd41962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 --2- 172.21.15.137:0/2105188259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd4103fd0 0x7f2cd4196820 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:07.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 >> 172.21.15.137:0/2105188259 conn(0x7f2cd40fb0b0 msgr2=0x7f2cd40fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:07.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 shutdown_connections 2024-06-15T01:04:07.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:07.346+0000 7f2cd9fa2700 1 -- 172.21.15.137:0/2105188259 wait complete. 2024-06-15T01:04:08.030 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-15T01:04:08.237 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:08.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:08 smithi137 ceph-mon[29354]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:08.629 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:08 smithi137 ceph-mon[29354]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:08.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:08 smithi142 ceph-mon[37379]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:08.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:08 smithi142 ceph-mon[37379]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:09.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 -- 172.21.15.137:0/2716194998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c100bc0 msgr2=0x7ff92c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 --2- 172.21.15.137:0/2716194998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c100bc0 0x7ff92c100fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff928009910 tx=0x7ff928009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 -- 172.21.15.137:0/2716194998 shutdown_connections 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 --2- 172.21.15.137:0/2716194998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c101dc0 0x7ff92c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 --2- 172.21.15.137:0/2716194998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c100bc0 0x7ff92c100fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 -- 172.21.15.137:0/2716194998 >> 172.21.15.137:0/2716194998 conn(0x7ff92c0fc140 msgr2=0x7ff92c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 -- 172.21.15.137:0/2716194998 shutdown_connections 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.729+0000 7ff933f62700 1 -- 172.21.15.137:0/2716194998 wait complete. 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 Processor -- start 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 -- start start 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c100bc0 0x7ff92c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c196f00 con 0x7ff92c101dc0 2024-06-15T01:04:09.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff933f62700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c197040 con 0x7ff92c100bc0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff9314fd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff9314fd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59924/0 (socket says 172.21.15.137:59924) 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff9314fd700 1 -- 172.21.15.137:0/2784575668 learned_addr learned my addr 172.21.15.137:0/2784575668 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.730+0000 7ff931cfe700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c100bc0 0x7ff92c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff9314fd700 1 -- 172.21.15.137:0/2784575668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c100bc0 msgr2=0x7ff92c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff9314fd700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c100bc0 0x7ff92c1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff9314fd700 1 -- 172.21.15.137:0/2784575668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280095f0 con 0x7ff92c101dc0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff9314fd700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b4b0 tx=0x7ff92000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff920011800 con 0x7ff92c101dc0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c19baf0 con 0x7ff92c101dc0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff920011df0 con 0x7ff92c101dc0 2024-06-15T01:04:09.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff920010e50 con 0x7ff92c101dc0 2024-06-15T01:04:09.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.731+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c19c0c0 con 0x7ff92c101dc0 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.732+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c109660 con 0x7ff92c101dc0 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.732+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff920010380 con 0x7ff92c101dc0 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.733+0000 7ff91effd700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 0x7ff91806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.733+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff92008b000 con 0x7ff92c101dc0 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.733+0000 7ff931cfe700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 0x7ff91806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:09.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.733+0000 7ff931cfe700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 0x7ff91806e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff928006010 tx=0x7ff92800b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:09.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.737+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff9200595b0 con 0x7ff92c101dc0 2024-06-15T01:04:09.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.935+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff92c106710 con 0x7ff91806c4e0 2024-06-15T01:04:09.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.940+0000 7ff91effd700 1 -- 172.21.15.137:0/2784575668 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7ff92c106710 con 0x7ff91806c4e0 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter 2/2 2m ago 6m * 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:crash 2/2 2m ago 6m * 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 2/2 2m ago 6m count:2 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:mon 2/2 2m ago 5m smithi137:172.21.15.137=smithi137;smithi142:172.21.15.142=smithi142;count:2 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:osd 6 2m ago - 2024-06-15T01:04:09.941 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 msgr2=0x7ff91806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 0x7ff91806e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff928006010 tx=0x7ff92800b540 comp rx=0 tx=0).stop 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 msgr2=0x7ff92c1968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b4b0 tx=0x7ff92000b7c0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 shutdown_connections 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff91806c4e0 0x7ff91806e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff92c100bc0 0x7ff92c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 --2- 172.21.15.137:0/2784575668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff92c101dc0 0x7ff92c1968e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 >> 172.21.15.137:0/2784575668 conn(0x7ff92c0fc140 msgr2=0x7ff92c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:09.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 shutdown_connections 2024-06-15T01:04:09.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:09.942+0000 7ff933f62700 1 -- 172.21.15.137:0/2784575668 wait complete. 2024-06-15T01:04:10.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-15T01:04:10.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:10 smithi137 ceph-mon[29354]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:10.743 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:10.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:10 smithi142 ceph-mon[37379]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:11.804 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:11 smithi137 ceph-mon[29354]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:11.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:11 smithi142 ceph-mon[37379]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.324+0000 7f8d12e80700 1 -- 172.21.15.137:0/910395529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c100be0 msgr2=0x7f8d0c101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.324+0000 7f8d12e80700 1 --2- 172.21.15.137:0/910395529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c100be0 0x7f8d0c101000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009910 tx=0x7f8d00009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- 172.21.15.137:0/910395529 shutdown_connections 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 --2- 172.21.15.137:0/910395529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c101de0 0x7f8d0c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 --2- 172.21.15.137:0/910395529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c100be0 0x7f8d0c101000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- 172.21.15.137:0/910395529 >> 172.21.15.137:0/910395529 conn(0x7f8d0c0fc140 msgr2=0x7f8d0c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- 172.21.15.137:0/910395529 shutdown_connections 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- 172.21.15.137:0/910395529 wait complete. 2024-06-15T01:04:12.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 Processor -- start 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- start start 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c100be0 0x7f8d0c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c196da0 con 0x7f8d0c101de0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.325+0000 7f8d12e80700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c196ee0 con 0x7f8d0c100be0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59950/0 (socket says 172.21.15.137:59950) 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 -- 172.21.15.137:0/3084834936 learned_addr learned my addr 172.21.15.137:0/3084834936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d10c1c700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c100be0 0x7f8d0c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 -- 172.21.15.137:0/3084834936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c100be0 msgr2=0x7f8d0c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c100be0 0x7f8d0c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 -- 172.21.15.137:0/3084834936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d000095f0 con 0x7f8d0c101de0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d0bfff700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c930 tx=0x7f8cf800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf8007ad0 con 0x7f8d0c101de0 2024-06-15T01:04:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c19b9a0 con 0x7f8d0c101de0 2024-06-15T01:04:12.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8cf800ce80 con 0x7f8d0c101de0 2024-06-15T01:04:12.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf8018630 con 0x7f8d0c101de0 2024-06-15T01:04:12.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.326+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c19bf40 con 0x7f8d0c101de0 2024-06-15T01:04:12.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.327+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c109600 con 0x7f8d0c101de0 2024-06-15T01:04:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.328+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8cf8007c30 con 0x7f8d0c101de0 2024-06-15T01:04:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.328+0000 7f8d09ffb700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 0x7f8cfc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.328+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8cf808b480 con 0x7f8d0c101de0 2024-06-15T01:04:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.328+0000 7f8d10c1c700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 0x7f8cfc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.329+0000 7f8d10c1c700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 0x7f8cfc06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d000051b0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.333+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8cf8059a30 con 0x7f8d0c101de0 2024-06-15T01:04:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.528+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d0c106730 con 0x7f8cfc06c4e0 2024-06-15T01:04:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.530+0000 7f8d09ffb700 1 -- 172.21.15.137:0/3084834936 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f8d0c106730 con 0x7f8cfc06c4e0 2024-06-15T01:04:12.530 INFO:teuthology.orchestra.run.smithi137.stdout:HOST ADDR LABELS STATUS 2024-06-15T01:04:12.531 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 172.21.15.137 2024-06-15T01:04:12.531 INFO:teuthology.orchestra.run.smithi137.stdout:smithi142 172.21.15.142 2024-06-15T01:04:12.531 INFO:teuthology.orchestra.run.smithi137.stdout:2 hosts in cluster 2024-06-15T01:04:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 msgr2=0x7f8cfc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 0x7f8cfc06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d000051b0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 msgr2=0x7f8d0c196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c930 tx=0x7f8cf800cc40 comp rx=0 tx=0).stop 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 shutdown_connections 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8cfc06c4e0 0x7f8cfc06e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d0c100be0 0x7f8d0c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 --2- 172.21.15.137:0/3084834936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d0c101de0 0x7f8d0c196810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 >> 172.21.15.137:0/3084834936 conn(0x7f8d0c0fc140 msgr2=0x7f8d0c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 shutdown_connections 2024-06-15T01:04:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:12.532+0000 7f8d12e80700 1 -- 172.21.15.137:0/3084834936 wait complete. 2024-06-15T01:04:12.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:12 smithi142 ceph-mon[37379]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:12.835 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:12 smithi137 ceph-mon[29354]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:13.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-15T01:04:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:13.788 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:13 smithi137 ceph-mon[29354]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:13.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:13 smithi142 ceph-mon[37379]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:14.584 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:14 smithi137 ceph-mon[29354]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:14.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 -- 172.21.15.137:0/1862793088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 msgr2=0x7f9b800ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:14.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 --2- 172.21.15.137:0/1862793088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b800ff630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400aa20 tx=0x7f9b7400ad30 comp rx=0 tx=0).stop 2024-06-15T01:04:14.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 -- 172.21.15.137:0/1862793088 shutdown_connections 2024-06-15T01:04:14.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 --2- 172.21.15.137:0/1862793088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 0x7f9b80100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:14.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 --2- 172.21.15.137:0/1862793088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b800ff630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.797+0000 7f9b86e14700 1 -- 172.21.15.137:0/1862793088 >> 172.21.15.137:0/1862793088 conn(0x7f9b800fa7b0 msgr2=0x7f9b800fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 -- 172.21.15.137:0/1862793088 shutdown_connections 2024-06-15T01:04:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 -- 172.21.15.137:0/1862793088 wait complete. 2024-06-15T01:04:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 Processor -- start 2024-06-15T01:04:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 -- start start 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 0x7f9b80196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80196d90 con 0x7f9b800ff210 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b86e14700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80196ed0 con 0x7f9b80100410 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b84bb0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b84bb0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59974/0 (socket says 172.21.15.137:59974) 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.798+0000 7f9b84bb0700 1 -- 172.21.15.137:0/325738116 learned_addr learned my addr 172.21.15.137:0/325738116 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b7ffff700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 0x7f9b80196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b84bb0700 1 -- 172.21.15.137:0/325738116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 msgr2=0x7f9b80196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b84bb0700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 0x7f9b80196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b84bb0700 1 -- 172.21.15.137:0/325738116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7400a700 con 0x7f9b800ff210 2024-06-15T01:04:14.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b84bb0700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b74000c00 tx=0x7f9b7400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b74004ca0 con 0x7f9b800ff210 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b74004e00 con 0x7f9b800ff210 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8019b920 con 0x7f9b800ff210 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b74016940 con 0x7f9b800ff210 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.799+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8019be90 con 0x7f9b800ff210 2024-06-15T01:04:14.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.800+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b801901e0 con 0x7f9b800ff210 2024-06-15T01:04:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.801+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9b7400ecc0 con 0x7f9b800ff210 2024-06-15T01:04:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.801+0000 7f9b7dffb700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 0x7f9b7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.801+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9b74012070 con 0x7f9b800ff210 2024-06-15T01:04:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.801+0000 7f9b7ffff700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 0x7f9b7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.802+0000 7f9b7ffff700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 0x7f9b7006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005950 tx=0x7f9b6c00a300 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:14.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:14.806+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9b7405bc50 con 0x7f9b800ff210 2024-06-15T01:04:14.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:14 smithi142 ceph-mon[37379]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:15.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.003+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b800611d0 con 0x7f9b7006c4e0 2024-06-15T01:04:15.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.005+0000 7f9b7dffb700 1 -- 172.21.15.137:0/325738116 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f9b800611d0 con 0x7f9b7006c4e0 2024-06-15T01:04:15.005 INFO:teuthology.orchestra.run.smithi137.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-15T01:04:15.005 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-06-15T01:04:15.005 INFO:teuthology.orchestra.run.smithi137.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 70s ago Has a FileSystem, LVM detected 2024-06-15T01:04:15.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 msgr2=0x7f9b7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:15.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 0x7f9b7006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005950 tx=0x7f9b6c00a300 comp rx=0 tx=0).stop 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 msgr2=0x7f9b80196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b74000c00 tx=0x7f9b7400cd90 comp rx=0 tx=0).stop 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 shutdown_connections 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f9b7006c4e0 0x7f9b7006e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b800ff210 0x7f9b80196230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 --2- 172.21.15.137:0/325738116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b80100410 0x7f9b80196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 >> 172.21.15.137:0/325738116 conn(0x7f9b800fa7b0 msgr2=0x7f9b80103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 shutdown_connections 2024-06-15T01:04:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:15.007+0000 7f9b86e14700 1 -- 172.21.15.137:0/325738116 wait complete. 2024-06-15T01:04:15.598 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:15.607 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:15.607 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-06-15T01:04:15.818 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:15.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:15 smithi142 ceph-mon[37379]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:15.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:15 smithi142 ceph-mon[37379]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:15.939 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:15 smithi137 ceph-mon[29354]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:15.939 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:15 smithi137 ceph-mon[29354]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:17.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 -- 172.21.15.137:0/1322967348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4380ffab0 msgr2=0x7fd4380ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:17.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 --2- 172.21.15.137:0/1322967348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4380ffab0 0x7fd4380ffed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd428009910 tx=0x7fd428009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:17.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 -- 172.21.15.137:0/1322967348 shutdown_connections 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 --2- 172.21.15.137:0/1322967348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd438100cb0 0x7fd438101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 --2- 172.21.15.137:0/1322967348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4380ffab0 0x7fd4380ffed0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 -- 172.21.15.137:0/1322967348 >> 172.21.15.137:0/1322967348 conn(0x7fd4380fb030 msgr2=0x7fd4380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 -- 172.21.15.137:0/1322967348 shutdown_connections 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 -- 172.21.15.137:0/1322967348 wait complete. 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.305+0000 7fd43cda0700 1 Processor -- start 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43cda0700 1 -- start start 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43cda0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 0x7fd438196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43cda0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43cda0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438196db0 con 0x7fd438100cb0 2024-06-15T01:04:17.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43cda0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438196ef0 con 0x7fd4380ffab0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52928/0 (socket says 172.21.15.137:52928) 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 -- 172.21.15.137:0/3116198995 learned_addr learned my addr 172.21.15.137:0/3116198995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43659c700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 0x7fd438196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 -- 172.21.15.137:0/3116198995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 msgr2=0x7fd438196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 0x7fd438196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd42dd9b700 1 -- 172.21.15.137:0/3116198995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280095f0 con 0x7fd438100cb0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.306+0000 7fd43659c700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 0x7fd438196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd42dd9b700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c8a0 tx=0x7fd42000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42000cea0 con 0x7fd438100cb0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd420004830 con 0x7fd438100cb0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43819b9a0 con 0x7fd438100cb0 2024-06-15T01:04:17.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd420005540 con 0x7fd438100cb0 2024-06-15T01:04:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.307+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43819bf70 con 0x7fd438100cb0 2024-06-15T01:04:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.308+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4381084d0 con 0x7fd438100cb0 2024-06-15T01:04:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.308+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd4200049a0 con 0x7fd438100cb0 2024-06-15T01:04:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.309+0000 7fd42ffff700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 0x7fd42406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.309+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd42008b1a0 con 0x7fd438100cb0 2024-06-15T01:04:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.309+0000 7fd43659c700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 0x7fd42406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.309+0000 7fd43659c700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 0x7fd42406e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd428006010 tx=0x7fd428005920 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.313+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd420059750 con 0x7fd438100cb0 2024-06-15T01:04:17.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:17.531+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd438068b20 con 0x7fd42406c4e0 2024-06-15T01:04:18.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:18 smithi137 ceph-mon[29354]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:18.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:18 smithi137 ceph-mon[29354]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:18.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-06-15T01:04:18.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:18.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:18 smithi142 ceph-mon[37379]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:18.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:18 smithi142 ceph-mon[37379]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:04:18.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-06-15T01:04:18.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:19.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.441+0000 7fd42ffff700 1 -- 172.21.15.137:0/3116198995 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd438068b20 con 0x7fd42406c4e0 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 msgr2=0x7fd42406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 0x7fd42406e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd428006010 tx=0x7fd428005920 comp rx=0 tx=0).stop 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 msgr2=0x7fd438196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c8a0 tx=0x7fd42000cc60 comp rx=0 tx=0).stop 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 shutdown_connections 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fd42406c4e0 0x7fd42406e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4380ffab0 0x7fd438196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 --2- 172.21.15.137:0/3116198995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd438100cb0 0x7fd438196790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 >> 172.21.15.137:0/3116198995 conn(0x7fd4380fb030 msgr2=0x7fd438068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.445+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 shutdown_connections 2024-06-15T01:04:19.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:19.446+0000 7fd43cda0700 1 -- 172.21.15.137:0/3116198995 wait complete. 2024-06-15T01:04:19.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:19 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-06-15T01:04:19.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:19 smithi137 ceph-mon[29354]: osdmap e42: 6 total, 6 up, 6 in 2024-06-15T01:04:19.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:19 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-06-15T01:04:19.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:19 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[29350]: 2024-06-15T01:04:19.418+0000 7f0e4442a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:04:19.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:19 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-06-15T01:04:19.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:19 smithi142 ceph-mon[37379]: osdmap e42: 6 total, 6 up, 6 in 2024-06-15T01:04:19.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:19 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-06-15T01:04:20.008 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-15T01:04:20.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: pgmap v173: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: osdmap e43: 6 total, 6 up, 6 in 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: osdmap e44: 6 total, 6 up, 6 in 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: fsmap cephfs:0 2024-06-15T01:04:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: Saving service mds.cephfs spec with placement count:4 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:20.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:20 smithi137 ceph-mon[29354]: Deploying daemon mds.cephfs.smithi137.nsntjq on smithi137 2024-06-15T01:04:20.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: pgmap v173: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: osdmap e43: 6 total, 6 up, 6 in 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: osdmap e44: 6 total, 6 up, 6 in 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: fsmap cephfs:0 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: Saving service mds.cephfs spec with placement count:4 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:20.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:20.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:20 smithi142 ceph-mon[37379]: Deploying daemon mds.cephfs.smithi137.nsntjq on smithi137 2024-06-15T01:04:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:21.625 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:21 smithi137 ceph-mon[29354]: osdmap e45: 6 total, 6 up, 6 in 2024-06-15T01:04:21.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:21 smithi142 ceph-mon[37379]: osdmap e45: 6 total, 6 up, 6 in 2024-06-15T01:04:22.702 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:22 smithi137 ceph-mon[29354]: pgmap v177: 65 pgs: 1 creating+activating, 10 creating+peering, 21 active+clean, 33 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:22.703 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:22 smithi137 ceph-mon[29354]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-15T01:04:22.703 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:22 smithi137 ceph-mon[29354]: osdmap e46: 6 total, 6 up, 6 in 2024-06-15T01:04:22.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:22 smithi142 ceph-mon[37379]: pgmap v177: 65 pgs: 1 creating+activating, 10 creating+peering, 21 active+clean, 33 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:22.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:22 smithi142 ceph-mon[37379]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-15T01:04:22.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:22 smithi142 ceph-mon[37379]: osdmap e46: 6 total, 6 up, 6 in 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.380+0000 7f11463ed700 1 -- 172.21.15.137:0/2868969045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1140101e70 msgr2=0x7f11401022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.380+0000 7f11463ed700 1 --2- 172.21.15.137:0/2868969045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1140101e70 0x7f11401022d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1134009910 tx=0x7f1134009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 -- 172.21.15.137:0/2868969045 shutdown_connections 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 --2- 172.21.15.137:0/2868969045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1140101e70 0x7f11401022d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 --2- 172.21.15.137:0/2868969045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11400fee90 0x7f11400ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 -- 172.21.15.137:0/2868969045 >> 172.21.15.137:0/2868969045 conn(0x7f11400fa7f0 msgr2=0x7f11400fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 -- 172.21.15.137:0/2868969045 shutdown_connections 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 -- 172.21.15.137:0/2868969045 wait complete. 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 Processor -- start 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.381+0000 7f11463ed700 1 -- start start 2024-06-15T01:04:23.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f11463ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f11463ed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140101e70 0x7f11401967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f11463ed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140196df0 con 0x7f11400fee90 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f11463ed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140196f30 con 0x7f1140101e70 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52976/0 (socket says 172.21.15.137:52976) 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 -- 172.21.15.137:0/602845803 learned_addr learned my addr 172.21.15.137:0/602845803 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113f7fe700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140101e70 0x7f11401967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 -- 172.21.15.137:0/602845803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140101e70 msgr2=0x7f11401967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140101e70 0x7f11401967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 -- 172.21.15.137:0/602845803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340095f0 con 0x7f11400fee90 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113ffff700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f113000d900 tx=0x7f113000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11300041f0 con 0x7f11400fee90 2024-06-15T01:04:23.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1130004350 con 0x7f11400fee90 2024-06-15T01:04:23.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114019b9e0 con 0x7f11400fee90 2024-06-15T01:04:23.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.382+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1130003bf0 con 0x7f11400fee90 2024-06-15T01:04:23.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.383+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114019bfb0 con 0x7f11400fee90 2024-06-15T01:04:23.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.384+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11401095f0 con 0x7f11400fee90 2024-06-15T01:04:23.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.384+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1130009730 con 0x7f11400fee90 2024-06-15T01:04:23.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.384+0000 7f113d7fa700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 0x7f112806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:23.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.385+0000 7f113f7fe700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 0x7f112806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:23.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.385+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f113008b6d0 con 0x7f11400fee90 2024-06-15T01:04:23.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.385+0000 7f113f7fe700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 0x7f112806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1134006010 tx=0x7f113400b540 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:23.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.389+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f11300599c0 con 0x7f11400fee90 2024-06-15T01:04:23.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.631+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f114019c6b0 con 0x7f11400fee90 2024-06-15T01:04:23.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.631+0000 7f113d7fa700 1 -- 172.21.15.137:0/602845803 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7f1130059550 con 0x7f11400fee90 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:e4 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 4 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:04:23.440259+0000 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:04:23.632 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 0 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 1 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:in 0 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480} 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:04:23.633 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:04:23.634 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:04:23.634 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 0 2024-06-15T01:04:23.634 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:creating seq 1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:04:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 msgr2=0x7f112806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 0x7f112806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1134006010 tx=0x7f113400b540 comp rx=0 tx=0).stop 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 msgr2=0x7f1140196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f113000d900 tx=0x7f113000dc10 comp rx=0 tx=0).stop 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 shutdown_connections 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f112806c4e0 0x7f112806e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11400fee90 0x7f1140196290 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 --2- 172.21.15.137:0/602845803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1140101e70 0x7f11401967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 >> 172.21.15.137:0/602845803 conn(0x7f11400fa7f0 msgr2=0x7f11401050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 shutdown_connections 2024-06-15T01:04:23.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:23.634+0000 7f11463ed700 1 -- 172.21.15.137:0/602845803 wait complete. 2024-06-15T01:04:23.636 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 4 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: pgmap v179: 65 pgs: 1 creating+activating, 10 creating+peering, 42 active+clean, 12 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:24.229 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: Deploying daemon mds.cephfs.smithi142.examgd on smithi142 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:boot 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: daemon mds.cephfs.smithi137.nsntjq assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: Cluster is now healthy 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: fsmap cephfs:0 1 up:standby 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:creating} 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: daemon mds.cephfs.smithi137.nsntjq is now active in filesystem cephfs as rank 0 2024-06-15T01:04:24.230 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:24 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/602845803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: pgmap v179: 65 pgs: 1 creating+activating, 10 creating+peering, 42 active+clean, 12 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: Deploying daemon mds.cephfs.smithi142.examgd on smithi142 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:boot 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: daemon mds.cephfs.smithi137.nsntjq assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-15T01:04:24.322 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: Cluster is now healthy 2024-06-15T01:04:24.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: fsmap cephfs:0 1 up:standby 2024-06-15T01:04:24.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:04:24.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:creating} 2024-06-15T01:04:24.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: daemon mds.cephfs.smithi137.nsntjq is now active in filesystem cephfs as rank 0 2024-06-15T01:04:24.323 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:24 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/602845803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:04:24.345 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:24.358 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:24.358 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-06-15T01:04:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:25.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:25 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:active 2024-06-15T01:04:25.628 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:25 smithi137 ceph-mon[29354]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:active} 2024-06-15T01:04:25.711 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:25 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:active 2024-06-15T01:04:25.711 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:25 smithi142 ceph-mon[37379]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:active} 2024-06-15T01:04:26.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 -- 172.21.15.137:0/3663072432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 msgr2=0x7f00801022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:26.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 --2- 172.21.15.137:0/3663072432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801022c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0070009910 tx=0x7f0070009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:26.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 -- 172.21.15.137:0/3663072432 shutdown_connections 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 --2- 172.21.15.137:0/3663072432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801022c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 --2- 172.21.15.137:0/3663072432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 0x7f00800ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.158+0000 7f0087a4a700 1 -- 172.21.15.137:0/3663072432 >> 172.21.15.137:0/3663072432 conn(0x7f00800fa7c0 msgr2=0x7f00800fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.159+0000 7f0087a4a700 1 -- 172.21.15.137:0/3663072432 shutdown_connections 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.159+0000 7f0087a4a700 1 -- 172.21.15.137:0/3663072432 wait complete. 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.159+0000 7f0087a4a700 1 Processor -- start 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.159+0000 7f0087a4a700 1 -- start start 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.159+0000 7f0087a4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 0x7f00801962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0087a4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0087a4a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0080196e10 con 0x7f0080101e60 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0087a4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0080196f50 con 0x7f00800fee80 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:26.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57796/0 (socket says 172.21.15.137:57796) 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 -- 172.21.15.137:0/2527323006 learned_addr learned my addr 172.21.15.137:0/2527323006 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f00857e6700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 0x7f00801962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 -- 172.21.15.137:0/2527323006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 msgr2=0x7f00801962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 0x7f00801962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 -- 172.21.15.137:0/2527323006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700095f0 con 0x7f0080101e60 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0084fe5700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f007000bde0 tx=0x7f007000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f007001d070 con 0x7f0080101e60 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008019b9a0 con 0x7f0080101e60 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0070005dc0 con 0x7f0080101e60 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0070003b10 con 0x7f0080101e60 2024-06-15T01:04:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.160+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008019beb0 con 0x7f0080101e60 2024-06-15T01:04:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.162+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00801095e0 con 0x7f0080101e60 2024-06-15T01:04:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.162+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0070003c70 con 0x7f0080101e60 2024-06-15T01:04:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.162+0000 7f00767fc700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 0x7f006c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.163+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f007008cfc0 con 0x7f0080101e60 2024-06-15T01:04:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.163+0000 7f00857e6700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 0x7f006c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.163+0000 7f00857e6700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 0x7f006c06e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0080068210 tx=0x7f007c009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.167+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f007005c160 con 0x7f0080101e60 2024-06-15T01:04:26.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.410+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f008019c5b0 con 0x7f0080101e60 2024-06-15T01:04:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.453+0000 7f00767fc700 1 -- 172.21.15.137:0/2527323006 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f007005bcf0 con 0x7f0080101e60 2024-06-15T01:04:26.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 msgr2=0x7f006c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:26.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 0x7f006c06e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0080068210 tx=0x7f007c009260 comp rx=0 tx=0).stop 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 msgr2=0x7f00801967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f007000bde0 tx=0x7f007000be10 comp rx=0 tx=0).stop 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 shutdown_connections 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f006c06c4e0 0x7f006c06e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00800fee80 0x7f00801962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 --2- 172.21.15.137:0/2527323006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101e60 0x7f00801967f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 >> 172.21.15.137:0/2527323006 conn(0x7f00800fa7c0 msgr2=0x7f0080105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 shutdown_connections 2024-06-15T01:04:26.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:26.455+0000 7f0087a4a700 1 -- 172.21.15.137:0/2527323006 wait complete. 2024-06-15T01:04:26.676 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: pgmap v180: 65 pgs: 1 creating+activating, 64 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:26.676 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:26.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:26 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2527323006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: pgmap v180: 65 pgs: 1 creating+activating, 64 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:26.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:26.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:26.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:26 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2527323006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: Deploying daemon mds.cephfs.smithi137.afplli on smithi137 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.142:6824/2333305414,v1:172.21.15.142:6825/2333305414] up:boot 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2527323006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: daemon mds.cephfs.smithi142.examgd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-06-15T01:04:27.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-15T01:04:27.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:active} 1 up:standby 2024-06-15T01:04:27.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:04:27.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:creating} 2024-06-15T01:04:27.480 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:27 smithi137 ceph-mon[29354]: daemon mds.cephfs.smithi142.examgd is now active in filesystem cephfs as rank 1 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: Deploying daemon mds.cephfs.smithi137.afplli on smithi137 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.142:6824/2333305414,v1:172.21.15.142:6825/2333305414] up:boot 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2527323006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: daemon mds.cephfs.smithi142.examgd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: fsmap cephfs:1 {0=cephfs.smithi137.nsntjq=up:active} 1 up:standby 2024-06-15T01:04:27.768 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:04:27.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:creating} 2024-06-15T01:04:27.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:27 smithi142 ceph-mon[37379]: daemon mds.cephfs.smithi142.examgd is now active in filesystem cephfs as rank 1 2024-06-15T01:04:27.984 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:27.992 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:27.993 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-06-15T01:04:28.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:28 smithi137 ceph-mon[29354]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-15T01:04:28.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:28 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.142:6824/2333305414,v1:172.21.15.142:6825/2333305414] up:active 2024-06-15T01:04:28.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:28 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2024-06-15T01:04:28.613 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:28.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:28 smithi142 ceph-mon[37379]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-15T01:04:28.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:28 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.142:6824/2333305414,v1:172.21.15.142:6825/2333305414] up:active 2024-06-15T01:04:28.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:28 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2024-06-15T01:04:29.881 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: Deploying daemon mds.cephfs.smithi142.uczriv on smithi142 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s wr, 8 op/s 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: Cluster is now healthy 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:boot 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 1 up:standby 2024-06-15T01:04:29.882 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:04:30.034 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:30.034 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: Deploying daemon mds.cephfs.smithi142.uczriv on smithi142 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s wr, 8 op/s 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: Cluster is now healthy 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:boot 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 1 up:standby 2024-06-15T01:04:30.035 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 -- 172.21.15.137:0/972292202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150100be0 msgr2=0x7fa150101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 --2- 172.21.15.137:0/972292202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150100be0 0x7fa150101000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa140009910 tx=0x7fa140009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 -- 172.21.15.137:0/972292202 shutdown_connections 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 --2- 172.21.15.137:0/972292202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150101de0 0x7fa150102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 --2- 172.21.15.137:0/972292202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150100be0 0x7fa150101000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 -- 172.21.15.137:0/972292202 >> 172.21.15.137:0/972292202 conn(0x7fa1500fc140 msgr2=0x7fa1500fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:30.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 -- 172.21.15.137:0/972292202 shutdown_connections 2024-06-15T01:04:30.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.129+0000 7fa1561d8700 1 -- 172.21.15.137:0/972292202 wait complete. 2024-06-15T01:04:30.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.130+0000 7fa1561d8700 1 Processor -- start 2024-06-15T01:04:30.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.130+0000 7fa1561d8700 1 -- start start 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150100be0 0x7fa1501962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150196e30 con 0x7fa150101de0 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150196f70 con 0x7fa150100be0 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150100be0 0x7fa1501962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57842/0 (socket says 172.21.15.137:57842) 2024-06-15T01:04:30.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 -- 172.21.15.137:0/629454013 learned_addr learned my addr 172.21.15.137:0/629454013 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 -- 172.21.15.137:0/629454013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150100be0 msgr2=0x7fa1501962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150100be0 0x7fa1501962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 -- 172.21.15.137:0/629454013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400095f0 con 0x7fa150101de0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14effd700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d900 tx=0x7fa14400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1440041f0 con 0x7fa150101de0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15019ba20 con 0x7fa150101de0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa144004cc0 con 0x7fa150101de0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa144005660 con 0x7fa150101de0 2024-06-15T01:04:30.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.131+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15019bf70 con 0x7fa150101de0 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.132+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150109750 con 0x7fa150101de0 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.133+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa144004830 con 0x7fa150101de0 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.133+0000 7fa14cff9700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 0x7fa13806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.134+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa14408b690 con 0x7fa150101de0 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.134+0000 7fa14f7fe700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 0x7fa13806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:30.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.134+0000 7fa14f7fe700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 0x7fa13806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa140005ec0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:30.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.138+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa144059980 con 0x7fa150101de0 2024-06-15T01:04:30.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.368+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa15019c640 con 0x7fa150101de0 2024-06-15T01:04:30.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.667+0000 7fa14cff9700 1 -- 172.21.15.137:0/629454013 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1440160c0 con 0x7fa150101de0 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.669+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 msgr2=0x7fa13806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.669+0000 7fa1561d8700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 0x7fa13806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa140005ec0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.669+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 msgr2=0x7fa150196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.669+0000 7fa1561d8700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d900 tx=0x7fa14400dc10 comp rx=0 tx=0).stop 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 shutdown_connections 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fa13806c4e0 0x7fa13806e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa150100be0 0x7fa1501962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 --2- 172.21.15.137:0/629454013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150101de0 0x7fa150196810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:30.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 >> 172.21.15.137:0/629454013 conn(0x7fa1500fc140 msgr2=0x7fa150105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:30.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 shutdown_connections 2024-06-15T01:04:30.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:30.670+0000 7fa1561d8700 1 -- 172.21.15.137:0/629454013 wait complete. 2024-06-15T01:04:30.914 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:30 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/629454013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-06-15T01:04:30.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:30 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/629454013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-06-15T01:04:31.323 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:31.332 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:31.332 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-06-15T01:04:31.554 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/629454013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 1 up:standby 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-06-15T01:04:31.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:31 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/629454013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 1 up:standby 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.960 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.961 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:31.961 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-06-15T01:04:31.961 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:31 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:32.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:32 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] up:boot 2024-06-15T01:04:32.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:32 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:active 2024-06-15T01:04:32.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:32 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2 up:standby 2024-06-15T01:04:32.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:32 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:04:32.959 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:32 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] up:boot 2024-06-15T01:04:32.959 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:32 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] up:active 2024-06-15T01:04:32.959 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:32 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2 up:standby 2024-06-15T01:04:32.959 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:32 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:04:33.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 -- 172.21.15.137:0/2046897448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 msgr2=0x7fe0540ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:33.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 --2- 172.21.15.137:0/2046897448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe0540ffed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe044009910 tx=0x7fe044009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 -- 172.21.15.137:0/2046897448 shutdown_connections 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 --2- 172.21.15.137:0/2046897448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe054101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 --2- 172.21.15.137:0/2046897448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe0540ffed0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 -- 172.21.15.137:0/2046897448 >> 172.21.15.137:0/2046897448 conn(0x7fe0540fb010 msgr2=0x7fe0540fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 -- 172.21.15.137:0/2046897448 shutdown_connections 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.927+0000 7fe05a57c700 1 -- 172.21.15.137:0/2046897448 wait complete. 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 Processor -- start 2024-06-15T01:04:33.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 -- start start 2024-06-15T01:04:33.929 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:standby 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: Dropping low affinity active daemon mds.cephfs.smithi142.examgd in favor of higher affinity standby. 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: Replacing daemon mds.cephfs.smithi142.examgd as rank 1 with standby daemon mds.cephfs.smithi137.afplli 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2 up:standby 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: osdmap e47: 6 total, 6 up, 6 in 2024-06-15T01:04:33.930 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:33 smithi137 ceph-mon[29354]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:replay} 1 up:standby 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe05419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419b6b0 con 0x7fe0540ffab0 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.928+0000 7fe05a57c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419b820 con 0x7fe054100cb0 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57860/0 (socket says 172.21.15.137:57860) 2024-06-15T01:04:33.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 -- 172.21.15.137:0/3021496647 learned_addr learned my addr 172.21.15.137:0/3021496647 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe0537fe700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe05419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 -- 172.21.15.137:0/3021496647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 msgr2=0x7fe05419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe05419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 -- 172.21.15.137:0/3021496647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0440095f0 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe0537fe700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe05419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe053fff700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe04400bda0 tx=0x7fe04400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04401d070 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419baa0 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe044022470 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04400f690 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.929+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05419c010 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.930+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054108550 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.931+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe04400f900 con 0x7fe0540ffab0 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.931+0000 7fe0517fa700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:33.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.931+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe04408cfc0 con 0x7fe0540ffab0 2024-06-15T01:04:33.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.931+0000 7fe0537fe700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:33.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.932+0000 7fe0537fe700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe054069ed0 tx=0x7fe048008040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:33.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:33.936+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe04405b410 con 0x7fe0540ffab0 2024-06-15T01:04:34.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:34.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:standby 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: Dropping low affinity active daemon mds.cephfs.smithi142.examgd in favor of higher affinity standby. 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: Replacing daemon mds.cephfs.smithi142.examgd as rank 1 with standby daemon mds.cephfs.smithi137.afplli 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi142.examgd=up:active} 2 up:standby 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: osdmap e47: 6 total, 6 up, 6 in 2024-06-15T01:04:34.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:33 smithi142 ceph-mon[37379]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:replay} 1 up:standby 2024-06-15T01:04:34.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.185+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fe054108740 con 0x7fe0540ffab0 2024-06-15T01:04:34.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.400+0000 7fe0517fa700 1 -- 172.21.15.137:0/3021496647 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v14) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe0440270b0 con 0x7fe0540ffab0 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 msgr2=0x7fe03c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe054069ed0 tx=0x7fe048008040 comp rx=0 tx=0).stop 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 msgr2=0x7fe054069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe04400bda0 tx=0x7fe04400be80 comp rx=0 tx=0).stop 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 shutdown_connections 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffab0 0x7fe054069970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:34.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 --2- 172.21.15.137:0/3021496647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe054100cb0 0x7fe05419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:34.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 >> 172.21.15.137:0/3021496647 conn(0x7fe0540fb010 msgr2=0x7fe054103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:34.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 shutdown_connections 2024-06-15T01:04:34.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:34.403+0000 7fe05a57c700 1 -- 172.21.15.137:0/3021496647 wait complete. 2024-06-15T01:04:34.404 INFO:teuthology.orchestra.run.smithi137.stderr:inline data disabled 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3021496647' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:resolve 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] up:boot 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3021496647' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:04:34.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3021496647' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:resolve 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] up:boot 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3021496647' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:04:35.071 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:04:35.392 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:04:35.402 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:04:35.403 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-15T01:04:36.178 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:36.634 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:36 smithi137 ceph-mon[29354]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 2.6 KiB/s wr, 8 op/s 2024-06-15T01:04:36.634 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:36 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:reconnect 2024-06-15T01:04:36.635 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:36 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] up:standby 2024-06-15T01:04:36.635 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:36 smithi137 ceph-mon[29354]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:reconnect} 2 up:standby 2024-06-15T01:04:36.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:36 smithi142 ceph-mon[37379]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 2.6 KiB/s wr, 8 op/s 2024-06-15T01:04:36.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:36 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:reconnect 2024-06-15T01:04:36.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:36 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] up:standby 2024-06-15T01:04:36.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:36 smithi142 ceph-mon[37379]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:reconnect} 2 up:standby 2024-06-15T01:04:37.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:37 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:rejoin 2024-06-15T01:04:37.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:37 smithi137 ceph-mon[29354]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:rejoin} 2 up:standby 2024-06-15T01:04:37.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:37 smithi137 ceph-mon[29354]: daemon mds.cephfs.smithi137.afplli is now active in filesystem cephfs as rank 1 2024-06-15T01:04:37.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:37 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:rejoin 2024-06-15T01:04:37.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:37 smithi142 ceph-mon[37379]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:rejoin} 2 up:standby 2024-06-15T01:04:37.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:37 smithi142 ceph-mon[37379]: daemon mds.cephfs.smithi137.afplli is now active in filesystem cephfs as rank 1 2024-06-15T01:04:38.612 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:38 smithi137 ceph-mon[29354]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-15T01:04:38.612 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:38 smithi137 ceph-mon[29354]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-15T01:04:38.612 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:38 smithi137 ceph-mon[29354]: Cluster is now healthy 2024-06-15T01:04:38.612 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:38 smithi137 ceph-mon[29354]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:active 2024-06-15T01:04:38.613 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:38 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:04:38.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:38 smithi142 ceph-mon[37379]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 1.6 KiB/s wr, 8 op/s 2024-06-15T01:04:38.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:38 smithi142 ceph-mon[37379]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-15T01:04:38.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:38 smithi142 ceph-mon[37379]: Cluster is now healthy 2024-06-15T01:04:38.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:38 smithi142 ceph-mon[37379]: mds.? [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] up:active 2024-06-15T01:04:38.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:38 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:04:39.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 -- 172.21.15.137:0/2167792186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00241021a0 msgr2=0x7f00241025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:39.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 --2- 172.21.15.137:0/2167792186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00241021a0 0x7f00241025c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0014009910 tx=0x7f0014009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 -- 172.21.15.137:0/2167792186 shutdown_connections 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 --2- 172.21.15.137:0/2167792186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00240fec20 0x7f00240ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 --2- 172.21.15.137:0/2167792186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00241021a0 0x7f00241025c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 -- 172.21.15.137:0/2167792186 >> 172.21.15.137:0/2167792186 conn(0x7f00240fa7c0 msgr2=0x7f00240fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 -- 172.21.15.137:0/2167792186 shutdown_connections 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.127+0000 7f002a112700 1 -- 172.21.15.137:0/2167792186 wait complete. 2024-06-15T01:04:39.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.128+0000 7f002a112700 1 Processor -- start 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.128+0000 7f002a112700 1 -- start start 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f002a112700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f002a112700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00241021a0 0x7f00241969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f002a112700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024196fc0 con 0x7f00240fec20 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f002a112700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024197100 con 0x7f00241021a0 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46256/0 (socket says 172.21.15.137:46256) 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 -- 172.21.15.137:0/1906083795 learned_addr learned my addr 172.21.15.137:0/1906083795 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:39.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 -- 172.21.15.137:0/1906083795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00241021a0 msgr2=0x7f00241969a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00241021a0 0x7f00241969a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.129+0000 7f00237fe700 1 -- 172.21.15.137:0/1906083795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140095f0 con 0x7f00240fec20 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f00237fe700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00140092e0 tx=0x7f001400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001401d070 con 0x7f00240fec20 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002419bb50 con 0x7f00240fec20 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f001400f460 con 0x7f00240fec20 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002419c040 con 0x7f00240fec20 2024-06-15T01:04:39.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.130+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0014021640 con 0x7f00240fec20 2024-06-15T01:04:39.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.131+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0010005320 con 0x7f00240fec20 2024-06-15T01:04:39.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.132+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f001402b440 con 0x7f00240fec20 2024-06-15T01:04:39.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.133+0000 7f0020ff9700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 0x7f000c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:39.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.133+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f001408cbd0 con 0x7f00240fec20 2024-06-15T01:04:39.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.133+0000 7f0022ffd700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 0x7f000c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:39.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.133+0000 7f0022ffd700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 0x7f000c06e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0018009730 tx=0x7f0018006cb0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:39.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.137+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f001405ae50 con 0x7f00240fec20 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.408+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0010006200 con 0x7f00240fec20 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout:e17 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:39.408 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 17 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:04:37.406114+0000 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:04:39.409 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:04:39.410 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:39.411 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:39.411 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:04:39.411 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:39.411 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:04:39.411 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.408+0000 7f0020ff9700 1 -- 172.21.15.137:0/1906083795 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f001405a9e0 con 0x7f00240fec20 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 msgr2=0x7f000c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 0x7f000c06e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0018009730 tx=0x7f0018006cb0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 msgr2=0x7f0024196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f00140092e0 tx=0x7f001400bf60 comp rx=0 tx=0).stop 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 shutdown_connections 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.411+0000 7f002a112700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f000c06c4e0 0x7f000c06e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.412+0000 7f002a112700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00240fec20 0x7f0024196460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.412+0000 7f002a112700 1 --2- 172.21.15.137:0/1906083795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00241021a0 0x7f00241969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.412+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 >> 172.21.15.137:0/1906083795 conn(0x7f00240fa7c0 msgr2=0x7f0024105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.412+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 shutdown_connections 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:39.412+0000 7f002a112700 1 -- 172.21.15.137:0/1906083795 wait complete. 2024-06-15T01:04:39.412 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 17 2024-06-15T01:04:40.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-06-15T01:04:40.318 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:40.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 409 B/s wr, 4 op/s 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1906083795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:04:40.321 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:40 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 409 B/s wr, 4 op/s 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1906083795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:04:40.414 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:40 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:04:42.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 -- 172.21.15.137:0/3259820698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 msgr2=0x7fc5900ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 --2- 172.21.15.137:0/3259820698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc5900ff690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009910 tx=0x7fc58c009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 -- 172.21.15.137:0/3259820698 shutdown_connections 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 --2- 172.21.15.137:0/3259820698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 0x7fc5901008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 --2- 172.21.15.137:0/3259820698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc5900ff690 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 -- 172.21.15.137:0/3259820698 >> 172.21.15.137:0/3259820698 conn(0x7fc5900fa810 msgr2=0x7fc5900fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 -- 172.21.15.137:0/3259820698 shutdown_connections 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.619+0000 7fc5975f7700 1 -- 172.21.15.137:0/3259820698 wait complete. 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 Processor -- start 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 -- start start 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 0x7fc590198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:42.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc590199090 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc5975f7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901991d0 con 0x7fc590100470 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46278/0 (socket says 172.21.15.137:46278) 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 -- 172.21.15.137:0/1022735390 learned_addr learned my addr 172.21.15.137:0/1022735390 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc594b92700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 0x7fc590198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 -- 172.21.15.137:0/1022735390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 msgr2=0x7fc590198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 0x7fc590198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 -- 172.21.15.137:0/1022735390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c0095f0 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.620+0000 7fc595393700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0092e0 tx=0x7fc58c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.621+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc58c01d070 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.621+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc58c005dc0 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.621+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59019dc20 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.621+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc58c00fe90 con 0x7fc5900ff270 2024-06-15T01:04:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.621+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59019e110 con 0x7fc5900ff270 2024-06-15T01:04:42.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.622+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc590107c90 con 0x7fc5900ff270 2024-06-15T01:04:42.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.622+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc58c00f6d0 con 0x7fc5900ff270 2024-06-15T01:04:42.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.622+0000 7fc5867fc700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 0x7fc57c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:42.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.623+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc58c08ca50 con 0x7fc5900ff270 2024-06-15T01:04:42.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.623+0000 7fc594b92700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 0x7fc57c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:42.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.623+0000 7fc594b92700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 0x7fc57c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc580005950 tx=0x7fc58000b410 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.627+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc58c05ad80 con 0x7fc5900ff270 2024-06-15T01:04:42.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:42 smithi137 ceph-mon[29354]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-06-15T01:04:42.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:42 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:42.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:42 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:42.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:42 smithi142 ceph-mon[37379]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-06-15T01:04:42.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:42 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:42.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:42 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:42.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.862+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc59019e750 con 0x7fc5900ff270 2024-06-15T01:04:42.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.863+0000 7fc5867fc700 1 -- 172.21.15.137:0/1022735390 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4757 (secure 0 0 0) 0x7fc58c05a910 con 0x7fc5900ff270 2024-06-15T01:04:42.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 msgr2=0x7fc57c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 0x7fc57c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc580005950 tx=0x7fc58000b410 comp rx=0 tx=0).stop 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 msgr2=0x7fc590198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0092e0 tx=0x7fc58c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 shutdown_connections 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc57c06c4e0 0x7fc57c06e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5900ff270 0x7fc590198530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 --2- 172.21.15.137:0/1022735390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc590100470 0x7fc590198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 >> 172.21.15.137:0/1022735390 conn(0x7fc5900fa810 msgr2=0x7fc5901036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 shutdown_connections 2024-06-15T01:04:42.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:42.865+0000 7fc5975f7700 1 -- 172.21.15.137:0/1022735390 wait complete. 2024-06-15T01:04:42.866 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 17 2024-06-15T01:04:42.875 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:04:43.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:43 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1022735390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:04:43.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:43 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1022735390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:04:43.988 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-06-15T01:04:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:44.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:44 smithi137 ceph-mon[29354]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-15T01:04:44.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:44 smithi142 ceph-mon[37379]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-15T01:04:46.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:46 smithi137 ceph-mon[29354]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-15T01:04:46.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:46 smithi142 ceph-mon[37379]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-15T01:04:47.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 -- 172.21.15.137:0/3651532997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 msgr2=0x7f0f281014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/3651532997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f281014f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000b0a0 tx=0x7f0f2000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 -- 172.21.15.137:0/3651532997 shutdown_connections 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/3651532997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f281001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/3651532997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f281014f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 -- 172.21.15.137:0/3651532997 >> 172.21.15.137:0/3651532997 conn(0x7f0f280fa7c0 msgr2=0x7f0f280fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 -- 172.21.15.137:0/3651532997 shutdown_connections 2024-06-15T01:04:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.271+0000 7f0f2e17b700 1 -- 172.21.15.137:0/3651532997 wait complete. 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 Processor -- start 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 -- start start 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f280ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819b6c0 con 0x7f0f281010d0 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f2e17b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819b830 con 0x7f0f280ffd30 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43814/0 (socket says 172.21.15.137:43814) 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 -- 172.21.15.137:0/1303078054 learned_addr learned my addr 172.21.15.137:0/1303078054 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f277fe700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f280ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 -- 172.21.15.137:0/1303078054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 msgr2=0x7f0f280ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f280ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f26ffd700 1 -- 172.21.15.137:0/1303078054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009d00 con 0x7f0f281010d0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.272+0000 7f0f277fe700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f280ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f26ffd700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000b4b0 tx=0x7f0f1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f10011800 con 0x7f0f281010d0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0f10011df0 con 0x7f0f281010d0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2819bb10 con 0x7f0f281010d0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f10010e40 con 0x7f0f281010d0 2024-06-15T01:04:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.273+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2819c0e0 con 0x7f0f281010d0 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.274+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28108570 con 0x7f0f281010d0 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.274+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0f10011960 con 0x7f0f281010d0 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.275+0000 7f0f24ff9700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 0x7f0f1406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.275+0000 7f0f277fe700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 0x7f0f1406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.275+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0f1008b4e0 con 0x7f0f281010d0 2024-06-15T01:04:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.275+0000 7f0f277fe700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 0x7f0f1406e950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f200062a0 tx=0x7f0f20006210 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:47.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.280+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f100596f0 con 0x7f0f281010d0 2024-06-15T01:04:47.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.560+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f0f2819c7b0 con 0x7f0f281010d0 2024-06-15T01:04:47.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.561+0000 7f0f24ff9700 1 -- 172.21.15.137:0/1303078054 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7f0f10059280 con 0x7f0f281010d0 2024-06-15T01:04:47.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 msgr2=0x7f0f1406e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 0x7f0f1406e950 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f200062a0 tx=0x7f0f20006210 comp rx=0 tx=0).stop 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 msgr2=0x7f0f2819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000b4b0 tx=0x7f0f1000b870 comp rx=0 tx=0).stop 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 shutdown_connections 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f0f1406c490 0x7f0f1406e950 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f280ffd30 0x7f0f280ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 --2- 172.21.15.137:0/1303078054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f281010d0 0x7f0f2819b0f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 >> 172.21.15.137:0/1303078054 conn(0x7f0f280fa7c0 msgr2=0x7f0f28103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 shutdown_connections 2024-06-15T01:04:47.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:47.563+0000 7f0f2e17b700 1 -- 172.21.15.137:0/1303078054 wait complete. 2024-06-15T01:04:47.573 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:04:48.196 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-06-15T01:04:48.208 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-15T01:04:48.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 -- 172.21.15.137:0/1679763631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40fe740 msgr2=0x7f49e40feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/1679763631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40fe740 0x7f49e40feb60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009880 tx=0x7f49d402bce0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 -- 172.21.15.137:0/1679763631 shutdown_connections 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/1679763631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40ff930 0x7f49e40ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/1679763631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40fe740 0x7f49e40feb60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.342+0000 7f49eaa1d640 1 -- 172.21.15.137:0/1679763631 >> 172.21.15.137:0/1679763631 conn(0x7f49e40f9cb0 msgr2=0x7f49e40fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 -- 172.21.15.137:0/1679763631 shutdown_connections 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 -- 172.21.15.137:0/1679763631 wait complete. 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 Processor -- start 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 -- start start 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40fe740 0x7f49e41016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e419aa60 con 0x7f49e40ff930 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49eaa1d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e419aba0 con 0x7f49e40fe740 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43830/0 (socket says 172.21.15.137:43830) 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.343+0000 7f49dbfff640 1 -- 172.21.15.137:0/2318470538 learned_addr learned my addr 172.21.15.137:0/2318470538 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49dbfff640 1 -- 172.21.15.137:0/2318470538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40fe740 msgr2=0x7f49e41016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49dbfff640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40fe740 0x7f49e41016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49dbfff640 1 -- 172.21.15.137:0/2318470538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009560 con 0x7f49e40ff930 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49dbfff640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c8a0 tx=0x7f49cc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc007b80 con 0x7f49e40ff930 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f49cc007ce0 con 0x7f49e40ff930 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e419ae80 con 0x7f49e40ff930 2024-06-15T01:04:48.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc0187f0 con 0x7f49e40ff930 2024-06-15T01:04:48.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.344+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e419b430 con 0x7f49e40ff930 2024-06-15T01:04:48.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.345+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e41071e0 con 0x7f49e40ff930 2024-06-15T01:04:48.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.345+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f49cc018950 con 0x7f49e40ff930 2024-06-15T01:04:48.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.346+0000 7f49e1ffb640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 0x7f49d006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.346+0000 7f49e3fff640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 0x7f49d006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.346+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f49cc08ba20 con 0x7f49e40ff930 2024-06-15T01:04:48.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.346+0000 7f49e3fff640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 0x7f49d006e9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036040 tx=0x7f49d4005d50 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.350+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49cc0561a0 con 0x7f49e40ff930 2024-06-15T01:04:48.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-15T01:04:48.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:48.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:48.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1303078054' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-06-15T01:04:48.479 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:48.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.523+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49e419b6b0 con 0x7f49e40ff930 2024-06-15T01:04:48.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.524+0000 7f49e1ffb640 1 -- 172.21.15.137:0/2318470538 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15204 (secure 0 0 0) 0x7f49cc0597c0 con 0x7f49e40ff930 2024-06-15T01:04:48.523 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:48.524 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":47,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:04:33.393640+0000","last_up_change":"2024-06-15T01:03:02.945497+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-06-15T01:04:17.764733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-06-15T01:04:18.637750+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6803","nonce":1195724024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6805","nonce":1195724024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6809","nonce":1195724024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1195724024},{"type":"v1","addr":"172.21.15.137:6807","nonce":1195724024}]},"public_addr":"172.21.15.137:6803/1195724024","cluster_addr":"172.21.15.137:6805/1195724024","heartbeat_back_addr":"172.21.15.137:6809/1195724024","heartbeat_front_addr":"172.21.15.137:6807/1195724024","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6811","nonce":4277122690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6813","nonce":4277122690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6817","nonce":4277122690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4277122690},{"type":"v1","addr":"172.21.15.137:6815","nonce":4277122690}]},"public_addr":"172.21.15.137:6811/4277122690","cluster_addr":"172.21.15.137:6813/4277122690","heartbeat_back_addr":"172.21.15.137:6817/4277122690","heartbeat_front_addr":"172.21.15.137:6815/4277122690","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6819","nonce":2443099131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6821","nonce":2443099131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6825","nonce":2443099131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2443099131},{"type":"v1","addr":"172.21.15.137:6823","nonce":2443099131}]},"public_addr":"172.21.15.137:6819/2443099131","cluster_addr":"172.21.15.137:6821/2443099131","heartbeat_back_addr":"172.21.15.137:6825/2443099131","heartbeat_front_addr":"172.21.15.137:6823/2443099131","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6801","nonce":3385845597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6803","nonce":3385845597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6807","nonce":3385845597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3385845597},{"type":"v1","addr":"172.21.15.142:6805","nonce":3385845597}]},"public_addr":"172.21.15.142:6801/3385845597","cluster_addr":"172.21.15.142:6803/3385845597","heartbeat_back_addr":"172.21.15.142:6807/3385845597","heartbeat_front_addr":"172.21.15.142:6805/3385845597","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6809","nonce":3568534083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6811","nonce":3568534083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6815","nonce":3568534083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3568534083},{"type":"v1","addr":"172.21.15.142:6813","nonce":3568534083}]},"public_addr":"172.21.15.142:6809/3568534083","cluster_addr":"172.21.15.142:6811/3568534083","heartbeat_back_addr":"172.21.15.142:6815/3568534083","heartbeat_front_addr":"172.21.15.142:6813/3568534083","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6817","nonce":3801228766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6819","nonce":3801228766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6823","nonce":3801228766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3801228766},{"type":"v1","addr":"172.21.15.142:6821","nonce":3801228766}]},"public_addr":"172.21.15.142:6817/3801228766","cluster_addr":"172.21.15.142:6819/3801228766","heartbeat_back_addr":"172.21.15.142:6823/3801228766","heartbeat_front_addr":"172.21.15.142:6821/3801228766","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.142:6824/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.142:6825/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:04:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 msgr2=0x7f49d006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 0x7f49d006e9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036040 tx=0x7f49d4005d50 comp rx=0 tx=0).stop 2024-06-15T01:04:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 msgr2=0x7f49e419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c8a0 tx=0x7f49cc00cd60 comp rx=0 tx=0).stop 2024-06-15T01:04:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 shutdown_connections 2024-06-15T01:04:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f49d006c510 0x7f49d006e9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e40ff930 0x7f49e419a520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 --2- 172.21.15.137:0/2318470538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e40fe740 0x7f49e41016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 >> 172.21.15.137:0/2318470538 conn(0x7f49e40f9cb0 msgr2=0x7f49e4103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:48.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 shutdown_connections 2024-06-15T01:04:48.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.526+0000 7f49eaa1d640 1 -- 172.21.15.137:0/2318470538 wait complete. 2024-06-15T01:04:48.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-15T01:01:49.347551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-06-15T01:04:17.764733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-06-15T01:04:18.637750+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-15T01:04:48.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-15T01:04:48.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/924661631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 msgr2=0x7f7c68102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/924661631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68102d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098b0 tx=0x7f7c6402bcd0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/924661631 shutdown_connections 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/924661631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 0x7f7c68105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/924661631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68102d50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/924661631 >> 172.21.15.137:0/924661631 conn(0x7f7c680fa370 msgr2=0x7f7c680fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/924661631 shutdown_connections 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.641+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/924661631 wait complete. 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 Processor -- start 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 -- start start 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 0x7f7c68195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68196250 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c681963c0 con 0x7f7c68103290 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43842/0 (socket says 172.21.15.137:43842) 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 -- 172.21.15.137:0/1793392197 learned_addr learned my addr 172.21.15.137:0/1793392197 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d19d640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 0x7f7c68195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 -- 172.21.15.137:0/1793392197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 msgr2=0x7f7c68195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 0x7f7c68195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 -- 172.21.15.137:0/1793392197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c64009560 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6d99e640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009880 tx=0x7f7c64003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c64003d00 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7c64003e60 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6819adf0 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.642+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c640348d0 con 0x7f7c68100960 2024-06-15T01:04:48.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.643+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6819b350 con 0x7f7c68100960 2024-06-15T01:04:48.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.644+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c68108b10 con 0x7f7c68100960 2024-06-15T01:04:48.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.644+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7c64034a80 con 0x7f7c68100960 2024-06-15T01:04:48.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.644+0000 7f7c5effd640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 0x7f7c5406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.645+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c640ae8d0 con 0x7f7c68100960 2024-06-15T01:04:48.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.645+0000 7f7c6d19d640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 0x7f7c5406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.645+0000 7f7c6d19d640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 0x7f7c5406e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005fd0 tx=0x7f7c58005e20 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.649+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7c64079170 con 0x7f7c68100960 2024-06-15T01:04:48.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.819+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c6819b570 con 0x7f7c68100960 2024-06-15T01:04:48.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.820+0000 7f7c5effd640 1 -- 172.21.15.137:0/1793392197 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c640430a0 con 0x7f7c68100960 2024-06-15T01:04:48.820 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-06-15T01:04:48.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-15T01:04:48.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:04:48.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:04:48.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:04:48.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1303078054' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-06-15T01:04:48.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 msgr2=0x7f7c5406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 0x7f7c5406e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005fd0 tx=0x7f7c58005e20 comp rx=0 tx=0).stop 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 msgr2=0x7f7c68195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009880 tx=0x7f7c64003670 comp rx=0 tx=0).stop 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 shutdown_connections 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f7c5406c450 0x7f7c5406e910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c68103290 0x7f7c68195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 --2- 172.21.15.137:0/1793392197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c68100960 0x7f7c68195740 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 >> 172.21.15.137:0/1793392197 conn(0x7f7c680fa370 msgr2=0x7f7c680fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:48.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 shutdown_connections 2024-06-15T01:04:48.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.822+0000 7f7c6e9a0640 1 -- 172.21.15.137:0/1793392197 wait complete. 2024-06-15T01:04:48.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-06-15T01:04:48.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.935+0000 7f798503a640 1 -- 172.21.15.137:0/456890717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79801005b0 msgr2=0x7f79801009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.935+0000 7f798503a640 1 --2- 172.21.15.137:0/456890717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79801005b0 0x7f79801009d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7968009880 tx=0x7f796802bce0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.935+0000 7f798503a640 1 -- 172.21.15.137:0/456890717 shutdown_connections 2024-06-15T01:04:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 --2- 172.21.15.137:0/456890717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79800ff1b0 0x7f79800ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 --2- 172.21.15.137:0/456890717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79801005b0 0x7f79801009d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 -- 172.21.15.137:0/456890717 >> 172.21.15.137:0/456890717 conn(0x7f79800f9cc0 msgr2=0x7f79800fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 -- 172.21.15.137:0/456890717 shutdown_connections 2024-06-15T01:04:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 -- 172.21.15.137:0/456890717 wait complete. 2024-06-15T01:04:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 Processor -- start 2024-06-15T01:04:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.936+0000 7f798503a640 1 -- start start 2024-06-15T01:04:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 0x7f798019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798019abb0 con 0x7f79800ff1b0 2024-06-15T01:04:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798019acf0 con 0x7f79801005b0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43856/0 (socket says 172.21.15.137:43856) 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 -- 172.21.15.137:0/3588658694 learned_addr learned my addr 172.21.15.137:0/3588658694 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797e59c640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 0x7f798019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 -- 172.21.15.137:0/3588658694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 msgr2=0x7f798019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 0x7f798019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 -- 172.21.15.137:0/3588658694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009560 con 0x7f79800ff1b0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797e59c640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 0x7f798019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f797ed9d640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7968000c00 tx=0x7f7968005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7968003c10 con 0x7f79800ff1b0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7968003d70 con 0x7f79800ff1b0 2024-06-15T01:04:48.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798019af40 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7968034830 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.937+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798019b4c0 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.938+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980107c00 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.939+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f796803e450 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.939+0000 7f7977fff640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 0x7f796c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.939+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f79680ae520 con 0x7f79800ff1b0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.939+0000 7f797e59c640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 0x7f796c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.940+0000 7f797e59c640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 0x7f796c06e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79800fe980 tx=0x7f7970005e50 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:48.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:48.944+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7968078ca0 con 0x7f79800ff1b0 2024-06-15T01:04:49.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.111+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f798019b8e0 con 0x7f79800ff1b0 2024-06-15T01:04:49.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.111+0000 7f7977fff640 1 -- 172.21.15.137:0/3588658694 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f796807c2c0 con 0x7f79800ff1b0 2024-06-15T01:04:49.111 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:04:49.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 msgr2=0x7f796c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 0x7f796c06e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79800fe980 tx=0x7f7970005e50 comp rx=0 tx=0).stop 2024-06-15T01:04:49.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 msgr2=0x7f79800fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7968000c00 tx=0x7f7968005e60 comp rx=0 tx=0).stop 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 shutdown_connections 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f796c06c450 0x7f796c06e910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79801005b0 0x7f798019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 --2- 172.21.15.137:0/3588658694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79800ff1b0 0x7f79800fec10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 >> 172.21.15.137:0/3588658694 conn(0x7f79800f9cc0 msgr2=0x7f79801033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 shutdown_connections 2024-06-15T01:04:49.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.113+0000 7f798503a640 1 -- 172.21.15.137:0/3588658694 wait complete. 2024-06-15T01:04:49.124 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 -- 172.21.15.137:0/2470716303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 msgr2=0x7ff2440ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 --2- 172.21.15.137:0/2470716303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff2440ff3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff234009c20 tx=0x7ff23402c1b0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 -- 172.21.15.137:0/2470716303 shutdown_connections 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 --2- 172.21.15.137:0/2470716303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 0x7ff244100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 --2- 172.21.15.137:0/2470716303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff2440ff3b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 -- 172.21.15.137:0/2470716303 >> 172.21.15.137:0/2470716303 conn(0x7ff2440fa4e0 msgr2=0x7ff2440fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:49.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 -- 172.21.15.137:0/2470716303 shutdown_connections 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.227+0000 7ff249069640 1 -- 172.21.15.137:0/2470716303 wait complete. 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 Processor -- start 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 -- start start 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 0x7ff24419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24419aad0 con 0x7ff2440fef90 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.228+0000 7ff249069640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24419ac10 con 0x7ff244100180 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43872/0 (socket says 172.21.15.137:43872) 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 -- 172.21.15.137:0/3585493889 learned_addr learned my addr 172.21.15.137:0/3585493889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff24259c640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 0x7ff24419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 -- 172.21.15.137:0/3585493889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 msgr2=0x7ff24419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 0x7ff24419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 -- 172.21.15.137:0/3585493889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234009900 con 0x7ff2440fef90 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff242d9d640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff234037040 tx=0x7ff234032aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff234034820 con 0x7ff2440fef90 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff234003e00 con 0x7ff2440fef90 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24419ae90 con 0x7ff2440fef90 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff234040450 con 0x7ff2440fef90 2024-06-15T01:04:49.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.229+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24419b360 con 0x7ff2440fef90 2024-06-15T01:04:49.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.230+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244107a30 con 0x7ff2440fef90 2024-06-15T01:04:49.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.231+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff23400a040 con 0x7ff2440fef90 2024-06-15T01:04:49.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.231+0000 7ff23bfff640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 0x7ff23006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.231+0000 7ff24259c640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 0x7ff23006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.231+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff2340ae970 con 0x7ff2440fef90 2024-06-15T01:04:49.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.231+0000 7ff24259c640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 0x7ff23006e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff244067c50 tx=0x7ff22c00a2d0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:49.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.236+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff2340790f0 con 0x7ff2440fef90 2024-06-15T01:04:49.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.413+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff24419b580 con 0x7ff2440fef90 2024-06-15T01:04:49.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.413+0000 7ff23bfff640 1 -- 172.21.15.137:0/3585493889 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff23407c710 con 0x7ff2440fef90 2024-06-15T01:04:49.413 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:04:49.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 msgr2=0x7ff23006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 0x7ff23006e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff244067c50 tx=0x7ff22c00a2d0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 msgr2=0x7ff244067ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff234037040 tx=0x7ff234032aa0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 shutdown_connections 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7ff23006c450 0x7ff23006e910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244100180 0x7ff24419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 --2- 172.21.15.137:0/3585493889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2440fef90 0x7ff244067ee0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 >> 172.21.15.137:0/3585493889 conn(0x7ff2440fa4e0 msgr2=0x7ff2441033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 shutdown_connections 2024-06-15T01:04:49.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.416+0000 7ff249069640 1 -- 172.21.15.137:0/3585493889 wait complete. 2024-06-15T01:04:49.426 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2318470538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:04:49.426 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1793392197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:04:49.426 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:49 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3588658694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:04:49.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.526+0000 7fec74deb640 1 -- 172.21.15.137:0/1798212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 msgr2=0x7fec70100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.526+0000 7fec74deb640 1 --2- 172.21.15.137:0/1798212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70100630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fec58009880 tx=0x7fec5802bce0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- 172.21.15.137:0/1798212823 shutdown_connections 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 --2- 172.21.15.137:0/1798212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70100630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 --2- 172.21.15.137:0/1798212823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 0x7fec700ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- 172.21.15.137:0/1798212823 >> 172.21.15.137:0/1798212823 conn(0x7fec700fa530 msgr2=0x7fec700fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- 172.21.15.137:0/1798212823 shutdown_connections 2024-06-15T01:04:49.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- 172.21.15.137:0/1798212823 wait complete. 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 Processor -- start 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- start start 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 0x7fec70194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70195250 con 0x7fec701001b0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.527+0000 7fec74deb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701953c0 con 0x7fec700fefc0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43890/0 (socket says 172.21.15.137:43890) 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 -- 172.21.15.137:0/2071298069 learned_addr learned my addr 172.21.15.137:0/2071298069 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6e59c640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 0x7fec70194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 -- 172.21.15.137:0/2071298069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 msgr2=0x7fec70194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 0x7fec70194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 -- 172.21.15.137:0/2071298069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58009560 con 0x7fec701001b0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec6dd9b640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec58009230 tx=0x7fec58003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec58003ba0 con 0x7fec701001b0 2024-06-15T01:04:49.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70199df0 con 0x7fec701001b0 2024-06-15T01:04:49.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fec58003d00 con 0x7fec701001b0 2024-06-15T01:04:49.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec58034700 con 0x7fec701001b0 2024-06-15T01:04:49.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.528+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7019a3a0 con 0x7fec701001b0 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.529+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70107a80 con 0x7fec701001b0 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.530+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fec5803e450 con 0x7fec701001b0 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.530+0000 7fec677fe640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 0x7fec5c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.530+0000 7fec6e59c640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 0x7fec5c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.530+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fec580ae470 con 0x7fec701001b0 2024-06-15T01:04:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.531+0000 7fec6e59c640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 0x7fec5c06e9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec70100010 tx=0x7fec68009870 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.535+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fec58078bf0 con 0x7fec701001b0 2024-06-15T01:04:49.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.737+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7019a530 con 0x7fec701001b0 2024-06-15T01:04:49.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.737+0000 7fec677fe640 1 -- 172.21.15.137:0/2071298069 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4757 (secure 0 0 0) 0x7fec5807c210 con 0x7fec701001b0 2024-06-15T01:04:49.737 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:04:49.737 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:04:37.406114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":14496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6827/183082556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":183082556},{"type":"v1","addr":"172.21.15.137:6827","nonce":183082556}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14496":{"gid":14496,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6829/937501599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":937501599},{"type":"v1","addr":"172.21.15.137:6829","nonce":937501599}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:04:49.737 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 17 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 msgr2=0x7fec5c06e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 0x7fec5c06e9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec70100010 tx=0x7fec68009870 comp rx=0 tx=0).stop 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 msgr2=0x7fec70194c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec58009230 tx=0x7fec58003820 comp rx=0 tx=0).stop 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 shutdown_connections 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fec5c06c510 0x7fec5c06e9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec701001b0 0x7fec70194c80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 --2- 172.21.15.137:0/2071298069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec700fefc0 0x7fec70194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:49.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 >> 172.21.15.137:0/2071298069 conn(0x7fec700fa530 msgr2=0x7fec701033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:49.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 shutdown_connections 2024-06-15T01:04:49.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:49.739+0000 7fec74deb640 1 -- 172.21.15.137:0/2071298069 wait complete. 2024-06-15T01:04:49.750 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-06-15T01:04:49.750 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-06-15T01:04:49.775 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-06-15T01:04:49.776 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-06-15T01:04:49.776 INFO:tasks.ceph_fuse:client.0 config is {} 2024-06-15T01:04:49.776 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-15T01:04:49.776 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:49.776 INFO:tasks.ceph_fuse:client.1 config is {} 2024-06-15T01:04:49.776 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-15T01:04:49.776 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:49.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:49.776 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-06-15T01:04:49.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:49.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-06-15T01:04:49.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:49 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2318470538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:04:49.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:49 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1793392197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:04:49.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:49 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3588658694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:04:49.860 INFO:teuthology.orchestra.run.smithi137.stderr:Cannot find device "ceph-brx" 2024-06-15T01:04:49.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:04:49.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:49.862 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-06-15T01:04:49.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:49.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-06-15T01:04:49.959 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-06-15T01:04:49.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:04:49.961 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-06-15T01:04:49.961 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph fs ls 2024-06-15T01:04:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:50.510 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:50 smithi137 ceph-mon[29354]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-15T01:04:50.510 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:50 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3585493889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:04:50.511 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:50 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2071298069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:04:50.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:50 smithi142 ceph-mon[37379]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-15T01:04:50.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:50 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3585493889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:04:50.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:50 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2071298069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 -- 172.21.15.137:0/4214859347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 msgr2=0x7fcbdc100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 --2- 172.21.15.137:0/4214859347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc100050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009910 tx=0x7fcbd8009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 -- 172.21.15.137:0/4214859347 shutdown_connections 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 --2- 172.21.15.137:0/4214859347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc100050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 --2- 172.21.15.137:0/4214859347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 0x7fcbdc105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 -- 172.21.15.137:0/4214859347 >> 172.21.15.137:0/4214859347 conn(0x7fcbdc0755a0 msgr2=0x7fcbdc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:51.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 -- 172.21.15.137:0/4214859347 shutdown_connections 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.695+0000 7fcbe4730700 1 -- 172.21.15.137:0/4214859347 wait complete. 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 Processor -- start 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 -- start start 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 0x7fcbdc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc1971f0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe4730700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc197330 con 0x7fcbdc105a40 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe24cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe24cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43906/0 (socket says 172.21.15.137:43906) 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe24cc700 1 -- 172.21.15.137:0/1342050034 learned_addr learned my addr 172.21.15.137:0/1342050034 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.696+0000 7fcbe1ccb700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 0x7fcbdc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe24cc700 1 -- 172.21.15.137:0/1342050034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 msgr2=0x7fcbdc196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe24cc700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 0x7fcbdc196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe24cc700 1 -- 172.21.15.137:0/1342050034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80095f0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe24cc700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c930 tx=0x7fcbcc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbcc007ad0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcbcc00ce80 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc19b180 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbcc018630 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.697+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc19b6d0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.698+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc107fd0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.699+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fcbcc007c30 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.699+0000 7fcbd37fe700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 0x7fcbc806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.699+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcbcc08b810 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.699+0000 7fcbe1ccb700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 0x7fcbc806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:51.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.700+0000 7fcbe1ccb700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 0x7fcbc806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc197c80 tx=0x7fcbd801a040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:51.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.704+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcbcc055f90 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.939+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcbdc04ea90 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.939+0000 7fcbd37fe700 1 -- 172.21.15.137:0/1342050034 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fcbcc0595b0 con 0x7fcbdc0ffbd0 2024-06-15T01:04:51.939 INFO:teuthology.orchestra.run.smithi137.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 msgr2=0x7fcbc806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 0x7fcbc806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc197c80 tx=0x7fcbd801a040 comp rx=0 tx=0).stop 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 msgr2=0x7fcbdc196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c930 tx=0x7fcbcc00cc40 comp rx=0 tx=0).stop 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 shutdown_connections 2024-06-15T01:04:51.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fcbc806c4e0 0x7fcbc806e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbdc0ffbd0 0x7fcbdc196660 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 --2- 172.21.15.137:0/1342050034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbdc105a40 0x7fcbdc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:51.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 >> 172.21.15.137:0/1342050034 conn(0x7fcbdc0755a0 msgr2=0x7fcbdc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:51.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.941+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 shutdown_connections 2024-06-15T01:04:51.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:51.942+0000 7fcbe4730700 1 -- 172.21.15.137:0/1342050034 wait complete. 2024-06-15T01:04:52.589 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:52 smithi137 ceph-mon[29354]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-15T01:04:52.590 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:52 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1342050034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-15T01:04:52.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-15T01:04:52.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-15T01:04:52.612 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-15T01:04:52.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:52.612 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:52.629 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 42322sec preferred_lft 42322sec 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-06-15T01:04:52.630 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:52.631 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-15T01:04:52.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add name ceph-brx type bridge 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr flush dev ceph-brx 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set ceph-brx up 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-15T01:04:52.631 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-06-15T01:04:52.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:52.794 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-15T01:04:52.818 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-06-15T01:04:52.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:52.820 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-06-15T01:04:52.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:52 smithi142 ceph-mon[37379]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-15T01:04:52.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:52 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1342050034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-15T01:04:52.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-15T01:04:52.899 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-06-15T01:04:53.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.126 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-06-15T01:04:53.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.142 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-06-15T01:04:53.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.198 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-06-15T01:04:53.198 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-06-15T01:04:53.198 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-15T01:04:53.198 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-15T01:04:53.198 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-06-15T01:04:53.277 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-15T01:04:53.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-15T01:04:53.278 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-06-15T01:04:53.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.415 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-06-15T01:04:53.415 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-06-15T01:04:53.415 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.0 up 2024-06-15T01:04:53.415 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.0 master ceph-brx 2024-06-15T01:04:53.415 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-06-15T01:04:53.452 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-06-15T01:04:53.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-15T01:04:53.452 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:53.507 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-15T01:04:53.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-15T01:04:53.508 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:53.563 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-06-15T01:04:53.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-06-15T01:04:53.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-06-15T01:04:53.693 INFO:teuthology.orchestra.run.smithi137.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-06-15T01:04:53.694 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-06-15T01:04:53.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.695 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-06-15T01:04:53.751 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-15T01:04:53.751 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-06-15T01:04:53.793 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-06-15T01:04:53.817 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-06-15T01:04:53.817 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-06-15T01:04:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-06-15T01:04:53.882 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-06-15T01:04:53.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:53.883 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-06-15T01:04:53.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:2024-06-15T01:04:53.889+0000 7fe7f7c0d480 -1 init, newargv = 0x55a5dc88e350 newargc=15 2024-06-15T01:04:53.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[77226]: starting ceph client 2024-06-15T01:04:53.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[77226]: starting fuse 2024-06-15T01:04:53.903 INFO:teuthology.orchestra.run.smithi137.stdout:164 2024-06-15T01:04:53.903 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [164] 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> import glob 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> import re 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> import os 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> import subprocess 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> def _find_admin_socket(client_name): 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> files = glob.glob(asok_path) 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> # Given a non-glob path, it better be there 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> if "*" not in asok_path: 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> assert(len(files) == 1) 2024-06-15T01:04:53.904 DEBUG:teuthology.orchestra.run.smithi137:> return files[0] 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> for f in files: 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> contents = proc_f.read() 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> if mountpoint in contents: 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> return f 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> print(_find_admin_socket("client.0")) 2024-06-15T01:04:53.905 DEBUG:teuthology.orchestra.run.smithi137:> ' 2024-06-15T01:04:54.017 INFO:teuthology.orchestra.run.smithi137.stdout:/var/run/ceph/ceph-client.0.77226.asok 2024-06-15T01:04:54.204 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77226.asok 2024-06-15T01:04:54.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:54.204 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77226.asok status 2024-06-15T01:04:54.338 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "metadata": { 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "entity_id": "0", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": "smithi137", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "pid": "77226", 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "root": "/" 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_count": 0, 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_pinned_count": 0, 2024-06-15T01:04:54.339 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 14550, 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "inst": { 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "name": { 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "client", 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "num": 14550 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 345370474 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-06-15T01:04:54.340 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 345370474 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "inst_str": "client.14550 192.168.0.1:0/345370474", 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "addr_str": "192.168.0.1:0/345370474", 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "inode_count": 1, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_epoch": 17, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch": 47, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch_barrier": 0, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "blocklisted": false, 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs" 2024-06-15T01:04:54.341 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:04:54.347 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- ceph fs ls 2024-06-15T01:04:54.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:54 smithi137 ceph-mon[29354]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-06-15T01:04:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:04:54.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:54 smithi142 ceph-mon[37379]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 -- 172.21.15.137:0/3778787658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 msgr2=0x7f5140105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 --2- 172.21.15.137:0/3778787658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140105f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5130009910 tx=0x7f5130009c20 comp rx=0 tx=0).stop 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 -- 172.21.15.137:0/3778787658 shutdown_connections 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 --2- 172.21.15.137:0/3778787658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f51400ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 --2- 172.21.15.137:0/3778787658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140105f00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 -- 172.21.15.137:0/3778787658 >> 172.21.15.137:0/3778787658 conn(0x7f5140075e30 msgr2=0x7f51400fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 -- 172.21.15.137:0/3778787658 shutdown_connections 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.213+0000 7f514546a700 1 -- 172.21.15.137:0/3778787658 wait complete. 2024-06-15T01:04:56.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 Processor -- start 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 -- start start 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f5140196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5140197280 con 0x7f5140105b20 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f514546a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514019b010 con 0x7f51400ff050 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52204/0 (socket says 172.21.15.137:52204) 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513e7fc700 1 -- 172.21.15.137:0/3356278704 learned_addr learned my addr 172.21.15.137:0/3356278704 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513effd700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f5140196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513e7fc700 1 -- 172.21.15.137:0/3356278704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 msgr2=0x7f5140196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.214+0000 7f513e7fc700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f5140196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f513e7fc700 1 -- 172.21.15.137:0/3356278704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300095f0 con 0x7f5140105b20 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f513effd700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f5140196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:04:56.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f513e7fc700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f513400e920 tx=0x7f513400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513400c9d0 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5134010430 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514019b2f0 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5134018630 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.215+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514019b840 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.216+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5140109060 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.216+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f513400cb30 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.217+0000 7f5127fff700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 0x7f512806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.217+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5134014070 con 0x7f5140105b20 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.217+0000 7f513effd700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 0x7f512806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:04:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.217+0000 7f513effd700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 0x7f512806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5130000c00 tx=0x7f5130009f70 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:04:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.221+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51340520e0 con 0x7f5140105b20 2024-06-15T01:04:56.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.456+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f514019ba60 con 0x7f5140105b20 2024-06-15T01:04:56.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.457+0000 7f5127fff700 1 -- 172.21.15.137:0/3356278704 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f513408de60 con 0x7f5140105b20 2024-06-15T01:04:56.456 INFO:teuthology.orchestra.run.smithi137.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-06-15T01:04:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 msgr2=0x7f512806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 0x7f512806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5130000c00 tx=0x7f5130009f70 comp rx=0 tx=0).stop 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 msgr2=0x7f5140196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f513400e920 tx=0x7f513400ec30 comp rx=0 tx=0).stop 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 shutdown_connections 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f512806c4e0 0x7f512806e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51400ff050 0x7f5140196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 --2- 172.21.15.137:0/3356278704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140105b20 0x7f5140196ba0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 >> 172.21.15.137:0/3356278704 conn(0x7f5140075e30 msgr2=0x7f51400fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 shutdown_connections 2024-06-15T01:04:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:04:56.459+0000 7f514546a700 1 -- 172.21.15.137:0/3356278704 wait complete. 2024-06-15T01:04:56.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:56 smithi137 ceph-mon[29354]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-15T01:04:56.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:56 smithi142 ceph-mon[37379]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-06-15T01:04:57.070 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-15T01:04:57.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-15T01:04:57.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-06-15T01:04:57.070 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-06-15T01:04:57.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:57.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-15T01:04:57.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-15T01:04:57.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-15T01:04:57.071 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-06-15T01:04:57.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.071 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:57.091 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-06-15T01:04:57.092 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-06-15T01:04:57.093 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-15T01:04:57.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-15T01:04:57.093 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-06-15T01:04:57.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.275 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-15T01:04:57.300 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-06-15T01:04:57.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.302 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-06-15T01:04:57.370 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-06-15T01:04:57.371 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-15T01:04:57.371 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-15T01:04:57.371 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-15T01:04:57.371 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-15T01:04:57.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-15T01:04:57.371 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-06-15T01:04:57.433 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:57 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3356278704' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-15T01:04:57.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.632 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-06-15T01:04:57.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.688 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-06-15T01:04:57.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:57 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3356278704' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-15T01:04:57.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.743 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-06-15T01:04:57.743 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-06-15T01:04:57.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-06-15T01:04:57.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-06-15T01:04:57.743 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-06-15T01:04:57.823 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-06-15T01:04:57.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-06-15T01:04:57.823 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-06-15T01:04:57.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:57.959 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-06-15T01:04:57.959 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-06-15T01:04:57.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-06-15T01:04:57.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2024-06-15T01:04:57.959 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-06-15T01:04:58.002 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-06-15T01:04:58.002 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-15T01:04:58.002 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:58.058 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-06-15T01:04:58.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-15T01:04:58.058 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:58.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-06-15T01:04:58.178 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-06-15T01:04:58.178 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-06-15T01:04:58.242 INFO:teuthology.orchestra.run.smithi142.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-06-15T01:04:58.243 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-06-15T01:04:58.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.243 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-06-15T01:04:58.298 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-15T01:04:58.299 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-06-15T01:04:58.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-06-15T01:04:58.364 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-06-15T01:04:58.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-06-15T01:04:58.428 INFO:teuthology.orchestra.run.smithi142.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-06-15T01:04:58.429 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-06-15T01:04:58.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.430 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-06-15T01:04:58.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-06-15T01:04:58.445+0000 7fc6e9c3d480 -1 init, newargv = 0x557b0be64bf0 newargc=15 2024-06-15T01:04:58.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[63011]: starting ceph client 2024-06-15T01:04:58.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[63011]: starting fuse 2024-06-15T01:04:58.460 INFO:teuthology.orchestra.run.smithi142.stdout:128 2024-06-15T01:04:58.461 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [128] 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-06-15T01:04:58.461 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-06-15T01:04:58.462 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-06-15T01:04:58.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:04:58 smithi142 ceph-mon[37379]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:04:58.575 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.63011.asok 2024-06-15T01:04:58.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:04:58 smithi137 ceph-mon[29354]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:04:58.761 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63011.asok 2024-06-15T01:04:58.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63011.asok status 2024-06-15T01:04:58.895 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-06-15T01:04:58.895 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-06-15T01:04:58.895 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "63011", 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 24309, 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-06-15T01:04:58.896 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 24309 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2246087572 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2246087572 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.24309 192.168.0.1:0/2246087572", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/2246087572", 2024-06-15T01:04:58.897 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 17, 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 47, 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-06-15T01:04:58.898 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-06-15T01:04:58.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.904 DEBUG:teuthology.orchestra.run.smithi137:> stat --file-system '--printf=%T 2024-06-15T01:04:58.904 DEBUG:teuthology.orchestra.run.smithi137:> ' -- /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:58.925 INFO:teuthology.orchestra.run.smithi137.stdout:fuseblk 2024-06-15T01:04:58.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:58.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-15T01:04:58.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:58.998 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-06-15T01:04:58.998 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:59.014 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-06-15T01:04:59.014 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-06-15T01:04:59.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:04:59.015 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-06-15T01:05:00.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:00 smithi137 ceph-mon[29354]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-06-15T01:05:00.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:00 smithi142 ceph-mon[37379]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-06-15T01:05:02.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:02 smithi137 ceph-mon[29354]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-15T01:05:02.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:02 smithi142 ceph-mon[37379]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-15T01:05:03.076 INFO:teuthology.run_tasks:Running task print... 2024-06-15T01:05:03.086 INFO:teuthology.task.print:**** done client 2024-06-15T01:05:03.086 INFO:teuthology.run_tasks:Running task parallel... 2024-06-15T01:05:03.145 INFO:teuthology.task.parallel:starting parallel... 2024-06-15T01:05:03.145 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-15T01:05:03.146 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-15T01:05:03.146 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:05:03.146 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-15T01:05:03.147 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-15T01:05:03.147 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-06-15T01:05:03.151 INFO:tasks.workunit:Pulling workunits from ref e3daf20513a97cf7004cd9e7bfdba8f5a015acbc 2024-06-15T01:05:03.152 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-15T01:05:03.152 INFO:tasks.workunit:timeout=3h 2024-06-15T01:05:03.152 INFO:tasks.workunit:cleanup=True 2024-06-15T01:05:03.152 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Device: a4h/164d Inode: 1 Links: 2 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:fusefs_t:s0 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-06-15 01:04:23.444734014 +0000 2024-06-15T01:05:03.173 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-06-15 01:04:58.997362354 +0000 2024-06-15T01:05:03.174 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-06-15T01:05:03.174 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-15T01:05:03.174 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-15T01:05:03.246 DEBUG:teuthology.orchestra.run.smithi142:> stat -- /home/ubuntu/cephtest/mnt.1 2024-06-15T01:05:03.267 INFO:teuthology.orchestra.run.smithi142.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 80h/128d Inode: 1 Links: 3 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fusefs_t:s0 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-06-15 01:05:03.242069755 +0000 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-06-15 01:05:03.242069755 +0000 2024-06-15T01:05:03.268 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-06-15T01:05:03.268 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-06-15T01:05:03.268 DEBUG:teuthology.orchestra.run.smithi142:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-06-15T01:05:03.344 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e3daf20513a97cf7004cd9e7bfdba8f5a015acbc 2024-06-15T01:05:03.344 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout e3daf20513a97cf7004cd9e7bfdba8f5a015acbc 2024-06-15T01:05:03.361 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-15T01:05:03.377 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:05:03.402 INFO:tasks.workunit.client.1.smithi142.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-06-15T01:05:03.478 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:03.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:04.679 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:04 smithi137 ceph-mon[29354]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:04.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:04 smithi142 ceph-mon[37379]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:04.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 -- 172.21.15.137:0/404799511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 msgr2=0x7fbf080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:04.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/404799511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080ffed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009910 tx=0x7fbf04009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:04.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 -- 172.21.15.137:0/404799511 shutdown_connections 2024-06-15T01:05:04.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/404799511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 0x7fbf08101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:04.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/404799511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080ffed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:04.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 -- 172.21.15.137:0/404799511 >> 172.21.15.137:0/404799511 conn(0x7fbf080fb010 msgr2=0x7fbf080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:04.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 -- 172.21.15.137:0/404799511 shutdown_connections 2024-06-15T01:05:04.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.898+0000 7fbf0f83c700 1 -- 172.21.15.137:0/404799511 wait complete. 2024-06-15T01:05:04.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 Processor -- start 2024-06-15T01:05:04.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 -- start start 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 0x7fbf0819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0819b5f0 con 0x7fbf080ffab0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0f83c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0819b730 con 0x7fbf08100cb0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52212/0 (socket says 172.21.15.137:52212) 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 -- 172.21.15.137:0/2909995806 learned_addr learned my addr 172.21.15.137:0/2909995806 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0cdd7700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 0x7fbf0819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 -- 172.21.15.137:0/2909995806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 msgr2=0x7fbf0819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 0x7fbf0819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 -- 172.21.15.137:0/2909995806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040095f0 con 0x7fbf080ffab0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.899+0000 7fbf0d5d8700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400bd70 tx=0x7fbf0400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.900+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf0401d070 con 0x7fbf080ffab0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.900+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbf0400f460 con 0x7fbf080ffab0 2024-06-15T01:05:04.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.900+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf04021640 con 0x7fbf080ffab0 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.900+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0819b9b0 con 0x7fbf080ffab0 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.900+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0819bf20 con 0x7fbf080ffab0 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.901+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf081084d0 con 0x7fbf080ffab0 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.901+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbf0400f9e0 con 0x7fbf080ffab0 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.902+0000 7fbefe7fc700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 0x7fbef406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:04.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.902+0000 7fbf0cdd7700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 0x7fbef406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:04.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.902+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbf0408ca80 con 0x7fbf080ffab0 2024-06-15T01:05:04.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.902+0000 7fbf0cdd7700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 0x7fbef406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:04.906+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbf0405ad10 con 0x7fbf080ffab0 2024-06-15T01:05:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.099+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fbf081086c0 con 0x7fbf080ffab0 2024-06-15T01:05:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.100+0000 7fbefe7fc700 1 -- 172.21.15.137:0/2909995806 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fbf081086c0 con 0x7fbf080ffab0 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 msgr2=0x7fbef406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 0x7fbef406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).stop 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 msgr2=0x7fbf080699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400bd70 tx=0x7fbf0400be50 comp rx=0 tx=0).stop 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 shutdown_connections 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbef406c4e0 0x7fbef406e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf080ffab0 0x7fbf080699b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 --2- 172.21.15.137:0/2909995806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf08100cb0 0x7fbf0819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 >> 172.21.15.137:0/2909995806 conn(0x7fbf080fb010 msgr2=0x7fbf08103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 shutdown_connections 2024-06-15T01:05:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:05.103+0000 7fbf0f83c700 1 -- 172.21.15.137:0/2909995806 wait complete. 2024-06-15T01:05:05.744 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-15T01:05:06.002 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:05:06.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:06 smithi137 ceph-mon[29354]: pgmap v201: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:06.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:06 smithi142 ceph-mon[37379]: pgmap v201: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 -- 172.21.15.137:0/4259495563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac0fee80 msgr2=0x7f74ac0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 --2- 172.21.15.137:0/4259495563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac0fee80 0x7f74ac0ff2a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009910 tx=0x7f74a0009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 -- 172.21.15.137:0/4259495563 shutdown_connections 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 --2- 172.21.15.137:0/4259495563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac101e60 0x7f74ac1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 --2- 172.21.15.137:0/4259495563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac0fee80 0x7f74ac0ff2a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 -- 172.21.15.137:0/4259495563 >> 172.21.15.137:0/4259495563 conn(0x7f74ac0fa7c0 msgr2=0x7f74ac0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 -- 172.21.15.137:0/4259495563 shutdown_connections 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.922+0000 7f74b3147700 1 -- 172.21.15.137:0/4259495563 wait complete. 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 Processor -- start 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 -- start start 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 0x7f74ac1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac100350 con 0x7f74ac101e60 2024-06-15T01:05:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b3147700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac100490 con 0x7f74ac0fee80 2024-06-15T01:05:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54578/0 (socket says 172.21.15.137:54578) 2024-06-15T01:05:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74b0ee3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 0x7f74ac1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.923+0000 7f74abfff700 1 -- 172.21.15.137:0/2532407845 learned_addr learned my addr 172.21.15.137:0/2532407845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:07.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74abfff700 1 -- 172.21.15.137:0/2532407845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 msgr2=0x7f74ac1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:07.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74abfff700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 0x7f74ac1017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:07.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74abfff700 1 -- 172.21.15.137:0/2532407845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00095f0 con 0x7f74ac101e60 2024-06-15T01:05:07.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74b0ee3700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 0x7f74ac1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74abfff700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f749800d8d0 tx=0x7f749800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74980098a0 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac100770 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7498009e90 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749800f5f0 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.924+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac19e330 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.925+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac1095e0 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.926+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7498010460 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.926+0000 7f74a9ffb700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 0x7f749c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.926+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f749808b750 con 0x7f74ac101e60 2024-06-15T01:05:07.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.926+0000 7f74b0ee3700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 0x7f749c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:07.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.927+0000 7f74b0ee3700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 0x7f749c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a000b8b0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:07.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:07.931+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7498059960 con 0x7f74ac101e60 2024-06-15T01:05:08.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.145+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f74ac19e8e0 con 0x7f74ac101e60 2024-06-15T01:05:08.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.145+0000 7f74a9ffb700 1 -- 172.21.15.137:0/2532407845 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f7498010710 con 0x7f74ac101e60 2024-06-15T01:05:08.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 msgr2=0x7f749c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 0x7f749c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a000b8b0 comp rx=0 tx=0).stop 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 msgr2=0x7f74ac0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f749800d8d0 tx=0x7f749800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 shutdown_connections 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f749c06c4e0 0x7f749c06e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0fee80 0x7f74ac1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 --2- 172.21.15.137:0/2532407845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74ac101e60 0x7f74ac0ffe10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.147+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 >> 172.21.15.137:0/2532407845 conn(0x7f74ac0fa7c0 msgr2=0x7f74ac105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.148+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 shutdown_connections 2024-06-15T01:05:08.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:08.148+0000 7f74b3147700 1 -- 172.21.15.137:0/2532407845 wait complete. 2024-06-15T01:05:08.715 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:08 smithi142 ceph-mon[37379]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:08.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:08 smithi137 ceph-mon[29354]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:08.927 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-15T01:05:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:05:10.687 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:10 smithi137 ceph-mon[29354]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-15T01:05:10.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:10 smithi142 ceph-mon[37379]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.945+0000 7f66f5795700 1 -- 172.21.15.137:0/2354691672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0101dc0 msgr2=0x7f66f0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.945+0000 7f66f5795700 1 --2- 172.21.15.137:0/2354691672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0101dc0 0x7f66f0102220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009910 tx=0x7f66e4009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.945+0000 7f66f5795700 1 -- 172.21.15.137:0/2354691672 shutdown_connections 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.945+0000 7f66f5795700 1 --2- 172.21.15.137:0/2354691672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0101dc0 0x7f66f0102220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.946+0000 7f66f5795700 1 --2- 172.21.15.137:0/2354691672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0100bc0 0x7f66f0100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.946+0000 7f66f5795700 1 -- 172.21.15.137:0/2354691672 >> 172.21.15.137:0/2354691672 conn(0x7f66f00fc140 msgr2=0x7f66f00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.946+0000 7f66f5795700 1 -- 172.21.15.137:0/2354691672 shutdown_connections 2024-06-15T01:05:10.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.946+0000 7f66f5795700 1 -- 172.21.15.137:0/2354691672 wait complete. 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.946+0000 7f66f5795700 1 Processor -- start 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66f5795700 1 -- start start 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66f5795700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66f5795700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0101dc0 0x7f66f0196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66f5795700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0196e20 con 0x7f66f0100bc0 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66f5795700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0196f60 con 0x7f66f0101dc0 2024-06-15T01:05:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54592/0 (socket says 172.21.15.137:54592) 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 -- 172.21.15.137:0/1960753493 learned_addr learned my addr 172.21.15.137:0/1960753493 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66ee7fc700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0101dc0 0x7f66f0196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 -- 172.21.15.137:0/1960753493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0101dc0 msgr2=0x7f66f0196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0101dc0 0x7f66f0196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.947+0000 7f66eeffd700 1 -- 172.21.15.137:0/1960753493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40095f0 con 0x7f66f0100bc0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66eeffd700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e8f0 tx=0x7f66e000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e0009790 con 0x7f66f0100bc0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f66e000c810 con 0x7f66f0100bc0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f019ba10 con 0x7f66f0100bc0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e0003c20 con 0x7f66f0100bc0 2024-06-15T01:05:10.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.948+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f019bfe0 con 0x7f66f0100bc0 2024-06-15T01:05:10.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.949+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f01095e0 con 0x7f66f0100bc0 2024-06-15T01:05:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.950+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f66e000c980 con 0x7f66f0100bc0 2024-06-15T01:05:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.950+0000 7f66d7fff700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 0x7f66d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.950+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f66e0014070 con 0x7f66f0100bc0 2024-06-15T01:05:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.950+0000 7f66ee7fc700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 0x7f66d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.951+0000 7f66ee7fc700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 0x7f66d806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e400b5a0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:10.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:10.955+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f66e0059d60 con 0x7f66f0100bc0 2024-06-15T01:05:11.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.186+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f66f01097a0 con 0x7f66f0100bc0 2024-06-15T01:05:11.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.186+0000 7f66d7fff700 1 -- 172.21.15.137:0/1960753493 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f66e00598f0 con 0x7f66f0100bc0 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.188+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 msgr2=0x7f66d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.188+0000 7f66f5795700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 0x7f66d806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e400b5a0 comp rx=0 tx=0).stop 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.188+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 msgr2=0x7f66f01962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e8f0 tx=0x7f66e000ec00 comp rx=0 tx=0).stop 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 shutdown_connections 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f66d806c4e0 0x7f66d806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f0100bc0 0x7f66f01962c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 --2- 172.21.15.137:0/1960753493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66f0101dc0 0x7f66f0196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 >> 172.21.15.137:0/1960753493 conn(0x7f66f00fc140 msgr2=0x7f66f0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 shutdown_connections 2024-06-15T01:05:11.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:11.189+0000 7f66f5795700 1 -- 172.21.15.137:0/1960753493 wait complete. 2024-06-15T01:05:12.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:12 smithi137 ceph-mon[29354]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-06-15T01:05:12.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:12 smithi142 ceph-mon[37379]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-06-15T01:05:14.230 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-06-15T01:05:14.448 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:05:14.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:14 smithi137 ceph-mon[29354]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-06-15T01:05:14.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:14 smithi142 ceph-mon[37379]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-06-15T01:05:16.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 -- 172.21.15.137:0/292533547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c0fee90 msgr2=0x7fb02c0ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 --2- 172.21.15.137:0/292533547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c0fee90 0x7fb02c0ff2b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb020009910 tx=0x7fb020009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 -- 172.21.15.137:0/292533547 shutdown_connections 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 --2- 172.21.15.137:0/292533547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c101e70 0x7fb02c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 --2- 172.21.15.137:0/292533547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c0fee90 0x7fb02c0ff2b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 -- 172.21.15.137:0/292533547 >> 172.21.15.137:0/292533547 conn(0x7fb02c0fa7f0 msgr2=0x7fb02c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 -- 172.21.15.137:0/292533547 shutdown_connections 2024-06-15T01:05:16.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.286+0000 7fb0313b8700 1 -- 172.21.15.137:0/292533547 wait complete. 2024-06-15T01:05:16.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 Processor -- start 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 -- start start 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 0x7fb02c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 0x7fb02c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c196e20 con 0x7fb02c101e70 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.287+0000 7fb0313b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c196f60 con 0x7fb02c0fee90 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 0x7fb02c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 0x7fb02c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 0x7fb02c196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33180/0 (socket says 172.21.15.137:33180) 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02a7fc700 1 -- 172.21.15.137:0/3125268148 learned_addr learned my addr 172.21.15.137:0/3125268148 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02affd700 1 -- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 msgr2=0x7fb02c196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02affd700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 0x7fb02c196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:16.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02affd700 1 -- 172.21.15.137:0/3125268148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200095f0 con 0x7fb02c0fee90 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb02affd700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 0x7fb02c1962c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb02000bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02001d070 con 0x7fb02c0fee90 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb020005710 con 0x7fb02c0fee90 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c19b9b0 con 0x7fb02c0fee90 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb020003980 con 0x7fb02c0fee90 2024-06-15T01:05:16.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.288+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c19bec0 con 0x7fb02c0fee90 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.290+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c1095f0 con 0x7fb02c0fee90 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.290+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb020004ac0 con 0x7fb02c0fee90 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.291+0000 7fb013fff700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 0x7fb01406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.291+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb02008cb20 con 0x7fb02c0fee90 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.291+0000 7fb02a7fc700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 0x7fb01406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.291+0000 7fb02a7fc700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 0x7fb01406e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009d20 tx=0x7fb01c009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.296+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb02005ad30 con 0x7fb02c0fee90 2024-06-15T01:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:16.521+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02c106870 con 0x7fb01406c4e0 2024-06-15T01:05:16.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:16 smithi137 ceph-mon[29354]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-15T01:05:16.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:16 smithi142 ceph-mon[37379]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-15T01:05:17.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:17 smithi137 ceph-mon[29354]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:17.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:17 smithi142 ceph-mon[37379]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:18.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:18 smithi137 ceph-mon[29354]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-06-15T01:05:18.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:18.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:18 smithi142 ceph-mon[37379]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-06-15T01:05:18.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:20.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:20 smithi137 ceph-mon[29354]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-06-15T01:05:20.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:20 smithi142 ceph-mon[37379]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-06-15T01:05:22.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:22 smithi137 ceph-mon[29354]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-15T01:05:22.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:22 smithi142 ceph-mon[37379]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-06-15T01:05:24.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:24 smithi137 ceph-mon[29354]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:24.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:24 smithi142 ceph-mon[37379]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:26.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:26 smithi137 ceph-mon[29354]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:26.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:26 smithi142 ceph-mon[37379]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:28.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:28 smithi137 ceph-mon[29354]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:28.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:28 smithi142 ceph-mon[37379]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:05:30.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:30 smithi137 ceph-mon[29354]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-15T01:05:30.728 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:30 smithi142 ceph-mon[37379]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-06-15T01:05:32.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:32 smithi137 ceph-mon[29354]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:32.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:32 smithi142 ceph-mon[37379]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:33.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:33.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:34.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:34 smithi137 ceph-mon[29354]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:34.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:34 smithi142 ceph-mon[37379]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:36.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:36 smithi137 ceph-mon[29354]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:36.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:36 smithi142 ceph-mon[37379]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:38.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:38 smithi137 ceph-mon[29354]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:38.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:38 smithi142 ceph-mon[37379]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:40.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:40 smithi137 ceph-mon[29354]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:40.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:40 smithi142 ceph-mon[37379]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:42.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:42 smithi137 ceph-mon[29354]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:42.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:42 smithi142 ceph-mon[37379]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:44.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:44 smithi137 ceph-mon[29354]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:44.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:44 smithi142 ceph-mon[37379]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:46.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:46 smithi137 ceph-mon[29354]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:46.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:46 smithi142 ceph-mon[37379]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:46.885 INFO:teuthology.orchestra.run.smithi137.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:05:46.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.886+0000 7fb013fff700 1 -- 172.21.15.137:0/3125268148 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb01c01e070 con 0x7fb01406c4e0 2024-06-15T01:05:46.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 msgr2=0x7fb01406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:46.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 0x7fb01406e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009d20 tx=0x7fb01c009260 comp rx=0 tx=0).stop 2024-06-15T01:05:46.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 msgr2=0x7fb02c1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 0x7fb02c1962c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb02000bde0 comp rx=0 tx=0).stop 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 shutdown_connections 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb01406c4e0 0x7fb01406e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02c0fee90 0x7fb02c1962c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 --2- 172.21.15.137:0/3125268148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb02c101e70 0x7fb02c196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.888+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 >> 172.21.15.137:0/3125268148 conn(0x7fb02c0fa7f0 msgr2=0x7fb02c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.889+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 shutdown_connections 2024-06-15T01:05:46.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:46.889+0000 7fb0313b8700 1 -- 172.21.15.137:0/3125268148 wait complete. 2024-06-15T01:05:47.521 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-06-15T01:05:47.781 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:05:48.089 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:05:48.090 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:47 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:05:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:05:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:05:48.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:05:48.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:47 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:48.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:48 smithi137 ceph-mon[29354]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:05:48.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:48 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:49.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:48 smithi142 ceph-mon[37379]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:05:49.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:48 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:05:50.208 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:49 smithi137 ceph-mon[29354]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:50.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:49 smithi142 ceph-mon[37379]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:50.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 -- 172.21.15.137:0/1462059575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0101e00 msgr2=0x7f18a0102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:50.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1462059575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0101e00 0x7f18a0102260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1890009910 tx=0x7f1890009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:50.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 -- 172.21.15.137:0/1462059575 shutdown_connections 2024-06-15T01:05:50.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1462059575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0101e00 0x7f18a0102260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1462059575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0100c00 0x7f18a0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 -- 172.21.15.137:0/1462059575 >> 172.21.15.137:0/1462059575 conn(0x7f18a00fc180 msgr2=0x7f18a00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.735+0000 7f18a7f98700 1 -- 172.21.15.137:0/1462059575 shutdown_connections 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.736+0000 7f18a7f98700 1 -- 172.21.15.137:0/1462059575 wait complete. 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.736+0000 7f18a7f98700 1 Processor -- start 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.736+0000 7f18a7f98700 1 -- start start 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.736+0000 7f18a7f98700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a7f98700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0101e00 0x7f18a0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a7f98700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00736c0 con 0x7f18a0100c00 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a7f98700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0073800 con 0x7f18a0101e00 2024-06-15T01:05:50.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49084/0 (socket says 172.21.15.137:49084) 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 -- 172.21.15.137:0/1556920718 learned_addr learned my addr 172.21.15.137:0/1556920718 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5533700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0101e00 0x7f18a0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 -- 172.21.15.137:0/1556920718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0101e00 msgr2=0x7f18a0073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0101e00 0x7f18a0073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 -- 172.21.15.137:0/1556920718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900095f0 con 0x7f18a0100c00 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.737+0000 7f18a5d34700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e8d0 tx=0x7f189c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:50.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.738+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c009770 con 0x7f18a0100c00 2024-06-15T01:05:50.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.738+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f189c00c810 con 0x7f18a0100c00 2024-06-15T01:05:50.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.738+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c003cf0 con 0x7f18a0100c00 2024-06-15T01:05:50.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.738+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a0073ae0 con 0x7f18a0100c00 2024-06-15T01:05:50.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.738+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a019e2f0 con 0x7f18a0100c00 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.739+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a0109640 con 0x7f18a0100c00 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.739+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f189c00c980 con 0x7f18a0100c00 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.740+0000 7f1896ffd700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 0x7f188c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.740+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f189c014070 con 0x7f18a0100c00 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.740+0000 7f18a5533700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 0x7f188c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:50.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.741+0000 7f18a5533700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 0x7f188c06e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f189000b5a0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:50.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:50.744+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f189c059d30 con 0x7f18a0100c00 2024-06-15T01:05:51.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.076+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a0106750 con 0x7f188c06c4e0 2024-06-15T01:05:51.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.078+0000 7f1896ffd700 1 -- 172.21.15.137:0/1556920718 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f18a0106750 con 0x7f188c06c4e0 2024-06-15T01:05:51.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 msgr2=0x7f188c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 0x7f188c06e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f189000b5a0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 msgr2=0x7f18a0074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e8d0 tx=0x7f189c00ebe0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 shutdown_connections 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f188c06c4e0 0x7f188c06e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a0100c00 0x7f18a0074b30 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e8d0 tx=0x7f189c00ebe0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 --2- 172.21.15.137:0/1556920718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a0101e00 0x7f18a0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 >> 172.21.15.137:0/1556920718 conn(0x7f18a00fc180 msgr2=0x7f18a0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 shutdown_connections 2024-06-15T01:05:51.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.080+0000 7f18a7f98700 1 -- 172.21.15.137:0/1556920718 wait complete. 2024-06-15T01:05:51.093 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 -- 172.21.15.137:0/2946754342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 msgr2=0x7f29381063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 --2- 172.21.15.137:0/2946754342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f29381063e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f292c009910 tx=0x7f292c009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 -- 172.21.15.137:0/2946754342 shutdown_connections 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 --2- 172.21.15.137:0/2946754342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f29381063e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 --2- 172.21.15.137:0/2946754342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 0x7f2938103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 -- 172.21.15.137:0/2946754342 >> 172.21.15.137:0/2946754342 conn(0x7f29380fb030 msgr2=0x7f29380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:51.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 -- 172.21.15.137:0/2946754342 shutdown_connections 2024-06-15T01:05:51.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.222+0000 7f293ca66700 1 -- 172.21.15.137:0/2946754342 wait complete. 2024-06-15T01:05:51.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 Processor -- start 2024-06-15T01:05:51.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 -- start start 2024-06-15T01:05:51.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 0x7f2938196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938196db0 con 0x7f2938103ff0 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293ca66700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938196ef0 con 0x7f29381016c0 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f293659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 0x7f2938196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49100/0 (socket says 172.21.15.137:49100) 2024-06-15T01:05:51.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.223+0000 7f2935d9b700 1 -- 172.21.15.137:0/4270454950 learned_addr learned my addr 172.21.15.137:0/4270454950 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f2935d9b700 1 -- 172.21.15.137:0/4270454950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 msgr2=0x7f2938196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f2935d9b700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 0x7f2938196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f2935d9b700 1 -- 172.21.15.137:0/4270454950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0095f0 con 0x7f2938103ff0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f2935d9b700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f292c00bde0 tx=0x7f292c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292c01d070 con 0x7f2938103ff0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293819b940 con 0x7f2938103ff0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.224+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f292c005dc0 con 0x7f2938103ff0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.225+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292c003b10 con 0x7f2938103ff0 2024-06-15T01:05:51.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.225+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293819be50 con 0x7f2938103ff0 2024-06-15T01:05:51.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.226+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29381903e0 con 0x7f2938103ff0 2024-06-15T01:05:51.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.226+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f292c0048e0 con 0x7f2938103ff0 2024-06-15T01:05:51.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.227+0000 7f29277fe700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 0x7f292006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.227+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f292c08cf50 con 0x7f2938103ff0 2024-06-15T01:05:51.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.227+0000 7f293659c700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 0x7f292006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.228+0000 7f293659c700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 0x7f292006e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2928009c60 tx=0x7f2928009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.235+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f292c05b140 con 0x7f2938103ff0 2024-06-15T01:05:51.447 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:51.448 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: Upgrade: Target is version 18.2.2-1443-g6f4b0956 (reef) 2024-06-15T01:05:51.448 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0'] 2024-06-15T01:05:51.448 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:05:51.448 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:51.448 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:51 smithi137 ceph-mon[29354]: Upgrade: Need to upgrade myself (mgr.smithi137.nbzbyc) 2024-06-15T01:05:51.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.467+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29380611d0 con 0x7f292006c4e0 2024-06-15T01:05:51.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.469+0000 7f29277fe700 1 -- 172.21.15.137:0/4270454950 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f29380611d0 con 0x7f292006c4e0 2024-06-15T01:05:51.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 msgr2=0x7f292006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 0x7f292006e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2928009c60 tx=0x7f2928009260 comp rx=0 tx=0).stop 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 msgr2=0x7f2938196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f292c00bde0 tx=0x7f292c00be10 comp rx=0 tx=0).stop 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 shutdown_connections 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f292006c4e0 0x7f292006e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29381016c0 0x7f2938196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 --2- 172.21.15.137:0/4270454950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2938103ff0 0x7f2938196790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 >> 172.21.15.137:0/4270454950 conn(0x7f29380fb030 msgr2=0x7f29380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.472+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 shutdown_connections 2024-06-15T01:05:51.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.473+0000 7f293ca66700 1 -- 172.21.15.137:0/4270454950 wait complete. 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: Upgrade: Target is version 18.2.2-1443-g6f4b0956 (reef) 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0'] 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:05:51.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:51 smithi142 ceph-mon[37379]: Upgrade: Need to upgrade myself (mgr.smithi137.nbzbyc) 2024-06-15T01:05:51.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 -- 172.21.15.137:0/710013004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98180ffd50 msgr2=0x7f98181001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 --2- 172.21.15.137:0/710013004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98180ffd50 0x7f98181001b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9808009910 tx=0x7f9808009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 -- 172.21.15.137:0/710013004 shutdown_connections 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 --2- 172.21.15.137:0/710013004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98180ffd50 0x7f98181001b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 --2- 172.21.15.137:0/710013004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98181010a0 0x7f98181014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 -- 172.21.15.137:0/710013004 >> 172.21.15.137:0/710013004 conn(0x7f98180fa7c0 msgr2=0x7f98180fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 -- 172.21.15.137:0/710013004 shutdown_connections 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.610+0000 7f981f665700 1 -- 172.21.15.137:0/710013004 wait complete. 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 Processor -- start 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 -- start start 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818195950 0x7f981819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818195e60 con 0x7f98181010a0 2024-06-15T01:05:51.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981f665700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818195fd0 con 0x7f9818195950 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981d401700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981d401700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49132/0 (socket says 172.21.15.137:49132) 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981d401700 1 -- 172.21.15.137:0/298227018 learned_addr learned my addr 172.21.15.137:0/298227018 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.611+0000 7f981cc00700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818195950 0x7f981819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.612+0000 7f981d401700 1 -- 172.21.15.137:0/298227018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818195950 msgr2=0x7f981819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.612+0000 7f981d401700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818195950 0x7f981819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.612+0000 7f981d401700 1 -- 172.21.15.137:0/298227018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080095f0 con 0x7f98181010a0 2024-06-15T01:05:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.612+0000 7f981d401700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98140098f0 tx=0x7f9814009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.612+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9814012070 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.613+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f98140044b0 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.613+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9814019480 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.613+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180feca0 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.613+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180ff270 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.615+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98181084e0 con 0x7f98181010a0 2024-06-15T01:05:51.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.615+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9814016030 con 0x7f98181010a0 2024-06-15T01:05:51.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.615+0000 7f980e7fc700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 0x7f980406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:51.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.616+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f981408cc20 con 0x7f98181010a0 2024-06-15T01:05:51.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.616+0000 7f981cc00700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 0x7f980406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:51.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.617+0000 7f981cc00700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 0x7f980406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9808009fd0 tx=0x7f9808005e80 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:51.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.623+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f981405ae10 con 0x7f98181010a0 2024-06-15T01:05:51.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:51.991+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f98181056b0 con 0x7f980406c4e0 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.010+0000 7f980e7fc700 1 -- 172.21.15.137:0/298227018 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f98181056b0 con 0x7f980406c4e0 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (5m) 64s ago 6m 25.7M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (7m) 64s ago 7m 7516k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (6m) 70s ago 6m 7235k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:05:52.008 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (7m) 64s ago 7m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (6m) 70s ago 6m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (5m) 64s ago 6m 83.9M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (83s) 64s ago 83s 17.1M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (89s) 64s ago 88s 18.9M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (86s) 70s ago 86s 11.3M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (81s) 70s ago 81s 14.3M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:9283,8765,8443 running (8m) 64s ago 8m 488M - 18.2.1 5be31c24972a 61486e562eb9 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (5m) 70s ago 5m 443M - 18.2.1 5be31c24972a e531298bfce9 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (8m) 64s ago 8m 45.0M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (5m) 70s ago 5m 35.5M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (7m) 64s ago 7m 16.6M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 70s ago 6m 16.9M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:05:52.009 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (5m) 64s ago 5m 47.6M 3946M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (4m) 64s ago 4m 50.9M 3946M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (4m) 64s ago 4m 45.7M 3946M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (3m) 70s ago 3m 47.1M 4970M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (3m) 70s ago 3m 44.2M 4970M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (2m) 70s ago 2m 43.3M 4970M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:05:52.010 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (5m) 64s ago 6m 54.5M - 2.43.0 a07b618ecd1d 904ef44cd0a5 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 msgr2=0x7f980406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 0x7f980406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9808009fd0 tx=0x7f9808005e80 comp rx=0 tx=0).stop 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 msgr2=0x7f9818195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98140098f0 tx=0x7f9814009c00 comp rx=0 tx=0).stop 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 shutdown_connections 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f980406c4e0 0x7f980406e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98181010a0 0x7f9818195410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 --2- 172.21.15.137:0/298227018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818195950 0x7f981819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 >> 172.21.15.137:0/298227018 conn(0x7f98180fa7c0 msgr2=0x7f9818103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:52.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 shutdown_connections 2024-06-15T01:05:52.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.012+0000 7f981f665700 1 -- 172.21.15.137:0/298227018 wait complete. 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.159+0000 7f8f48dc2700 1 -- 172.21.15.137:0/1878963528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 msgr2=0x7f8f44100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.159+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/1878963528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f44100230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009910 tx=0x7f8f38009c20 comp rx=0 tx=0).stop 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.159+0000 7f8f48dc2700 1 -- 172.21.15.137:0/1878963528 shutdown_connections 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.160+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/1878963528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f44100230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.160+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/1878963528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.160+0000 7f8f48dc2700 1 -- 172.21.15.137:0/1878963528 >> 172.21.15.137:0/1878963528 conn(0x7f8f440fa800 msgr2=0x7f8f440fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.160+0000 7f8f48dc2700 1 -- 172.21.15.137:0/1878963528 shutdown_connections 2024-06-15T01:05:52.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.160+0000 7f8f48dc2700 1 -- 172.21.15.137:0/1878963528 wait complete. 2024-06-15T01:05:52.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.161+0000 7f8f48dc2700 1 Processor -- start 2024-06-15T01:05:52.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.161+0000 7f8f48dc2700 1 -- start start 2024-06-15T01:05:52.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.161+0000 7f8f48dc2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f441962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f48dc2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f48dc2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44196e50 con 0x7f8f440ffdb0 2024-06-15T01:05:52.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f48dc2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44196f90 con 0x7f8f44101100 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:60966/0 (socket says 172.21.15.137:60966) 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 -- 172.21.15.137:0/3507005102 learned_addr learned my addr 172.21.15.137:0/3507005102 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f4259c700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f441962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 -- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 msgr2=0x7f8f441962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f441962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.162+0000 7f8f41d9b700 1 -- 172.21.15.137:0/3507005102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380095f0 con 0x7f8f44101100 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f41d9b700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f38000c00 tx=0x7f8f3800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3801d070 con 0x7f8f44101100 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4419b9e0 con 0x7f8f44101100 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4419bef0 con 0x7f8f44101100 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8f38005710 con 0x7f8f44101100 2024-06-15T01:05:52.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.163+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f380039c0 con 0x7f8f44101100 2024-06-15T01:05:52.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.164+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44109650 con 0x7f8f44101100 2024-06-15T01:05:52.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.165+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8f38003c00 con 0x7f8f44101100 2024-06-15T01:05:52.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.166+0000 7f8f337fe700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 0x7f8f2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.166+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8f3808cea0 con 0x7f8f44101100 2024-06-15T01:05:52.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.166+0000 7f8f4259c700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 0x7f8f2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.167+0000 7f8f4259c700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 0x7f8f2c06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f44068b70 tx=0x7f8f34009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:52.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.171+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f3805b120 con 0x7f8f44101100 2024-06-15T01:05:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:52 smithi137 ceph-mon[29354]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 on smithi142 2024-06-15T01:05:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:52 smithi137 ceph-mon[29354]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:52 smithi137 ceph-mon[29354]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:52.477 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:52 smithi137 ceph-mon[29354]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:52.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:52 smithi142 ceph-mon[37379]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 on smithi142 2024-06-15T01:05:52.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:52 smithi142 ceph-mon[37379]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:52.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:52 smithi142 ceph-mon[37379]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:52.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:52 smithi142 ceph-mon[37379]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.660+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f44109870 con 0x7f8f44101100 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.661+0000 7f8f337fe700 1 -- 172.21.15.137:0/3507005102 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f8f3805acb0 con 0x7f8f44101100 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:05:52.659 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:05:52.660 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 msgr2=0x7f8f2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 0x7f8f2c06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f44068b70 tx=0x7f8f34009260 comp rx=0 tx=0).stop 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 msgr2=0x7f8f44196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f38000c00 tx=0x7f8f3800bde0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 shutdown_connections 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f8f2c06c4e0 0x7f8f2c06e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f440ffdb0 0x7f8f441962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 --2- 172.21.15.137:0/3507005102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f44101100 0x7f8f44196830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.664+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 >> 172.21.15.137:0/3507005102 conn(0x7f8f440fa800 msgr2=0x7f8f44103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.665+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 shutdown_connections 2024-06-15T01:05:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.665+0000 7f8f48dc2700 1 -- 172.21.15.137:0/3507005102 wait complete. 2024-06-15T01:05:52.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.819+0000 7f3ca77ce700 1 -- 172.21.15.137:0/3221375011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 msgr2=0x7f3ca00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.819+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/3221375011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca00ff660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00aa20 tx=0x7f3c9c00ad30 comp rx=0 tx=0).stop 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.820+0000 7f3ca77ce700 1 -- 172.21.15.137:0/3221375011 shutdown_connections 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.820+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/3221375011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 0x7f3ca01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.820+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/3221375011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca00ff660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.820+0000 7f3ca77ce700 1 -- 172.21.15.137:0/3221375011 >> 172.21.15.137:0/3221375011 conn(0x7f3ca00fa7c0 msgr2=0x7f3ca00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.821+0000 7f3ca77ce700 1 -- 172.21.15.137:0/3221375011 shutdown_connections 2024-06-15T01:05:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.821+0000 7f3ca77ce700 1 -- 172.21.15.137:0/3221375011 wait complete. 2024-06-15T01:05:52.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 Processor -- start 2024-06-15T01:05:52.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 -- start start 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 0x7f3ca0194610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0194c30 con 0x7f3ca00ff240 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca77ce700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0194d70 con 0x7f3ca0100440 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca556a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca556a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49174/0 (socket says 172.21.15.137:49174) 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca556a700 1 -- 172.21.15.137:0/1163142618 learned_addr learned my addr 172.21.15.137:0/1163142618 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:05:52.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.822+0000 7f3ca4d69700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 0x7f3ca0194610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3ca556a700 1 -- 172.21.15.137:0/1163142618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 msgr2=0x7f3ca0194610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3ca556a700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 0x7f3ca0194610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3ca556a700 1 -- 172.21.15.137:0/1163142618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c00a700 con 0x7f3ca00ff240 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3ca556a700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00af00 tx=0x7f3c9c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c9c0051d0 con 0x7f3ca00ff240 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca01977b0 con 0x7f3ca00ff240 2024-06-15T01:05:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3c9c005700 con 0x7f3ca00ff240 2024-06-15T01:05:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.823+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c9c016760 con 0x7f3ca00ff240 2024-06-15T01:05:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.824+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0197d20 con 0x7f3ca00ff240 2024-06-15T01:05:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.825+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca01073d0 con 0x7f3ca00ff240 2024-06-15T01:05:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.826+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3c9c00eb70 con 0x7f3ca00ff240 2024-06-15T01:05:52.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.826+0000 7f3c967fc700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 0x7f3c8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:05:52.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.826+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3c9c012070 con 0x7f3ca00ff240 2024-06-15T01:05:52.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.826+0000 7f3ca4d69700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 0x7f3c8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:05:52.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.827+0000 7f3ca4d69700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 0x7f3c8c06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009bf0 tx=0x7f3c90009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:05:52.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:52.835+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3c9c05bb40 con 0x7f3ca00ff240 2024-06-15T01:05:53.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.197+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ca0104d90 con 0x7f3c8c06c4e0 2024-06-15T01:05:53.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.199+0000 7f3c967fc700 1 -- 172.21.15.137:0/1163142618 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3ca0104d90 con 0x7f3c8c06c4e0 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [], 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "0/2 daemons upgraded", 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 image on host smithi142", 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:05:53.213 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 msgr2=0x7f3c8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 0x7f3c8c06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009bf0 tx=0x7f3c90009260 comp rx=0 tx=0).stop 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 msgr2=0x7f3ca01940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00af00 tx=0x7f3c9c005dc0 comp rx=0 tx=0).stop 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 shutdown_connections 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f3c8c06c4e0 0x7f3c8c06e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca00ff240 0x7f3ca01940d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 --2- 172.21.15.137:0/1163142618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ca0100440 0x7f3ca0194610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:05:53.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 >> 172.21.15.137:0/1163142618 conn(0x7f3ca00fa7c0 msgr2=0x7f3ca0103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:05:53.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.201+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 shutdown_connections 2024-06-15T01:05:53.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:05:53.202+0000 7f3ca77ce700 1 -- 172.21.15.137:0/1163142618 wait complete. 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:Note: switching to 'e3daf20513a97cf7004cd9e7bfdba8f5a015acbc'. 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:state without impacting any branches by switching back to a branch. 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr: git switch -c 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.244 INFO:tasks.workunit.client.1.smithi142.stderr:Or undo this operation with: 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr: git switch - 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr: 2024-06-15T01:05:53.245 INFO:tasks.workunit.client.1.smithi142.stderr:HEAD is now at e3daf20513a qa/distros: removed rhel 8 from supported distros 2024-06-15T01:05:53.251 DEBUG:teuthology.orchestra.run.smithi142:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-06-15T01:05:53.312 INFO:tasks.workunit.client.1.smithi142.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-15T01:05:53.314 INFO:tasks.workunit.client.1.smithi142.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-06-15T01:05:53.314 INFO:tasks.workunit.client.1.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-15T01:05:53.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:53 smithi137 ceph-mon[29354]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:53.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:53 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3507005102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:05:53.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:53 smithi142 ceph-mon[37379]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:53.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:53 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3507005102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:05:54.207 INFO:tasks.workunit.client.1.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-15T01:05:54.267 INFO:tasks.workunit.client.1.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-15T01:05:54.313 INFO:tasks.workunit.client.1.smithi142.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-06-15T01:05:54.315 INFO:tasks.workunit.client.1.smithi142.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-06-15T01:05:54.316 INFO:tasks.workunit.client.1.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-15T01:05:54.366 INFO:tasks.workunit.client.1.smithi142.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-06-15T01:05:54.369 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-06-15T01:05:54.369 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-06-15T01:05:54.427 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-06-15T01:05:54.428 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-06-15T01:05:54.429 DEBUG:teuthology.orchestra.run.smithi142:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e3daf20513a97cf7004cd9e7bfdba8f5a015acbc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-06-15T01:05:54.496 INFO:tasks.workunit.client.1.smithi142.stderr:+ mkdir -p fsstress 2024-06-15T01:05:54.498 INFO:tasks.workunit.client.1.smithi142.stderr:+ pushd fsstress 2024-06-15T01:05:54.500 INFO:tasks.workunit.client.1.smithi142.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-15T01:05:54.500 INFO:tasks.workunit.client.1.smithi142.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-06-15T01:05:54.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:54 smithi137 ceph-mon[29354]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:54.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:54 smithi137 ceph-mon[29354]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:54.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:54 smithi142 ceph-mon[37379]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:54.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:54 smithi142 ceph-mon[37379]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:05:55.277 INFO:tasks.workunit.client.1.smithi142.stderr:+ tar xzf ltp-full.tgz 2024-06-15T01:05:56.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:56 smithi137 ceph-mon[29354]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:56.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:56 smithi142 ceph-mon[37379]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:05:58.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:05:58 smithi137 ceph-mon[29354]: pgmap v227: 65 pgs: 65 active+clean; 7.0 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 547 KiB/s wr, 3 op/s 2024-06-15T01:05:58.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:05:58 smithi142 ceph-mon[37379]: pgmap v227: 65 pgs: 65 active+clean; 7.0 MiB data, 173 MiB used, 536 GiB / 536 GiB avail; 547 KiB/s wr, 3 op/s 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to 'e3daf20513a97cf7004cd9e7bfdba8f5a015acbc'. 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.562 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-06-15T01:05:59.563 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at e3daf20513a qa/distros: removed rhel 8 from supported distros 2024-06-15T01:05:59.567 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-15T01:05:59.621 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-15T01:05:59.623 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-15T01:05:59.623 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-15T01:06:00.571 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-15T01:06:00.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:00 smithi137 ceph-mon[29354]: pgmap v228: 65 pgs: 65 active+clean; 14 MiB data, 204 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s wr, 54 op/s 2024-06-15T01:06:00.737 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-15T01:06:00.785 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-15T01:06:00.786 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-15T01:06:00.786 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-15T01:06:00.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:00 smithi142 ceph-mon[37379]: pgmap v228: 65 pgs: 65 active+clean; 14 MiB data, 204 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s wr, 54 op/s 2024-06-15T01:06:00.828 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-15T01:06:00.832 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:06:00.832 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-15T01:06:00.848 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-06-15T01:06:00.849 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-06-15T01:06:00.849 DEBUG:teuthology.orchestra.run.smithi137:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e3daf20513a97cf7004cd9e7bfdba8f5a015acbc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-06-15T01:06:00.916 INFO:tasks.workunit.client.0.smithi137.stderr:+ mkdir -p fsstress 2024-06-15T01:06:00.918 INFO:tasks.workunit.client.0.smithi137.stderr:+ pushd fsstress 2024-06-15T01:06:00.919 INFO:tasks.workunit.client.0.smithi137.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-15T01:06:00.920 INFO:tasks.workunit.client.0.smithi137.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-06-15T01:06:01.620 INFO:tasks.workunit.client.0.smithi137.stderr:+ tar xzf ltp-full.tgz 2024-06-15T01:06:02.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:02 smithi137 ceph-mon[29354]: pgmap v229: 65 pgs: 65 active+clean; 16 MiB data, 225 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s wr, 104 op/s 2024-06-15T01:06:02.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:02 smithi142 ceph-mon[37379]: pgmap v229: 65 pgs: 65 active+clean; 16 MiB data, 225 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s wr, 104 op/s 2024-06-15T01:06:03.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:03 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:03.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:03 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:04.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:04 smithi137 ceph-mon[29354]: pgmap v230: 65 pgs: 65 active+clean; 23 MiB data, 256 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s wr, 147 op/s 2024-06-15T01:06:04.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:04 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:06:04.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:04 smithi142 ceph-mon[37379]: pgmap v230: 65 pgs: 65 active+clean; 23 MiB data, 256 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s wr, 147 op/s 2024-06-15T01:06:04.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:04 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:06:06.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:06 smithi142 ceph-mon[37379]: pgmap v231: 65 pgs: 65 active+clean; 30 MiB data, 288 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 208 op/s 2024-06-15T01:06:06.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:06 smithi137 ceph-mon[29354]: pgmap v231: 65 pgs: 65 active+clean; 30 MiB data, 288 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 208 op/s 2024-06-15T01:06:08.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:08 smithi137 ceph-mon[29354]: pgmap v232: 65 pgs: 65 active+clean; 43 MiB data, 336 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 273 op/s 2024-06-15T01:06:08.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:08 smithi142 ceph-mon[37379]: pgmap v232: 65 pgs: 65 active+clean; 43 MiB data, 336 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 273 op/s 2024-06-15T01:06:10.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:10 smithi137 ceph-mon[29354]: pgmap v233: 65 pgs: 65 active+clean; 55 MiB data, 413 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 405 op/s 2024-06-15T01:06:10.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:10 smithi142 ceph-mon[37379]: pgmap v233: 65 pgs: 65 active+clean; 55 MiB data, 413 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 405 op/s 2024-06-15T01:06:12.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:12 smithi137 ceph-mon[29354]: pgmap v234: 65 pgs: 65 active+clean; 68 MiB data, 452 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 427 op/s 2024-06-15T01:06:12.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:12 smithi142 ceph-mon[37379]: pgmap v234: 65 pgs: 65 active+clean; 68 MiB data, 452 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 427 op/s 2024-06-15T01:06:14.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:14 smithi137 ceph-mon[29354]: pgmap v235: 65 pgs: 65 active+clean; 76 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s wr, 427 op/s 2024-06-15T01:06:14.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:14 smithi142 ceph-mon[37379]: pgmap v235: 65 pgs: 65 active+clean; 76 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s wr, 427 op/s 2024-06-15T01:06:16.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:16 smithi142 ceph-mon[37379]: pgmap v236: 65 pgs: 65 active+clean; 83 MiB data, 493 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 459 op/s 2024-06-15T01:06:16.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:16 smithi137 ceph-mon[29354]: pgmap v236: 65 pgs: 65 active+clean; 83 MiB data, 493 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 459 op/s 2024-06-15T01:06:18.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:18 smithi137 ceph-mon[29354]: pgmap v237: 65 pgs: 65 active+clean; 86 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 462 op/s 2024-06-15T01:06:18.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:18 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:18.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:18 smithi142 ceph-mon[37379]: pgmap v237: 65 pgs: 65 active+clean; 86 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 462 op/s 2024-06-15T01:06:18.840 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:18 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:20.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:20 smithi137 ceph-mon[29354]: pgmap v238: 65 pgs: 65 active+clean; 103 MiB data, 533 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 526 op/s 2024-06-15T01:06:20.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:06:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:06:20.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:20 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:06:20.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:20 smithi142 ceph-mon[37379]: pgmap v238: 65 pgs: 65 active+clean; 103 MiB data, 533 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 526 op/s 2024-06-15T01:06:20.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:20.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:06:20.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:06:20.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:20 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:06:21.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:21 smithi137 ceph-mon[29354]: Upgrade: Updating mgr.smithi142.dllryb 2024-06-15T01:06:21.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:21 smithi137 ceph-mon[29354]: Deploying daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:06:21.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:21 smithi142 ceph-mon[37379]: Upgrade: Updating mgr.smithi142.dllryb 2024-06-15T01:06:21.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:21 smithi142 ceph-mon[37379]: Deploying daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:06:22.663 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:22 smithi142 ceph-mon[37379]: pgmap v239: 65 pgs: 65 active+clean; 106 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 446 op/s 2024-06-15T01:06:22.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:22 smithi137 ceph-mon[29354]: pgmap v239: 65 pgs: 65 active+clean; 106 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 446 op/s 2024-06-15T01:06:23.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 -- 172.21.15.137:0/2932266247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc390103ff0 msgr2=0x7fc3901063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 --2- 172.21.15.137:0/2932266247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc390103ff0 0x7fc3901063e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc384009910 tx=0x7fc384009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:23.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 -- 172.21.15.137:0/2932266247 shutdown_connections 2024-06-15T01:06:23.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 --2- 172.21.15.137:0/2932266247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc390103ff0 0x7fc3901063e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 --2- 172.21.15.137:0/2932266247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3901016c0 0x7fc390103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.368+0000 7fc395281700 1 -- 172.21.15.137:0/2932266247 >> 172.21.15.137:0/2932266247 conn(0x7fc3900fb030 msgr2=0x7fc3900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.369+0000 7fc395281700 1 -- 172.21.15.137:0/2932266247 shutdown_connections 2024-06-15T01:06:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.369+0000 7fc395281700 1 -- 172.21.15.137:0/2932266247 wait complete. 2024-06-15T01:06:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.369+0000 7fc395281700 1 Processor -- start 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.370+0000 7fc395281700 1 -- start start 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.370+0000 7fc395281700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.370+0000 7fc395281700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 0x7fc3900ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.370+0000 7fc38effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.370+0000 7fc38effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52344/0 (socket says 172.21.15.137:52344) 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38effd700 1 -- 172.21.15.137:0/1701717479 learned_addr learned my addr 172.21.15.137:0/1701717479 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:23.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900ffbb0 con 0x7fc3901016c0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900ffcf0 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38e7fc700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 0x7fc3900ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38e7fc700 1 -- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 msgr2=0x7fc390101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38e7fc700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38e7fc700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3840095f0 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38effd700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.371+0000 7fc38e7fc700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 0x7fc3900ff670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc384009910 tx=0x7fc38400bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.372+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38401d070 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.372+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc3840056f0 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.372+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3900fff70 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.372+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3840039d0 con 0x7fc390103ff0 2024-06-15T01:06:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.372+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc390100350 con 0x7fc390103ff0 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.373+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3901903e0 con 0x7fc390103ff0 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.374+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc384003ba0 con 0x7fc390103ff0 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.374+0000 7fc377fff700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 0x7fc37806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.375+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc38408ce40 con 0x7fc390103ff0 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.375+0000 7fc38effd700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 0x7fc37806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.375+0000 7fc38effd700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 0x7fc37806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc380011550 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:23.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.380+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc38405b0c0 con 0x7fc390103ff0 2024-06-15T01:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.774+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3900611d0 con 0x7fc37806c4e0 2024-06-15T01:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.776+0000 7fc377fff700 1 -- 172.21.15.137:0/1701717479 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fc3900611d0 con 0x7fc37806c4e0 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 msgr2=0x7fc37806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 0x7fc37806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc380011550 tx=0x7fc380009260 comp rx=0 tx=0).stop 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 msgr2=0x7fc3900ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 0x7fc3900ff670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc384009910 tx=0x7fc38400bde0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 shutdown_connections 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc37806c4e0 0x7fc37806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3901016c0 0x7fc390101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 --2- 172.21.15.137:0/1701717479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc390103ff0 0x7fc3900ff670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 >> 172.21.15.137:0/1701717479 conn(0x7fc3900fb030 msgr2=0x7fc3900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:23.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 shutdown_connections 2024-06-15T01:06:23.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.780+0000 7fc395281700 1 -- 172.21.15.137:0/1701717479 wait complete. 2024-06-15T01:06:23.793 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:06:23.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.944+0000 7fc552bf1700 1 -- 172.21.15.137:0/3460532582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 msgr2=0x7fc54c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.944+0000 7fc552bf1700 1 --2- 172.21.15.137:0/3460532582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c101030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc540009910 tx=0x7fc540009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:23.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 -- 172.21.15.137:0/3460532582 shutdown_connections 2024-06-15T01:06:23.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 --2- 172.21.15.137:0/3460532582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 --2- 172.21.15.137:0/3460532582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c101030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 -- 172.21.15.137:0/3460532582 >> 172.21.15.137:0/3460532582 conn(0x7fc54c0fc1b0 msgr2=0x7fc54c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:23.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 -- 172.21.15.137:0/3460532582 shutdown_connections 2024-06-15T01:06:23.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.945+0000 7fc552bf1700 1 -- 172.21.15.137:0/3460532582 wait complete. 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 Processor -- start 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 -- start start 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc55098d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c199f20 con 0x7fc54c100c10 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc55098d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52362/0 (socket says 172.21.15.137:52362) 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc55098d700 1 -- 172.21.15.137:0/2838338331 learned_addr learned my addr 172.21.15.137:0/2838338331 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:23.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c19a060 con 0x7fc54c101e10 2024-06-15T01:06:23.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.946+0000 7fc54bfff700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc54bfff700 1 -- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 msgr2=0x7fc54c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc54bfff700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc54bfff700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400095f0 con 0x7fc54c101e10 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc54bfff700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c073180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e970 tx=0x7fc53800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc53800c9d0 con 0x7fc54c101e10 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc538010430 con 0x7fc54c101e10 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c0736c0 con 0x7fc54c101e10 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.947+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc538018630 con 0x7fc54c101e10 2024-06-15T01:06:23.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.948+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c073c60 con 0x7fc54c101e10 2024-06-15T01:06:23.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.949+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c073ed0 con 0x7fc54c101e10 2024-06-15T01:06:23.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.950+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc5380105a0 con 0x7fc54c101e10 2024-06-15T01:06:23.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.950+0000 7fc549ffb700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 0x7fc53c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:23.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.950+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc538014070 con 0x7fc54c101e10 2024-06-15T01:06:23.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.951+0000 7fc55098d700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 0x7fc53c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:23.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.951+0000 7fc55098d700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 0x7fc53c06e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc54000b600 tx=0x7fc540009f70 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:23.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:23.956+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc5380520e0 con 0x7fc54c101e10 2024-06-15T01:06:24.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.314+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc54c106760 con 0x7fc53c06c530 2024-06-15T01:06:24.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.315+0000 7fc549ffb700 1 -- 172.21.15.137:0/2838338331 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc54c106760 con 0x7fc53c06c530 2024-06-15T01:06:24.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 msgr2=0x7fc53c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 0x7fc53c06e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc54000b600 tx=0x7fc540009f70 comp rx=0 tx=0).stop 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 msgr2=0x7fc54c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c073180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e970 tx=0x7fc53800ec80 comp rx=0 tx=0).stop 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 shutdown_connections 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fc53c06c530 0x7fc53c06e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54c100c10 0x7fc54c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 --2- 172.21.15.137:0/2838338331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc54c101e10 0x7fc54c073180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 >> 172.21.15.137:0/2838338331 conn(0x7fc54c0fc1b0 msgr2=0x7fc54c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 shutdown_connections 2024-06-15T01:06:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.319+0000 7fc552bf1700 1 -- 172.21.15.137:0/2838338331 wait complete. 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2778533761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 msgr2=0x7f41bc1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2778533761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc1063e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009910 tx=0x7f41ac009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2778533761 shutdown_connections 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2778533761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc1063e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2778533761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2778533761 >> 172.21.15.137:0/2778533761 conn(0x7f41bc0fb030 msgr2=0x7f41bc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2778533761 shutdown_connections 2024-06-15T01:06:24.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.468+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2778533761 wait complete. 2024-06-15T01:06:24.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 Processor -- start 2024-06-15T01:06:24.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 -- start start 2024-06-15T01:06:24.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc19d3b0 con 0x7f41bc103ff0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.469+0000 7f41c3ce0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc19d4f0 con 0x7f41bc1016c0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c127b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:32770/0 (socket says 172.21.15.137:32770) 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 -- 172.21.15.137:0/2574772354 learned_addr learned my addr 172.21.15.137:0/2574772354 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c127b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52392/0 (socket says 172.21.15.137:52392) 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 -- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 msgr2=0x7f41bc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 -- 172.21.15.137:0/2574772354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8009520 con 0x7f41bc1016c0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.470+0000 7f41c1a7c700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eab0 tx=0x7f41b800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.471+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b800cba0 con 0x7f41bc1016c0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.471+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac0095f0 con 0x7f41bc1016c0 2024-06-15T01:06:24.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.471+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1a2350 con 0x7f41bc1016c0 2024-06-15T01:06:24.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.471+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f41b80044b0 con 0x7f41bc1016c0 2024-06-15T01:06:24.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.471+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b8005020 con 0x7f41bc1016c0 2024-06-15T01:06:24.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.472+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a0005320 con 0x7f41bc1016c0 2024-06-15T01:06:24.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.473+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f41b8005210 con 0x7f41bc1016c0 2024-06-15T01:06:24.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.473+0000 7f41b2ffd700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 0x7f41a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.474+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f41b8014070 con 0x7f41bc1016c0 2024-06-15T01:06:24.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.474+0000 7f41c127b700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 0x7f41a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:24.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.474+0000 7f41c127b700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 0x7f41a806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000c00 tx=0x7f41ac012040 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:24.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.480+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f41b8059860 con 0x7f41bc1016c0 2024-06-15T01:06:24.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:24 smithi142 ceph-mon[37379]: pgmap v240: 65 pgs: 65 active+clean; 111 MiB data, 589 MiB used, 536 GiB / 536 GiB avail; 3.8 MiB/s wr, 411 op/s 2024-06-15T01:06:24.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:24.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:24.569 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:24 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:24.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:24 smithi137 ceph-mon[29354]: pgmap v240: 65 pgs: 65 active+clean; 111 MiB data, 589 MiB used, 536 GiB / 536 GiB avail; 3.8 MiB/s wr, 411 op/s 2024-06-15T01:06:24.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:24.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:24.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:24 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:24.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.789+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a0000bf0 con 0x7f41a806c530 2024-06-15T01:06:24.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.806+0000 7f41b2ffd700 1 -- 172.21.15.137:0/2574772354 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3576 (secure 0 0 0) 0x7f41a0000bf0 con 0x7f41a806c530 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (6m) 97s ago 7m 25.7M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (7m) 97s ago 7m 7516k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (6m) 103s ago 6m 7235k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (7m) 97s ago 7m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (6m) 103s ago 6m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (5m) 97s ago 7m 83.9M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:06:24.804 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (116s) 97s ago 116s 17.1M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (2m) 97s ago 2m 18.9M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (119s) 103s ago 119s 11.3M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (114s) 103s ago 113s 14.3M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:9283,8765,8443 running (9m) 97s ago 9m 488M - 18.2.1 5be31c24972a 61486e562eb9 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 starting - - - - 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (9m) 97s ago 9m 45.0M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (6m) 103s ago 6m 35.5M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (7m) 97s ago 7m 16.6M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 103s ago 6m 16.9M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (5m) 97s ago 5m 47.6M 3946M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (5m) 97s ago 5m 50.9M 3946M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (4m) 97s ago 4m 45.7M 3946M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:06:24.805 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (4m) 103s ago 4m 47.1M 4970M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:06:24.806 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (3m) 103s ago 3m 44.2M 4970M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:06:24.806 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (3m) 103s ago 3m 43.3M 4970M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:06:24.806 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (5m) 97s ago 6m 54.5M - 2.43.0 a07b618ecd1d 904ef44cd0a5 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 msgr2=0x7f41a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 0x7f41a806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000c00 tx=0x7f41ac012040 comp rx=0 tx=0).stop 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 msgr2=0x7f41bc19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eab0 tx=0x7f41b800edc0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 shutdown_connections 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7f41a806c530 0x7f41a806e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41bc1016c0 0x7f41bc19c850 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 --2- 172.21.15.137:0/2574772354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc103ff0 0x7f41bc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 >> 172.21.15.137:0/2574772354 conn(0x7f41bc0fb030 msgr2=0x7f41bc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:24.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 shutdown_connections 2024-06-15T01:06:24.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.809+0000 7f41c3ce0700 1 -- 172.21.15.137:0/2574772354 wait complete. 2024-06-15T01:06:24.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 -- 172.21.15.137:0/3288427944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e80ffab0 msgr2=0x7fb9e80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/3288427944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e80ffab0 0x7fb9e80ffed0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009910 tx=0x7fb9dc009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:24.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 -- 172.21.15.137:0/3288427944 shutdown_connections 2024-06-15T01:06:24.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/3288427944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e8100cb0 0x7fb9e8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/3288427944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e80ffab0 0x7fb9e80ffed0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 -- 172.21.15.137:0/3288427944 >> 172.21.15.137:0/3288427944 conn(0x7fb9e80fb030 msgr2=0x7fb9e80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 -- 172.21.15.137:0/3288427944 shutdown_connections 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.958+0000 7fb9eec4e700 1 -- 172.21.15.137:0/3288427944 wait complete. 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 Processor -- start 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 -- start start 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e80ffab0 0x7fb9e8195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8195cd0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9eec4e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8195e10 con 0x7fb9e80ffab0 2024-06-15T01:06:24.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52412/0 (socket says 172.21.15.137:52412) 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 -- 172.21.15.137:0/2354561439 learned_addr learned my addr 172.21.15.137:0/2354561439 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 -- 172.21.15.137:0/2354561439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e80ffab0 msgr2=0x7fb9e8195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e80ffab0 0x7fb9e8195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.959+0000 7fb9e7fff700 1 -- 172.21.15.137:0/2354561439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009520 con 0x7fb9e8100cb0 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9e7fff700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009f60 tx=0x7fb9d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:24.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9d4009880 con 0x7fb9e8100cb0 2024-06-15T01:06:24.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb9d40044b0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc0095f0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9d4005020 con 0x7fb9e8100cb0 2024-06-15T01:06:24.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.960+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e80688b0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.962+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e81084d0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.962+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb9d4004020 con 0x7fb9e8100cb0 2024-06-15T01:06:24.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.963+0000 7fb9e5ffb700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 0x7fb9d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:24.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.963+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb9d408dbf0 con 0x7fb9e8100cb0 2024-06-15T01:06:24.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.963+0000 7fb9ec9ea700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 0x7fb9d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:24.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.964+0000 7fb9ec9ea700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 0x7fb9d806e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005e20 tx=0x7fb9dc009f70 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:24.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:24.969+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9d405be00 con 0x7fb9e8100cb0 2024-06-15T01:06:25.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.420+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9e819a7a0 con 0x7fb9e8100cb0 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.420+0000 7fb9e5ffb700 1 -- 172.21.15.137:0/2354561439 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fb9d405b990 con 0x7fb9e8100cb0 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:06:25.418 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:06:25.419 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 msgr2=0x7fb9d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 0x7fb9d806e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005e20 tx=0x7fb9dc009f70 comp rx=0 tx=0).stop 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 msgr2=0x7fb9e81956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009f60 tx=0x7fb9d4009f90 comp rx=0 tx=0).stop 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 shutdown_connections 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fb9d806c4e0 0x7fb9d806e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e80ffab0 0x7fb9e8195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 --2- 172.21.15.137:0/2354561439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9e8100cb0 0x7fb9e81956b0 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009f60 tx=0x7fb9d4009f90 comp rx=0 tx=0).stop 2024-06-15T01:06:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 >> 172.21.15.137:0/2354561439 conn(0x7fb9e80fb030 msgr2=0x7fb9e8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:25.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 shutdown_connections 2024-06-15T01:06:25.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.424+0000 7fb9eec4e700 1 -- 172.21.15.137:0/2354561439 wait complete. 2024-06-15T01:06:25.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 -- 172.21.15.137:0/1535126086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 msgr2=0x7fbc5c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 --2- 172.21.15.137:0/1535126086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c103ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009910 tx=0x7fbc58009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 -- 172.21.15.137:0/1535126086 shutdown_connections 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 --2- 172.21.15.137:0/1535126086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 0x7fbc5c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 --2- 172.21.15.137:0/1535126086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c103ad0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.577+0000 7fbc638f6700 1 -- 172.21.15.137:0/1535126086 >> 172.21.15.137:0/1535126086 conn(0x7fbc5c0fb050 msgr2=0x7fbc5c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.578+0000 7fbc638f6700 1 -- 172.21.15.137:0/1535126086 shutdown_connections 2024-06-15T01:06:25.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.578+0000 7fbc638f6700 1 -- 172.21.15.137:0/1535126086 wait complete. 2024-06-15T01:06:25.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 Processor -- start 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 -- start start 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 0x7fbc5c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c199060 con 0x7fbc5c1016e0 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc638f6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1991a0 con 0x7fbc5c104010 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc61692700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:25.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc61692700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52424/0 (socket says 172.21.15.137:52424) 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.579+0000 7fbc61692700 1 -- 172.21.15.137:0/94497176 learned_addr learned my addr 172.21.15.137:0/94497176 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.580+0000 7fbc60e91700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 0x7fbc5c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.580+0000 7fbc61692700 1 -- 172.21.15.137:0/94497176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 msgr2=0x7fbc5c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.580+0000 7fbc61692700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 0x7fbc5c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.580+0000 7fbc61692700 1 -- 172.21.15.137:0/94497176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580095f0 con 0x7fbc5c1016e0 2024-06-15T01:06:25.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.580+0000 7fbc61692700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b5c0 tx=0x7fbc5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.581+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5801d070 con 0x7fbc5c1016e0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.581+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c19dbf0 con 0x7fbc5c1016e0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.581+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbc5800f460 con 0x7fbc5c1016e0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.581+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc58021640 con 0x7fbc5c1016e0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.581+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c19e0e0 con 0x7fbc5c1016e0 2024-06-15T01:06:25.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.583+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c0ffa80 con 0x7fbc5c1016e0 2024-06-15T01:06:25.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.583+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbc5802b440 con 0x7fbc5c1016e0 2024-06-15T01:06:25.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.584+0000 7fbc527fc700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 0x7fbc4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:25.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.584+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbc5808cb30 con 0x7fbc5c1016e0 2024-06-15T01:06:25.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.584+0000 7fbc60e91700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 0x7fbc4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:25.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.584+0000 7fbc60e91700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 0x7fbc4806e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007be0 tx=0x7fbc4c0073d0 comp rx=0 tx=0).ready entity=mgr.14219 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:25.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.590+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc5805aee0 con 0x7fbc5c1016e0 2024-06-15T01:06:25.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:25 smithi137 ceph-mon[29354]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:25.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:25 smithi137 ceph-mon[29354]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:25.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:25 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2354561439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:06:25.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:25 smithi142 ceph-mon[37379]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:25.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:25 smithi142 ceph-mon[37379]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:25.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:25 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2354561439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:06:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.905+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 --> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc5c0fc490 con 0x7fbc4806c4e0 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.907+0000 7fbc527fc700 1 -- 172.21.15.137:0/94497176 <== mgr.14219 v2:172.21.15.137:6800/3216905676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fbc5c0fc490 con 0x7fbc4806c4e0 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [], 2024-06-15T01:06:25.904 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "", 2024-06-15T01:06:25.905 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mgr daemons", 2024-06-15T01:06:25.905 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:06:25.905 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 msgr2=0x7fbc4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 0x7fbc4806e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007be0 tx=0x7fbc4c0073d0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 msgr2=0x7fbc5c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b5c0 tx=0x7fbc5800bd70 comp rx=0 tx=0).stop 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 shutdown_connections 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:6800/3216905676,v1:172.21.15.137:6801/3216905676] conn(0x7fbc4806c4e0 0x7fbc4806e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc5c1016e0 0x7fbc5c198500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 --2- 172.21.15.137:0/94497176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc5c104010 0x7fbc5c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.910+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 >> 172.21.15.137:0/94497176 conn(0x7fbc5c0fb050 msgr2=0x7fbc5c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.911+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 shutdown_connections 2024-06-15T01:06:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:25.911+0000 7fbc638f6700 1 -- 172.21.15.137:0/94497176 wait complete. 2024-06-15T01:06:26.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:26 smithi137 ceph-mon[29354]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:26.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:26 smithi137 ceph-mon[29354]: pgmap v241: 65 pgs: 65 active+clean; 113 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s wr, 412 op/s 2024-06-15T01:06:26.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:26 smithi142 ceph-mon[37379]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:26.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:26 smithi142 ceph-mon[37379]: pgmap v241: 65 pgs: 65 active+clean; 113 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s wr, 412 op/s 2024-06-15T01:06:27.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:27 smithi137 ceph-mon[29354]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:27.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:27 smithi142 ceph-mon[37379]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:28.714 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:28 smithi142 ceph-mon[37379]: pgmap v242: 65 pgs: 65 active+clean; 121 MiB data, 641 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 397 op/s 2024-06-15T01:06:28.714 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:28 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:28.714 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:28 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:28.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:28 smithi137 ceph-mon[29354]: pgmap v242: 65 pgs: 65 active+clean; 121 MiB data, 641 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 397 op/s 2024-06-15T01:06:28.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:28 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:28.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:28 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:30.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:30.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:29 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:30.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:29 smithi142 ceph-mon[37379]: pgmap v243: 65 pgs: 65 active+clean; 135 MiB data, 698 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 473 op/s 2024-06-15T01:06:30.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:30.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:29 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:30.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:29 smithi137 ceph-mon[29354]: pgmap v243: 65 pgs: 65 active+clean; 135 MiB data, 698 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 473 op/s 2024-06-15T01:06:31.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:06:31.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb restarted 2024-06-15T01:06:31.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:06:31.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:06:31.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:06:31.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:06:31.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:31 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:06:31.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:06:31.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb restarted 2024-06-15T01:06:31.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:06:31.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:06:31.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:06:31.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:06:31.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:31 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:06:32.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:32 smithi142 ceph-mon[37379]: mgrmap e20: smithi137.nbzbyc(active, since 6m), standbys: smithi142.dllryb 2024-06-15T01:06:32.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:32 smithi142 ceph-mon[37379]: pgmap v244: 65 pgs: 65 active+clean; 141 MiB data, 725 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 418 op/s 2024-06-15T01:06:32.476 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:32 smithi137 ceph-mon[29354]: mgrmap e20: smithi137.nbzbyc(active, since 6m), standbys: smithi142.dllryb 2024-06-15T01:06:32.476 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:32 smithi137 ceph-mon[29354]: pgmap v244: 65 pgs: 65 active+clean; 141 MiB data, 725 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 418 op/s 2024-06-15T01:06:33.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:33 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:33.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:33 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:34.443 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: pgmap v245: 65 pgs: 65 active+clean; 150 MiB data, 738 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s wr, 419 op/s 2024-06-15T01:06:34.443 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:34.444 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:34.444 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:06:34.444 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:06:34.444 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:34 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:34.704 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: pgmap v245: 65 pgs: 65 active+clean; 150 MiB data, 738 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s wr, 419 op/s 2024-06-15T01:06:34.704 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:34.705 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:34.705 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:06:34.705 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:06:34.705 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:34 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:06:35.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr fail", "who": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:06:35.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: osdmap e48: 6 total, 6 up, 6 in 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "mgr fail", "who": "smithi137.nbzbyc"}]': finished 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: mgrmap e21: smithi142.dllryb(active, starting, since 0.0101586s) 2024-06-15T01:06:35.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: Manager daemon smithi142.dllryb is now available 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/trash_purge_schedule"}]: dispatch 2024-06-15T01:06:35.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:35 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/trash_purge_schedule"}]: dispatch 2024-06-15T01:06:35.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:35.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr fail", "who": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:06:35.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:06:35.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:06:35.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:06:35.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:06:35.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:06:35.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:06:35.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:06:35.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:06:35.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:06:35.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:06:35.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:06:35.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:06:35.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:06:35.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: osdmap e48: 6 total, 6 up, 6 in 2024-06-15T01:06:35.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.14219 172.21.15.137:0/1430656548' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "mgr fail", "who": "smithi137.nbzbyc"}]': finished 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: mgrmap e21: smithi142.dllryb(active, starting, since 0.0101586s) 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: Manager daemon smithi142.dllryb is now available 2024-06-15T01:06:35.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:06:35.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:35.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:06:35.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:06:35.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/trash_purge_schedule"}]: dispatch 2024-06-15T01:06:35.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:35 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.dllryb/trash_purge_schedule"}]: dispatch 2024-06-15T01:06:36.694 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:36 smithi137 ceph-mon[29354]: Deploying cephadm binary to smithi142 2024-06-15T01:06:36.694 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:36 smithi137 ceph-mon[29354]: Deploying cephadm binary to smithi137 2024-06-15T01:06:36.694 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:36 smithi137 ceph-mon[29354]: mgrmap e22: smithi142.dllryb(active, since 1.00966s) 2024-06-15T01:06:36.694 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:36 smithi137 ceph-mon[29354]: pgmap v3: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:36.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:36 smithi142 ceph-mon[37379]: Deploying cephadm binary to smithi142 2024-06-15T01:06:36.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:36 smithi142 ceph-mon[37379]: Deploying cephadm binary to smithi137 2024-06-15T01:06:36.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:36 smithi142 ceph-mon[37379]: mgrmap e22: smithi142.dllryb(active, since 1.00966s) 2024-06-15T01:06:36.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:36 smithi142 ceph-mon[37379]: pgmap v3: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:37.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: pgmap v4: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: [15/Jun/2024:01:06:36] ENGINE Bus STARTING 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: [15/Jun/2024:01:06:36] ENGINE Serving on https://172.21.15.142:7150 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: [15/Jun/2024:01:06:36] ENGINE Error in HTTPServer.serve 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Traceback (most recent call last): 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-15T01:06:37.654 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._run(expiration_interval) 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-15T01:06:37.655 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: s = self.context.wrap_socket( 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: return self.sslsocket_class._create( 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self.do_handshake() 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._sslobj.do_handshake() 2024-06-15T01:06:37.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-15T01:06:37.657 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: [15/Jun/2024:01:06:36] ENGINE Serving on http://172.21.15.142:8765 2024-06-15T01:06:37.657 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:37 smithi142 ceph-mon[37379]: [15/Jun/2024:01:06:36] ENGINE Bus STARTED 2024-06-15T01:06:37.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: pgmap v4: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: [15/Jun/2024:01:06:36] ENGINE Bus STARTING 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: [15/Jun/2024:01:06:36] ENGINE Serving on https://172.21.15.142:7150 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: [15/Jun/2024:01:06:36] ENGINE Error in HTTPServer.serve 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Traceback (most recent call last): 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-15T01:06:37.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-06-15T01:06:37.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-15T01:06:37.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._run(expiration_interval) 2024-06-15T01:06:37.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-15T01:06:37.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-15T01:06:37.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: s = self.context.wrap_socket( 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: return self.sslsocket_class._create( 2024-06-15T01:06:37.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self.do_handshake() 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._sslobj.do_handshake() 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: [15/Jun/2024:01:06:36] ENGINE Serving on http://172.21.15.142:8765 2024-06-15T01:06:37.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:37 smithi137 ceph-mon[29354]: [15/Jun/2024:01:06:36] ENGINE Bus STARTED 2024-06-15T01:06:38.476 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:38 smithi137 ceph-mon[29354]: mgrmap e23: smithi142.dllryb(active, since 3s) 2024-06-15T01:06:38.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:38 smithi142 ceph-mon[37379]: mgrmap e23: smithi142.dllryb(active, since 3s) 2024-06-15T01:06:39.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:39 smithi137 ceph-mon[29354]: pgmap v5: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:39.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:39 smithi142 ceph-mon[37379]: pgmap v5: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:41.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:41 smithi137 ceph-mon[29354]: pgmap v6: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:41.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:41 smithi142 ceph-mon[37379]: pgmap v6: 65 pgs: 65 active+clean; 172 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-06-15T01:06:43.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:43 smithi137 ceph-mon[29354]: pgmap v7: 65 pgs: 65 active+clean; 191 MiB data, 892 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-06-15T01:06:43.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:43 smithi142 ceph-mon[37379]: pgmap v7: 65 pgs: 65 active+clean; 191 MiB data, 892 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-06-15T01:06:44.725 INFO:tasks.workunit.client.1.smithi142.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-06-15T01:06:44.739 INFO:tasks.workunit.client.1.smithi142.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-15T01:06:44.740 INFO:tasks.workunit.client.1.smithi142.stderr:+ make 2024-06-15T01:06:44.929 INFO:tasks.workunit.client.1.smithi142.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: Standby manager daemon smithi137.nbzbyc started 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/crt"}]: dispatch 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/key"}]: dispatch 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:44 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: Standby manager daemon smithi137.nbzbyc started 2024-06-15T01:06:45.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/crt"}]: dispatch 2024-06-15T01:06:45.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/key"}]: dispatch 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/1583485813' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:45.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:44 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.203 INFO:tasks.workunit.client.0.smithi137.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-06-15T01:06:46.207 INFO:tasks.workunit.client.0.smithi137.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-15T01:06:46.208 INFO:tasks.workunit.client.0.smithi137.stderr:+ make 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: pgmap v8: 65 pgs: 65 active+clean; 191 MiB data, 892 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 2.2 MiB/s wr, 227 op/s 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: mgrmap e24: smithi142.dllryb(active, since 10s), standbys: smithi137.nbzbyc 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:45 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: pgmap v8: 65 pgs: 65 active+clean; 191 MiB data, 892 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 2.2 MiB/s wr, 227 op/s 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: mgrmap e24: smithi142.dllryb(active, since 10s), standbys: smithi137.nbzbyc 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:06:46.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:45 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:06:46.562 INFO:tasks.workunit.client.0.smithi137.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-06-15T01:06:46.953 INFO:tasks.workunit.client.1.smithi142.stderr:++ readlink -f fsstress 2024-06-15T01:06:46.954 INFO:tasks.workunit.client.1.smithi142.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-06-15T01:06:46.955 INFO:tasks.workunit.client.1.smithi142.stderr:+ popd 2024-06-15T01:06:46.955 INFO:tasks.workunit.client.1.smithi142.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-06-15T01:06:46.956 INFO:tasks.workunit.client.1.smithi142.stderr:+ popd 2024-06-15T01:06:46.956 INFO:tasks.workunit.client.1.smithi142.stdout:~/cephtest/mnt.1/client.1/tmp 2024-06-15T01:06:46.956 INFO:tasks.workunit.client.1.smithi142.stderr:++ mktemp -d -p . 2024-06-15T01:06:46.960 INFO:tasks.workunit.client.1.smithi142.stderr:+ T=./tmp.K3vUaNGYGU 2024-06-15T01:06:46.960 INFO:tasks.workunit.client.1.smithi142.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.K3vUaNGYGU -l 1 -n 1000 -p 10 -v 2024-06-15T01:06:46.963 INFO:tasks.workunit.client.1.smithi142.stdout:seed = 1717710691 2024-06-15T01:06:46.970 INFO:tasks.workunit.client.1.smithi142.stdout:0/0: creat f0 x:0 0 0 2024-06-15T01:06:46.971 INFO:tasks.workunit.client.1.smithi142.stdout:2/0: read - no filename 2024-06-15T01:06:46.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/1: write - no filename 2024-06-15T01:06:46.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/2: write - no filename 2024-06-15T01:06:46.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/3: chown . 25162796 1 2024-06-15T01:06:46.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/4: unlink - no file 2024-06-15T01:06:46.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/5: dwrite - no filename 2024-06-15T01:06:46.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/6: dread - no filename 2024-06-15T01:06:46.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/7: read - no filename 2024-06-15T01:06:46.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/8: dwrite - no filename 2024-06-15T01:06:46.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/9: dwrite - no filename 2024-06-15T01:06:46.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/10: dwrite - no filename 2024-06-15T01:06:46.974 INFO:tasks.workunit.client.1.smithi142.stdout:1/0: creat f0 x:0 0 0 2024-06-15T01:06:46.976 INFO:tasks.workunit.client.1.smithi142.stdout:4/0: rmdir - no directory 2024-06-15T01:06:46.976 INFO:tasks.workunit.client.1.smithi142.stdout:4/1: rmdir - no directory 2024-06-15T01:06:46.977 INFO:tasks.workunit.client.1.smithi142.stdout:1/1: link f0 f1 0 2024-06-15T01:06:46.979 INFO:tasks.workunit.client.1.smithi142.stdout:5/0: chown . 92432065 1 2024-06-15T01:06:46.979 INFO:tasks.workunit.client.1.smithi142.stdout:5/1: chown . 43435830 1 2024-06-15T01:06:46.979 INFO:tasks.workunit.client.1.smithi142.stdout:5/2: readlink - no filename 2024-06-15T01:06:46.980 INFO:tasks.workunit.client.1.smithi142.stdout:3/0: mkdir d0 0 2024-06-15T01:06:46.981 INFO:tasks.workunit.client.1.smithi142.stdout:1/2: mknod c2 0 2024-06-15T01:06:46.981 INFO:tasks.workunit.client.1.smithi142.stdout:1/3: dread - f1 zero size 2024-06-15T01:06:46.982 INFO:tasks.workunit.client.1.smithi142.stdout:4/2: symlink l0 0 2024-06-15T01:06:46.982 INFO:tasks.workunit.client.1.smithi142.stdout:4/3: write - no filename 2024-06-15T01:06:46.982 INFO:tasks.workunit.client.1.smithi142.stdout:4/4: truncate - no filename 2024-06-15T01:06:46.983 INFO:tasks.workunit.client.1.smithi142.stdout:4/5: truncate - no filename 2024-06-15T01:06:46.983 INFO:tasks.workunit.client.1.smithi142.stdout:4/6: dwrite - no filename 2024-06-15T01:06:46.983 INFO:tasks.workunit.client.1.smithi142.stdout:4/7: write - no filename 2024-06-15T01:06:46.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/0: link - no file 2024-06-15T01:06:46.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/1: rename - no filename 2024-06-15T01:06:46.987 INFO:tasks.workunit.client.1.smithi142.stdout:1/4: creat f3 x:0 0 0 2024-06-15T01:06:46.987 INFO:tasks.workunit.client.1.smithi142.stdout:1/5: dread - f3 zero size 2024-06-15T01:06:46.988 INFO:tasks.workunit.client.1.smithi142.stdout:4/8: creat f1 x:0 0 0 2024-06-15T01:06:46.989 INFO:tasks.workunit.client.1.smithi142.stdout:5/3: getdents . 0 2024-06-15T01:06:46.989 INFO:tasks.workunit.client.1.smithi142.stdout:5/4: read - no filename 2024-06-15T01:06:46.990 INFO:tasks.workunit.client.1.smithi142.stdout:3/1: symlink d0/l1 0 2024-06-15T01:06:46.990 INFO:tasks.workunit.client.1.smithi142.stdout:1/6: rename f0 to f4 0 2024-06-15T01:06:46.991 INFO:tasks.workunit.client.1.smithi142.stdout:1/7: dread - f1 zero size 2024-06-15T01:06:46.991 INFO:tasks.workunit.client.1.smithi142.stdout:4/9: creat f2 x:0 0 0 2024-06-15T01:06:46.993 INFO:tasks.workunit.client.1.smithi142.stdout:8/0: write - no filename 2024-06-15T01:06:46.993 INFO:tasks.workunit.client.1.smithi142.stdout:6/2: creat f0 x:0 0 0 2024-06-15T01:06:46.997 INFO:tasks.workunit.client.1.smithi142.stdout:1/8: unlink f1 0 2024-06-15T01:06:46.999 INFO:tasks.workunit.client.1.smithi142.stdout:5/5: symlink l0 0 2024-06-15T01:06:46.999 INFO:tasks.workunit.client.1.smithi142.stdout:5/6: dwrite - no filename 2024-06-15T01:06:46.999 INFO:tasks.workunit.client.1.smithi142.stdout:5/7: fsync - no filename 2024-06-15T01:06:47.000 INFO:tasks.workunit.client.1.smithi142.stdout:5/8: dread - no filename 2024-06-15T01:06:47.000 INFO:tasks.workunit.client.1.smithi142.stdout:5/9: write - no filename 2024-06-15T01:06:47.000 INFO:tasks.workunit.client.1.smithi142.stdout:4/10: dwrite f2 [0,4194304] 0 2024-06-15T01:06:47.000 INFO:tasks.workunit.client.1.smithi142.stdout:6/3: mknod c1 0 2024-06-15T01:06:47.004 INFO:tasks.workunit.client.1.smithi142.stdout:5/10: rename l0 to l1 0 2024-06-15T01:06:47.004 INFO:tasks.workunit.client.1.smithi142.stdout:5/11: dread - no filename 2024-06-15T01:06:47.004 INFO:tasks.workunit.client.1.smithi142.stdout:4/11: dread f2 [0,4194304] 0 2024-06-15T01:06:47.005 INFO:tasks.workunit.client.1.smithi142.stdout:6/4: creat f2 x:0 0 0 2024-06-15T01:06:47.005 INFO:tasks.workunit.client.1.smithi142.stdout:6/5: fdatasync f2 0 2024-06-15T01:06:47.006 INFO:tasks.workunit.client.1.smithi142.stdout:6/6: read - f0 zero size 2024-06-15T01:06:47.007 INFO:tasks.workunit.client.1.smithi142.stdout:9/0: symlink l0 0 2024-06-15T01:06:47.007 INFO:tasks.workunit.client.1.smithi142.stdout:9/1: dread - no filename 2024-06-15T01:06:47.007 INFO:tasks.workunit.client.1.smithi142.stdout:9/2: write - no filename 2024-06-15T01:06:47.008 INFO:tasks.workunit.client.1.smithi142.stdout:4/12: creat f3 x:0 0 0 2024-06-15T01:06:47.009 INFO:tasks.workunit.client.1.smithi142.stdout:6/7: mknod c3 0 2024-06-15T01:06:47.010 INFO:tasks.workunit.client.1.smithi142.stdout:9/3: mknod c1 0 2024-06-15T01:06:47.011 INFO:tasks.workunit.client.1.smithi142.stdout:9/4: dread - no filename 2024-06-15T01:06:47.011 INFO:tasks.workunit.client.1.smithi142.stdout:9/5: truncate - no filename 2024-06-15T01:06:47.011 INFO:tasks.workunit.client.1.smithi142.stdout:4/13: creat f4 x:0 0 0 2024-06-15T01:06:47.012 INFO:tasks.workunit.client.1.smithi142.stdout:6/8: creat f4 x:0 0 0 2024-06-15T01:06:47.014 INFO:tasks.workunit.client.1.smithi142.stdout:9/6: unlink l0 0 2024-06-15T01:06:47.015 INFO:tasks.workunit.client.1.smithi142.stdout:4/14: mkdir d5 0 2024-06-15T01:06:47.015 INFO:tasks.workunit.client.1.smithi142.stdout:4/15: read - f3 zero size 2024-06-15T01:06:47.016 INFO:tasks.workunit.client.1.smithi142.stdout:6/9: creat f5 x:0 0 0 2024-06-15T01:06:47.016 INFO:tasks.workunit.client.1.smithi142.stdout:4/16: write f3 [658919,12414] 0 2024-06-15T01:06:47.016 INFO:tasks.workunit.client.1.smithi142.stdout:4/17: truncate f1 904943 0 2024-06-15T01:06:47.017 INFO:tasks.workunit.client.1.smithi142.stdout:9/7: creat f2 x:0 0 0 2024-06-15T01:06:47.017 INFO:tasks.workunit.client.1.smithi142.stdout:9/8: chown c1 30 1 2024-06-15T01:06:47.019 INFO:tasks.workunit.client.1.smithi142.stdout:9/9: creat f3 x:0 0 0 2024-06-15T01:06:47.046 INFO:tasks.workunit.client.1.smithi142.stdout:4/18: dwrite f3 [0,4194304] 0 2024-06-15T01:06:47.064 INFO:tasks.workunit.client.1.smithi142.stdout:9/10: mknod c4 0 2024-06-15T01:06:47.064 INFO:tasks.workunit.client.1.smithi142.stdout:9/11: fdatasync f2 0 2024-06-15T01:06:47.070 INFO:tasks.workunit.client.1.smithi142.stdout:4/19: creat d5/f6 x:0 0 0 2024-06-15T01:06:47.071 INFO:tasks.workunit.client.1.smithi142.stdout:9/12: dwrite f2 [0,4194304] 0 2024-06-15T01:06:47.072 INFO:tasks.workunit.client.1.smithi142.stdout:4/20: unlink f4 0 2024-06-15T01:06:47.072 INFO:tasks.workunit.client.1.smithi142.stdout:9/13: chown f2 36440139 1 2024-06-15T01:06:47.073 INFO:tasks.workunit.client.1.smithi142.stdout:4/21: creat d5/f7 x:0 0 0 2024-06-15T01:06:47.075 INFO:tasks.workunit.client.1.smithi142.stdout:9/14: rename f3 to f5 0 2024-06-15T01:06:47.078 INFO:tasks.workunit.client.1.smithi142.stdout:4/22: rename f1 to d5/f8 0 2024-06-15T01:06:47.080 INFO:tasks.workunit.client.1.smithi142.stdout:9/15: dread f2 [0,4194304] 0 2024-06-15T01:06:47.080 INFO:tasks.workunit.client.1.smithi142.stdout:4/23: mkdir d5/d9 0 2024-06-15T01:06:47.081 INFO:tasks.workunit.client.1.smithi142.stdout:4/24: chown f3 339 1 2024-06-15T01:06:47.082 INFO:tasks.workunit.client.1.smithi142.stdout:9/16: mkdir d6 0 2024-06-15T01:06:47.082 INFO:tasks.workunit.client.1.smithi142.stdout:9/17: chown f5 25824278 1 2024-06-15T01:06:47.084 INFO:tasks.workunit.client.1.smithi142.stdout:4/25: mkdir d5/d9/da 0 2024-06-15T01:06:47.085 INFO:tasks.workunit.client.1.smithi142.stdout:4/26: creat d5/fb x:0 0 0 2024-06-15T01:06:47.087 INFO:tasks.workunit.client.1.smithi142.stdout:4/27: rename d5/fb to d5/fc 0 2024-06-15T01:06:47.088 INFO:tasks.workunit.client.1.smithi142.stdout:9/18: link c4 d6/c7 0 2024-06-15T01:06:47.091 INFO:tasks.workunit.client.1.smithi142.stdout:4/28: link d5/f8 d5/d9/fd 0 2024-06-15T01:06:47.092 INFO:tasks.workunit.client.1.smithi142.stdout:9/19: symlink d6/l8 0 2024-06-15T01:06:47.093 INFO:tasks.workunit.client.1.smithi142.stdout:4/29: write d5/f8 [382680,4348] 0 2024-06-15T01:06:47.094 INFO:tasks.workunit.client.1.smithi142.stdout:9/20: creat d6/f9 x:0 0 0 2024-06-15T01:06:47.096 INFO:tasks.workunit.client.1.smithi142.stdout:4/30: mknod d5/d9/da/ce 0 2024-06-15T01:06:47.105 INFO:tasks.workunit.client.1.smithi142.stdout:4/31: dwrite d5/f8 [0,4194304] 0 2024-06-15T01:06:47.106 INFO:tasks.workunit.client.1.smithi142.stdout:4/32: mkdir d5/df 0 2024-06-15T01:06:47.109 INFO:tasks.workunit.client.1.smithi142.stdout:4/33: link d5/f7 d5/f10 0 2024-06-15T01:06:47.123 INFO:tasks.workunit.client.1.smithi142.stdout:4/34: dread f2 [0,4194304] 0 2024-06-15T01:06:47.171 INFO:tasks.workunit.client.1.smithi142.stdout:0/1: fsync f0 0 2024-06-15T01:06:47.183 INFO:tasks.workunit.client.1.smithi142.stdout:3/2: getdents d0 0 2024-06-15T01:06:47.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:46 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi137 to 1216M 2024-06-15T01:06:47.303 INFO:tasks.workunit.client.1.smithi142.stdout:0/2: unlink f0 0 2024-06-15T01:06:47.303 INFO:tasks.workunit.client.1.smithi142.stdout:0/3: fsync - no filename 2024-06-15T01:06:47.304 INFO:tasks.workunit.client.1.smithi142.stdout:1/9: chown f4 365 1 2024-06-15T01:06:47.304 INFO:tasks.workunit.client.1.smithi142.stdout:1/10: truncate f3 506652 0 2024-06-15T01:06:47.305 INFO:tasks.workunit.client.1.smithi142.stdout:4/35: mknod d5/df/c11 0 2024-06-15T01:06:47.305 INFO:tasks.workunit.client.1.smithi142.stdout:4/36: write d5/f6 [6233,104603] 0 2024-06-15T01:06:47.306 INFO:tasks.workunit.client.1.smithi142.stdout:4/37: truncate d5/fc 496213 0 2024-06-15T01:06:47.306 INFO:tasks.workunit.client.1.smithi142.stdout:3/3: getdents d0 0 2024-06-15T01:06:47.308 INFO:tasks.workunit.client.1.smithi142.stdout:3/4: mknod d0/c2 0 2024-06-15T01:06:47.309 INFO:tasks.workunit.client.1.smithi142.stdout:4/38: mkdir d5/df/d12 0 2024-06-15T01:06:47.310 INFO:tasks.workunit.client.1.smithi142.stdout:4/39: chown l0 60866896 1 2024-06-15T01:06:47.310 INFO:tasks.workunit.client.1.smithi142.stdout:3/5: mkdir d0/d3 0 2024-06-15T01:06:47.310 INFO:tasks.workunit.client.1.smithi142.stdout:3/6: dwrite - no filename 2024-06-15T01:06:47.310 INFO:tasks.workunit.client.1.smithi142.stdout:3/7: dread - no filename 2024-06-15T01:06:47.311 INFO:tasks.workunit.client.1.smithi142.stdout:3/8: readlink d0/l1 0 2024-06-15T01:06:47.313 INFO:tasks.workunit.client.1.smithi142.stdout:3/9: rename d0/l1 to d0/d3/l4 0 2024-06-15T01:06:47.313 INFO:tasks.workunit.client.1.smithi142.stdout:3/10: dwrite - no filename 2024-06-15T01:06:47.314 INFO:tasks.workunit.client.1.smithi142.stdout:4/40: dread d5/f8 [0,4194304] 0 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/11: symlink d0/d3/l5 0 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/12: write - no filename 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/13: dread - no filename 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/14: stat d0/d3 0 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/15: dread - no filename 2024-06-15T01:06:47.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/16: write - no filename 2024-06-15T01:06:47.316 INFO:tasks.workunit.client.1.smithi142.stdout:4/41: mknod d5/c13 0 2024-06-15T01:06:47.317 INFO:tasks.workunit.client.1.smithi142.stdout:3/17: creat d0/d3/f6 x:0 0 0 2024-06-15T01:06:47.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:46 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi137 to 1216M 2024-06-15T01:06:47.318 INFO:tasks.workunit.client.1.smithi142.stdout:3/18: read - d0/d3/f6 zero size 2024-06-15T01:06:47.318 INFO:tasks.workunit.client.1.smithi142.stdout:3/19: fdatasync d0/d3/f6 0 2024-06-15T01:06:47.320 INFO:tasks.workunit.client.1.smithi142.stdout:3/20: symlink d0/l7 0 2024-06-15T01:06:47.320 INFO:tasks.workunit.client.1.smithi142.stdout:3/21: chown d0/d3 325953841 1 2024-06-15T01:06:47.321 INFO:tasks.workunit.client.1.smithi142.stdout:3/22: truncate d0/d3/f6 670922 0 2024-06-15T01:06:47.321 INFO:tasks.workunit.client.1.smithi142.stdout:3/23: stat d0/l7 0 2024-06-15T01:06:47.322 INFO:tasks.workunit.client.1.smithi142.stdout:3/24: stat d0/c2 0 2024-06-15T01:06:47.323 INFO:tasks.workunit.client.1.smithi142.stdout:3/25: symlink d0/d3/l8 0 2024-06-15T01:06:47.330 INFO:tasks.workunit.client.1.smithi142.stdout:3/26: dwrite d0/d3/f6 [0,4194304] 0 2024-06-15T01:06:47.332 INFO:tasks.workunit.client.1.smithi142.stdout:3/27: unlink d0/l7 0 2024-06-15T01:06:47.334 INFO:tasks.workunit.client.1.smithi142.stdout:3/28: creat d0/f9 x:0 0 0 2024-06-15T01:06:47.353 INFO:tasks.workunit.client.1.smithi142.stdout:1/11: write f4 [155008,52567] 0 2024-06-15T01:06:47.355 INFO:tasks.workunit.client.1.smithi142.stdout:1/12: rename f4 to f5 0 2024-06-15T01:06:47.357 INFO:tasks.workunit.client.1.smithi142.stdout:1/13: dread f5 [0,4194304] 0 2024-06-15T01:06:47.359 INFO:tasks.workunit.client.1.smithi142.stdout:1/14: mkdir d6 0 2024-06-15T01:06:47.363 INFO:tasks.workunit.client.1.smithi142.stdout:1/15: write f3 [1546378,107150] 0 2024-06-15T01:06:47.368 INFO:tasks.workunit.client.1.smithi142.stdout:4/42: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:06:47.370 INFO:tasks.workunit.client.1.smithi142.stdout:1/16: read f3 [1165875,95467] 0 2024-06-15T01:06:47.370 INFO:tasks.workunit.client.1.smithi142.stdout:3/29: dwrite d0/d3/f6 [4194304,4194304] 0 2024-06-15T01:06:47.375 INFO:tasks.workunit.client.1.smithi142.stdout:4/43: write d5/f7 [4563730,113125] 0 2024-06-15T01:06:47.375 INFO:tasks.workunit.client.1.smithi142.stdout:3/30: mkdir d0/d3/da 0 2024-06-15T01:06:47.379 INFO:tasks.workunit.client.1.smithi142.stdout:3/31: write d0/d3/f6 [3304847,33818] 0 2024-06-15T01:06:47.379 INFO:tasks.workunit.client.1.smithi142.stdout:1/17: mknod d6/c7 0 2024-06-15T01:06:47.380 INFO:tasks.workunit.client.1.smithi142.stdout:4/44: dread d5/f6 [0,4194304] 0 2024-06-15T01:06:47.383 INFO:tasks.workunit.client.1.smithi142.stdout:1/18: read f3 [552900,21174] 0 2024-06-15T01:06:47.383 INFO:tasks.workunit.client.1.smithi142.stdout:3/32: rename d0/c2 to d0/d3/da/cb 0 2024-06-15T01:06:47.387 INFO:tasks.workunit.client.1.smithi142.stdout:3/33: creat d0/d3/da/fc x:0 0 0 2024-06-15T01:06:47.390 INFO:tasks.workunit.client.1.smithi142.stdout:3/34: truncate d0/f9 392042 0 2024-06-15T01:06:47.390 INFO:tasks.workunit.client.1.smithi142.stdout:4/45: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:06:47.397 INFO:tasks.workunit.client.1.smithi142.stdout:1/19: dwrite f3 [0,4194304] 0 2024-06-15T01:06:47.401 INFO:tasks.workunit.client.1.smithi142.stdout:1/20: link f3 d6/f8 0 2024-06-15T01:06:47.403 INFO:tasks.workunit.client.1.smithi142.stdout:3/35: getdents d0/d3/da 0 2024-06-15T01:06:47.404 INFO:tasks.workunit.client.1.smithi142.stdout:1/21: dread f3 [0,4194304] 0 2024-06-15T01:06:47.405 INFO:tasks.workunit.client.1.smithi142.stdout:1/22: chown f3 26 1 2024-06-15T01:06:47.405 INFO:tasks.workunit.client.1.smithi142.stdout:1/23: chown d6 40238 1 2024-06-15T01:06:47.469 INFO:tasks.workunit.client.1.smithi142.stdout:3/36: mknod d0/d3/da/cd 0 2024-06-15T01:06:47.472 INFO:tasks.workunit.client.1.smithi142.stdout:4/46: truncate d5/f10 723335 0 2024-06-15T01:06:47.476 INFO:tasks.workunit.client.1.smithi142.stdout:4/47: symlink d5/df/d12/l14 0 2024-06-15T01:06:47.478 INFO:tasks.workunit.client.1.smithi142.stdout:4/48: mkdir d5/df/d15 0 2024-06-15T01:06:47.478 INFO:tasks.workunit.client.1.smithi142.stdout:4/49: write f3 [3575946,59186] 0 2024-06-15T01:06:47.479 INFO:tasks.workunit.client.1.smithi142.stdout:4/50: chown d5/df 91372 1 2024-06-15T01:06:47.480 INFO:tasks.workunit.client.1.smithi142.stdout:4/51: fsync d5/d9/fd 0 2024-06-15T01:06:47.480 INFO:tasks.workunit.client.1.smithi142.stdout:4/52: stat d5/f10 0 2024-06-15T01:06:47.481 INFO:tasks.workunit.client.1.smithi142.stdout:4/53: mknod d5/d9/c16 0 2024-06-15T01:06:47.483 INFO:tasks.workunit.client.1.smithi142.stdout:4/54: creat d5/df/d15/f17 x:0 0 0 2024-06-15T01:06:47.486 INFO:tasks.workunit.client.1.smithi142.stdout:1/24: getdents d6 0 2024-06-15T01:06:47.487 INFO:tasks.workunit.client.1.smithi142.stdout:3/37: rmdir d0/d3 39 2024-06-15T01:06:47.488 INFO:tasks.workunit.client.1.smithi142.stdout:1/25: creat d6/f9 x:0 0 0 2024-06-15T01:06:47.490 INFO:tasks.workunit.client.1.smithi142.stdout:1/26: write d6/f9 [113554,39416] 0 2024-06-15T01:06:47.491 INFO:tasks.workunit.client.1.smithi142.stdout:3/38: dread - d0/d3/da/fc zero size 2024-06-15T01:06:47.492 INFO:tasks.workunit.client.1.smithi142.stdout:1/27: mknod d6/ca 0 2024-06-15T01:06:47.495 INFO:tasks.workunit.client.1.smithi142.stdout:1/28: mknod d6/cb 0 2024-06-15T01:06:47.495 INFO:tasks.workunit.client.1.smithi142.stdout:4/55: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:06:47.496 INFO:tasks.workunit.client.1.smithi142.stdout:3/39: rename d0/f9 to d0/d3/da/fe 0 2024-06-15T01:06:47.497 INFO:tasks.workunit.client.1.smithi142.stdout:4/56: mkdir d5/d9/d18 0 2024-06-15T01:06:47.499 INFO:tasks.workunit.client.1.smithi142.stdout:4/57: write d5/f8 [770482,100364] 0 2024-06-15T01:06:47.501 INFO:tasks.workunit.client.1.smithi142.stdout:4/58: fdatasync f2 0 2024-06-15T01:06:47.501 INFO:tasks.workunit.client.1.smithi142.stdout:3/40: dread d0/d3/f6 [4194304,4194304] 0 2024-06-15T01:06:47.502 INFO:tasks.workunit.client.1.smithi142.stdout:4/59: rmdir d5/df 39 2024-06-15T01:06:47.505 INFO:tasks.workunit.client.1.smithi142.stdout:4/60: unlink d5/fc 0 2024-06-15T01:06:47.506 INFO:tasks.workunit.client.1.smithi142.stdout:3/41: mknod d0/cf 0 2024-06-15T01:06:47.508 INFO:tasks.workunit.client.1.smithi142.stdout:3/42: write d0/d3/da/fe [1116172,90413] 0 2024-06-15T01:06:47.508 INFO:tasks.workunit.client.1.smithi142.stdout:4/61: symlink d5/df/d12/l19 0 2024-06-15T01:06:47.509 INFO:tasks.workunit.client.1.smithi142.stdout:1/29: truncate f3 74755 0 2024-06-15T01:06:47.510 INFO:tasks.workunit.client.1.smithi142.stdout:3/43: mknod d0/d3/da/c10 0 2024-06-15T01:06:47.512 INFO:tasks.workunit.client.1.smithi142.stdout:3/44: creat d0/d3/da/f11 x:0 0 0 2024-06-15T01:06:47.514 INFO:tasks.workunit.client.1.smithi142.stdout:4/62: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:06:47.514 INFO:tasks.workunit.client.1.smithi142.stdout:3/45: creat d0/d3/f12 x:0 0 0 2024-06-15T01:06:47.516 INFO:tasks.workunit.client.1.smithi142.stdout:3/46: truncate d0/d3/da/fc 175205 0 2024-06-15T01:06:47.516 INFO:tasks.workunit.client.1.smithi142.stdout:4/63: mkdir d5/d9/d1a 0 2024-06-15T01:06:47.518 INFO:tasks.workunit.client.1.smithi142.stdout:3/47: unlink d0/d3/f6 0 2024-06-15T01:06:47.519 INFO:tasks.workunit.client.1.smithi142.stdout:3/48: rename d0 to d0/d3/d13 22 2024-06-15T01:06:47.522 INFO:tasks.workunit.client.1.smithi142.stdout:4/64: dread d5/f10 [0,4194304] 0 2024-06-15T01:06:47.526 INFO:tasks.workunit.client.1.smithi142.stdout:4/65: truncate d5/f10 2536650 0 2024-06-15T01:06:47.527 INFO:tasks.workunit.client.1.smithi142.stdout:4/66: mknod d5/d9/d1a/c1b 0 2024-06-15T01:06:47.529 INFO:tasks.workunit.client.1.smithi142.stdout:4/67: mkdir d5/d1c 0 2024-06-15T01:06:47.530 INFO:tasks.workunit.client.1.smithi142.stdout:1/30: link f3 d6/fc 0 2024-06-15T01:06:47.530 INFO:tasks.workunit.client.1.smithi142.stdout:4/68: chown d5/d9/c16 265616928 1 2024-06-15T01:06:47.532 INFO:tasks.workunit.client.1.smithi142.stdout:4/69: rename d5/d9/da/ce to d5/d9/d18/c1d 0 2024-06-15T01:06:47.534 INFO:tasks.workunit.client.1.smithi142.stdout:4/70: symlink d5/df/d12/l1e 0 2024-06-15T01:06:47.538 INFO:tasks.workunit.client.1.smithi142.stdout:4/71: dread d5/d9/fd [0,4194304] 0 2024-06-15T01:06:47.540 INFO:tasks.workunit.client.1.smithi142.stdout:4/72: creat d5/d9/f1f x:0 0 0 2024-06-15T01:06:47.541 INFO:tasks.workunit.client.1.smithi142.stdout:1/31: dwrite f3 [0,4194304] 0 2024-06-15T01:06:47.542 INFO:tasks.workunit.client.1.smithi142.stdout:4/73: mkdir d5/df/d20 0 2024-06-15T01:06:47.544 INFO:tasks.workunit.client.1.smithi142.stdout:1/32: rename f3 to d6/fd 0 2024-06-15T01:06:47.545 INFO:tasks.workunit.client.1.smithi142.stdout:4/74: fsync d5/d9/fd 0 2024-06-15T01:06:47.546 INFO:tasks.workunit.client.1.smithi142.stdout:4/75: read d5/f10 [2172350,71375] 0 2024-06-15T01:06:47.547 INFO:tasks.workunit.client.1.smithi142.stdout:1/33: rename d6/c7 to d6/ce 0 2024-06-15T01:06:47.549 INFO:tasks.workunit.client.1.smithi142.stdout:4/76: rename d5/df/d15/f17 to d5/df/d15/f21 0 2024-06-15T01:06:47.551 INFO:tasks.workunit.client.1.smithi142.stdout:4/77: creat d5/d9/f22 x:0 0 0 2024-06-15T01:06:47.551 INFO:tasks.workunit.client.1.smithi142.stdout:1/34: truncate d6/fd 769346 0 2024-06-15T01:06:47.557 INFO:tasks.workunit.client.1.smithi142.stdout:4/78: truncate d5/f10 431713 0 2024-06-15T01:06:47.558 INFO:tasks.workunit.client.1.smithi142.stdout:1/35: rename d6/f8 to d6/ff 0 2024-06-15T01:06:47.561 INFO:tasks.workunit.client.1.smithi142.stdout:4/79: mkdir d5/d9/d23 0 2024-06-15T01:06:47.561 INFO:tasks.workunit.client.1.smithi142.stdout:4/80: chown d5/d9/da 360554757 1 2024-06-15T01:06:47.562 INFO:tasks.workunit.client.1.smithi142.stdout:1/36: rename d6/fd to d6/f10 0 2024-06-15T01:06:47.562 INFO:tasks.workunit.client.1.smithi142.stdout:1/37: symlink d6/l11 0 2024-06-15T01:06:47.563 INFO:tasks.workunit.client.1.smithi142.stdout:1/38: read d6/f9 [40037,113116] 0 2024-06-15T01:06:47.564 INFO:tasks.workunit.client.1.smithi142.stdout:4/81: dread d5/f10 [0,4194304] 0 2024-06-15T01:06:47.565 INFO:tasks.workunit.client.1.smithi142.stdout:4/82: truncate f2 5024552 0 2024-06-15T01:06:47.565 INFO:tasks.workunit.client.1.smithi142.stdout:1/39: dread d6/f9 [0,4194304] 0 2024-06-15T01:06:47.567 INFO:tasks.workunit.client.1.smithi142.stdout:1/40: mkdir d6/d12 0 2024-06-15T01:06:47.571 INFO:tasks.workunit.client.1.smithi142.stdout:1/41: getdents d6 0 2024-06-15T01:06:47.572 INFO:tasks.workunit.client.1.smithi142.stdout:4/83: truncate f2 4959645 0 2024-06-15T01:06:47.573 INFO:tasks.workunit.client.1.smithi142.stdout:1/42: creat d6/f13 x:0 0 0 2024-06-15T01:06:47.573 INFO:tasks.workunit.client.1.smithi142.stdout:1/43: read d6/f10 [632290,113701] 0 2024-06-15T01:06:47.573 INFO:tasks.workunit.client.1.smithi142.stdout:4/84: creat d5/df/d20/f24 x:0 0 0 2024-06-15T01:06:47.576 INFO:tasks.workunit.client.1.smithi142.stdout:1/44: rename d6/ca to d6/c14 0 2024-06-15T01:06:47.577 INFO:tasks.workunit.client.1.smithi142.stdout:4/85: dread f3 [0,4194304] 0 2024-06-15T01:06:47.578 INFO:tasks.workunit.client.1.smithi142.stdout:4/86: read d5/f8 [521315,52460] 0 2024-06-15T01:06:47.579 INFO:tasks.workunit.client.1.smithi142.stdout:1/45: rename d6/ce to d6/d12/c15 0 2024-06-15T01:06:47.579 INFO:tasks.workunit.client.1.smithi142.stdout:1/46: readlink d6/l11 0 2024-06-15T01:06:47.579 INFO:tasks.workunit.client.1.smithi142.stdout:1/47: stat d6/ff 0 2024-06-15T01:06:47.580 INFO:tasks.workunit.client.1.smithi142.stdout:1/48: fdatasync d6/f13 0 2024-06-15T01:06:47.580 INFO:tasks.workunit.client.1.smithi142.stdout:1/49: stat d6/f13 0 2024-06-15T01:06:47.580 INFO:tasks.workunit.client.1.smithi142.stdout:1/50: dread - d6/f13 zero size 2024-06-15T01:06:47.580 INFO:tasks.workunit.client.1.smithi142.stdout:4/87: rmdir d5/d9 39 2024-06-15T01:06:47.582 INFO:tasks.workunit.client.1.smithi142.stdout:4/88: fdatasync d5/d9/f22 0 2024-06-15T01:06:47.582 INFO:tasks.workunit.client.1.smithi142.stdout:1/51: mkdir d6/d12/d16 0 2024-06-15T01:06:47.583 INFO:tasks.workunit.client.1.smithi142.stdout:4/89: dread d5/f6 [0,4194304] 0 2024-06-15T01:06:47.583 INFO:tasks.workunit.client.1.smithi142.stdout:1/52: read - d6/f13 zero size 2024-06-15T01:06:47.584 INFO:tasks.workunit.client.1.smithi142.stdout:4/90: chown d5/df/d15/f21 10982 1 2024-06-15T01:06:47.584 INFO:tasks.workunit.client.1.smithi142.stdout:1/53: read d6/ff [423347,52717] 0 2024-06-15T01:06:47.586 INFO:tasks.workunit.client.1.smithi142.stdout:4/91: link d5/df/d12/l1e d5/d9/d18/l25 0 2024-06-15T01:06:47.588 INFO:tasks.workunit.client.1.smithi142.stdout:4/92: write d5/f6 [330330,106556] 0 2024-06-15T01:06:47.591 INFO:tasks.workunit.client.1.smithi142.stdout:1/54: dwrite d6/f9 [0,4194304] 0 2024-06-15T01:06:47.592 INFO:tasks.workunit.client.1.smithi142.stdout:1/55: chown d6/f10 1365569 1 2024-06-15T01:06:47.595 INFO:tasks.workunit.client.1.smithi142.stdout:4/93: dwrite d5/f6 [0,4194304] 0 2024-06-15T01:06:47.596 INFO:tasks.workunit.client.1.smithi142.stdout:1/56: rmdir d6/d12/d16 0 2024-06-15T01:06:47.597 INFO:tasks.workunit.client.1.smithi142.stdout:4/94: symlink d5/d9/l26 0 2024-06-15T01:06:47.598 INFO:tasks.workunit.client.1.smithi142.stdout:4/95: dread - d5/df/d15/f21 zero size 2024-06-15T01:06:47.600 INFO:tasks.workunit.client.1.smithi142.stdout:4/96: fsync d5/f10 0 2024-06-15T01:06:47.604 INFO:tasks.workunit.client.1.smithi142.stdout:4/97: symlink d5/d1c/l27 0 2024-06-15T01:06:47.606 INFO:tasks.workunit.client.1.smithi142.stdout:4/98: fsync d5/df/d15/f21 0 2024-06-15T01:06:47.607 INFO:tasks.workunit.client.1.smithi142.stdout:4/99: truncate f3 4959519 0 2024-06-15T01:06:47.610 INFO:tasks.workunit.client.1.smithi142.stdout:4/100: symlink d5/d9/d18/l28 0 2024-06-15T01:06:47.610 INFO:tasks.workunit.client.1.smithi142.stdout:1/57: dwrite d6/fc [0,4194304] 0 2024-06-15T01:06:47.623 INFO:tasks.workunit.client.1.smithi142.stdout:1/58: dwrite f5 [0,4194304] 0 2024-06-15T01:06:47.623 INFO:tasks.workunit.client.1.smithi142.stdout:4/101: dwrite d5/f7 [0,4194304] 0 2024-06-15T01:06:47.625 INFO:tasks.workunit.client.1.smithi142.stdout:1/59: mknod d6/c17 0 2024-06-15T01:06:47.629 INFO:tasks.workunit.client.1.smithi142.stdout:4/102: getdents d5/df/d15 0 2024-06-15T01:06:47.633 INFO:tasks.workunit.client.1.smithi142.stdout:4/103: creat d5/d9/d23/f29 x:0 0 0 2024-06-15T01:06:47.636 INFO:tasks.workunit.client.1.smithi142.stdout:4/104: mkdir d5/d1c/d2a 0 2024-06-15T01:06:47.637 INFO:tasks.workunit.client.1.smithi142.stdout:1/60: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:06:47.638 INFO:tasks.workunit.client.1.smithi142.stdout:4/105: write d5/df/d15/f21 [988935,103917] 0 2024-06-15T01:06:47.640 INFO:tasks.workunit.client.1.smithi142.stdout:4/106: symlink d5/d1c/d2a/l2b 0 2024-06-15T01:06:47.640 INFO:tasks.workunit.client.1.smithi142.stdout:4/107: dread - d5/d9/f22 zero size 2024-06-15T01:06:47.641 INFO:tasks.workunit.client.1.smithi142.stdout:4/108: readlink d5/d9/l26 0 2024-06-15T01:06:47.641 INFO:tasks.workunit.client.1.smithi142.stdout:4/109: chown d5/d9/d23 1009318 1 2024-06-15T01:06:47.642 INFO:tasks.workunit.client.1.smithi142.stdout:4/110: rmdir d5/df 39 2024-06-15T01:06:47.643 INFO:tasks.workunit.client.1.smithi142.stdout:4/111: truncate d5/d9/d23/f29 23950 0 2024-06-15T01:06:47.653 INFO:tasks.workunit.client.1.smithi142.stdout:4/112: dwrite d5/df/d20/f24 [0,4194304] 0 2024-06-15T01:06:47.654 INFO:tasks.workunit.client.1.smithi142.stdout:4/113: write d5/df/d20/f24 [3675654,58476] 0 2024-06-15T01:06:47.654 INFO:tasks.workunit.client.1.smithi142.stdout:4/114: fsync d5/d9/f22 0 2024-06-15T01:06:47.655 INFO:tasks.workunit.client.1.smithi142.stdout:4/115: read - d5/d9/f22 zero size 2024-06-15T01:06:47.656 INFO:tasks.workunit.client.1.smithi142.stdout:4/116: fdatasync f3 0 2024-06-15T01:06:47.657 INFO:tasks.workunit.client.1.smithi142.stdout:4/117: symlink d5/d9/d23/l2c 0 2024-06-15T01:06:47.671 INFO:tasks.workunit.client.1.smithi142.stdout:4/118: fdatasync d5/df/d20/f24 0 2024-06-15T01:06:47.672 INFO:tasks.workunit.client.1.smithi142.stdout:4/119: rmdir d5/d9/d1a 39 2024-06-15T01:06:47.672 INFO:tasks.workunit.client.1.smithi142.stdout:4/120: dread - d5/d9/f22 zero size 2024-06-15T01:06:47.676 INFO:tasks.workunit.client.1.smithi142.stdout:4/121: dread d5/f7 [0,4194304] 0 2024-06-15T01:06:47.677 INFO:tasks.workunit.client.1.smithi142.stdout:4/122: rmdir d5/d1c 39 2024-06-15T01:06:47.679 INFO:tasks.workunit.client.1.smithi142.stdout:4/123: mkdir d5/df/d2d 0 2024-06-15T01:06:47.680 INFO:tasks.workunit.client.1.smithi142.stdout:4/124: mkdir d5/d9/da/d2e 0 2024-06-15T01:06:47.682 INFO:tasks.workunit.client.1.smithi142.stdout:4/125: creat d5/d9/d18/f2f x:0 0 0 2024-06-15T01:06:47.685 INFO:tasks.workunit.client.1.smithi142.stdout:4/126: mkdir d5/d1c/d2a/d30 0 2024-06-15T01:06:47.691 INFO:tasks.workunit.client.1.smithi142.stdout:4/127: dwrite d5/d9/d18/f2f [0,4194304] 0 2024-06-15T01:06:47.694 INFO:tasks.workunit.client.1.smithi142.stdout:4/128: fdatasync f2 0 2024-06-15T01:06:47.697 INFO:tasks.workunit.client.1.smithi142.stdout:4/129: symlink d5/d9/da/l31 0 2024-06-15T01:06:47.697 INFO:tasks.workunit.client.1.smithi142.stdout:4/130: write d5/d9/f1f [148115,39751] 0 2024-06-15T01:06:47.698 INFO:tasks.workunit.client.1.smithi142.stdout:4/131: chown d5/d9/f1f 2 1 2024-06-15T01:06:47.705 INFO:tasks.workunit.client.1.smithi142.stdout:4/132: truncate f2 4879692 0 2024-06-15T01:06:47.705 INFO:tasks.workunit.client.1.smithi142.stdout:4/133: read d5/f7 [848614,67843] 0 2024-06-15T01:06:47.706 INFO:tasks.workunit.client.1.smithi142.stdout:4/134: fsync d5/d9/f22 0 2024-06-15T01:06:47.708 INFO:tasks.workunit.client.1.smithi142.stdout:4/135: mkdir d5/d9/d23/d32 0 2024-06-15T01:06:47.709 INFO:tasks.workunit.client.1.smithi142.stdout:4/136: write d5/f8 [1655916,4523] 0 2024-06-15T01:06:47.711 INFO:tasks.workunit.client.1.smithi142.stdout:4/137: symlink d5/d9/d18/l33 0 2024-06-15T01:06:47.714 INFO:tasks.workunit.client.1.smithi142.stdout:4/138: dread d5/f7 [0,4194304] 0 2024-06-15T01:06:47.715 INFO:tasks.workunit.client.1.smithi142.stdout:4/139: read d5/f6 [2785737,68642] 0 2024-06-15T01:06:47.724 INFO:tasks.workunit.client.1.smithi142.stdout:4/140: dread f2 [0,4194304] 0 2024-06-15T01:06:47.727 INFO:tasks.workunit.client.1.smithi142.stdout:4/141: dread d5/d9/fd [0,4194304] 0 2024-06-15T01:06:47.729 INFO:tasks.workunit.client.1.smithi142.stdout:4/142: rename d5/df/d20/f24 to d5/d1c/f34 0 2024-06-15T01:06:47.732 INFO:tasks.workunit.client.1.smithi142.stdout:4/143: dread f2 [0,4194304] 0 2024-06-15T01:06:47.735 INFO:tasks.workunit.client.1.smithi142.stdout:4/144: dread d5/d1c/f34 [0,4194304] 0 2024-06-15T01:06:47.737 INFO:tasks.workunit.client.1.smithi142.stdout:4/145: creat d5/df/d2d/f35 x:0 0 0 2024-06-15T01:06:47.737 INFO:tasks.workunit.client.1.smithi142.stdout:4/146: write d5/d9/d18/f2f [1220363,55214] 0 2024-06-15T01:06:47.739 INFO:tasks.workunit.client.1.smithi142.stdout:4/147: write d5/f7 [907165,62195] 0 2024-06-15T01:06:47.743 INFO:tasks.workunit.client.1.smithi142.stdout:4/148: link f3 d5/d9/da/d2e/f36 0 2024-06-15T01:06:47.746 INFO:tasks.workunit.client.1.smithi142.stdout:4/149: truncate f3 2639870 0 2024-06-15T01:06:47.747 INFO:tasks.workunit.client.1.smithi142.stdout:4/150: creat d5/df/d12/f37 x:0 0 0 2024-06-15T01:06:47.750 INFO:tasks.workunit.client.1.smithi142.stdout:4/151: link d5/df/c11 d5/d9/da/c38 0 2024-06-15T01:06:47.750 INFO:tasks.workunit.client.1.smithi142.stdout:4/152: chown d5/d1c/l27 3494 1 2024-06-15T01:06:48.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:47 smithi137 ceph-mon[29354]: pgmap v9: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 379 op/s 2024-06-15T01:06:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:47 smithi142 ceph-mon[37379]: pgmap v9: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 379 op/s 2024-06-15T01:06:48.505 INFO:tasks.workunit.client.0.smithi137.stderr:++ readlink -f fsstress 2024-06-15T01:06:48.508 INFO:tasks.workunit.client.0.smithi137.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-06-15T01:06:48.508 INFO:tasks.workunit.client.0.smithi137.stderr:+ popd 2024-06-15T01:06:48.509 INFO:tasks.workunit.client.0.smithi137.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-15T01:06:48.510 INFO:tasks.workunit.client.0.smithi137.stderr:+ popd 2024-06-15T01:06:48.511 INFO:tasks.workunit.client.0.smithi137.stdout:~/cephtest/mnt.0/client.0/tmp 2024-06-15T01:06:48.511 INFO:tasks.workunit.client.0.smithi137.stderr:++ mktemp -d -p . 2024-06-15T01:06:48.515 INFO:tasks.workunit.client.0.smithi137.stderr:+ T=./tmp.5akVWFJBP9 2024-06-15T01:06:48.515 INFO:tasks.workunit.client.0.smithi137.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.5akVWFJBP9 -l 1 -n 1000 -p 10 -v 2024-06-15T01:06:48.519 INFO:tasks.workunit.client.0.smithi137.stdout:seed = 1718294163 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/0: link - no file 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/1: dread - no filename 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/2: write - no filename 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/3: readlink - no filename 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/4: dread - no filename 2024-06-15T01:06:48.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/5: truncate - no filename 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/0: readlink - no filename 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/1: write - no filename 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:0/6: getdents . 0 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:0/7: dread - no filename 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/2: chown . 10350 1 2024-06-15T01:06:48.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/3: link - no file 2024-06-15T01:06:48.527 INFO:tasks.workunit.client.0.smithi137.stdout:2/0: dread - no filename 2024-06-15T01:06:48.528 INFO:tasks.workunit.client.0.smithi137.stdout:1/4: mkdir d0 0 2024-06-15T01:06:48.529 INFO:tasks.workunit.client.0.smithi137.stdout:0/8: symlink l0 0 2024-06-15T01:06:48.529 INFO:tasks.workunit.client.0.smithi137.stdout:0/9: rmdir - no directory 2024-06-15T01:06:48.529 INFO:tasks.workunit.client.0.smithi137.stdout:0/10: write - no filename 2024-06-15T01:06:48.529 INFO:tasks.workunit.client.0.smithi137.stdout:0/11: dread - no filename 2024-06-15T01:06:48.531 INFO:tasks.workunit.client.0.smithi137.stdout:3/0: dread - no filename 2024-06-15T01:06:48.532 INFO:tasks.workunit.client.0.smithi137.stdout:0/12: mkdir d1 0 2024-06-15T01:06:48.533 INFO:tasks.workunit.client.0.smithi137.stdout:2/1: getdents . 0 2024-06-15T01:06:48.533 INFO:tasks.workunit.client.0.smithi137.stdout:1/5: mkdir d0/d1 0 2024-06-15T01:06:48.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/6: dwrite - no filename 2024-06-15T01:06:48.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/7: write - no filename 2024-06-15T01:06:48.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/8: truncate - no filename 2024-06-15T01:06:48.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/9: truncate - no filename 2024-06-15T01:06:48.536 INFO:tasks.workunit.client.0.smithi137.stdout:1/10: creat d0/f2 x:0 0 0 2024-06-15T01:06:48.537 INFO:tasks.workunit.client.0.smithi137.stdout:0/13: mkdir d1/d2 0 2024-06-15T01:06:48.537 INFO:tasks.workunit.client.0.smithi137.stdout:0/14: write - no filename 2024-06-15T01:06:48.537 INFO:tasks.workunit.client.0.smithi137.stdout:0/15: truncate - no filename 2024-06-15T01:06:48.538 INFO:tasks.workunit.client.0.smithi137.stdout:5/0: write - no filename 2024-06-15T01:06:48.539 INFO:tasks.workunit.client.0.smithi137.stdout:2/2: mkdir d0 0 2024-06-15T01:06:48.540 INFO:tasks.workunit.client.0.smithi137.stdout:1/11: mkdir d0/d3 0 2024-06-15T01:06:48.541 INFO:tasks.workunit.client.0.smithi137.stdout:1/12: write d0/f2 [528765,121851] 0 2024-06-15T01:06:48.542 INFO:tasks.workunit.client.0.smithi137.stdout:1/13: write d0/f2 [1025253,48769] 0 2024-06-15T01:06:48.544 INFO:tasks.workunit.client.0.smithi137.stdout:4/0: mknod c0 0 2024-06-15T01:06:48.546 INFO:tasks.workunit.client.0.smithi137.stdout:5/1: creat f0 x:0 0 0 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/0: dwrite - no filename 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/1: truncate - no filename 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/2: dread - no filename 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/3: link - no file 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/4: stat - no entries 2024-06-15T01:06:48.548 INFO:tasks.workunit.client.0.smithi137.stdout:1/14: symlink d0/l4 0 2024-06-15T01:06:48.549 INFO:tasks.workunit.client.0.smithi137.stdout:2/3: creat d0/f1 x:0 0 0 2024-06-15T01:06:48.549 INFO:tasks.workunit.client.0.smithi137.stdout:2/4: fdatasync d0/f1 0 2024-06-15T01:06:48.549 INFO:tasks.workunit.client.0.smithi137.stdout:4/1: mknod c1 0 2024-06-15T01:06:48.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/2: write - no filename 2024-06-15T01:06:48.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/3: stat c0 0 2024-06-15T01:06:48.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/4: read - no filename 2024-06-15T01:06:48.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/5: write - no filename 2024-06-15T01:06:48.550 INFO:tasks.workunit.client.0.smithi137.stdout:0/16: link l0 d1/d2/l3 0 2024-06-15T01:06:48.551 INFO:tasks.workunit.client.0.smithi137.stdout:5/2: creat f1 x:0 0 0 2024-06-15T01:06:48.553 INFO:tasks.workunit.client.0.smithi137.stdout:8/0: rename - no filename 2024-06-15T01:06:48.554 INFO:tasks.workunit.client.0.smithi137.stdout:2/5: mknod d0/c2 0 2024-06-15T01:06:48.554 INFO:tasks.workunit.client.0.smithi137.stdout:1/15: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:48.554 INFO:tasks.workunit.client.0.smithi137.stdout:4/6: mknod c2 0 2024-06-15T01:06:48.556 INFO:tasks.workunit.client.0.smithi137.stdout:5/3: unlink f0 0 2024-06-15T01:06:48.556 INFO:tasks.workunit.client.0.smithi137.stdout:7/5: mknod c0 0 2024-06-15T01:06:48.556 INFO:tasks.workunit.client.0.smithi137.stdout:7/6: dwrite - no filename 2024-06-15T01:06:48.556 INFO:tasks.workunit.client.0.smithi137.stdout:7/7: fsync - no filename 2024-06-15T01:06:48.560 INFO:tasks.workunit.client.0.smithi137.stdout:9/0: unlink - no file 2024-06-15T01:06:48.560 INFO:tasks.workunit.client.0.smithi137.stdout:9/1: write - no filename 2024-06-15T01:06:48.560 INFO:tasks.workunit.client.0.smithi137.stdout:2/6: creat d0/f3 x:0 0 0 2024-06-15T01:06:48.560 INFO:tasks.workunit.client.0.smithi137.stdout:4/7: symlink l3 0 2024-06-15T01:06:48.561 INFO:tasks.workunit.client.0.smithi137.stdout:0/17: creat d1/d2/f4 x:0 0 0 2024-06-15T01:06:48.562 INFO:tasks.workunit.client.0.smithi137.stdout:1/16: write d0/f2 [1364724,50892] 0 2024-06-15T01:06:48.563 INFO:tasks.workunit.client.0.smithi137.stdout:8/1: creat f0 x:0 0 0 2024-06-15T01:06:48.563 INFO:tasks.workunit.client.0.smithi137.stdout:2/7: creat d0/f4 x:0 0 0 2024-06-15T01:06:48.565 INFO:tasks.workunit.client.0.smithi137.stdout:4/8: symlink l4 0 2024-06-15T01:06:48.565 INFO:tasks.workunit.client.0.smithi137.stdout:4/9: dread - no filename 2024-06-15T01:06:48.565 INFO:tasks.workunit.client.0.smithi137.stdout:4/10: rmdir - no directory 2024-06-15T01:06:48.567 INFO:tasks.workunit.client.0.smithi137.stdout:1/17: write d0/f2 [1662229,55291] 0 2024-06-15T01:06:48.567 INFO:tasks.workunit.client.0.smithi137.stdout:8/2: chown f0 5755478 1 2024-06-15T01:06:48.567 INFO:tasks.workunit.client.0.smithi137.stdout:4/11: chown c2 21080352 1 2024-06-15T01:06:48.567 INFO:tasks.workunit.client.0.smithi137.stdout:2/8: chown d0/f4 0 1 2024-06-15T01:06:48.570 INFO:tasks.workunit.client.0.smithi137.stdout:8/3: dread - f0 zero size 2024-06-15T01:06:48.570 INFO:tasks.workunit.client.0.smithi137.stdout:9/2: creat f0 x:0 0 0 2024-06-15T01:06:48.571 INFO:tasks.workunit.client.0.smithi137.stdout:9/3: dread - f0 zero size 2024-06-15T01:06:48.572 INFO:tasks.workunit.client.0.smithi137.stdout:4/12: creat f5 x:0 0 0 2024-06-15T01:06:48.576 INFO:tasks.workunit.client.0.smithi137.stdout:8/4: creat f1 x:0 0 0 2024-06-15T01:06:48.576 INFO:tasks.workunit.client.0.smithi137.stdout:5/4: dwrite f1 [0,4194304] 0 2024-06-15T01:06:48.577 INFO:tasks.workunit.client.0.smithi137.stdout:4/13: truncate f5 344786 0 2024-06-15T01:06:48.577 INFO:tasks.workunit.client.0.smithi137.stdout:2/9: creat d0/f5 x:0 0 0 2024-06-15T01:06:48.580 INFO:tasks.workunit.client.0.smithi137.stdout:9/4: creat f1 x:0 0 0 2024-06-15T01:06:48.580 INFO:tasks.workunit.client.0.smithi137.stdout:2/10: dread - d0/f3 zero size 2024-06-15T01:06:48.582 INFO:tasks.workunit.client.0.smithi137.stdout:9/5: read - f0 zero size 2024-06-15T01:06:48.583 INFO:tasks.workunit.client.0.smithi137.stdout:2/11: read - d0/f4 zero size 2024-06-15T01:06:48.585 INFO:tasks.workunit.client.0.smithi137.stdout:9/6: dread - f0 zero size 2024-06-15T01:06:48.587 INFO:tasks.workunit.client.0.smithi137.stdout:0/18: dwrite d1/d2/f4 [0,4194304] 0 2024-06-15T01:06:48.590 INFO:tasks.workunit.client.0.smithi137.stdout:2/12: chown d0/c2 8 1 2024-06-15T01:06:48.590 INFO:tasks.workunit.client.0.smithi137.stdout:1/18: link d0/f2 d0/d3/f5 0 2024-06-15T01:06:48.590 INFO:tasks.workunit.client.0.smithi137.stdout:8/5: mknod c2 0 2024-06-15T01:06:48.593 INFO:tasks.workunit.client.0.smithi137.stdout:5/5: rename f1 to f2 0 2024-06-15T01:06:48.593 INFO:tasks.workunit.client.0.smithi137.stdout:2/13: write d0/f5 [707775,82891] 0 2024-06-15T01:06:48.595 INFO:tasks.workunit.client.0.smithi137.stdout:0/19: write d1/d2/f4 [793956,76677] 0 2024-06-15T01:06:48.598 INFO:tasks.workunit.client.0.smithi137.stdout:4/14: dwrite f5 [0,4194304] 0 2024-06-15T01:06:48.599 INFO:tasks.workunit.client.0.smithi137.stdout:2/14: truncate d0/f1 109455 0 2024-06-15T01:06:48.599 INFO:tasks.workunit.client.0.smithi137.stdout:9/7: rename f1 to f2 0 2024-06-15T01:06:48.600 INFO:tasks.workunit.client.0.smithi137.stdout:9/8: write f0 [522288,84305] 0 2024-06-15T01:06:48.602 INFO:tasks.workunit.client.0.smithi137.stdout:9/9: write f0 [1103323,8450] 0 2024-06-15T01:06:48.611 INFO:tasks.workunit.client.0.smithi137.stdout:4/15: symlink l6 0 2024-06-15T01:06:48.618 INFO:tasks.workunit.client.0.smithi137.stdout:5/6: dwrite f2 [0,4194304] 0 2024-06-15T01:06:48.621 INFO:tasks.workunit.client.0.smithi137.stdout:1/19: creat d0/d1/f6 x:0 0 0 2024-06-15T01:06:48.622 INFO:tasks.workunit.client.0.smithi137.stdout:4/16: dread f5 [0,4194304] 0 2024-06-15T01:06:48.623 INFO:tasks.workunit.client.0.smithi137.stdout:7/8: getdents . 0 2024-06-15T01:06:48.626 INFO:tasks.workunit.client.0.smithi137.stdout:9/10: dwrite f0 [0,4194304] 0 2024-06-15T01:06:48.628 INFO:tasks.workunit.client.0.smithi137.stdout:5/7: creat f3 x:0 0 0 2024-06-15T01:06:48.629 INFO:tasks.workunit.client.0.smithi137.stdout:7/9: creat f1 x:0 0 0 2024-06-15T01:06:48.630 INFO:tasks.workunit.client.0.smithi137.stdout:0/20: rmdir d1/d2 39 2024-06-15T01:06:48.630 INFO:tasks.workunit.client.0.smithi137.stdout:7/10: dread - f1 zero size 2024-06-15T01:06:48.630 INFO:tasks.workunit.client.0.smithi137.stdout:0/21: chown d1 129585 1 2024-06-15T01:06:48.631 INFO:tasks.workunit.client.0.smithi137.stdout:4/17: mknod c7 0 2024-06-15T01:06:48.632 INFO:tasks.workunit.client.0.smithi137.stdout:4/18: chown c0 2274 1 2024-06-15T01:06:48.636 INFO:tasks.workunit.client.0.smithi137.stdout:5/8: creat f4 x:0 0 0 2024-06-15T01:06:48.638 INFO:tasks.workunit.client.0.smithi137.stdout:7/11: mkdir d2 0 2024-06-15T01:06:48.639 INFO:tasks.workunit.client.0.smithi137.stdout:8/6: fsync f1 0 2024-06-15T01:06:48.641 INFO:tasks.workunit.client.0.smithi137.stdout:8/7: chown f0 0 1 2024-06-15T01:06:48.641 INFO:tasks.workunit.client.0.smithi137.stdout:4/19: write f5 [3729285,79232] 0 2024-06-15T01:06:48.641 INFO:tasks.workunit.client.0.smithi137.stdout:2/15: getdents d0 0 2024-06-15T01:06:48.645 INFO:tasks.workunit.client.0.smithi137.stdout:2/16: write d0/f3 [412454,112028] 0 2024-06-15T01:06:48.645 INFO:tasks.workunit.client.0.smithi137.stdout:9/11: dwrite f2 [0,4194304] 0 2024-06-15T01:06:48.648 INFO:tasks.workunit.client.0.smithi137.stdout:5/9: link f2 f5 0 2024-06-15T01:06:48.651 INFO:tasks.workunit.client.0.smithi137.stdout:8/8: mknod c3 0 2024-06-15T01:06:48.653 INFO:tasks.workunit.client.0.smithi137.stdout:1/20: getdents d0 0 2024-06-15T01:06:48.653 INFO:tasks.workunit.client.0.smithi137.stdout:8/9: truncate f0 949218 0 2024-06-15T01:06:48.655 INFO:tasks.workunit.client.0.smithi137.stdout:8/10: dread - f1 zero size 2024-06-15T01:06:48.656 INFO:tasks.workunit.client.0.smithi137.stdout:9/12: creat f3 x:0 0 0 2024-06-15T01:06:48.657 INFO:tasks.workunit.client.0.smithi137.stdout:2/17: dwrite d0/f5 [0,4194304] 0 2024-06-15T01:06:48.657 INFO:tasks.workunit.client.0.smithi137.stdout:5/10: symlink l6 0 2024-06-15T01:06:48.657 INFO:tasks.workunit.client.0.smithi137.stdout:2/18: chown d0 4 1 2024-06-15T01:06:48.659 INFO:tasks.workunit.client.0.smithi137.stdout:7/12: rmdir d2 0 2024-06-15T01:06:48.662 INFO:tasks.workunit.client.0.smithi137.stdout:8/11: dread f0 [0,4194304] 0 2024-06-15T01:06:48.663 INFO:tasks.workunit.client.0.smithi137.stdout:9/13: symlink l4 0 2024-06-15T01:06:48.666 INFO:tasks.workunit.client.0.smithi137.stdout:5/11: rename f4 to f7 0 2024-06-15T01:06:48.668 INFO:tasks.workunit.client.0.smithi137.stdout:2/19: creat d0/f6 x:0 0 0 2024-06-15T01:06:48.669 INFO:tasks.workunit.client.0.smithi137.stdout:7/13: creat f3 x:0 0 0 2024-06-15T01:06:48.670 INFO:tasks.workunit.client.0.smithi137.stdout:2/20: read d0/f3 [442424,119015] 0 2024-06-15T01:06:48.671 INFO:tasks.workunit.client.0.smithi137.stdout:8/12: symlink l4 0 2024-06-15T01:06:48.671 INFO:tasks.workunit.client.0.smithi137.stdout:7/14: write f1 [639126,103154] 0 2024-06-15T01:06:48.671 INFO:tasks.workunit.client.0.smithi137.stdout:9/14: dwrite f0 [0,4194304] 0 2024-06-15T01:06:48.672 INFO:tasks.workunit.client.0.smithi137.stdout:2/21: chown d0/f1 165 1 2024-06-15T01:06:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:9/15: write f3 [132616,33581] 0 2024-06-15T01:06:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:5/12: symlink l8 0 2024-06-15T01:06:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:9/16: rmdir - no directory 2024-06-15T01:06:48.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/15: mknod c4 0 2024-06-15T01:06:48.683 INFO:tasks.workunit.client.0.smithi137.stdout:1/21: write d0/d3/f5 [567693,64691] 0 2024-06-15T01:06:48.724 INFO:tasks.workunit.client.0.smithi137.stdout:8/13: rename c2 to c5 0 2024-06-15T01:06:48.725 INFO:tasks.workunit.client.0.smithi137.stdout:2/22: mknod d0/c7 0 2024-06-15T01:06:48.726 INFO:tasks.workunit.client.0.smithi137.stdout:9/17: creat f5 x:0 0 0 2024-06-15T01:06:48.728 INFO:tasks.workunit.client.0.smithi137.stdout:5/13: rename f2 to f9 0 2024-06-15T01:06:48.730 INFO:tasks.workunit.client.0.smithi137.stdout:1/22: rename d0/d1 to d0/d3/d7 0 2024-06-15T01:06:48.731 INFO:tasks.workunit.client.0.smithi137.stdout:1/23: chown d0/d3/f5 39 1 2024-06-15T01:06:48.734 INFO:tasks.workunit.client.0.smithi137.stdout:2/23: creat d0/f8 x:0 0 0 2024-06-15T01:06:48.736 INFO:tasks.workunit.client.0.smithi137.stdout:7/16: link f1 f5 0 2024-06-15T01:06:48.736 INFO:tasks.workunit.client.0.smithi137.stdout:7/17: chown f1 363562726 1 2024-06-15T01:06:48.736 INFO:tasks.workunit.client.0.smithi137.stdout:7/18: rmdir - no directory 2024-06-15T01:06:48.736 INFO:tasks.workunit.client.0.smithi137.stdout:7/19: readlink - no filename 2024-06-15T01:06:48.738 INFO:tasks.workunit.client.0.smithi137.stdout:2/24: creat d0/f9 x:0 0 0 2024-06-15T01:06:48.739 INFO:tasks.workunit.client.0.smithi137.stdout:2/25: write d0/f5 [1802979,88988] 0 2024-06-15T01:06:48.740 INFO:tasks.workunit.client.0.smithi137.stdout:7/20: mknod c6 0 2024-06-15T01:06:48.745 INFO:tasks.workunit.client.0.smithi137.stdout:7/21: symlink l7 0 2024-06-15T01:06:48.746 INFO:tasks.workunit.client.0.smithi137.stdout:2/26: dread d0/f3 [0,4194304] 0 2024-06-15T01:06:48.748 INFO:tasks.workunit.client.0.smithi137.stdout:7/22: write f5 [1109520,82294] 0 2024-06-15T01:06:48.749 INFO:tasks.workunit.client.0.smithi137.stdout:7/23: dread - f3 zero size 2024-06-15T01:06:48.750 INFO:tasks.workunit.client.0.smithi137.stdout:2/27: mkdir d0/da 0 2024-06-15T01:06:48.751 INFO:tasks.workunit.client.0.smithi137.stdout:2/28: write d0/f9 [545787,77566] 0 2024-06-15T01:06:48.759 INFO:tasks.workunit.client.0.smithi137.stdout:5/14: getdents . 0 2024-06-15T01:06:48.760 INFO:tasks.workunit.client.0.smithi137.stdout:1/24: getdents d0/d3 0 2024-06-15T01:06:48.761 INFO:tasks.workunit.client.0.smithi137.stdout:8/14: write f0 [1964788,52249] 0 2024-06-15T01:06:48.763 INFO:tasks.workunit.client.0.smithi137.stdout:5/15: truncate f5 3972479 0 2024-06-15T01:06:48.763 INFO:tasks.workunit.client.0.smithi137.stdout:5/16: rmdir - no directory 2024-06-15T01:06:48.763 INFO:tasks.workunit.client.0.smithi137.stdout:1/25: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:48.764 INFO:tasks.workunit.client.0.smithi137.stdout:8/15: creat f6 x:0 0 0 2024-06-15T01:06:48.767 INFO:tasks.workunit.client.0.smithi137.stdout:1/26: mknod d0/c8 0 2024-06-15T01:06:48.767 INFO:tasks.workunit.client.0.smithi137.stdout:1/27: chown d0/d3/f5 2619 1 2024-06-15T01:06:48.768 INFO:tasks.workunit.client.0.smithi137.stdout:8/16: creat f7 x:0 0 0 2024-06-15T01:06:48.775 INFO:tasks.workunit.client.0.smithi137.stdout:8/17: dwrite f0 [0,4194304] 0 2024-06-15T01:06:48.776 INFO:tasks.workunit.client.0.smithi137.stdout:8/18: stat l4 0 2024-06-15T01:06:48.776 INFO:tasks.workunit.client.0.smithi137.stdout:8/19: dread - f6 zero size 2024-06-15T01:06:48.779 INFO:tasks.workunit.client.0.smithi137.stdout:1/28: truncate d0/f2 1565897 0 2024-06-15T01:06:48.780 INFO:tasks.workunit.client.0.smithi137.stdout:1/29: dread - d0/d3/d7/f6 zero size 2024-06-15T01:06:48.782 INFO:tasks.workunit.client.0.smithi137.stdout:8/20: link f6 f8 0 2024-06-15T01:06:48.782 INFO:tasks.workunit.client.0.smithi137.stdout:8/21: readlink l4 0 2024-06-15T01:06:48.782 INFO:tasks.workunit.client.0.smithi137.stdout:5/17: link f9 fa 0 2024-06-15T01:06:48.785 INFO:tasks.workunit.client.0.smithi137.stdout:8/22: mkdir d9 0 2024-06-15T01:06:48.785 INFO:tasks.workunit.client.0.smithi137.stdout:8/23: stat c3 0 2024-06-15T01:06:48.788 INFO:tasks.workunit.client.0.smithi137.stdout:8/24: rename l4 to d9/la 0 2024-06-15T01:06:48.790 INFO:tasks.workunit.client.0.smithi137.stdout:5/18: link l6 lb 0 2024-06-15T01:06:48.790 INFO:tasks.workunit.client.0.smithi137.stdout:8/25: mknod d9/cb 0 2024-06-15T01:06:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:5/19: dwrite f7 [0,4194304] 0 2024-06-15T01:06:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:5/20: creat fc x:0 0 0 2024-06-15T01:06:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:1/30: truncate d0/f2 362999 0 2024-06-15T01:06:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:1/31: write d0/d3/d7/f6 [833342,40293] 0 2024-06-15T01:06:48.808 INFO:tasks.workunit.client.0.smithi137.stdout:5/21: write f5 [502559,18543] 0 2024-06-15T01:06:48.812 INFO:tasks.workunit.client.0.smithi137.stdout:5/22: getdents . 0 2024-06-15T01:06:48.814 INFO:tasks.workunit.client.0.smithi137.stdout:5/23: write f5 [4178332,119890] 0 2024-06-15T01:06:48.817 INFO:tasks.workunit.client.0.smithi137.stdout:5/24: creat fd x:0 0 0 2024-06-15T01:06:48.818 INFO:tasks.workunit.client.0.smithi137.stdout:5/25: stat fc 0 2024-06-15T01:06:48.819 INFO:tasks.workunit.client.0.smithi137.stdout:1/32: dwrite d0/d3/f5 [0,4194304] 0 2024-06-15T01:06:48.820 INFO:tasks.workunit.client.0.smithi137.stdout:5/26: rename fa to fe 0 2024-06-15T01:06:48.823 INFO:tasks.workunit.client.0.smithi137.stdout:1/33: creat d0/f9 x:0 0 0 2024-06-15T01:06:48.823 INFO:tasks.workunit.client.0.smithi137.stdout:5/27: mkdir df 0 2024-06-15T01:06:48.826 INFO:tasks.workunit.client.0.smithi137.stdout:1/34: creat d0/d3/fa x:0 0 0 2024-06-15T01:06:48.828 INFO:tasks.workunit.client.0.smithi137.stdout:1/35: mkdir d0/db 0 2024-06-15T01:06:48.829 INFO:tasks.workunit.client.0.smithi137.stdout:5/28: creat df/f10 x:0 0 0 2024-06-15T01:06:48.830 INFO:tasks.workunit.client.0.smithi137.stdout:1/36: unlink d0/d3/fa 0 2024-06-15T01:06:48.831 INFO:tasks.workunit.client.0.smithi137.stdout:5/29: mkdir df/d11 0 2024-06-15T01:06:48.832 INFO:tasks.workunit.client.0.smithi137.stdout:1/37: creat d0/d3/fc x:0 0 0 2024-06-15T01:06:48.835 INFO:tasks.workunit.client.0.smithi137.stdout:1/38: link d0/c8 d0/cd 0 2024-06-15T01:06:48.838 INFO:tasks.workunit.client.0.smithi137.stdout:1/39: symlink d0/d3/le 0 2024-06-15T01:06:48.839 INFO:tasks.workunit.client.0.smithi137.stdout:5/30: dwrite fd [0,4194304] 0 2024-06-15T01:06:48.840 INFO:tasks.workunit.client.0.smithi137.stdout:1/40: read d0/d3/f5 [2252515,119224] 0 2024-06-15T01:06:48.840 INFO:tasks.workunit.client.0.smithi137.stdout:5/31: write f7 [1902442,32968] 0 2024-06-15T01:06:48.841 INFO:tasks.workunit.client.0.smithi137.stdout:1/41: write d0/f9 [405388,77437] 0 2024-06-15T01:06:48.845 INFO:tasks.workunit.client.0.smithi137.stdout:5/32: rmdir df 39 2024-06-15T01:06:48.849 INFO:tasks.workunit.client.0.smithi137.stdout:5/33: stat df 0 2024-06-15T01:06:48.851 INFO:tasks.workunit.client.0.smithi137.stdout:1/42: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:48.852 INFO:tasks.workunit.client.0.smithi137.stdout:5/34: getdents df 0 2024-06-15T01:06:48.853 INFO:tasks.workunit.client.0.smithi137.stdout:5/35: creat df/f12 x:0 0 0 2024-06-15T01:06:48.854 INFO:tasks.workunit.client.0.smithi137.stdout:5/36: chown f3 0 1 2024-06-15T01:06:48.856 INFO:tasks.workunit.client.0.smithi137.stdout:5/37: link fe df/d11/f13 0 2024-06-15T01:06:48.857 INFO:tasks.workunit.client.0.smithi137.stdout:5/38: stat fe 0 2024-06-15T01:06:48.857 INFO:tasks.workunit.client.0.smithi137.stdout:5/39: truncate f3 537692 0 2024-06-15T01:06:48.857 INFO:tasks.workunit.client.0.smithi137.stdout:1/43: truncate d0/f9 417359 0 2024-06-15T01:06:48.860 INFO:tasks.workunit.client.0.smithi137.stdout:5/40: mkdir df/d11/d14 0 2024-06-15T01:06:48.860 INFO:tasks.workunit.client.0.smithi137.stdout:5/41: fdatasync df/f10 0 2024-06-15T01:06:48.861 INFO:tasks.workunit.client.0.smithi137.stdout:5/42: dread - fc zero size 2024-06-15T01:06:48.861 INFO:tasks.workunit.client.0.smithi137.stdout:5/43: dread - fc zero size 2024-06-15T01:06:48.861 INFO:tasks.workunit.client.0.smithi137.stdout:1/44: rename d0/c8 to d0/db/cf 0 2024-06-15T01:06:48.865 INFO:tasks.workunit.client.0.smithi137.stdout:5/44: creat df/d11/d14/f15 x:0 0 0 2024-06-15T01:06:48.865 INFO:tasks.workunit.client.0.smithi137.stdout:5/45: chown df 3107437 1 2024-06-15T01:06:48.865 INFO:tasks.workunit.client.0.smithi137.stdout:5/46: chown df/d11/f13 1 1 2024-06-15T01:06:48.866 INFO:tasks.workunit.client.0.smithi137.stdout:5/47: fdatasync fd 0 2024-06-15T01:06:48.866 INFO:tasks.workunit.client.0.smithi137.stdout:5/48: chown df/d11/d14 377 1 2024-06-15T01:06:48.869 INFO:tasks.workunit.client.0.smithi137.stdout:5/49: creat df/d11/d14/f16 x:0 0 0 2024-06-15T01:06:48.871 INFO:tasks.workunit.client.0.smithi137.stdout:1/45: rename d0/db to d0/d10 0 2024-06-15T01:06:48.874 INFO:tasks.workunit.client.0.smithi137.stdout:5/50: link fe df/d11/f17 0 2024-06-15T01:06:48.880 INFO:tasks.workunit.client.0.smithi137.stdout:1/46: truncate d0/d3/d7/f6 111702 0 2024-06-15T01:06:48.882 INFO:tasks.workunit.client.0.smithi137.stdout:5/51: dwrite f3 [0,4194304] 0 2024-06-15T01:06:48.887 INFO:tasks.workunit.client.0.smithi137.stdout:1/47: write d0/d3/f5 [4912542,109398] 0 2024-06-15T01:06:48.888 INFO:tasks.workunit.client.0.smithi137.stdout:5/52: dread fd [0,4194304] 0 2024-06-15T01:06:48.889 INFO:tasks.workunit.client.0.smithi137.stdout:1/48: chown d0/d3/f5 328541091 1 2024-06-15T01:06:48.891 INFO:tasks.workunit.client.0.smithi137.stdout:5/53: rmdir df/d11/d14 39 2024-06-15T01:06:48.892 INFO:tasks.workunit.client.0.smithi137.stdout:1/49: rename d0/d3/le to d0/d3/l11 0 2024-06-15T01:06:48.893 INFO:tasks.workunit.client.0.smithi137.stdout:5/54: mknod df/d11/c18 0 2024-06-15T01:06:48.895 INFO:tasks.workunit.client.0.smithi137.stdout:5/55: rmdir df/d11 39 2024-06-15T01:06:48.896 INFO:tasks.workunit.client.0.smithi137.stdout:1/50: truncate d0/f9 32670 0 2024-06-15T01:06:48.896 INFO:tasks.workunit.client.0.smithi137.stdout:5/56: write f3 [1836707,58427] 0 2024-06-15T01:06:48.896 INFO:tasks.workunit.client.0.smithi137.stdout:5/57: chown df/f10 831890222 1 2024-06-15T01:06:48.898 INFO:tasks.workunit.client.0.smithi137.stdout:1/51: creat d0/d10/f12 x:0 0 0 2024-06-15T01:06:48.900 INFO:tasks.workunit.client.0.smithi137.stdout:5/58: truncate df/d11/d14/f15 440836 0 2024-06-15T01:06:48.901 INFO:tasks.workunit.client.0.smithi137.stdout:1/52: rename d0/d10/f12 to d0/d3/f13 0 2024-06-15T01:06:48.902 INFO:tasks.workunit.client.0.smithi137.stdout:5/59: dread df/d11/d14/f15 [0,4194304] 0 2024-06-15T01:06:48.903 INFO:tasks.workunit.client.0.smithi137.stdout:5/60: chown f9 3 1 2024-06-15T01:06:48.905 INFO:tasks.workunit.client.0.smithi137.stdout:5/61: fdatasync fe 0 2024-06-15T01:06:48.912 INFO:tasks.workunit.client.0.smithi137.stdout:5/62: dwrite fd [0,4194304] 0 2024-06-15T01:06:48.918 INFO:tasks.workunit.client.0.smithi137.stdout:1/53: dwrite d0/f9 [0,4194304] 0 2024-06-15T01:06:48.920 INFO:tasks.workunit.client.0.smithi137.stdout:5/63: write df/d11/f13 [1926645,32847] 0 2024-06-15T01:06:48.920 INFO:tasks.workunit.client.0.smithi137.stdout:1/54: write d0/d3/f5 [4704878,5095] 0 2024-06-15T01:06:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:5/64: creat df/d11/d14/f19 x:0 0 0 2024-06-15T01:06:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:1/55: mkdir d0/d3/d14 0 2024-06-15T01:06:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:1/56: mknod d0/c15 0 2024-06-15T01:06:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:1/57: symlink d0/d10/l16 0 2024-06-15T01:06:48.931 INFO:tasks.workunit.client.0.smithi137.stdout:5/65: dwrite f5 [0,4194304] 0 2024-06-15T01:06:48.934 INFO:tasks.workunit.client.0.smithi137.stdout:5/66: write df/d11/f13 [2493505,128259] 0 2024-06-15T01:06:48.938 INFO:tasks.workunit.client.0.smithi137.stdout:1/58: dwrite d0/f9 [0,4194304] 0 2024-06-15T01:06:48.943 INFO:tasks.workunit.client.0.smithi137.stdout:5/67: dread df/d11/f13 [0,4194304] 0 2024-06-15T01:06:48.946 INFO:tasks.workunit.client.0.smithi137.stdout:5/68: symlink df/d11/l1a 0 2024-06-15T01:06:48.947 INFO:tasks.workunit.client.0.smithi137.stdout:5/69: write fc [547351,26803] 0 2024-06-15T01:06:48.949 INFO:tasks.workunit.client.0.smithi137.stdout:1/59: truncate d0/d3/f5 1012886 0 2024-06-15T01:06:48.952 INFO:tasks.workunit.client.0.smithi137.stdout:1/60: mknod d0/d3/d7/c17 0 2024-06-15T01:06:48.954 INFO:tasks.workunit.client.0.smithi137.stdout:1/61: fsync d0/d3/f13 0 2024-06-15T01:06:48.954 INFO:tasks.workunit.client.0.smithi137.stdout:5/70: dwrite df/d11/d14/f19 [0,4194304] 0 2024-06-15T01:06:48.957 INFO:tasks.workunit.client.0.smithi137.stdout:1/62: rmdir d0/d10 39 2024-06-15T01:06:48.958 INFO:tasks.workunit.client.0.smithi137.stdout:5/71: symlink df/l1b 0 2024-06-15T01:06:48.961 INFO:tasks.workunit.client.0.smithi137.stdout:1/63: mknod d0/d3/d7/c18 0 2024-06-15T01:06:48.963 INFO:tasks.workunit.client.0.smithi137.stdout:5/72: symlink df/d11/d14/l1c 0 2024-06-15T01:06:48.966 INFO:tasks.workunit.client.0.smithi137.stdout:1/64: creat d0/d3/d14/f19 x:0 0 0 2024-06-15T01:06:48.967 INFO:tasks.workunit.client.0.smithi137.stdout:5/73: symlink df/d11/l1d 0 2024-06-15T01:06:48.971 INFO:tasks.workunit.client.0.smithi137.stdout:5/74: dread f7 [0,4194304] 0 2024-06-15T01:06:48.971 INFO:tasks.workunit.client.0.smithi137.stdout:1/65: rename d0/d3/l11 to d0/d3/d14/l1a 0 2024-06-15T01:06:48.972 INFO:tasks.workunit.client.0.smithi137.stdout:1/66: mkdir d0/d3/d1b 0 2024-06-15T01:06:48.973 INFO:tasks.workunit.client.0.smithi137.stdout:5/75: creat df/d11/d14/f1e x:0 0 0 2024-06-15T01:06:48.975 INFO:tasks.workunit.client.0.smithi137.stdout:1/67: rename d0/l4 to d0/d3/l1c 0 2024-06-15T01:06:48.975 INFO:tasks.workunit.client.0.smithi137.stdout:5/76: mknod df/d11/d14/c1f 0 2024-06-15T01:06:48.977 INFO:tasks.workunit.client.0.smithi137.stdout:1/68: mkdir d0/d10/d1d 0 2024-06-15T01:06:48.978 INFO:tasks.workunit.client.0.smithi137.stdout:1/69: dread - d0/d3/d14/f19 zero size 2024-06-15T01:06:48.979 INFO:tasks.workunit.client.0.smithi137.stdout:5/77: creat df/d11/f20 x:0 0 0 2024-06-15T01:06:48.983 INFO:tasks.workunit.client.0.smithi137.stdout:5/78: rename df/d11/f17 to df/d11/d14/f21 0 2024-06-15T01:06:48.984 INFO:tasks.workunit.client.0.smithi137.stdout:5/79: write df/d11/d14/f21 [256551,36649] 0 2024-06-15T01:06:48.985 INFO:tasks.workunit.client.0.smithi137.stdout:5/80: chown df/d11/d14/f1e 67784 1 2024-06-15T01:06:48.993 INFO:tasks.workunit.client.0.smithi137.stdout:1/70: write d0/d3/f5 [854543,99777] 0 2024-06-15T01:06:48.995 INFO:tasks.workunit.client.0.smithi137.stdout:1/71: symlink d0/d3/d14/l1e 0 2024-06-15T01:06:48.998 INFO:tasks.workunit.client.0.smithi137.stdout:1/72: mkdir d0/d1f 0 2024-06-15T01:06:48.998 INFO:tasks.workunit.client.0.smithi137.stdout:5/81: dwrite df/d11/d14/f15 [0,4194304] 0 2024-06-15T01:06:49.001 INFO:tasks.workunit.client.0.smithi137.stdout:5/82: symlink df/d11/l22 0 2024-06-15T01:06:49.002 INFO:tasks.workunit.client.0.smithi137.stdout:5/83: write f7 [4099136,131060] 0 2024-06-15T01:06:49.004 INFO:tasks.workunit.client.0.smithi137.stdout:5/84: readlink l6 0 2024-06-15T01:06:49.004 INFO:tasks.workunit.client.0.smithi137.stdout:5/85: fdatasync df/d11/f20 0 2024-06-15T01:06:49.006 INFO:tasks.workunit.client.0.smithi137.stdout:1/73: truncate d0/f2 297877 0 2024-06-15T01:06:49.014 INFO:tasks.workunit.client.0.smithi137.stdout:1/74: dwrite d0/d3/d14/f19 [0,4194304] 0 2024-06-15T01:06:49.015 INFO:tasks.workunit.client.0.smithi137.stdout:5/86: truncate df/d11/d14/f19 2625789 0 2024-06-15T01:06:49.016 INFO:tasks.workunit.client.0.smithi137.stdout:1/75: fdatasync d0/d3/fc 0 2024-06-15T01:06:49.017 INFO:tasks.workunit.client.0.smithi137.stdout:5/87: truncate df/d11/f20 499370 0 2024-06-15T01:06:49.022 INFO:tasks.workunit.client.0.smithi137.stdout:5/88: symlink df/d11/l23 0 2024-06-15T01:06:49.023 INFO:tasks.workunit.client.0.smithi137.stdout:5/89: creat df/d11/f24 x:0 0 0 2024-06-15T01:06:49.024 INFO:tasks.workunit.client.0.smithi137.stdout:5/90: truncate df/d11/d14/f1e 859700 0 2024-06-15T01:06:49.030 INFO:tasks.workunit.client.0.smithi137.stdout:5/91: dwrite fd [0,4194304] 0 2024-06-15T01:06:49.032 INFO:tasks.workunit.client.0.smithi137.stdout:1/76: fdatasync d0/f2 0 2024-06-15T01:06:49.033 INFO:tasks.workunit.client.0.smithi137.stdout:1/77: chown d0/d3/d7/c18 0 1 2024-06-15T01:06:49.037 INFO:tasks.workunit.client.0.smithi137.stdout:5/92: dread fe [0,4194304] 0 2024-06-15T01:06:49.042 INFO:tasks.workunit.client.0.smithi137.stdout:1/78: dwrite d0/d3/f13 [0,4194304] 0 2024-06-15T01:06:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:1/79: symlink d0/d3/d1b/l20 0 2024-06-15T01:06:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:1/80: dread d0/d3/d7/f6 [0,4194304] 0 2024-06-15T01:06:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:1/81: rename d0 to d0/d10/d1d/d21 22 2024-06-15T01:06:49.053 INFO:tasks.workunit.client.0.smithi137.stdout:1/82: rename d0/d3/d7/f6 to d0/d3/f22 0 2024-06-15T01:06:49.054 INFO:tasks.workunit.client.0.smithi137.stdout:1/83: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:49.057 INFO:tasks.workunit.client.0.smithi137.stdout:1/84: stat d0/d3/d14/l1a 0 2024-06-15T01:06:49.059 INFO:tasks.workunit.client.0.smithi137.stdout:5/93: dwrite df/d11/d14/f19 [0,4194304] 0 2024-06-15T01:06:49.059 INFO:tasks.workunit.client.0.smithi137.stdout:1/85: creat d0/d10/f23 x:0 0 0 2024-06-15T01:06:49.061 INFO:tasks.workunit.client.0.smithi137.stdout:5/94: rename lb to df/d11/d14/l25 0 2024-06-15T01:06:49.062 INFO:tasks.workunit.client.0.smithi137.stdout:1/86: mknod d0/d3/c24 0 2024-06-15T01:06:49.065 INFO:tasks.workunit.client.0.smithi137.stdout:5/95: link df/d11/l1a df/d11/d14/l26 0 2024-06-15T01:06:49.069 INFO:tasks.workunit.client.0.smithi137.stdout:5/96: symlink df/d11/l27 0 2024-06-15T01:06:49.069 INFO:tasks.workunit.client.0.smithi137.stdout:1/87: dwrite d0/d3/f13 [0,4194304] 0 2024-06-15T01:06:49.074 INFO:tasks.workunit.client.0.smithi137.stdout:1/88: dread d0/f9 [0,4194304] 0 2024-06-15T01:06:49.076 INFO:tasks.workunit.client.0.smithi137.stdout:1/89: mknod d0/d3/d14/c25 0 2024-06-15T01:06:49.079 INFO:tasks.workunit.client.0.smithi137.stdout:1/90: mknod d0/c26 0 2024-06-15T01:06:49.080 INFO:tasks.workunit.client.0.smithi137.stdout:5/97: truncate f3 2192314 0 2024-06-15T01:06:49.084 INFO:tasks.workunit.client.0.smithi137.stdout:1/91: dread d0/f9 [0,4194304] 0 2024-06-15T01:06:49.085 INFO:tasks.workunit.client.0.smithi137.stdout:1/92: rmdir d0/d3/d7 39 2024-06-15T01:06:49.087 INFO:tasks.workunit.client.0.smithi137.stdout:1/93: write d0/d3/f13 [4379385,46013] 0 2024-06-15T01:06:49.087 INFO:tasks.workunit.client.0.smithi137.stdout:5/98: dwrite df/d11/f13 [0,4194304] 0 2024-06-15T01:06:49.088 INFO:tasks.workunit.client.0.smithi137.stdout:1/94: readlink d0/d3/d14/l1e 0 2024-06-15T01:06:49.088 INFO:tasks.workunit.client.0.smithi137.stdout:1/95: readlink d0/d3/d14/l1a 0 2024-06-15T01:06:49.093 INFO:tasks.workunit.client.0.smithi137.stdout:5/99: getdents df/d11 0 2024-06-15T01:06:49.097 INFO:tasks.workunit.client.0.smithi137.stdout:5/100: symlink df/d11/l28 0 2024-06-15T01:06:49.097 INFO:tasks.workunit.client.0.smithi137.stdout:1/96: dwrite d0/f9 [0,4194304] 0 2024-06-15T01:06:49.098 INFO:tasks.workunit.client.0.smithi137.stdout:1/97: dread - d0/d10/f23 zero size 2024-06-15T01:06:49.099 INFO:tasks.workunit.client.0.smithi137.stdout:1/98: readlink d0/d3/d1b/l20 0 2024-06-15T01:06:49.101 INFO:tasks.workunit.client.0.smithi137.stdout:1/99: unlink d0/d3/fc 0 2024-06-15T01:06:49.106 INFO:tasks.workunit.client.0.smithi137.stdout:1/100: link d0/d3/f5 d0/d3/d14/f27 0 2024-06-15T01:06:49.108 INFO:tasks.workunit.client.0.smithi137.stdout:1/101: dread d0/d3/d14/f27 [0,4194304] 0 2024-06-15T01:06:49.111 INFO:tasks.workunit.client.0.smithi137.stdout:1/102: symlink d0/d3/d1b/l28 0 2024-06-15T01:06:49.111 INFO:tasks.workunit.client.0.smithi137.stdout:5/101: symlink df/d11/l29 0 2024-06-15T01:06:49.113 INFO:tasks.workunit.client.0.smithi137.stdout:1/103: symlink d0/d3/d7/l29 0 2024-06-15T01:06:49.114 INFO:tasks.workunit.client.0.smithi137.stdout:1/104: rename d0/d1f to d0/d1f/d2a 22 2024-06-15T01:06:49.115 INFO:tasks.workunit.client.0.smithi137.stdout:1/105: read d0/d3/f22 [28828,51802] 0 2024-06-15T01:06:49.121 INFO:tasks.workunit.client.0.smithi137.stdout:1/106: dwrite d0/f9 [4194304,4194304] 0 2024-06-15T01:06:49.122 INFO:tasks.workunit.client.0.smithi137.stdout:1/107: stat d0/f2 0 2024-06-15T01:06:49.122 INFO:tasks.workunit.client.0.smithi137.stdout:1/108: chown d0 227 1 2024-06-15T01:06:49.124 INFO:tasks.workunit.client.0.smithi137.stdout:1/109: creat d0/d3/d7/f2b x:0 0 0 2024-06-15T01:06:49.131 INFO:tasks.workunit.client.0.smithi137.stdout:1/110: dwrite d0/d10/f23 [0,4194304] 0 2024-06-15T01:06:49.132 INFO:tasks.workunit.client.0.smithi137.stdout:1/111: dread - d0/d3/d7/f2b zero size 2024-06-15T01:06:49.137 INFO:tasks.workunit.client.0.smithi137.stdout:1/112: write d0/d3/d14/f27 [719106,88688] 0 2024-06-15T01:06:49.143 INFO:tasks.workunit.client.0.smithi137.stdout:1/113: rmdir d0/d3 39 2024-06-15T01:06:49.145 INFO:tasks.workunit.client.0.smithi137.stdout:1/114: creat d0/d10/d1d/f2c x:0 0 0 2024-06-15T01:06:49.146 INFO:tasks.workunit.client.0.smithi137.stdout:1/115: chown d0/d10/d1d/f2c 0 1 2024-06-15T01:06:49.147 INFO:tasks.workunit.client.0.smithi137.stdout:1/116: mkdir d0/d2d 0 2024-06-15T01:06:49.149 INFO:tasks.workunit.client.0.smithi137.stdout:1/117: write d0/d3/f13 [1550397,116468] 0 2024-06-15T01:06:49.150 INFO:tasks.workunit.client.0.smithi137.stdout:1/118: fdatasync d0/d3/d7/f2b 0 2024-06-15T01:06:49.151 INFO:tasks.workunit.client.0.smithi137.stdout:1/119: fdatasync d0/d3/d14/f27 0 2024-06-15T01:06:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:1/120: rename d0/f9 to d0/d10/f2e 0 2024-06-15T01:06:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:1/121: rename d0/d10/d1d/f2c to d0/d3/f2f 0 2024-06-15T01:06:49.161 INFO:tasks.workunit.client.0.smithi137.stdout:1/122: mknod d0/d1f/c30 0 2024-06-15T01:06:49.163 INFO:tasks.workunit.client.0.smithi137.stdout:1/123: write d0/d3/f2f [168767,35580] 0 2024-06-15T01:06:49.166 INFO:tasks.workunit.client.0.smithi137.stdout:1/124: getdents d0/d3/d1b 0 2024-06-15T01:06:49.167 INFO:tasks.workunit.client.0.smithi137.stdout:1/125: mknod d0/d10/c31 0 2024-06-15T01:06:49.171 INFO:tasks.workunit.client.0.smithi137.stdout:1/126: dread d0/d10/f2e [0,4194304] 0 2024-06-15T01:06:49.172 INFO:tasks.workunit.client.0.smithi137.stdout:1/127: write d0/d10/f23 [304528,79169] 0 2024-06-15T01:06:49.174 INFO:tasks.workunit.client.0.smithi137.stdout:1/128: symlink d0/d3/l32 0 2024-06-15T01:06:49.176 INFO:tasks.workunit.client.0.smithi137.stdout:1/129: symlink d0/d3/d1b/l33 0 2024-06-15T01:06:49.177 INFO:tasks.workunit.client.0.smithi137.stdout:1/130: chown d0/d10/l16 12 1 2024-06-15T01:06:49.177 INFO:tasks.workunit.client.0.smithi137.stdout:1/131: fdatasync d0/f2 0 2024-06-15T01:06:49.181 INFO:tasks.workunit.client.0.smithi137.stdout:1/132: dread d0/d3/f13 [0,4194304] 0 2024-06-15T01:06:49.183 INFO:tasks.workunit.client.0.smithi137.stdout:1/133: mkdir d0/d3/d14/d34 0 2024-06-15T01:06:49.184 INFO:tasks.workunit.client.0.smithi137.stdout:1/134: mknod d0/d10/d1d/c35 0 2024-06-15T01:06:49.185 INFO:tasks.workunit.client.0.smithi137.stdout:1/135: chown d0/d3/l32 140 1 2024-06-15T01:06:49.187 INFO:tasks.workunit.client.0.smithi137.stdout:1/136: creat d0/d3/f36 x:0 0 0 2024-06-15T01:06:49.187 INFO:tasks.workunit.client.0.smithi137.stdout:1/137: chown d0/d1f 65593741 1 2024-06-15T01:06:49.188 INFO:tasks.workunit.client.0.smithi137.stdout:1/138: write d0/f2 [1298344,124709] 0 2024-06-15T01:06:49.190 INFO:tasks.workunit.client.0.smithi137.stdout:1/139: symlink d0/d10/l37 0 2024-06-15T01:06:49.194 INFO:tasks.workunit.client.0.smithi137.stdout:1/140: rename d0/d3/d14/f27 to d0/d3/d14/f38 0 2024-06-15T01:06:49.198 INFO:tasks.workunit.client.0.smithi137.stdout:1/141: rename d0/d3/d7/f2b to d0/d3/d14/d34/f39 0 2024-06-15T01:06:49.200 INFO:tasks.workunit.client.0.smithi137.stdout:1/142: symlink d0/d3/l3a 0 2024-06-15T01:06:49.201 INFO:tasks.workunit.client.0.smithi137.stdout:1/143: chown d0/d1f 15663220 1 2024-06-15T01:06:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:49 smithi137 ceph-mon[29354]: pgmap v10: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 347 op/s 2024-06-15T01:06:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:49 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:50.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:49 smithi142 ceph-mon[37379]: pgmap v10: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 347 op/s 2024-06-15T01:06:50.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:49 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:06:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:51 smithi142 ceph-mon[37379]: pgmap v11: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 347 op/s 2024-06-15T01:06:52.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:51 smithi137 ceph-mon[29354]: pgmap v11: 65 pgs: 65 active+clean; 220 MiB data, 1012 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 347 op/s 2024-06-15T01:06:52.613 INFO:tasks.workunit.client.1.smithi142.stdout:2/11: sync 2024-06-15T01:06:52.613 INFO:tasks.workunit.client.1.smithi142.stdout:2/12: chown . 171334 1 2024-06-15T01:06:52.613 INFO:tasks.workunit.client.1.smithi142.stdout:2/13: dread - no filename 2024-06-15T01:06:52.615 INFO:tasks.workunit.client.1.smithi142.stdout:2/14: symlink l0 0 2024-06-15T01:06:52.617 INFO:tasks.workunit.client.1.smithi142.stdout:2/15: symlink l1 0 2024-06-15T01:06:52.618 INFO:tasks.workunit.client.1.smithi142.stdout:2/16: mknod c2 0 2024-06-15T01:06:52.618 INFO:tasks.workunit.client.1.smithi142.stdout:2/17: dwrite - no filename 2024-06-15T01:06:52.620 INFO:tasks.workunit.client.1.smithi142.stdout:2/18: creat f3 x:0 0 0 2024-06-15T01:06:52.620 INFO:tasks.workunit.client.1.smithi142.stdout:2/19: write f3 [440180,125906] 0 2024-06-15T01:06:52.621 INFO:tasks.workunit.client.1.smithi142.stdout:2/20: chown c2 467871753 1 2024-06-15T01:06:52.628 INFO:tasks.workunit.client.1.smithi142.stdout:2/21: dwrite f3 [0,4194304] 0 2024-06-15T01:06:52.697 INFO:tasks.workunit.client.0.smithi137.stdout:5/102: sync 2024-06-15T01:06:52.697 INFO:tasks.workunit.client.0.smithi137.stdout:4/20: sync 2024-06-15T01:06:52.697 INFO:tasks.workunit.client.0.smithi137.stdout:1/144: sync 2024-06-15T01:06:52.697 INFO:tasks.workunit.client.0.smithi137.stdout:2/29: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:7/24: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:6/0: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:6/1: dwrite - no filename 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:8/26: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:0/22: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/1: sync 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/2: dread - no filename 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/3: dwrite - no filename 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/4: unlink - no file 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/5: write - no filename 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/6: fsync - no filename 2024-06-15T01:06:52.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/7: truncate - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:3/8: stat - no entries 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:9/18: sync 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/2: chown . 15234 1 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/3: dwrite - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/4: dwrite - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/5: dwrite - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/6: readlink - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/7: dread - no filename 2024-06-15T01:06:52.699 INFO:tasks.workunit.client.0.smithi137.stdout:6/8: read - no filename 2024-06-15T01:06:52.700 INFO:tasks.workunit.client.0.smithi137.stdout:8/27: chown f8 14253781 1 2024-06-15T01:06:52.700 INFO:tasks.workunit.client.0.smithi137.stdout:5/103: write fd [463911,7683] 0 2024-06-15T01:06:52.700 INFO:tasks.workunit.client.0.smithi137.stdout:2/30: chown d0/f6 57570764 1 2024-06-15T01:06:52.701 INFO:tasks.workunit.client.0.smithi137.stdout:8/28: dread - f8 zero size 2024-06-15T01:06:52.702 INFO:tasks.workunit.client.0.smithi137.stdout:2/31: dread d0/f1 [0,4194304] 0 2024-06-15T01:06:52.702 INFO:tasks.workunit.client.0.smithi137.stdout:4/21: creat f8 x:0 0 0 2024-06-15T01:06:52.703 INFO:tasks.workunit.client.0.smithi137.stdout:2/32: write d0/f6 [501054,20720] 0 2024-06-15T01:06:52.705 INFO:tasks.workunit.client.0.smithi137.stdout:2/33: dread - d0/f8 zero size 2024-06-15T01:06:52.705 INFO:tasks.workunit.client.0.smithi137.stdout:7/25: dread f5 [0,4194304] 0 2024-06-15T01:06:52.708 INFO:tasks.workunit.client.0.smithi137.stdout:3/9: mkdir d0 0 2024-06-15T01:06:52.710 INFO:tasks.workunit.client.0.smithi137.stdout:6/9: mkdir d0 0 2024-06-15T01:06:52.710 INFO:tasks.workunit.client.0.smithi137.stdout:9/19: truncate f5 843000 0 2024-06-15T01:06:52.711 INFO:tasks.workunit.client.0.smithi137.stdout:6/10: chown d0 15821 1 2024-06-15T01:06:52.711 INFO:tasks.workunit.client.0.smithi137.stdout:6/11: truncate - no filename 2024-06-15T01:06:52.711 INFO:tasks.workunit.client.0.smithi137.stdout:0/23: symlink d1/l5 0 2024-06-15T01:06:52.711 INFO:tasks.workunit.client.0.smithi137.stdout:8/29: rmdir d9 39 2024-06-15T01:06:52.714 INFO:tasks.workunit.client.0.smithi137.stdout:1/145: creat d0/f3b x:0 0 0 2024-06-15T01:06:52.714 INFO:tasks.workunit.client.0.smithi137.stdout:4/22: dwrite f5 [0,4194304] 0 2024-06-15T01:06:52.719 INFO:tasks.workunit.client.0.smithi137.stdout:2/34: creat d0/fb x:0 0 0 2024-06-15T01:06:52.719 INFO:tasks.workunit.client.0.smithi137.stdout:7/26: fdatasync f1 0 2024-06-15T01:06:52.721 INFO:tasks.workunit.client.0.smithi137.stdout:9/20: dread f0 [0,4194304] 0 2024-06-15T01:06:52.722 INFO:tasks.workunit.client.0.smithi137.stdout:5/104: creat df/f2a x:0 0 0 2024-06-15T01:06:52.723 INFO:tasks.workunit.client.0.smithi137.stdout:9/21: truncate f3 558272 0 2024-06-15T01:06:52.725 INFO:tasks.workunit.client.0.smithi137.stdout:1/146: dread d0/d3/d14/f19 [0,4194304] 0 2024-06-15T01:06:52.726 INFO:tasks.workunit.client.0.smithi137.stdout:1/147: dread - d0/f3b zero size 2024-06-15T01:06:52.727 INFO:tasks.workunit.client.0.smithi137.stdout:0/24: creat d1/d2/f6 x:0 0 0 2024-06-15T01:06:52.728 INFO:tasks.workunit.client.0.smithi137.stdout:0/25: read d1/d2/f4 [3833434,83690] 0 2024-06-15T01:06:52.729 INFO:tasks.workunit.client.0.smithi137.stdout:7/27: mknod c8 0 2024-06-15T01:06:52.731 INFO:tasks.workunit.client.0.smithi137.stdout:7/28: write f3 [318636,126835] 0 2024-06-15T01:06:52.733 INFO:tasks.workunit.client.0.smithi137.stdout:6/12: mknod d0/c1 0 2024-06-15T01:06:52.734 INFO:tasks.workunit.client.0.smithi137.stdout:7/29: dread f1 [0,4194304] 0 2024-06-15T01:06:52.734 INFO:tasks.workunit.client.0.smithi137.stdout:6/13: stat d0 0 2024-06-15T01:06:52.735 INFO:tasks.workunit.client.0.smithi137.stdout:2/35: mkdir d0/dc 0 2024-06-15T01:06:52.737 INFO:tasks.workunit.client.0.smithi137.stdout:8/30: creat d9/fc x:0 0 0 2024-06-15T01:06:52.738 INFO:tasks.workunit.client.0.smithi137.stdout:0/26: dread d1/d2/f4 [0,4194304] 0 2024-06-15T01:06:52.740 INFO:tasks.workunit.client.0.smithi137.stdout:4/23: fdatasync f5 0 2024-06-15T01:06:52.741 INFO:tasks.workunit.client.0.smithi137.stdout:9/22: symlink l6 0 2024-06-15T01:06:52.742 INFO:tasks.workunit.client.0.smithi137.stdout:1/148: mknod d0/d10/c3c 0 2024-06-15T01:06:52.743 INFO:tasks.workunit.client.0.smithi137.stdout:4/24: readlink l6 0 2024-06-15T01:06:52.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/0: sync 2024-06-15T01:06:52.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/1: rename - no filename 2024-06-15T01:06:52.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/2: link - no file 2024-06-15T01:06:52.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/3: dwrite - no filename 2024-06-15T01:06:52.744 INFO:tasks.workunit.client.1.smithi142.stdout:7/4: link - no file 2024-06-15T01:06:52.745 INFO:tasks.workunit.client.0.smithi137.stdout:0/27: write d1/d2/f4 [198017,124313] 0 2024-06-15T01:06:52.745 INFO:tasks.workunit.client.0.smithi137.stdout:6/14: rmdir d0 39 2024-06-15T01:06:52.746 INFO:tasks.workunit.client.1.smithi142.stdout:7/5: symlink l0 0 2024-06-15T01:06:52.746 INFO:tasks.workunit.client.1.smithi142.stdout:7/6: stat l0 0 2024-06-15T01:06:52.750 INFO:tasks.workunit.client.1.smithi142.stdout:7/7: creat f1 x:0 0 0 2024-06-15T01:06:52.750 INFO:tasks.workunit.client.0.smithi137.stdout:9/23: mknod c7 0 2024-06-15T01:06:52.752 INFO:tasks.workunit.client.0.smithi137.stdout:7/30: dwrite f3 [0,4194304] 0 2024-06-15T01:06:52.753 INFO:tasks.workunit.client.0.smithi137.stdout:2/36: dwrite d0/f8 [0,4194304] 0 2024-06-15T01:06:52.753 INFO:tasks.workunit.client.0.smithi137.stdout:4/25: creat f9 x:0 0 0 2024-06-15T01:06:52.754 INFO:tasks.workunit.client.0.smithi137.stdout:7/31: fdatasync f1 0 2024-06-15T01:06:52.755 INFO:tasks.workunit.client.0.smithi137.stdout:7/32: read f3 [3666646,54828] 0 2024-06-15T01:06:52.757 INFO:tasks.workunit.client.0.smithi137.stdout:1/149: dread d0/d3/f13 [0,4194304] 0 2024-06-15T01:06:52.757 INFO:tasks.workunit.client.0.smithi137.stdout:2/37: truncate d0/f6 778878 0 2024-06-15T01:06:52.757 INFO:tasks.workunit.client.0.smithi137.stdout:2/38: stat d0 0 2024-06-15T01:06:52.759 INFO:tasks.workunit.client.0.smithi137.stdout:5/105: creat df/f2b x:0 0 0 2024-06-15T01:06:52.761 INFO:tasks.workunit.client.0.smithi137.stdout:0/28: symlink d1/d2/l7 0 2024-06-15T01:06:52.762 INFO:tasks.workunit.client.0.smithi137.stdout:9/24: creat f8 x:0 0 0 2024-06-15T01:06:52.764 INFO:tasks.workunit.client.0.smithi137.stdout:9/25: fdatasync f5 0 2024-06-15T01:06:52.766 INFO:tasks.workunit.client.0.smithi137.stdout:4/26: creat fa x:0 0 0 2024-06-15T01:06:52.767 INFO:tasks.workunit.client.0.smithi137.stdout:7/33: dread f3 [0,4194304] 0 2024-06-15T01:06:52.767 INFO:tasks.workunit.client.0.smithi137.stdout:2/39: mknod d0/cd 0 2024-06-15T01:06:52.768 INFO:tasks.workunit.client.1.smithi142.stdout:2/22: truncate f3 728186 0 2024-06-15T01:06:52.769 INFO:tasks.workunit.client.0.smithi137.stdout:9/26: write f2 [3896374,84419] 0 2024-06-15T01:06:52.770 INFO:tasks.workunit.client.0.smithi137.stdout:7/34: stat c6 0 2024-06-15T01:06:52.771 INFO:tasks.workunit.client.1.smithi142.stdout:7/8: getdents . 0 2024-06-15T01:06:52.771 INFO:tasks.workunit.client.0.smithi137.stdout:9/27: write f5 [1645606,66964] 0 2024-06-15T01:06:52.771 INFO:tasks.workunit.client.1.smithi142.stdout:7/9: write f1 [659836,7869] 0 2024-06-15T01:06:52.772 INFO:tasks.workunit.client.1.smithi142.stdout:7/10: write f1 [1659214,21465] 0 2024-06-15T01:06:52.775 INFO:tasks.workunit.client.1.smithi142.stdout:7/11: unlink f1 0 2024-06-15T01:06:52.775 INFO:tasks.workunit.client.1.smithi142.stdout:7/12: read - no filename 2024-06-15T01:06:52.776 INFO:tasks.workunit.client.1.smithi142.stdout:2/23: write f3 [153106,89124] 0 2024-06-15T01:06:52.777 INFO:tasks.workunit.client.0.smithi137.stdout:2/40: mkdir d0/de 0 2024-06-15T01:06:52.778 INFO:tasks.workunit.client.1.smithi142.stdout:2/24: dread f3 [0,4194304] 0 2024-06-15T01:06:52.778 INFO:tasks.workunit.client.1.smithi142.stdout:2/25: chown f3 2547375 1 2024-06-15T01:06:52.779 INFO:tasks.workunit.client.1.smithi142.stdout:7/13: mkdir d2 0 2024-06-15T01:06:52.781 INFO:tasks.workunit.client.0.smithi137.stdout:9/28: rename c7 to c9 0 2024-06-15T01:06:52.781 INFO:tasks.workunit.client.1.smithi142.stdout:2/26: creat f4 x:0 0 0 2024-06-15T01:06:52.781 INFO:tasks.workunit.client.0.smithi137.stdout:5/106: dwrite fd [0,4194304] 0 2024-06-15T01:06:52.783 INFO:tasks.workunit.client.1.smithi142.stdout:2/27: write f3 [1516313,114504] 0 2024-06-15T01:06:52.784 INFO:tasks.workunit.client.0.smithi137.stdout:2/41: write d0/f9 [915045,56029] 0 2024-06-15T01:06:52.785 INFO:tasks.workunit.client.1.smithi142.stdout:7/14: creat d2/f3 x:0 0 0 2024-06-15T01:06:52.785 INFO:tasks.workunit.client.0.smithi137.stdout:9/29: write f2 [2269520,83065] 0 2024-06-15T01:06:52.787 INFO:tasks.workunit.client.0.smithi137.stdout:9/30: chown f3 410 1 2024-06-15T01:06:52.788 INFO:tasks.workunit.client.1.smithi142.stdout:7/15: symlink d2/l4 0 2024-06-15T01:06:52.789 INFO:tasks.workunit.client.0.smithi137.stdout:1/150: dwrite d0/d3/d14/f19 [0,4194304] 0 2024-06-15T01:06:52.789 INFO:tasks.workunit.client.1.smithi142.stdout:7/16: readlink d2/l4 0 2024-06-15T01:06:52.790 INFO:tasks.workunit.client.1.smithi142.stdout:7/17: rename d2 to d2/d5 22 2024-06-15T01:06:52.790 INFO:tasks.workunit.client.1.smithi142.stdout:2/28: dwrite f3 [0,4194304] 0 2024-06-15T01:06:52.793 INFO:tasks.workunit.client.0.smithi137.stdout:5/107: mknod df/d11/c2c 0 2024-06-15T01:06:52.794 INFO:tasks.workunit.client.1.smithi142.stdout:7/18: mkdir d2/d6 0 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:0/4: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:0/5: truncate - no filename 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:5/12: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:8/1: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:8/2: write - no filename 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:8/3: rename - no filename 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:6/10: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:4/153: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:1/61: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:3/49: sync 2024-06-15T01:06:52.795 INFO:tasks.workunit.client.1.smithi142.stdout:9/21: sync 2024-06-15T01:06:52.796 INFO:tasks.workunit.client.0.smithi137.stdout:7/35: dwrite f5 [0,4194304] 0 2024-06-15T01:06:52.796 INFO:tasks.workunit.client.1.smithi142.stdout:9/22: rename d6 to d6/da 22 2024-06-15T01:06:52.796 INFO:tasks.workunit.client.1.smithi142.stdout:3/50: chown d0/d3/l5 32160 1 2024-06-15T01:06:52.799 INFO:tasks.workunit.client.1.smithi142.stdout:3/51: dread - d0/d3/da/f11 zero size 2024-06-15T01:06:52.801 INFO:tasks.workunit.client.1.smithi142.stdout:0/6: creat f1 x:0 0 0 2024-06-15T01:06:52.806 INFO:tasks.workunit.client.0.smithi137.stdout:2/42: rename d0/f4 to d0/da/ff 0 2024-06-15T01:06:52.806 INFO:tasks.workunit.client.0.smithi137.stdout:1/151: mknod d0/d3/d14/d34/c3d 0 2024-06-15T01:06:52.806 INFO:tasks.workunit.client.0.smithi137.stdout:9/31: dwrite f5 [0,4194304] 0 2024-06-15T01:06:52.807 INFO:tasks.workunit.client.0.smithi137.stdout:7/36: creat f9 x:0 0 0 2024-06-15T01:06:52.813 INFO:tasks.workunit.client.0.smithi137.stdout:2/43: mkdir d0/de/d10 0 2024-06-15T01:06:52.813 INFO:tasks.workunit.client.0.smithi137.stdout:2/44: stat d0/de 0 2024-06-15T01:06:52.815 INFO:tasks.workunit.client.0.smithi137.stdout:9/32: link f3 fa 0 2024-06-15T01:06:52.825 INFO:tasks.workunit.client.0.smithi137.stdout:9/33: dwrite f0 [4194304,4194304] 0 2024-06-15T01:06:52.826 INFO:tasks.workunit.client.0.smithi137.stdout:7/37: fdatasync f1 0 2024-06-15T01:06:52.826 INFO:tasks.workunit.client.0.smithi137.stdout:9/34: chown f0 0 1 2024-06-15T01:06:52.826 INFO:tasks.workunit.client.0.smithi137.stdout:7/38: chown f3 162 1 2024-06-15T01:06:52.827 INFO:tasks.workunit.client.0.smithi137.stdout:7/39: chown f5 16541742 1 2024-06-15T01:06:52.829 INFO:tasks.workunit.client.0.smithi137.stdout:9/35: creat fb x:0 0 0 2024-06-15T01:06:52.832 INFO:tasks.workunit.client.0.smithi137.stdout:9/36: rename fb to fc 0 2024-06-15T01:06:52.833 INFO:tasks.workunit.client.0.smithi137.stdout:7/40: link c8 ca 0 2024-06-15T01:06:52.840 INFO:tasks.workunit.client.0.smithi137.stdout:9/37: dwrite f2 [0,4194304] 0 2024-06-15T01:06:52.840 INFO:tasks.workunit.client.0.smithi137.stdout:9/38: rmdir - no directory 2024-06-15T01:06:52.843 INFO:tasks.workunit.client.1.smithi142.stdout:5/13: mkdir d2 0 2024-06-15T01:06:52.843 INFO:tasks.workunit.client.1.smithi142.stdout:9/23: rename d6/l8 to d6/lb 0 2024-06-15T01:06:52.844 INFO:tasks.workunit.client.1.smithi142.stdout:4/154: symlink d5/df/d15/l39 0 2024-06-15T01:06:52.845 INFO:tasks.workunit.client.1.smithi142.stdout:5/14: rename d2 to d2/d3 22 2024-06-15T01:06:52.845 INFO:tasks.workunit.client.1.smithi142.stdout:2/29: link l1 l5 0 2024-06-15T01:06:52.848 INFO:tasks.workunit.client.1.smithi142.stdout:4/155: fsync d5/d9/f22 0 2024-06-15T01:06:52.850 INFO:tasks.workunit.client.0.smithi137.stdout:9/39: dwrite fc [0,4194304] 0 2024-06-15T01:06:52.850 INFO:tasks.workunit.client.0.smithi137.stdout:9/40: read fc [2452051,63988] 0 2024-06-15T01:06:52.852 INFO:tasks.workunit.client.0.smithi137.stdout:9/41: write f8 [259979,124868] 0 2024-06-15T01:06:52.901 INFO:tasks.workunit.client.1.smithi142.stdout:2/30: truncate f4 171941 0 2024-06-15T01:06:52.902 INFO:tasks.workunit.client.1.smithi142.stdout:7/19: dwrite d2/f3 [0,4194304] 0 2024-06-15T01:06:52.902 INFO:tasks.workunit.client.1.smithi142.stdout:8/4: mkdir d0 0 2024-06-15T01:06:52.902 INFO:tasks.workunit.client.1.smithi142.stdout:8/5: truncate - no filename 2024-06-15T01:06:52.902 INFO:tasks.workunit.client.1.smithi142.stdout:1/62: dwrite d6/fc [0,4194304] 0 2024-06-15T01:06:52.903 INFO:tasks.workunit.client.1.smithi142.stdout:9/24: mkdir d6/dc 0 2024-06-15T01:06:52.908 INFO:tasks.workunit.client.1.smithi142.stdout:2/31: chown l5 7113 1 2024-06-15T01:06:52.908 INFO:tasks.workunit.client.1.smithi142.stdout:5/15: creat d2/f4 x:0 0 0 2024-06-15T01:06:52.910 INFO:tasks.workunit.client.1.smithi142.stdout:1/63: rename d6/l11 to d6/l18 0 2024-06-15T01:06:52.911 INFO:tasks.workunit.client.1.smithi142.stdout:2/32: write f3 [1840915,125218] 0 2024-06-15T01:06:52.912 INFO:tasks.workunit.client.1.smithi142.stdout:8/6: mkdir d0/d1 0 2024-06-15T01:06:52.912 INFO:tasks.workunit.client.1.smithi142.stdout:8/7: link - no file 2024-06-15T01:06:52.912 INFO:tasks.workunit.client.1.smithi142.stdout:8/8: write - no filename 2024-06-15T01:06:52.913 INFO:tasks.workunit.client.1.smithi142.stdout:9/25: mkdir d6/dc/dd 0 2024-06-15T01:06:52.913 INFO:tasks.workunit.client.1.smithi142.stdout:3/52: dwrite d0/d3/da/f11 [0,4194304] 0 2024-06-15T01:06:52.916 INFO:tasks.workunit.client.1.smithi142.stdout:7/20: link d2/f3 d2/d6/f7 0 2024-06-15T01:06:52.920 INFO:tasks.workunit.client.1.smithi142.stdout:9/26: chown c4 252 1 2024-06-15T01:06:52.923 INFO:tasks.workunit.client.1.smithi142.stdout:3/53: mknod d0/d3/da/c14 0 2024-06-15T01:06:52.924 INFO:tasks.workunit.client.1.smithi142.stdout:4/156: link l0 d5/d9/d1a/l3a 0 2024-06-15T01:06:52.925 INFO:tasks.workunit.client.1.smithi142.stdout:5/16: rename d2/f4 to d2/f5 0 2024-06-15T01:06:52.926 INFO:tasks.workunit.client.1.smithi142.stdout:3/54: dread d0/d3/da/fc [0,4194304] 0 2024-06-15T01:06:52.927 INFO:tasks.workunit.client.1.smithi142.stdout:8/9: mknod d0/d1/c2 0 2024-06-15T01:06:52.927 INFO:tasks.workunit.client.1.smithi142.stdout:3/55: write d0/d3/da/f11 [4956971,13416] 0 2024-06-15T01:06:52.929 INFO:tasks.workunit.client.1.smithi142.stdout:3/56: chown d0/d3/da/c10 68441071 1 2024-06-15T01:06:52.931 INFO:tasks.workunit.client.1.smithi142.stdout:7/21: fsync d2/f3 0 2024-06-15T01:06:52.933 INFO:tasks.workunit.client.1.smithi142.stdout:5/17: creat d2/f6 x:0 0 0 2024-06-15T01:06:52.934 INFO:tasks.workunit.client.1.smithi142.stdout:9/27: symlink d6/dc/dd/le 0 2024-06-15T01:06:52.934 INFO:tasks.workunit.client.1.smithi142.stdout:4/157: dwrite d5/d9/f22 [0,4194304] 0 2024-06-15T01:06:52.936 INFO:tasks.workunit.client.1.smithi142.stdout:8/10: mknod d0/c3 0 2024-06-15T01:06:52.936 INFO:tasks.workunit.client.1.smithi142.stdout:8/11: dwrite - no filename 2024-06-15T01:06:52.936 INFO:tasks.workunit.client.1.smithi142.stdout:8/12: write - no filename 2024-06-15T01:06:52.939 INFO:tasks.workunit.client.1.smithi142.stdout:3/57: dwrite d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:52.940 INFO:tasks.workunit.client.1.smithi142.stdout:3/58: readlink d0/d3/l8 0 2024-06-15T01:06:52.940 INFO:tasks.workunit.client.1.smithi142.stdout:7/22: mkdir d2/d8 0 2024-06-15T01:06:52.942 INFO:tasks.workunit.client.1.smithi142.stdout:5/18: creat d2/f7 x:0 0 0 2024-06-15T01:06:52.943 INFO:tasks.workunit.client.1.smithi142.stdout:5/19: truncate d2/f6 447728 0 2024-06-15T01:06:52.943 INFO:tasks.workunit.client.1.smithi142.stdout:1/64: truncate d6/f9 3190046 0 2024-06-15T01:06:52.943 INFO:tasks.workunit.client.1.smithi142.stdout:5/20: dread - d2/f7 zero size 2024-06-15T01:06:52.944 INFO:tasks.workunit.client.1.smithi142.stdout:1/65: write d6/f13 [2123529,80511] 0 2024-06-15T01:06:52.945 INFO:tasks.workunit.client.1.smithi142.stdout:5/21: truncate d2/f7 333143 0 2024-06-15T01:06:52.945 INFO:tasks.workunit.client.1.smithi142.stdout:5/22: chown d2 354 1 2024-06-15T01:06:52.949 INFO:tasks.workunit.client.1.smithi142.stdout:8/13: creat d0/d1/f4 x:0 0 0 2024-06-15T01:06:52.951 INFO:tasks.workunit.client.1.smithi142.stdout:3/59: dread d0/d3/da/fc [0,4194304] 0 2024-06-15T01:06:52.953 INFO:tasks.workunit.client.1.smithi142.stdout:8/14: write d0/d1/f4 [96182,58085] 0 2024-06-15T01:06:52.953 INFO:tasks.workunit.client.1.smithi142.stdout:1/66: dread d6/fc [0,4194304] 0 2024-06-15T01:06:52.955 INFO:tasks.workunit.client.1.smithi142.stdout:8/15: stat d0/c3 0 2024-06-15T01:06:52.955 INFO:tasks.workunit.client.1.smithi142.stdout:3/60: write d0/d3/da/fe [358774,10790] 0 2024-06-15T01:06:52.956 INFO:tasks.workunit.client.1.smithi142.stdout:7/23: creat d2/d6/f9 x:0 0 0 2024-06-15T01:06:52.956 INFO:tasks.workunit.client.1.smithi142.stdout:1/67: truncate f5 4957878 0 2024-06-15T01:06:52.957 INFO:tasks.workunit.client.1.smithi142.stdout:5/23: dwrite d2/f6 [0,4194304] 0 2024-06-15T01:06:52.957 INFO:tasks.workunit.client.1.smithi142.stdout:7/24: chown d2/d8 46830 1 2024-06-15T01:06:52.958 INFO:tasks.workunit.client.1.smithi142.stdout:1/68: truncate d6/fc 5169401 0 2024-06-15T01:06:52.959 INFO:tasks.workunit.client.1.smithi142.stdout:8/16: creat d0/f5 x:0 0 0 2024-06-15T01:06:52.960 INFO:tasks.workunit.client.1.smithi142.stdout:7/25: truncate d2/d6/f9 933802 0 2024-06-15T01:06:52.960 INFO:tasks.workunit.client.1.smithi142.stdout:3/61: mkdir d0/d3/d15 0 2024-06-15T01:06:52.960 INFO:tasks.workunit.client.1.smithi142.stdout:7/26: chown d2/f3 0 1 2024-06-15T01:06:52.961 INFO:tasks.workunit.client.1.smithi142.stdout:1/69: symlink d6/l19 0 2024-06-15T01:06:52.963 INFO:tasks.workunit.client.1.smithi142.stdout:3/62: mknod d0/d3/c16 0 2024-06-15T01:06:52.967 INFO:tasks.workunit.client.1.smithi142.stdout:5/24: rename l1 to d2/l8 0 2024-06-15T01:06:52.967 INFO:tasks.workunit.client.1.smithi142.stdout:8/17: dwrite d0/d1/f4 [0,4194304] 0 2024-06-15T01:06:52.968 INFO:tasks.workunit.client.1.smithi142.stdout:8/18: truncate d0/f5 234618 0 2024-06-15T01:06:52.969 INFO:tasks.workunit.client.1.smithi142.stdout:8/19: fdatasync d0/f5 0 2024-06-15T01:06:52.970 INFO:tasks.workunit.client.1.smithi142.stdout:7/27: mkdir d2/d8/da 0 2024-06-15T01:06:52.971 INFO:tasks.workunit.client.1.smithi142.stdout:1/70: write d6/fc [3229593,23147] 0 2024-06-15T01:06:52.971 INFO:tasks.workunit.client.1.smithi142.stdout:7/28: fsync d2/d6/f7 0 2024-06-15T01:06:52.973 INFO:tasks.workunit.client.1.smithi142.stdout:3/63: readlink d0/d3/l4 0 2024-06-15T01:06:52.975 INFO:tasks.workunit.client.1.smithi142.stdout:8/20: mknod d0/d1/c6 0 2024-06-15T01:06:52.975 INFO:tasks.workunit.client.1.smithi142.stdout:7/29: write d2/d6/f7 [3283084,113078] 0 2024-06-15T01:06:52.975 INFO:tasks.workunit.client.1.smithi142.stdout:1/71: chown d6/ff 48716 1 2024-06-15T01:06:52.977 INFO:tasks.workunit.client.1.smithi142.stdout:7/30: chown l0 127644024 1 2024-06-15T01:06:52.978 INFO:tasks.workunit.client.1.smithi142.stdout:8/21: chown d0/d1/c2 0 1 2024-06-15T01:06:52.978 INFO:tasks.workunit.client.1.smithi142.stdout:7/31: readlink l0 0 2024-06-15T01:06:52.978 INFO:tasks.workunit.client.1.smithi142.stdout:7/32: readlink d2/l4 0 2024-06-15T01:06:52.978 INFO:tasks.workunit.client.1.smithi142.stdout:8/22: stat d0/d1/f4 0 2024-06-15T01:06:52.979 INFO:tasks.workunit.client.1.smithi142.stdout:4/158: dwrite d5/f6 [0,4194304] 0 2024-06-15T01:06:52.980 INFO:tasks.workunit.client.1.smithi142.stdout:7/33: read d2/d6/f7 [2780558,96019] 0 2024-06-15T01:06:52.980 INFO:tasks.workunit.client.1.smithi142.stdout:3/64: rename d0/cf to d0/d3/da/c17 0 2024-06-15T01:06:52.980 INFO:tasks.workunit.client.1.smithi142.stdout:7/34: fsync d2/d6/f9 0 2024-06-15T01:06:52.982 INFO:tasks.workunit.client.1.smithi142.stdout:1/72: creat d6/d12/f1a x:0 0 0 2024-06-15T01:06:52.983 INFO:tasks.workunit.client.1.smithi142.stdout:3/65: write d0/d3/da/f11 [4792582,40071] 0 2024-06-15T01:06:52.984 INFO:tasks.workunit.client.1.smithi142.stdout:8/23: creat d0/d1/f7 x:0 0 0 2024-06-15T01:06:52.985 INFO:tasks.workunit.client.1.smithi142.stdout:1/73: chown d6/cb 2649643 1 2024-06-15T01:06:52.988 INFO:tasks.workunit.client.1.smithi142.stdout:8/24: read d0/d1/f4 [3866851,29849] 0 2024-06-15T01:06:52.988 INFO:tasks.workunit.client.1.smithi142.stdout:3/66: fsync d0/d3/da/f11 0 2024-06-15T01:06:52.991 INFO:tasks.workunit.client.1.smithi142.stdout:4/159: dwrite d5/f10 [4194304,4194304] 0 2024-06-15T01:06:52.994 INFO:tasks.workunit.client.1.smithi142.stdout:3/67: mkdir d0/d3/da/d18 0 2024-06-15T01:06:52.997 INFO:tasks.workunit.client.1.smithi142.stdout:4/160: creat d5/d9/d23/f3b x:0 0 0 2024-06-15T01:06:53.000 INFO:tasks.workunit.client.1.smithi142.stdout:7/35: dwrite d2/d6/f9 [0,4194304] 0 2024-06-15T01:06:53.001 INFO:tasks.workunit.client.1.smithi142.stdout:4/161: readlink d5/df/d15/l39 0 2024-06-15T01:06:53.004 INFO:tasks.workunit.client.1.smithi142.stdout:1/74: dwrite f5 [4194304,4194304] 0 2024-06-15T01:06:53.007 INFO:tasks.workunit.client.1.smithi142.stdout:8/25: dwrite d0/f5 [0,4194304] 0 2024-06-15T01:06:53.010 INFO:tasks.workunit.client.1.smithi142.stdout:8/26: truncate d0/d1/f7 449848 0 2024-06-15T01:06:53.012 INFO:tasks.workunit.client.0.smithi137.stdout:3/10: sync 2024-06-15T01:06:53.012 INFO:tasks.workunit.client.0.smithi137.stdout:3/11: truncate - no filename 2024-06-15T01:06:53.012 INFO:tasks.workunit.client.0.smithi137.stdout:3/12: readlink - no filename 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/13: chown d0 39887 1 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/14: dwrite - no filename 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/15: read - no filename 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/16: dwrite - no filename 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/17: fdatasync - no filename 2024-06-15T01:06:53.013 INFO:tasks.workunit.client.0.smithi137.stdout:3/18: rename d0 to d0/d1 22 2024-06-15T01:06:53.014 INFO:tasks.workunit.client.0.smithi137.stdout:3/19: dwrite - no filename 2024-06-15T01:06:53.014 INFO:tasks.workunit.client.0.smithi137.stdout:3/20: dwrite - no filename 2024-06-15T01:06:53.014 INFO:tasks.workunit.client.0.smithi137.stdout:3/21: chown d0 4094631 1 2024-06-15T01:06:53.014 INFO:tasks.workunit.client.1.smithi142.stdout:8/27: dread d0/f5 [0,4194304] 0 2024-06-15T01:06:53.018 INFO:tasks.workunit.client.1.smithi142.stdout:7/36: creat d2/fb x:0 0 0 2024-06-15T01:06:53.019 INFO:tasks.workunit.client.1.smithi142.stdout:4/162: creat d5/d9/da/f3c x:0 0 0 2024-06-15T01:06:53.019 INFO:tasks.workunit.client.1.smithi142.stdout:1/75: readlink d6/l18 0 2024-06-15T01:06:53.020 INFO:tasks.workunit.client.1.smithi142.stdout:1/76: chown c2 0 1 2024-06-15T01:06:53.020 INFO:tasks.workunit.client.1.smithi142.stdout:1/77: readlink d6/l19 0 2024-06-15T01:06:53.022 INFO:tasks.workunit.client.0.smithi137.stdout:3/22: getdents d0 0 2024-06-15T01:06:53.023 INFO:tasks.workunit.client.1.smithi142.stdout:7/37: mknod d2/cc 0 2024-06-15T01:06:53.024 INFO:tasks.workunit.client.1.smithi142.stdout:4/163: fsync d5/f6 0 2024-06-15T01:06:53.025 INFO:tasks.workunit.client.0.smithi137.stdout:3/23: mkdir d0/d2 0 2024-06-15T01:06:53.027 INFO:tasks.workunit.client.1.smithi142.stdout:7/38: mkdir d2/d8/dd 0 2024-06-15T01:06:53.027 INFO:tasks.workunit.client.0.smithi137.stdout:3/24: mknod d0/d2/c3 0 2024-06-15T01:06:53.028 INFO:tasks.workunit.client.1.smithi142.stdout:1/78: dwrite d6/f10 [0,4194304] 0 2024-06-15T01:06:53.029 INFO:tasks.workunit.client.0.smithi137.stdout:3/25: creat d0/f4 x:0 0 0 2024-06-15T01:06:53.030 INFO:tasks.workunit.client.1.smithi142.stdout:4/164: creat d5/d1c/d2a/f3d x:0 0 0 2024-06-15T01:06:53.031 INFO:tasks.workunit.client.1.smithi142.stdout:1/79: dread - d6/d12/f1a zero size 2024-06-15T01:06:53.034 INFO:tasks.workunit.client.1.smithi142.stdout:8/28: dwrite d0/d1/f4 [0,4194304] 0 2024-06-15T01:06:53.034 INFO:tasks.workunit.client.1.smithi142.stdout:7/39: write d2/f3 [2813942,66514] 0 2024-06-15T01:06:53.038 INFO:tasks.workunit.client.1.smithi142.stdout:3/68: rename d0/d3/da/c17 to d0/d3/d15/c19 0 2024-06-15T01:06:53.038 INFO:tasks.workunit.client.1.smithi142.stdout:4/165: rename d5/df/d12 to d5/df/d12/d3e 22 2024-06-15T01:06:53.039 INFO:tasks.workunit.client.1.smithi142.stdout:1/80: mknod d6/d12/c1b 0 2024-06-15T01:06:53.040 INFO:tasks.workunit.client.0.smithi137.stdout:3/26: dwrite d0/f4 [0,4194304] 0 2024-06-15T01:06:53.042 INFO:tasks.workunit.client.1.smithi142.stdout:7/40: creat d2/d8/fe x:0 0 0 2024-06-15T01:06:53.043 INFO:tasks.workunit.client.1.smithi142.stdout:7/41: write d2/d8/fe [130712,119307] 0 2024-06-15T01:06:53.044 INFO:tasks.workunit.client.1.smithi142.stdout:7/42: write d2/d8/fe [445816,106788] 0 2024-06-15T01:06:53.047 INFO:tasks.workunit.client.0.smithi137.stdout:3/27: link d0/f4 d0/d2/f5 0 2024-06-15T01:06:53.048 INFO:tasks.workunit.client.1.smithi142.stdout:3/69: unlink d0/d3/da/fc 0 2024-06-15T01:06:53.049 INFO:tasks.workunit.client.1.smithi142.stdout:3/70: chown d0/d3/da/c14 4579643 1 2024-06-15T01:06:53.049 INFO:tasks.workunit.client.1.smithi142.stdout:3/71: readlink d0/d3/l8 0 2024-06-15T01:06:53.049 INFO:tasks.workunit.client.0.smithi137.stdout:3/28: mknod d0/d2/c6 0 2024-06-15T01:06:53.050 INFO:tasks.workunit.client.1.smithi142.stdout:3/72: write d0/d3/da/fe [946535,88402] 0 2024-06-15T01:06:53.054 INFO:tasks.workunit.client.0.smithi137.stdout:3/29: dread d0/d2/f5 [0,4194304] 0 2024-06-15T01:06:53.056 INFO:tasks.workunit.client.0.smithi137.stdout:3/30: symlink d0/d2/l7 0 2024-06-15T01:06:53.056 INFO:tasks.workunit.client.1.smithi142.stdout:7/43: creat d2/ff x:0 0 0 2024-06-15T01:06:53.058 INFO:tasks.workunit.client.1.smithi142.stdout:3/73: creat d0/d3/f1a x:0 0 0 2024-06-15T01:06:53.059 INFO:tasks.workunit.client.1.smithi142.stdout:3/74: dread - d0/d3/f1a zero size 2024-06-15T01:06:53.059 INFO:tasks.workunit.client.1.smithi142.stdout:4/166: creat d5/d1c/d2a/d30/f3f x:0 0 0 2024-06-15T01:06:53.064 INFO:tasks.workunit.client.1.smithi142.stdout:8/29: getdents d0/d1 0 2024-06-15T01:06:53.067 INFO:tasks.workunit.client.1.smithi142.stdout:3/75: mkdir d0/d1b 0 2024-06-15T01:06:53.068 INFO:tasks.workunit.client.1.smithi142.stdout:3/76: dread - d0/d3/f1a zero size 2024-06-15T01:06:53.068 INFO:tasks.workunit.client.1.smithi142.stdout:4/167: getdents d5/d9/d23/d32 0 2024-06-15T01:06:53.069 INFO:tasks.workunit.client.1.smithi142.stdout:1/81: rmdir d6 39 2024-06-15T01:06:53.076 INFO:tasks.workunit.client.1.smithi142.stdout:8/30: link d0/d1/f4 d0/f8 0 2024-06-15T01:06:53.079 INFO:tasks.workunit.client.1.smithi142.stdout:3/77: truncate d0/d3/da/f11 3254474 0 2024-06-15T01:06:53.079 INFO:tasks.workunit.client.1.smithi142.stdout:3/78: chown d0/d3/da/cb 669365 1 2024-06-15T01:06:53.080 INFO:tasks.workunit.client.1.smithi142.stdout:7/44: truncate d2/d8/fe 272480 0 2024-06-15T01:06:53.081 INFO:tasks.workunit.client.1.smithi142.stdout:1/82: mkdir d6/d12/d1c 0 2024-06-15T01:06:53.081 INFO:tasks.workunit.client.1.smithi142.stdout:1/83: chown d6/d12/f1a 497 1 2024-06-15T01:06:53.082 INFO:tasks.workunit.client.1.smithi142.stdout:1/84: chown d6/ff 8128804 1 2024-06-15T01:06:53.082 INFO:tasks.workunit.client.1.smithi142.stdout:1/85: stat d6/d12 0 2024-06-15T01:06:53.082 INFO:tasks.workunit.client.1.smithi142.stdout:4/168: creat d5/d9/da/d2e/f40 x:0 0 0 2024-06-15T01:06:53.084 INFO:tasks.workunit.client.1.smithi142.stdout:8/31: mknod d0/c9 0 2024-06-15T01:06:53.087 INFO:tasks.workunit.client.1.smithi142.stdout:1/86: creat d6/f1d x:0 0 0 2024-06-15T01:06:53.088 INFO:tasks.workunit.client.1.smithi142.stdout:3/79: mknod d0/d1b/c1c 0 2024-06-15T01:06:53.089 INFO:tasks.workunit.client.1.smithi142.stdout:3/80: chown d0/d3/da/cb 1123277 1 2024-06-15T01:06:53.095 INFO:tasks.workunit.client.1.smithi142.stdout:1/87: dwrite d6/d12/f1a [0,4194304] 0 2024-06-15T01:06:53.099 INFO:tasks.workunit.client.1.smithi142.stdout:3/81: symlink d0/l1d 0 2024-06-15T01:06:53.099 INFO:tasks.workunit.client.1.smithi142.stdout:7/45: dread d2/d8/fe [0,4194304] 0 2024-06-15T01:06:53.100 INFO:tasks.workunit.client.1.smithi142.stdout:4/169: rename d5/d9/da/d2e to d5/d9/d41 0 2024-06-15T01:06:53.100 INFO:tasks.workunit.client.1.smithi142.stdout:0/7: sync 2024-06-15T01:06:53.101 INFO:tasks.workunit.client.1.smithi142.stdout:6/11: sync 2024-06-15T01:06:53.103 INFO:tasks.workunit.client.1.smithi142.stdout:1/88: unlink d6/d12/f1a 0 2024-06-15T01:06:53.104 INFO:tasks.workunit.client.1.smithi142.stdout:1/89: chown d6/d12/c1b 149 1 2024-06-15T01:06:53.105 INFO:tasks.workunit.client.1.smithi142.stdout:6/12: dread - f2 zero size 2024-06-15T01:06:53.106 INFO:tasks.workunit.client.1.smithi142.stdout:0/8: mknod c2 0 2024-06-15T01:06:53.107 INFO:tasks.workunit.client.1.smithi142.stdout:0/9: truncate f1 1029835 0 2024-06-15T01:06:53.108 INFO:tasks.workunit.client.1.smithi142.stdout:8/32: rename d0/d1/f7 to d0/fa 0 2024-06-15T01:06:53.109 INFO:tasks.workunit.client.1.smithi142.stdout:4/170: dwrite d5/df/d2d/f35 [0,4194304] 0 2024-06-15T01:06:53.111 INFO:tasks.workunit.client.1.smithi142.stdout:1/90: unlink d6/ff 0 2024-06-15T01:06:53.111 INFO:tasks.workunit.client.1.smithi142.stdout:0/10: dread f1 [0,4194304] 0 2024-06-15T01:06:53.114 INFO:tasks.workunit.client.1.smithi142.stdout:0/11: dread f1 [0,4194304] 0 2024-06-15T01:06:53.114 INFO:tasks.workunit.client.1.smithi142.stdout:6/13: rename f5 to f6 0 2024-06-15T01:06:53.118 INFO:tasks.workunit.client.1.smithi142.stdout:7/46: creat d2/d8/dd/f10 x:0 0 0 2024-06-15T01:06:53.119 INFO:tasks.workunit.client.1.smithi142.stdout:8/33: write d0/f5 [2518454,45663] 0 2024-06-15T01:06:53.121 INFO:tasks.workunit.client.0.smithi137.stdout:0/29: getdents d1 0 2024-06-15T01:06:53.121 INFO:tasks.workunit.client.0.smithi137.stdout:0/30: readlink d1/l5 0 2024-06-15T01:06:53.122 INFO:tasks.workunit.client.1.smithi142.stdout:8/34: dread d0/d1/f4 [0,4194304] 0 2024-06-15T01:06:53.123 INFO:tasks.workunit.client.0.smithi137.stdout:6/15: creat d0/f2 x:0 0 0 2024-06-15T01:06:53.124 INFO:tasks.workunit.client.0.smithi137.stdout:6/16: rename d0 to d0/d3 22 2024-06-15T01:06:53.125 INFO:tasks.workunit.client.0.smithi137.stdout:6/17: chown d0/f2 12433421 1 2024-06-15T01:06:53.126 INFO:tasks.workunit.client.0.smithi137.stdout:6/18: truncate d0/f2 980959 0 2024-06-15T01:06:53.126 INFO:tasks.workunit.client.1.smithi142.stdout:4/171: mknod d5/d9/d41/c42 0 2024-06-15T01:06:53.127 INFO:tasks.workunit.client.0.smithi137.stdout:6/19: chown d0 0 1 2024-06-15T01:06:53.127 INFO:tasks.workunit.client.0.smithi137.stdout:0/31: dread d1/d2/f4 [0,4194304] 0 2024-06-15T01:06:53.128 INFO:tasks.workunit.client.1.smithi142.stdout:1/91: chown d6/c14 7437102 1 2024-06-15T01:06:53.128 INFO:tasks.workunit.client.0.smithi137.stdout:0/32: truncate d1/d2/f6 927875 0 2024-06-15T01:06:53.129 INFO:tasks.workunit.client.1.smithi142.stdout:0/12: mkdir d3 0 2024-06-15T01:06:53.129 INFO:tasks.workunit.client.0.smithi137.stdout:6/20: creat d0/f4 x:0 0 0 2024-06-15T01:06:53.130 INFO:tasks.workunit.client.0.smithi137.stdout:6/21: truncate d0/f2 1525394 0 2024-06-15T01:06:53.130 INFO:tasks.workunit.client.1.smithi142.stdout:6/14: creat f7 x:0 0 0 2024-06-15T01:06:53.130 INFO:tasks.workunit.client.1.smithi142.stdout:4/172: dread d5/f8 [0,4194304] 0 2024-06-15T01:06:53.131 INFO:tasks.workunit.client.0.smithi137.stdout:0/33: mknod d1/c8 0 2024-06-15T01:06:53.133 INFO:tasks.workunit.client.1.smithi142.stdout:7/47: mknod d2/c11 0 2024-06-15T01:06:53.133 INFO:tasks.workunit.client.0.smithi137.stdout:6/22: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:53.136 INFO:tasks.workunit.client.1.smithi142.stdout:3/82: truncate d0/d3/da/fe 418658 0 2024-06-15T01:06:53.137 INFO:tasks.workunit.client.1.smithi142.stdout:3/83: chown d0/d3 16090 1 2024-06-15T01:06:53.137 INFO:tasks.workunit.client.0.smithi137.stdout:0/34: creat d1/f9 x:0 0 0 2024-06-15T01:06:53.137 INFO:tasks.workunit.client.0.smithi137.stdout:4/27: fsync f9 0 2024-06-15T01:06:53.138 INFO:tasks.workunit.client.1.smithi142.stdout:8/35: mknod d0/d1/cb 0 2024-06-15T01:06:53.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/28: fsync fa 0 2024-06-15T01:06:53.139 INFO:tasks.workunit.client.1.smithi142.stdout:2/33: sync 2024-06-15T01:06:53.139 INFO:tasks.workunit.client.1.smithi142.stdout:9/28: sync 2024-06-15T01:06:53.140 INFO:tasks.workunit.client.1.smithi142.stdout:5/25: sync 2024-06-15T01:06:53.140 INFO:tasks.workunit.client.1.smithi142.stdout:2/34: write f4 [187489,120363] 0 2024-06-15T01:06:53.141 INFO:tasks.workunit.client.0.smithi137.stdout:0/35: rename d1 to d1/da 22 2024-06-15T01:06:53.142 INFO:tasks.workunit.client.0.smithi137.stdout:8/31: dwrite f0 [4194304,4194304] 0 2024-06-15T01:06:53.143 INFO:tasks.workunit.client.1.smithi142.stdout:9/29: dread f2 [0,4194304] 0 2024-06-15T01:06:53.144 INFO:tasks.workunit.client.1.smithi142.stdout:4/173: rmdir d5/df/d15 39 2024-06-15T01:06:53.144 INFO:tasks.workunit.client.1.smithi142.stdout:9/30: read - f5 zero size 2024-06-15T01:06:53.144 INFO:tasks.workunit.client.0.smithi137.stdout:4/29: symlink lb 0 2024-06-15T01:06:53.145 INFO:tasks.workunit.client.0.smithi137.stdout:4/30: write f5 [3151944,66245] 0 2024-06-15T01:06:53.151 INFO:tasks.workunit.client.0.smithi137.stdout:4/31: mknod cc 0 2024-06-15T01:06:53.153 INFO:tasks.workunit.client.1.smithi142.stdout:8/36: truncate d0/f8 2695362 0 2024-06-15T01:06:53.153 INFO:tasks.workunit.client.1.smithi142.stdout:1/92: fsync d6/f9 0 2024-06-15T01:06:53.154 INFO:tasks.workunit.client.1.smithi142.stdout:0/13: creat d3/f4 x:0 0 0 2024-06-15T01:06:53.155 INFO:tasks.workunit.client.0.smithi137.stdout:0/36: dwrite d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:53.156 INFO:tasks.workunit.client.1.smithi142.stdout:6/15: write f0 [1023081,79556] 0 2024-06-15T01:06:53.157 INFO:tasks.workunit.client.1.smithi142.stdout:6/16: read - f7 zero size 2024-06-15T01:06:53.157 INFO:tasks.workunit.client.1.smithi142.stdout:5/26: creat d2/f9 x:0 0 0 2024-06-15T01:06:53.157 INFO:tasks.workunit.client.1.smithi142.stdout:6/17: dread - f2 zero size 2024-06-15T01:06:53.158 INFO:tasks.workunit.client.1.smithi142.stdout:5/27: dread - d2/f5 zero size 2024-06-15T01:06:53.159 INFO:tasks.workunit.client.1.smithi142.stdout:4/174: creat d5/df/d12/f43 x:0 0 0 2024-06-15T01:06:53.160 INFO:tasks.workunit.client.1.smithi142.stdout:9/31: mknod d6/dc/cf 0 2024-06-15T01:06:53.161 INFO:tasks.workunit.client.1.smithi142.stdout:7/48: link d2/d8/dd/f10 d2/d8/dd/f12 0 2024-06-15T01:06:53.167 INFO:tasks.workunit.client.1.smithi142.stdout:0/14: mkdir d3/d5 0 2024-06-15T01:06:53.167 INFO:tasks.workunit.client.1.smithi142.stdout:0/15: readlink - no filename 2024-06-15T01:06:53.169 INFO:tasks.workunit.client.1.smithi142.stdout:5/28: mknod d2/ca 0 2024-06-15T01:06:53.170 INFO:tasks.workunit.client.1.smithi142.stdout:4/175: mknod d5/d1c/c44 0 2024-06-15T01:06:53.171 INFO:tasks.workunit.client.1.smithi142.stdout:9/32: symlink d6/l10 0 2024-06-15T01:06:53.171 INFO:tasks.workunit.client.1.smithi142.stdout:7/49: mknod d2/d6/c13 0 2024-06-15T01:06:53.172 INFO:tasks.workunit.client.1.smithi142.stdout:1/93: mkdir d6/d12/d1c/d1e 0 2024-06-15T01:06:53.173 INFO:tasks.workunit.client.1.smithi142.stdout:0/16: symlink d3/l6 0 2024-06-15T01:06:53.173 INFO:tasks.workunit.client.1.smithi142.stdout:7/50: fsync d2/ff 0 2024-06-15T01:06:53.173 INFO:tasks.workunit.client.1.smithi142.stdout:8/37: dwrite d0/fa [0,4194304] 0 2024-06-15T01:06:53.174 INFO:tasks.workunit.client.1.smithi142.stdout:5/29: symlink d2/lb 0 2024-06-15T01:06:53.174 INFO:tasks.workunit.client.1.smithi142.stdout:5/30: stat d2/lb 0 2024-06-15T01:06:53.175 INFO:tasks.workunit.client.1.smithi142.stdout:5/31: dread - d2/f9 zero size 2024-06-15T01:06:53.175 INFO:tasks.workunit.client.1.smithi142.stdout:4/176: write d5/d9/f1f [557441,130700] 0 2024-06-15T01:06:53.177 INFO:tasks.workunit.client.1.smithi142.stdout:9/33: fdatasync d6/f9 0 2024-06-15T01:06:53.178 INFO:tasks.workunit.client.1.smithi142.stdout:2/35: getdents . 0 2024-06-15T01:06:53.180 INFO:tasks.workunit.client.1.smithi142.stdout:5/32: dread d2/f7 [0,4194304] 0 2024-06-15T01:06:53.180 INFO:tasks.workunit.client.1.smithi142.stdout:1/94: creat d6/d12/d1c/f1f x:0 0 0 2024-06-15T01:06:53.181 INFO:tasks.workunit.client.1.smithi142.stdout:4/177: symlink d5/d9/d41/l45 0 2024-06-15T01:06:53.245 INFO:tasks.workunit.client.0.smithi137.stdout:1/152: sync 2024-06-15T01:06:53.245 INFO:tasks.workunit.client.0.smithi137.stdout:2/45: unlink d0/da/ff 0 2024-06-15T01:06:53.247 INFO:tasks.workunit.client.0.smithi137.stdout:0/37: symlink d1/d2/lb 0 2024-06-15T01:06:53.247 INFO:tasks.workunit.client.0.smithi137.stdout:5/108: truncate f5 722643 0 2024-06-15T01:06:53.248 INFO:tasks.workunit.client.0.smithi137.stdout:7/41: truncate f1 721924 0 2024-06-15T01:06:53.249 INFO:tasks.workunit.client.0.smithi137.stdout:3/31: rmdir d0 39 2024-06-15T01:06:53.253 INFO:tasks.workunit.client.0.smithi137.stdout:2/46: creat d0/da/f11 x:0 0 0 2024-06-15T01:06:53.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/38: mkdir d1/d2/dc 0 2024-06-15T01:06:53.255 INFO:tasks.workunit.client.0.smithi137.stdout:0/39: chown d1/d2/lb 42929 1 2024-06-15T01:06:53.255 INFO:tasks.workunit.client.0.smithi137.stdout:8/32: write f8 [236357,101695] 0 2024-06-15T01:06:53.256 INFO:tasks.workunit.client.0.smithi137.stdout:7/42: mknod cb 0 2024-06-15T01:06:53.257 INFO:tasks.workunit.client.0.smithi137.stdout:7/43: write f9 [929808,116986] 0 2024-06-15T01:06:53.259 INFO:tasks.workunit.client.0.smithi137.stdout:1/153: mkdir d0/d2d/d3e 0 2024-06-15T01:06:53.262 INFO:tasks.workunit.client.0.smithi137.stdout:5/109: mknod df/d11/c2d 0 2024-06-15T01:06:53.263 INFO:tasks.workunit.client.0.smithi137.stdout:4/32: truncate f5 3965868 0 2024-06-15T01:06:53.265 INFO:tasks.workunit.client.0.smithi137.stdout:8/33: symlink d9/ld 0 2024-06-15T01:06:53.268 INFO:tasks.workunit.client.0.smithi137.stdout:3/32: mkdir d0/d2/d8 0 2024-06-15T01:06:53.269 INFO:tasks.workunit.client.0.smithi137.stdout:7/44: fdatasync f3 0 2024-06-15T01:06:53.269 INFO:tasks.workunit.client.1.smithi142.stdout:2/36: creat f6 x:0 0 0 2024-06-15T01:06:53.270 INFO:tasks.workunit.client.0.smithi137.stdout:1/154: mknod d0/d3/d14/c3f 0 2024-06-15T01:06:53.271 INFO:tasks.workunit.client.1.smithi142.stdout:2/37: dread - f6 zero size 2024-06-15T01:06:53.271 INFO:tasks.workunit.client.1.smithi142.stdout:2/38: write f4 [538369,55591] 0 2024-06-15T01:06:53.274 INFO:tasks.workunit.client.1.smithi142.stdout:2/39: write f6 [767082,16310] 0 2024-06-15T01:06:53.274 INFO:tasks.workunit.client.1.smithi142.stdout:8/38: link d0/c3 d0/cc 0 2024-06-15T01:06:53.274 INFO:tasks.workunit.client.1.smithi142.stdout:8/39: readlink - no filename 2024-06-15T01:06:53.275 INFO:tasks.workunit.client.0.smithi137.stdout:4/33: mkdir dd 0 2024-06-15T01:06:53.275 INFO:tasks.workunit.client.0.smithi137.stdout:0/40: symlink d1/ld 0 2024-06-15T01:06:53.276 INFO:tasks.workunit.client.0.smithi137.stdout:0/41: readlink d1/d2/lb 0 2024-06-15T01:06:53.277 INFO:tasks.workunit.client.0.smithi137.stdout:8/34: mkdir d9/de 0 2024-06-15T01:06:53.277 INFO:tasks.workunit.client.1.smithi142.stdout:8/40: fsync d0/fa 0 2024-06-15T01:06:53.277 INFO:tasks.workunit.client.1.smithi142.stdout:0/17: write f1 [2047424,112000] 0 2024-06-15T01:06:53.280 INFO:tasks.workunit.client.1.smithi142.stdout:4/178: unlink d5/d9/d1a/l3a 0 2024-06-15T01:06:53.280 INFO:tasks.workunit.client.0.smithi137.stdout:3/33: creat d0/d2/f9 x:0 0 0 2024-06-15T01:06:53.281 INFO:tasks.workunit.client.1.smithi142.stdout:1/95: dread d6/f10 [0,4194304] 0 2024-06-15T01:06:53.282 INFO:tasks.workunit.client.1.smithi142.stdout:2/40: dread f6 [0,4194304] 0 2024-06-15T01:06:53.282 INFO:tasks.workunit.client.0.smithi137.stdout:7/45: creat fc x:0 0 0 2024-06-15T01:06:53.283 INFO:tasks.workunit.client.0.smithi137.stdout:1/155: creat d0/d10/f40 x:0 0 0 2024-06-15T01:06:53.285 INFO:tasks.workunit.client.0.smithi137.stdout:5/110: creat df/d11/d14/f2e x:0 0 0 2024-06-15T01:06:53.286 INFO:tasks.workunit.client.1.smithi142.stdout:3/84: dwrite d0/d3/da/f11 [0,4194304] 0 2024-06-15T01:06:53.286 INFO:tasks.workunit.client.1.smithi142.stdout:2/41: chown f3 76 1 2024-06-15T01:06:53.286 INFO:tasks.workunit.client.1.smithi142.stdout:6/18: dwrite f2 [0,4194304] 0 2024-06-15T01:06:53.286 INFO:tasks.workunit.client.1.smithi142.stdout:8/41: symlink d0/d1/ld 0 2024-06-15T01:06:53.287 INFO:tasks.workunit.client.0.smithi137.stdout:8/35: dwrite f8 [0,4194304] 0 2024-06-15T01:06:53.287 INFO:tasks.workunit.client.1.smithi142.stdout:0/18: mkdir d3/d7 0 2024-06-15T01:06:53.287 INFO:tasks.workunit.client.0.smithi137.stdout:2/47: write d0/f3 [805689,8983] 0 2024-06-15T01:06:53.287 INFO:tasks.workunit.client.0.smithi137.stdout:1/156: truncate d0/d3/f5 1486497 0 2024-06-15T01:06:53.289 INFO:tasks.workunit.client.0.smithi137.stdout:1/157: fdatasync d0/d3/f36 0 2024-06-15T01:06:53.290 INFO:tasks.workunit.client.0.smithi137.stdout:1/158: chown d0/d3/d7/c18 1215678463 1 2024-06-15T01:06:53.291 INFO:tasks.workunit.client.0.smithi137.stdout:3/34: creat d0/d2/fa x:0 0 0 2024-06-15T01:06:53.292 INFO:tasks.workunit.client.1.smithi142.stdout:4/179: write d5/d9/fd [1649079,94593] 0 2024-06-15T01:06:53.292 INFO:tasks.workunit.client.1.smithi142.stdout:2/42: mknod c7 0 2024-06-15T01:06:53.292 INFO:tasks.workunit.client.0.smithi137.stdout:3/35: dread - d0/d2/f9 zero size 2024-06-15T01:06:53.293 INFO:tasks.workunit.client.1.smithi142.stdout:0/19: write d3/f4 [301795,28214] 0 2024-06-15T01:06:53.293 INFO:tasks.workunit.client.1.smithi142.stdout:6/19: creat f8 x:0 0 0 2024-06-15T01:06:53.294 INFO:tasks.workunit.client.0.smithi137.stdout:3/36: write d0/d2/f9 [39831,52604] 0 2024-06-15T01:06:53.294 INFO:tasks.workunit.client.0.smithi137.stdout:8/36: dread f6 [0,4194304] 0 2024-06-15T01:06:53.295 INFO:tasks.workunit.client.1.smithi142.stdout:6/20: dread - f7 zero size 2024-06-15T01:06:53.295 INFO:tasks.workunit.client.1.smithi142.stdout:5/33: truncate d2/f6 3063241 0 2024-06-15T01:06:53.297 INFO:tasks.workunit.client.0.smithi137.stdout:8/37: truncate d9/fc 400560 0 2024-06-15T01:06:53.298 INFO:tasks.workunit.client.0.smithi137.stdout:8/38: chown d9/fc 0 1 2024-06-15T01:06:53.300 INFO:tasks.workunit.client.1.smithi142.stdout:1/96: dwrite d6/f1d [0,4194304] 0 2024-06-15T01:06:53.302 INFO:tasks.workunit.client.0.smithi137.stdout:4/34: creat dd/fe x:0 0 0 2024-06-15T01:06:53.303 INFO:tasks.workunit.client.1.smithi142.stdout:3/85: dwrite d0/d3/da/f11 [4194304,4194304] 0 2024-06-15T01:06:53.303 INFO:tasks.workunit.client.0.smithi137.stdout:1/159: unlink d0/c26 0 2024-06-15T01:06:53.304 INFO:tasks.workunit.client.0.smithi137.stdout:5/111: creat df/d11/d14/f2f x:0 0 0 2024-06-15T01:06:53.304 INFO:tasks.workunit.client.1.smithi142.stdout:8/42: rename d0/d1/cb to d0/ce 0 2024-06-15T01:06:53.305 INFO:tasks.workunit.client.1.smithi142.stdout:2/43: mkdir d8 0 2024-06-15T01:06:53.306 INFO:tasks.workunit.client.0.smithi137.stdout:4/35: truncate dd/fe 835986 0 2024-06-15T01:06:53.306 INFO:tasks.workunit.client.0.smithi137.stdout:3/37: rename d0/d2/f9 to d0/d2/d8/fb 0 2024-06-15T01:06:53.307 INFO:tasks.workunit.client.1.smithi142.stdout:6/21: stat f6 0 2024-06-15T01:06:53.308 INFO:tasks.workunit.client.0.smithi137.stdout:4/36: fdatasync f9 0 2024-06-15T01:06:53.308 INFO:tasks.workunit.client.0.smithi137.stdout:5/112: truncate df/f10 811444 0 2024-06-15T01:06:53.308 INFO:tasks.workunit.client.1.smithi142.stdout:6/22: chown f4 2 1 2024-06-15T01:06:53.310 INFO:tasks.workunit.client.1.smithi142.stdout:5/34: creat d2/fc x:0 0 0 2024-06-15T01:06:53.310 INFO:tasks.workunit.client.1.smithi142.stdout:3/86: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:53.311 INFO:tasks.workunit.client.1.smithi142.stdout:4/180: dread d5/d9/d41/f36 [0,4194304] 0 2024-06-15T01:06:53.311 INFO:tasks.workunit.client.0.smithi137.stdout:4/37: truncate f9 44255 0 2024-06-15T01:06:53.312 INFO:tasks.workunit.client.1.smithi142.stdout:1/97: rename d6/d12/d1c/f1f to d6/d12/d1c/d1e/f20 0 2024-06-15T01:06:53.313 INFO:tasks.workunit.client.1.smithi142.stdout:2/44: fdatasync f6 0 2024-06-15T01:06:53.313 INFO:tasks.workunit.client.1.smithi142.stdout:8/43: dread d0/fa [0,4194304] 0 2024-06-15T01:06:53.313 INFO:tasks.workunit.client.1.smithi142.stdout:3/87: truncate d0/d3/f1a 853352 0 2024-06-15T01:06:53.314 INFO:tasks.workunit.client.0.smithi137.stdout:8/39: dwrite f6 [0,4194304] 0 2024-06-15T01:06:53.315 INFO:tasks.workunit.client.1.smithi142.stdout:5/35: creat d2/fd x:0 0 0 2024-06-15T01:06:53.318 INFO:tasks.workunit.client.0.smithi137.stdout:1/160: mkdir d0/d10/d41 0 2024-06-15T01:06:53.319 INFO:tasks.workunit.client.1.smithi142.stdout:4/181: read d5/d1c/f34 [1906703,24813] 0 2024-06-15T01:06:53.319 INFO:tasks.workunit.client.0.smithi137.stdout:7/46: getdents . 0 2024-06-15T01:06:53.320 INFO:tasks.workunit.client.1.smithi142.stdout:1/98: mknod d6/c21 0 2024-06-15T01:06:53.323 INFO:tasks.workunit.client.1.smithi142.stdout:2/45: unlink f3 0 2024-06-15T01:06:53.324 INFO:tasks.workunit.client.0.smithi137.stdout:5/113: dwrite df/f10 [0,4194304] 0 2024-06-15T01:06:53.325 INFO:tasks.workunit.client.0.smithi137.stdout:1/161: dread - d0/d3/d14/d34/f39 zero size 2024-06-15T01:06:53.327 INFO:tasks.workunit.client.0.smithi137.stdout:4/38: symlink dd/lf 0 2024-06-15T01:06:53.328 INFO:tasks.workunit.client.1.smithi142.stdout:8/44: unlink d0/d1/c6 0 2024-06-15T01:06:53.329 INFO:tasks.workunit.client.0.smithi137.stdout:5/114: read df/f10 [3118711,52571] 0 2024-06-15T01:06:53.329 INFO:tasks.workunit.client.1.smithi142.stdout:3/88: mknod d0/d3/da/c1e 0 2024-06-15T01:06:53.330 INFO:tasks.workunit.client.1.smithi142.stdout:3/89: readlink d0/l1d 0 2024-06-15T01:06:53.331 INFO:tasks.workunit.client.1.smithi142.stdout:3/90: truncate d0/d3/f1a 1641334 0 2024-06-15T01:06:53.331 INFO:tasks.workunit.client.1.smithi142.stdout:6/23: link f0 f9 0 2024-06-15T01:06:53.331 INFO:tasks.workunit.client.1.smithi142.stdout:6/24: rmdir - no directory 2024-06-15T01:06:53.331 INFO:tasks.workunit.client.1.smithi142.stdout:6/25: rmdir - no directory 2024-06-15T01:06:53.334 INFO:tasks.workunit.client.0.smithi137.stdout:0/42: truncate d1/d2/f6 556653 0 2024-06-15T01:06:53.334 INFO:tasks.workunit.client.1.smithi142.stdout:5/36: mkdir d2/de 0 2024-06-15T01:06:53.335 INFO:tasks.workunit.client.1.smithi142.stdout:4/182: truncate d5/d1c/f34 5227204 0 2024-06-15T01:06:53.335 INFO:tasks.workunit.client.0.smithi137.stdout:7/47: dwrite f9 [0,4194304] 0 2024-06-15T01:06:53.335 INFO:tasks.workunit.client.0.smithi137.stdout:0/43: stat d1/d2/l7 0 2024-06-15T01:06:53.336 INFO:tasks.workunit.client.1.smithi142.stdout:5/37: chown d2/ca 65 1 2024-06-15T01:06:53.338 INFO:tasks.workunit.client.0.smithi137.stdout:8/40: creat d9/de/ff x:0 0 0 2024-06-15T01:06:53.339 INFO:tasks.workunit.client.0.smithi137.stdout:1/162: write d0/d3/f5 [387249,11024] 0 2024-06-15T01:06:53.340 INFO:tasks.workunit.client.0.smithi137.stdout:4/39: mkdir dd/d10 0 2024-06-15T01:06:53.341 INFO:tasks.workunit.client.1.smithi142.stdout:3/91: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:53.342 INFO:tasks.workunit.client.0.smithi137.stdout:5/115: unlink df/d11/c2c 0 2024-06-15T01:06:53.342 INFO:tasks.workunit.client.1.smithi142.stdout:3/92: chown d0/d3/da/cb 8730 1 2024-06-15T01:06:53.344 INFO:tasks.workunit.client.1.smithi142.stdout:3/93: chown d0/d3/da/c10 412719 1 2024-06-15T01:06:53.345 INFO:tasks.workunit.client.0.smithi137.stdout:1/163: fdatasync d0/d3/d14/d34/f39 0 2024-06-15T01:06:53.345 INFO:tasks.workunit.client.1.smithi142.stdout:0/20: truncate f1 1464087 0 2024-06-15T01:06:53.346 INFO:tasks.workunit.client.0.smithi137.stdout:1/164: chown d0/d3/f2f 1820220499 1 2024-06-15T01:06:53.347 INFO:tasks.workunit.client.1.smithi142.stdout:8/45: mknod d0/cf 0 2024-06-15T01:06:53.347 INFO:tasks.workunit.client.0.smithi137.stdout:7/48: truncate f3 1566274 0 2024-06-15T01:06:53.348 INFO:tasks.workunit.client.1.smithi142.stdout:8/46: chown d0/fa 0 1 2024-06-15T01:06:53.348 INFO:tasks.workunit.client.0.smithi137.stdout:1/165: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:06:53.349 INFO:tasks.workunit.client.1.smithi142.stdout:6/26: truncate f6 497143 0 2024-06-15T01:06:53.349 INFO:tasks.workunit.client.0.smithi137.stdout:7/49: chown l7 13 1 2024-06-15T01:06:53.349 INFO:tasks.workunit.client.1.smithi142.stdout:5/38: creat d2/ff x:0 0 0 2024-06-15T01:06:53.350 INFO:tasks.workunit.client.1.smithi142.stdout:4/183: mkdir d5/d1c/d2a/d30/d46 0 2024-06-15T01:06:53.352 INFO:tasks.workunit.client.1.smithi142.stdout:4/184: chown d5/d1c/c44 40 1 2024-06-15T01:06:53.352 INFO:tasks.workunit.client.0.smithi137.stdout:4/40: mknod dd/d10/c11 0 2024-06-15T01:06:53.353 INFO:tasks.workunit.client.0.smithi137.stdout:4/41: dread - fa zero size 2024-06-15T01:06:53.353 INFO:tasks.workunit.client.0.smithi137.stdout:4/42: chown c7 16 1 2024-06-15T01:06:53.354 INFO:tasks.workunit.client.1.smithi142.stdout:2/46: creat d8/f9 x:0 0 0 2024-06-15T01:06:53.354 INFO:tasks.workunit.client.1.smithi142.stdout:2/47: dread - d8/f9 zero size 2024-06-15T01:06:53.356 INFO:tasks.workunit.client.1.smithi142.stdout:3/94: write d0/d3/f12 [3600427,33143] 0 2024-06-15T01:06:53.356 INFO:tasks.workunit.client.0.smithi137.stdout:5/116: mkdir df/d11/d30 0 2024-06-15T01:06:53.357 INFO:tasks.workunit.client.1.smithi142.stdout:8/47: creat d0/f10 x:0 0 0 2024-06-15T01:06:53.357 INFO:tasks.workunit.client.1.smithi142.stdout:0/21: dwrite d3/f4 [0,4194304] 0 2024-06-15T01:06:53.357 INFO:tasks.workunit.client.1.smithi142.stdout:6/27: mknod ca 0 2024-06-15T01:06:53.358 INFO:tasks.workunit.client.0.smithi137.stdout:7/50: creat fd x:0 0 0 2024-06-15T01:06:53.359 INFO:tasks.workunit.client.1.smithi142.stdout:5/39: symlink d2/l10 0 2024-06-15T01:06:53.359 INFO:tasks.workunit.client.1.smithi142.stdout:1/99: link d6/f9 d6/d12/f22 0 2024-06-15T01:06:53.360 INFO:tasks.workunit.client.0.smithi137.stdout:4/43: creat dd/f12 x:0 0 0 2024-06-15T01:06:53.361 INFO:tasks.workunit.client.1.smithi142.stdout:4/185: mkdir d5/df/d12/d47 0 2024-06-15T01:06:53.361 INFO:tasks.workunit.client.1.smithi142.stdout:5/40: dread - d2/f9 zero size 2024-06-15T01:06:53.362 INFO:tasks.workunit.client.0.smithi137.stdout:8/41: getdents d9 0 2024-06-15T01:06:53.363 INFO:tasks.workunit.client.1.smithi142.stdout:6/28: mknod cb 0 2024-06-15T01:06:53.363 INFO:tasks.workunit.client.0.smithi137.stdout:7/51: creat fe x:0 0 0 2024-06-15T01:06:53.363 INFO:tasks.workunit.client.0.smithi137.stdout:8/42: readlink d9/la 0 2024-06-15T01:06:53.363 INFO:tasks.workunit.client.1.smithi142.stdout:5/41: read - d2/ff zero size 2024-06-15T01:06:53.364 INFO:tasks.workunit.client.0.smithi137.stdout:8/43: fsync d9/fc 0 2024-06-15T01:06:53.364 INFO:tasks.workunit.client.0.smithi137.stdout:7/52: write fd [152270,91187] 0 2024-06-15T01:06:53.366 INFO:tasks.workunit.client.0.smithi137.stdout:5/117: creat df/d11/f31 x:0 0 0 2024-06-15T01:06:53.366 INFO:tasks.workunit.client.1.smithi142.stdout:8/48: rename d0/fa to d0/d1/f11 0 2024-06-15T01:06:53.367 INFO:tasks.workunit.client.0.smithi137.stdout:5/118: dread - df/f12 zero size 2024-06-15T01:06:53.368 INFO:tasks.workunit.client.1.smithi142.stdout:3/95: mkdir d0/d1f 0 2024-06-15T01:06:53.368 INFO:tasks.workunit.client.1.smithi142.stdout:2/48: dwrite d8/f9 [0,4194304] 0 2024-06-15T01:06:53.368 INFO:tasks.workunit.client.0.smithi137.stdout:0/44: read d1/d2/f6 [382617,104471] 0 2024-06-15T01:06:53.369 INFO:tasks.workunit.client.1.smithi142.stdout:8/49: truncate d0/f10 547425 0 2024-06-15T01:06:53.369 INFO:tasks.workunit.client.1.smithi142.stdout:0/22: dread d3/f4 [0,4194304] 0 2024-06-15T01:06:53.370 INFO:tasks.workunit.client.1.smithi142.stdout:3/96: fdatasync d0/d3/f1a 0 2024-06-15T01:06:53.370 INFO:tasks.workunit.client.0.smithi137.stdout:3/38: truncate d0/d2/f5 2603925 0 2024-06-15T01:06:53.371 INFO:tasks.workunit.client.1.smithi142.stdout:2/49: write f6 [1289563,93264] 0 2024-06-15T01:06:53.371 INFO:tasks.workunit.client.1.smithi142.stdout:3/97: chown d0/d3/da/c14 199386875 1 2024-06-15T01:06:53.371 INFO:tasks.workunit.client.0.smithi137.stdout:8/44: symlink d9/l10 0 2024-06-15T01:06:53.372 INFO:tasks.workunit.client.1.smithi142.stdout:3/98: chown d0/d1b/c1c 2595 1 2024-06-15T01:06:53.373 INFO:tasks.workunit.client.0.smithi137.stdout:8/45: truncate d9/de/ff 137821 0 2024-06-15T01:06:53.373 INFO:tasks.workunit.client.0.smithi137.stdout:7/53: mkdir df 0 2024-06-15T01:06:53.373 INFO:tasks.workunit.client.1.smithi142.stdout:6/29: creat fc x:0 0 0 2024-06-15T01:06:53.376 INFO:tasks.workunit.client.1.smithi142.stdout:4/186: symlink d5/d9/da/l48 0 2024-06-15T01:06:53.377 INFO:tasks.workunit.client.1.smithi142.stdout:4/187: dread - d5/d1c/d2a/d30/f3f zero size 2024-06-15T01:06:53.379 INFO:tasks.workunit.client.0.smithi137.stdout:5/119: creat df/d11/d14/f32 x:0 0 0 2024-06-15T01:06:53.382 INFO:tasks.workunit.client.0.smithi137.stdout:8/46: dwrite d9/fc [0,4194304] 0 2024-06-15T01:06:53.382 INFO:tasks.workunit.client.1.smithi142.stdout:5/42: read d2/f7 [331516,60930] 0 2024-06-15T01:06:53.383 INFO:tasks.workunit.client.0.smithi137.stdout:3/39: creat d0/d2/fc x:0 0 0 2024-06-15T01:06:53.389 INFO:tasks.workunit.client.1.smithi142.stdout:8/50: mknod d0/d1/c12 0 2024-06-15T01:06:53.390 INFO:tasks.workunit.client.0.smithi137.stdout:5/120: unlink df/d11/d14/f1e 0 2024-06-15T01:06:53.391 INFO:tasks.workunit.client.0.smithi137.stdout:7/54: mkdir df/d10 0 2024-06-15T01:06:53.391 INFO:tasks.workunit.client.0.smithi137.stdout:4/44: getdents dd 0 2024-06-15T01:06:53.392 INFO:tasks.workunit.client.1.smithi142.stdout:5/43: dwrite d2/fd [0,4194304] 0 2024-06-15T01:06:53.392 INFO:tasks.workunit.client.1.smithi142.stdout:5/44: chown d2/fc 12316625 1 2024-06-15T01:06:53.393 INFO:tasks.workunit.client.1.smithi142.stdout:5/45: write d2/fd [3545232,22583] 0 2024-06-15T01:06:53.394 INFO:tasks.workunit.client.1.smithi142.stdout:4/188: truncate d5/d9/d23/f29 575611 0 2024-06-15T01:06:53.395 INFO:tasks.workunit.client.0.smithi137.stdout:0/45: write d1/d2/f6 [1051033,25156] 0 2024-06-15T01:06:53.397 INFO:tasks.workunit.client.0.smithi137.stdout:8/47: creat d9/f11 x:0 0 0 2024-06-15T01:06:53.402 INFO:tasks.workunit.client.0.smithi137.stdout:5/121: creat df/d11/d14/f33 x:0 0 0 2024-06-15T01:06:53.403 INFO:tasks.workunit.client.1.smithi142.stdout:3/99: mkdir d0/d1f/d20 0 2024-06-15T01:06:53.407 INFO:tasks.workunit.client.0.smithi137.stdout:4/45: creat dd/f13 x:0 0 0 2024-06-15T01:06:53.409 INFO:tasks.workunit.client.1.smithi142.stdout:8/51: truncate d0/f8 1841430 0 2024-06-15T01:06:53.410 INFO:tasks.workunit.client.0.smithi137.stdout:0/46: symlink d1/d2/le 0 2024-06-15T01:06:53.410 INFO:tasks.workunit.client.0.smithi137.stdout:0/47: write d1/f9 [342975,78218] 0 2024-06-15T01:06:53.411 INFO:tasks.workunit.client.1.smithi142.stdout:6/30: link c1 cd 0 2024-06-15T01:06:53.412 INFO:tasks.workunit.client.0.smithi137.stdout:0/48: write d1/d2/f6 [1348098,18588] 0 2024-06-15T01:06:53.412 INFO:tasks.workunit.client.0.smithi137.stdout:8/48: rmdir d9/de 39 2024-06-15T01:06:53.412 INFO:tasks.workunit.client.0.smithi137.stdout:8/49: fsync f1 0 2024-06-15T01:06:53.412 INFO:tasks.workunit.client.1.smithi142.stdout:1/100: write d6/d12/f22 [1661355,81836] 0 2024-06-15T01:06:53.413 INFO:tasks.workunit.client.1.smithi142.stdout:4/189: dread d5/d1c/f34 [0,4194304] 0 2024-06-15T01:06:53.418 INFO:tasks.workunit.client.1.smithi142.stdout:1/101: mknod d6/c23 0 2024-06-15T01:06:53.418 INFO:tasks.workunit.client.0.smithi137.stdout:4/46: creat dd/f14 x:0 0 0 2024-06-15T01:06:53.418 INFO:tasks.workunit.client.1.smithi142.stdout:8/52: rename d0/c3 to d0/c13 0 2024-06-15T01:06:53.419 INFO:tasks.workunit.client.0.smithi137.stdout:5/122: getdents df/d11/d30 0 2024-06-15T01:06:53.420 INFO:tasks.workunit.client.1.smithi142.stdout:6/31: dwrite f7 [0,4194304] 0 2024-06-15T01:06:53.421 INFO:tasks.workunit.client.0.smithi137.stdout:4/47: truncate dd/f14 696705 0 2024-06-15T01:06:53.423 INFO:tasks.workunit.client.0.smithi137.stdout:0/49: mknod d1/d2/dc/cf 0 2024-06-15T01:06:53.424 INFO:tasks.workunit.client.0.smithi137.stdout:8/50: dwrite f0 [4194304,4194304] 0 2024-06-15T01:06:53.424 INFO:tasks.workunit.client.1.smithi142.stdout:1/102: creat d6/d12/d1c/d1e/f24 x:0 0 0 2024-06-15T01:06:53.426 INFO:tasks.workunit.client.0.smithi137.stdout:4/48: mkdir dd/d15 0 2024-06-15T01:06:53.426 INFO:tasks.workunit.client.1.smithi142.stdout:1/103: write d6/d12/d1c/d1e/f24 [327834,69159] 0 2024-06-15T01:06:53.428 INFO:tasks.workunit.client.1.smithi142.stdout:1/104: fdatasync d6/f13 0 2024-06-15T01:06:53.430 INFO:tasks.workunit.client.1.smithi142.stdout:6/32: dwrite f8 [0,4194304] 0 2024-06-15T01:06:53.431 INFO:tasks.workunit.client.1.smithi142.stdout:8/53: read d0/f8 [1298192,104299] 0 2024-06-15T01:06:53.432 INFO:tasks.workunit.client.0.smithi137.stdout:0/50: rmdir d1/d2/dc 39 2024-06-15T01:06:53.436 INFO:tasks.workunit.client.0.smithi137.stdout:9/42: sync 2024-06-15T01:06:53.436 INFO:tasks.workunit.client.0.smithi137.stdout:5/123: mkdir df/d11/d34 0 2024-06-15T01:06:53.438 INFO:tasks.workunit.client.1.smithi142.stdout:3/100: truncate d0/d3/da/f11 7220111 0 2024-06-15T01:06:53.439 INFO:tasks.workunit.client.0.smithi137.stdout:9/43: chown f0 468211 1 2024-06-15T01:06:53.439 INFO:tasks.workunit.client.0.smithi137.stdout:0/51: read d1/f9 [206464,110331] 0 2024-06-15T01:06:53.440 INFO:tasks.workunit.client.1.smithi142.stdout:1/105: mkdir d6/d25 0 2024-06-15T01:06:53.441 INFO:tasks.workunit.client.1.smithi142.stdout:6/33: unlink ca 0 2024-06-15T01:06:53.442 INFO:tasks.workunit.client.1.smithi142.stdout:6/34: dread - fc zero size 2024-06-15T01:06:53.445 INFO:tasks.workunit.client.1.smithi142.stdout:8/54: write d0/d1/f11 [1080340,48701] 0 2024-06-15T01:06:53.446 INFO:tasks.workunit.client.0.smithi137.stdout:0/52: chown d1 14056725 1 2024-06-15T01:06:53.447 INFO:tasks.workunit.client.0.smithi137.stdout:8/51: dwrite f7 [0,4194304] 0 2024-06-15T01:06:53.451 INFO:tasks.workunit.client.0.smithi137.stdout:3/40: dwrite d0/d2/f5 [0,4194304] 0 2024-06-15T01:06:53.453 INFO:tasks.workunit.client.0.smithi137.stdout:4/49: dwrite dd/f12 [0,4194304] 0 2024-06-15T01:06:53.454 INFO:tasks.workunit.client.0.smithi137.stdout:5/124: mkdir df/d11/d34/d35 0 2024-06-15T01:06:53.454 INFO:tasks.workunit.client.1.smithi142.stdout:8/55: creat d0/f14 x:0 0 0 2024-06-15T01:06:53.455 INFO:tasks.workunit.client.0.smithi137.stdout:9/44: dread fa [0,4194304] 0 2024-06-15T01:06:53.455 INFO:tasks.workunit.client.1.smithi142.stdout:3/101: getdents d0/d3/d15 0 2024-06-15T01:06:53.457 INFO:tasks.workunit.client.0.smithi137.stdout:8/52: stat c5 0 2024-06-15T01:06:53.458 INFO:tasks.workunit.client.1.smithi142.stdout:8/56: creat d0/f15 x:0 0 0 2024-06-15T01:06:53.460 INFO:tasks.workunit.client.0.smithi137.stdout:8/53: chown f0 224 1 2024-06-15T01:06:53.460 INFO:tasks.workunit.client.0.smithi137.stdout:3/41: write d0/f4 [5116468,90809] 0 2024-06-15T01:06:53.462 INFO:tasks.workunit.client.0.smithi137.stdout:0/53: chown d1/d2/l3 434601135 1 2024-06-15T01:06:53.464 INFO:tasks.workunit.client.0.smithi137.stdout:7/55: dwrite f3 [0,4194304] 0 2024-06-15T01:06:53.465 INFO:tasks.workunit.client.1.smithi142.stdout:3/102: rename d0/d3/da/fe to d0/d3/da/d18/f21 0 2024-06-15T01:06:53.467 INFO:tasks.workunit.client.1.smithi142.stdout:3/103: chown d0/d3/f1a 29725424 1 2024-06-15T01:06:53.467 INFO:tasks.workunit.client.1.smithi142.stdout:8/57: dwrite d0/f15 [0,4194304] 0 2024-06-15T01:06:53.468 INFO:tasks.workunit.client.1.smithi142.stdout:3/104: stat d0 0 2024-06-15T01:06:53.468 INFO:tasks.workunit.client.1.smithi142.stdout:8/58: chown d0/f10 36801277 1 2024-06-15T01:06:53.470 INFO:tasks.workunit.client.0.smithi137.stdout:8/54: chown d9/de 397 1 2024-06-15T01:06:53.472 INFO:tasks.workunit.client.0.smithi137.stdout:0/54: readlink d1/l5 0 2024-06-15T01:06:53.473 INFO:tasks.workunit.client.1.smithi142.stdout:6/35: write f4 [323330,6266] 0 2024-06-15T01:06:53.474 INFO:tasks.workunit.client.0.smithi137.stdout:3/42: unlink d0/f4 0 2024-06-15T01:06:53.475 INFO:tasks.workunit.client.1.smithi142.stdout:6/36: mknod ce 0 2024-06-15T01:06:53.476 INFO:tasks.workunit.client.0.smithi137.stdout:7/56: symlink df/l11 0 2024-06-15T01:06:53.476 INFO:tasks.workunit.client.1.smithi142.stdout:1/106: dwrite d6/fc [0,4194304] 0 2024-06-15T01:06:53.476 INFO:tasks.workunit.client.1.smithi142.stdout:6/37: write f8 [201407,94447] 0 2024-06-15T01:06:53.477 INFO:tasks.workunit.client.1.smithi142.stdout:6/38: chown f8 1009547275 1 2024-06-15T01:06:53.478 INFO:tasks.workunit.client.0.smithi137.stdout:9/45: link f0 fd 0 2024-06-15T01:06:53.479 INFO:tasks.workunit.client.0.smithi137.stdout:8/55: dread f6 [0,4194304] 0 2024-06-15T01:06:53.480 INFO:tasks.workunit.client.0.smithi137.stdout:8/56: write f0 [5055557,120385] 0 2024-06-15T01:06:53.483 INFO:tasks.workunit.client.0.smithi137.stdout:8/57: truncate d9/f11 1008903 0 2024-06-15T01:06:53.483 INFO:tasks.workunit.client.1.smithi142.stdout:6/39: creat ff x:0 0 0 2024-06-15T01:06:53.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/59: rmdir d0 39 2024-06-15T01:06:53.485 INFO:tasks.workunit.client.0.smithi137.stdout:9/46: dread fd [4194304,4194304] 0 2024-06-15T01:06:53.486 INFO:tasks.workunit.client.0.smithi137.stdout:3/43: rename d0/d2/fa to d0/fd 0 2024-06-15T01:06:53.490 INFO:tasks.workunit.client.0.smithi137.stdout:3/44: read - d0/fd zero size 2024-06-15T01:06:53.490 INFO:tasks.workunit.client.0.smithi137.stdout:7/57: mknod df/c12 0 2024-06-15T01:06:53.490 INFO:tasks.workunit.client.1.smithi142.stdout:1/107: dwrite d6/fc [4194304,4194304] 0 2024-06-15T01:06:53.491 INFO:tasks.workunit.client.1.smithi142.stdout:1/108: readlink d6/l19 0 2024-06-15T01:06:53.491 INFO:tasks.workunit.client.1.smithi142.stdout:6/40: creat f10 x:0 0 0 2024-06-15T01:06:53.492 INFO:tasks.workunit.client.1.smithi142.stdout:8/60: rename d0/cf to d0/c16 0 2024-06-15T01:06:53.493 INFO:tasks.workunit.client.0.smithi137.stdout:7/58: truncate f3 4819230 0 2024-06-15T01:06:53.493 INFO:tasks.workunit.client.0.smithi137.stdout:0/55: mkdir d1/d2/dc/d10 0 2024-06-15T01:06:53.497 INFO:tasks.workunit.client.0.smithi137.stdout:8/58: dwrite d9/f11 [0,4194304] 0 2024-06-15T01:06:53.500 INFO:tasks.workunit.client.1.smithi142.stdout:6/41: dread f9 [0,4194304] 0 2024-06-15T01:06:53.500 INFO:tasks.workunit.client.0.smithi137.stdout:9/47: fdatasync fd 0 2024-06-15T01:06:53.501 INFO:tasks.workunit.client.1.smithi142.stdout:8/61: dwrite d0/f14 [0,4194304] 0 2024-06-15T01:06:53.507 INFO:tasks.workunit.client.0.smithi137.stdout:3/45: creat d0/d2/d8/fe x:0 0 0 2024-06-15T01:06:53.508 INFO:tasks.workunit.client.0.smithi137.stdout:5/125: dwrite f3 [0,4194304] 0 2024-06-15T01:06:53.510 INFO:tasks.workunit.client.0.smithi137.stdout:4/50: dwrite f5 [0,4194304] 0 2024-06-15T01:06:53.510 INFO:tasks.workunit.client.0.smithi137.stdout:3/46: write d0/d2/d8/fe [707882,8306] 0 2024-06-15T01:06:53.511 INFO:tasks.workunit.client.0.smithi137.stdout:7/59: mkdir df/d13 0 2024-06-15T01:06:53.514 INFO:tasks.workunit.client.0.smithi137.stdout:0/56: mknod d1/d2/dc/c11 0 2024-06-15T01:06:53.515 INFO:tasks.workunit.client.0.smithi137.stdout:5/126: rename df/d11 to df/d11/d34/d35/d36 22 2024-06-15T01:06:53.515 INFO:tasks.workunit.client.0.smithi137.stdout:7/60: write f9 [3439377,105306] 0 2024-06-15T01:06:53.528 INFO:tasks.workunit.client.1.smithi142.stdout:6/42: symlink l11 0 2024-06-15T01:06:53.531 INFO:tasks.workunit.client.0.smithi137.stdout:7/61: dread f1 [0,4194304] 0 2024-06-15T01:06:53.532 INFO:tasks.workunit.client.1.smithi142.stdout:8/62: truncate d0/d1/f4 1749177 0 2024-06-15T01:06:53.532 INFO:tasks.workunit.client.0.smithi137.stdout:9/48: dwrite f2 [0,4194304] 0 2024-06-15T01:06:53.536 INFO:tasks.workunit.client.1.smithi142.stdout:6/43: symlink l12 0 2024-06-15T01:06:53.539 INFO:tasks.workunit.client.0.smithi137.stdout:4/51: dwrite fa [0,4194304] 0 2024-06-15T01:06:53.541 INFO:tasks.workunit.client.0.smithi137.stdout:9/49: write f5 [1534110,112023] 0 2024-06-15T01:06:53.544 INFO:tasks.workunit.client.0.smithi137.stdout:3/47: dwrite d0/d2/d8/fe [0,4194304] 0 2024-06-15T01:06:53.545 INFO:tasks.workunit.client.1.smithi142.stdout:6/44: fsync f6 0 2024-06-15T01:06:53.547 INFO:tasks.workunit.client.0.smithi137.stdout:7/62: creat df/f14 x:0 0 0 2024-06-15T01:06:53.547 INFO:tasks.workunit.client.0.smithi137.stdout:5/127: write f9 [1196539,123489] 0 2024-06-15T01:06:53.551 INFO:tasks.workunit.client.0.smithi137.stdout:9/50: fdatasync fc 0 2024-06-15T01:06:53.551 INFO:tasks.workunit.client.0.smithi137.stdout:5/128: truncate df/f12 21569 0 2024-06-15T01:06:53.551 INFO:tasks.workunit.client.0.smithi137.stdout:7/63: dread - df/f14 zero size 2024-06-15T01:06:53.552 INFO:tasks.workunit.client.0.smithi137.stdout:0/57: dwrite d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:53.552 INFO:tasks.workunit.client.0.smithi137.stdout:9/51: fsync fd 0 2024-06-15T01:06:53.557 INFO:tasks.workunit.client.1.smithi142.stdout:6/45: dread f2 [0,4194304] 0 2024-06-15T01:06:53.559 INFO:tasks.workunit.client.0.smithi137.stdout:9/52: dread f3 [0,4194304] 0 2024-06-15T01:06:53.559 INFO:tasks.workunit.client.1.smithi142.stdout:6/46: dread f9 [0,4194304] 0 2024-06-15T01:06:53.564 INFO:tasks.workunit.client.0.smithi137.stdout:7/64: dread f9 [0,4194304] 0 2024-06-15T01:06:53.565 INFO:tasks.workunit.client.0.smithi137.stdout:4/52: fdatasync fa 0 2024-06-15T01:06:53.567 INFO:tasks.workunit.client.0.smithi137.stdout:9/53: mknod ce 0 2024-06-15T01:06:53.568 INFO:tasks.workunit.client.0.smithi137.stdout:5/129: dread fe [0,4194304] 0 2024-06-15T01:06:53.571 INFO:tasks.workunit.client.0.smithi137.stdout:3/48: mknod d0/d2/cf 0 2024-06-15T01:06:53.573 INFO:tasks.workunit.client.0.smithi137.stdout:7/65: rename c6 to df/d13/c15 0 2024-06-15T01:06:53.575 INFO:tasks.workunit.client.1.smithi142.stdout:8/63: write d0/d1/f4 [2720252,64325] 0 2024-06-15T01:06:53.577 INFO:tasks.workunit.client.0.smithi137.stdout:5/130: symlink df/d11/d34/l37 0 2024-06-15T01:06:53.578 INFO:tasks.workunit.client.1.smithi142.stdout:8/64: mkdir d0/d17 0 2024-06-15T01:06:53.579 INFO:tasks.workunit.client.0.smithi137.stdout:9/54: dwrite f8 [0,4194304] 0 2024-06-15T01:06:53.580 INFO:tasks.workunit.client.1.smithi142.stdout:7/51: sync 2024-06-15T01:06:53.580 INFO:tasks.workunit.client.1.smithi142.stdout:0/23: sync 2024-06-15T01:06:53.580 INFO:tasks.workunit.client.1.smithi142.stdout:9/34: sync 2024-06-15T01:06:53.580 INFO:tasks.workunit.client.0.smithi137.stdout:5/131: write f3 [2211945,41297] 0 2024-06-15T01:06:53.581 INFO:tasks.workunit.client.1.smithi142.stdout:6/47: dwrite f6 [0,4194304] 0 2024-06-15T01:06:53.581 INFO:tasks.workunit.client.0.smithi137.stdout:4/53: link f9 dd/f16 0 2024-06-15T01:06:53.581 INFO:tasks.workunit.client.1.smithi142.stdout:7/52: read - d2/fb zero size 2024-06-15T01:06:53.582 INFO:tasks.workunit.client.1.smithi142.stdout:6/48: stat f9 0 2024-06-15T01:06:53.583 INFO:tasks.workunit.client.1.smithi142.stdout:6/49: fdatasync f7 0 2024-06-15T01:06:53.583 INFO:tasks.workunit.client.0.smithi137.stdout:7/66: rmdir df 39 2024-06-15T01:06:53.585 INFO:tasks.workunit.client.0.smithi137.stdout:0/58: truncate d1/d2/f4 3573889 0 2024-06-15T01:06:53.586 INFO:tasks.workunit.client.1.smithi142.stdout:9/35: mknod d6/dc/dd/c11 0 2024-06-15T01:06:53.587 INFO:tasks.workunit.client.0.smithi137.stdout:9/55: mkdir df 0 2024-06-15T01:06:53.587 INFO:tasks.workunit.client.1.smithi142.stdout:6/50: dread f0 [0,4194304] 0 2024-06-15T01:06:53.589 INFO:tasks.workunit.client.0.smithi137.stdout:3/49: unlink d0/d2/cf 0 2024-06-15T01:06:53.590 INFO:tasks.workunit.client.1.smithi142.stdout:7/53: dread d2/d6/f9 [0,4194304] 0 2024-06-15T01:06:53.590 INFO:tasks.workunit.client.1.smithi142.stdout:6/51: write f6 [1025895,67467] 0 2024-06-15T01:06:53.591 INFO:tasks.workunit.client.0.smithi137.stdout:4/54: creat dd/f17 x:0 0 0 2024-06-15T01:06:53.591 INFO:tasks.workunit.client.1.smithi142.stdout:7/54: dread - d2/d8/dd/f12 zero size 2024-06-15T01:06:53.591 INFO:tasks.workunit.client.1.smithi142.stdout:6/52: chown ff 6975 1 2024-06-15T01:06:53.591 INFO:tasks.workunit.client.1.smithi142.stdout:8/65: truncate d0/f14 2438767 0 2024-06-15T01:06:53.595 INFO:tasks.workunit.client.0.smithi137.stdout:5/132: mknod df/d11/c38 0 2024-06-15T01:06:53.595 INFO:tasks.workunit.client.0.smithi137.stdout:3/50: truncate d0/d2/fc 795477 0 2024-06-15T01:06:53.596 INFO:tasks.workunit.client.0.smithi137.stdout:5/133: chown df/d11/d14/c1f 1287265358 1 2024-06-15T01:06:53.596 INFO:tasks.workunit.client.1.smithi142.stdout:6/53: unlink f2 0 2024-06-15T01:06:53.596 INFO:tasks.workunit.client.0.smithi137.stdout:4/55: creat dd/f18 x:0 0 0 2024-06-15T01:06:53.597 INFO:tasks.workunit.client.0.smithi137.stdout:7/67: symlink df/d13/l16 0 2024-06-15T01:06:53.598 INFO:tasks.workunit.client.1.smithi142.stdout:9/36: dwrite f2 [0,4194304] 0 2024-06-15T01:06:53.598 INFO:tasks.workunit.client.1.smithi142.stdout:7/55: mkdir d2/d8/dd/d14 0 2024-06-15T01:06:53.599 INFO:tasks.workunit.client.0.smithi137.stdout:0/59: symlink d1/d2/l12 0 2024-06-15T01:06:53.599 INFO:tasks.workunit.client.1.smithi142.stdout:9/37: chown c4 18497 1 2024-06-15T01:06:53.599 INFO:tasks.workunit.client.0.smithi137.stdout:7/68: dread f1 [0,4194304] 0 2024-06-15T01:06:53.600 INFO:tasks.workunit.client.0.smithi137.stdout:7/69: write df/f14 [110932,48556] 0 2024-06-15T01:06:53.605 INFO:tasks.workunit.client.1.smithi142.stdout:8/66: getdents d0 0 2024-06-15T01:06:53.605 INFO:tasks.workunit.client.1.smithi142.stdout:9/38: write f5 [243381,24282] 0 2024-06-15T01:06:53.608 INFO:tasks.workunit.client.0.smithi137.stdout:4/56: rename l3 to dd/d10/l19 0 2024-06-15T01:06:53.609 INFO:tasks.workunit.client.1.smithi142.stdout:7/56: dwrite d2/d6/f7 [0,4194304] 0 2024-06-15T01:06:53.610 INFO:tasks.workunit.client.1.smithi142.stdout:7/57: stat d2/d6/f7 0 2024-06-15T01:06:53.612 INFO:tasks.workunit.client.0.smithi137.stdout:9/56: link c9 df/c10 0 2024-06-15T01:06:53.614 INFO:tasks.workunit.client.1.smithi142.stdout:6/54: truncate f8 1095536 0 2024-06-15T01:06:53.615 INFO:tasks.workunit.client.1.smithi142.stdout:9/39: mknod d6/dc/c12 0 2024-06-15T01:06:53.615 INFO:tasks.workunit.client.1.smithi142.stdout:0/24: dwrite f1 [0,4194304] 0 2024-06-15T01:06:53.617 INFO:tasks.workunit.client.1.smithi142.stdout:8/67: link d0/f15 d0/f18 0 2024-06-15T01:06:53.619 INFO:tasks.workunit.client.0.smithi137.stdout:7/70: dwrite fc [0,4194304] 0 2024-06-15T01:06:53.620 INFO:tasks.workunit.client.1.smithi142.stdout:7/58: mkdir d2/d8/dd/d15 0 2024-06-15T01:06:53.621 INFO:tasks.workunit.client.0.smithi137.stdout:3/51: rename d0/d2/c3 to d0/d2/d8/c10 0 2024-06-15T01:06:53.622 INFO:tasks.workunit.client.0.smithi137.stdout:5/134: link df/d11/l23 df/d11/d34/l39 0 2024-06-15T01:06:53.624 INFO:tasks.workunit.client.1.smithi142.stdout:6/55: rename ff to f13 0 2024-06-15T01:06:53.624 INFO:tasks.workunit.client.1.smithi142.stdout:6/56: chown c3 2288 1 2024-06-15T01:06:53.625 INFO:tasks.workunit.client.0.smithi137.stdout:4/57: dwrite dd/fe [0,4194304] 0 2024-06-15T01:06:53.627 INFO:tasks.workunit.client.0.smithi137.stdout:7/71: stat ca 0 2024-06-15T01:06:53.630 INFO:tasks.workunit.client.1.smithi142.stdout:9/40: symlink d6/dc/l13 0 2024-06-15T01:06:53.630 INFO:tasks.workunit.client.1.smithi142.stdout:7/59: fdatasync d2/d6/f9 0 2024-06-15T01:06:53.633 INFO:tasks.workunit.client.0.smithi137.stdout:7/72: dread f1 [0,4194304] 0 2024-06-15T01:06:53.634 INFO:tasks.workunit.client.0.smithi137.stdout:3/52: rename d0/d2/d8/fe to d0/d2/d8/f11 0 2024-06-15T01:06:53.638 INFO:tasks.workunit.client.0.smithi137.stdout:4/58: dwrite dd/f12 [0,4194304] 0 2024-06-15T01:06:53.638 INFO:tasks.workunit.client.1.smithi142.stdout:4/190: sync 2024-06-15T01:06:53.639 INFO:tasks.workunit.client.1.smithi142.stdout:5/46: sync 2024-06-15T01:06:53.641 INFO:tasks.workunit.client.1.smithi142.stdout:2/50: sync 2024-06-15T01:06:53.642 INFO:tasks.workunit.client.1.smithi142.stdout:9/41: rename d6/l10 to d6/l14 0 2024-06-15T01:06:53.648 INFO:tasks.workunit.client.1.smithi142.stdout:5/47: mknod d2/c11 0 2024-06-15T01:06:53.650 INFO:tasks.workunit.client.1.smithi142.stdout:0/25: fsync f1 0 2024-06-15T01:06:53.651 INFO:tasks.workunit.client.1.smithi142.stdout:2/51: creat d8/fa x:0 0 0 2024-06-15T01:06:53.652 INFO:tasks.workunit.client.0.smithi137.stdout:0/60: dwrite d1/f9 [0,4194304] 0 2024-06-15T01:06:53.652 INFO:tasks.workunit.client.0.smithi137.stdout:4/59: link f8 dd/d15/f1a 0 2024-06-15T01:06:53.653 INFO:tasks.workunit.client.1.smithi142.stdout:9/42: creat d6/dc/f15 x:0 0 0 2024-06-15T01:06:53.654 INFO:tasks.workunit.client.1.smithi142.stdout:9/43: dread - d6/dc/f15 zero size 2024-06-15T01:06:53.654 INFO:tasks.workunit.client.0.smithi137.stdout:4/60: fsync f5 0 2024-06-15T01:06:53.655 INFO:tasks.workunit.client.0.smithi137.stdout:5/135: getdents df/d11/d14 0 2024-06-15T01:06:53.657 INFO:tasks.workunit.client.0.smithi137.stdout:4/61: chown l6 1598 1 2024-06-15T01:06:53.658 INFO:tasks.workunit.client.0.smithi137.stdout:7/73: getdents df 0 2024-06-15T01:06:53.658 INFO:tasks.workunit.client.1.smithi142.stdout:0/26: mknod d3/c8 0 2024-06-15T01:06:53.658 INFO:tasks.workunit.client.0.smithi137.stdout:4/62: truncate f8 257256 0 2024-06-15T01:06:53.659 INFO:tasks.workunit.client.0.smithi137.stdout:3/53: dwrite d0/fd [0,4194304] 0 2024-06-15T01:06:53.659 INFO:tasks.workunit.client.0.smithi137.stdout:0/61: mkdir d1/d2/d13 0 2024-06-15T01:06:53.661 INFO:tasks.workunit.client.0.smithi137.stdout:7/74: truncate f9 5076851 0 2024-06-15T01:06:53.661 INFO:tasks.workunit.client.1.smithi142.stdout:7/60: link d2/d8/fe d2/d8/dd/d15/f16 0 2024-06-15T01:06:53.662 INFO:tasks.workunit.client.1.smithi142.stdout:7/61: chown d2/d8/dd 709445 1 2024-06-15T01:06:53.662 INFO:tasks.workunit.client.0.smithi137.stdout:4/63: read - dd/f13 zero size 2024-06-15T01:06:53.663 INFO:tasks.workunit.client.0.smithi137.stdout:5/136: dread fe [0,4194304] 0 2024-06-15T01:06:53.663 INFO:tasks.workunit.client.0.smithi137.stdout:3/54: chown d0/fd 0 1 2024-06-15T01:06:53.664 INFO:tasks.workunit.client.1.smithi142.stdout:6/57: getdents . 0 2024-06-15T01:06:53.666 INFO:tasks.workunit.client.1.smithi142.stdout:5/48: rename d2/f6 to d2/f12 0 2024-06-15T01:06:53.668 INFO:tasks.workunit.client.1.smithi142.stdout:8/68: truncate d0/f18 3294824 0 2024-06-15T01:06:53.668 INFO:tasks.workunit.client.1.smithi142.stdout:8/69: fdatasync d0/f5 0 2024-06-15T01:06:53.670 INFO:tasks.workunit.client.0.smithi137.stdout:5/137: rename df/d11 to df/d11/d34/d3a 22 2024-06-15T01:06:53.671 INFO:tasks.workunit.client.0.smithi137.stdout:5/138: write df/f2a [662742,43427] 0 2024-06-15T01:06:53.671 INFO:tasks.workunit.client.1.smithi142.stdout:7/62: creat d2/d8/f17 x:0 0 0 2024-06-15T01:06:53.672 INFO:tasks.workunit.client.0.smithi137.stdout:4/64: symlink dd/d15/l1b 0 2024-06-15T01:06:53.672 INFO:tasks.workunit.client.1.smithi142.stdout:4/191: write d5/df/d12/f37 [204217,68150] 0 2024-06-15T01:06:53.673 INFO:tasks.workunit.client.1.smithi142.stdout:9/44: link f5 d6/dc/dd/f16 0 2024-06-15T01:06:53.674 INFO:tasks.workunit.client.1.smithi142.stdout:9/45: chown d6/dc/f15 425948 1 2024-06-15T01:06:53.674 INFO:tasks.workunit.client.1.smithi142.stdout:9/46: fsync f2 0 2024-06-15T01:06:53.675 INFO:tasks.workunit.client.1.smithi142.stdout:6/58: mknod c14 0 2024-06-15T01:06:53.675 INFO:tasks.workunit.client.0.smithi137.stdout:5/139: mkdir df/d11/d34/d3b 0 2024-06-15T01:06:53.675 INFO:tasks.workunit.client.0.smithi137.stdout:4/65: dread - dd/f13 zero size 2024-06-15T01:06:53.675 INFO:tasks.workunit.client.0.smithi137.stdout:0/62: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:53.676 INFO:tasks.workunit.client.1.smithi142.stdout:5/49: fsync d2/f5 0 2024-06-15T01:06:53.676 INFO:tasks.workunit.client.1.smithi142.stdout:5/50: write d2/f9 [425324,88128] 0 2024-06-15T01:06:53.677 INFO:tasks.workunit.client.0.smithi137.stdout:4/66: write dd/f14 [1684871,51963] 0 2024-06-15T01:06:53.677 INFO:tasks.workunit.client.0.smithi137.stdout:3/55: getdents d0/d2/d8 0 2024-06-15T01:06:53.679 INFO:tasks.workunit.client.0.smithi137.stdout:5/140: write f9 [228591,21555] 0 2024-06-15T01:06:53.680 INFO:tasks.workunit.client.1.smithi142.stdout:8/70: mknod d0/c19 0 2024-06-15T01:06:53.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/75: fsync f9 0 2024-06-15T01:06:53.684 INFO:tasks.workunit.client.1.smithi142.stdout:2/52: truncate d8/f9 3893542 0 2024-06-15T01:06:53.684 INFO:tasks.workunit.client.0.smithi137.stdout:5/141: chown df/d11/d14/f32 195920 1 2024-06-15T01:06:53.685 INFO:tasks.workunit.client.0.smithi137.stdout:6/23: sync 2024-06-15T01:06:53.685 INFO:tasks.workunit.client.0.smithi137.stdout:3/56: write d0/d2/f5 [4078987,60108] 0 2024-06-15T01:06:53.686 INFO:tasks.workunit.client.0.smithi137.stdout:4/67: link dd/d15/f1a dd/d10/f1c 0 2024-06-15T01:06:53.687 INFO:tasks.workunit.client.0.smithi137.stdout:7/76: symlink df/l17 0 2024-06-15T01:06:53.687 INFO:tasks.workunit.client.0.smithi137.stdout:0/63: dread d1/d2/f4 [0,4194304] 0 2024-06-15T01:06:53.688 INFO:tasks.workunit.client.1.smithi142.stdout:9/47: mknod d6/dc/dd/c17 0 2024-06-15T01:06:53.688 INFO:tasks.workunit.client.1.smithi142.stdout:9/48: truncate f5 580584 0 2024-06-15T01:06:53.688 INFO:tasks.workunit.client.0.smithi137.stdout:5/142: read - df/d11/f24 zero size 2024-06-15T01:06:53.689 INFO:tasks.workunit.client.1.smithi142.stdout:6/59: rename l12 to l15 0 2024-06-15T01:06:53.690 INFO:tasks.workunit.client.0.smithi137.stdout:3/57: symlink d0/l12 0 2024-06-15T01:06:53.690 INFO:tasks.workunit.client.0.smithi137.stdout:7/77: mkdir df/d18 0 2024-06-15T01:06:53.693 INFO:tasks.workunit.client.0.smithi137.stdout:0/64: chown d1/d2/l7 131974675 1 2024-06-15T01:06:53.696 INFO:tasks.workunit.client.1.smithi142.stdout:2/53: symlink d8/lb 0 2024-06-15T01:06:53.696 INFO:tasks.workunit.client.1.smithi142.stdout:0/27: truncate d3/f4 1721723 0 2024-06-15T01:06:53.697 INFO:tasks.workunit.client.1.smithi142.stdout:4/192: mknod d5/d9/d23/d32/c49 0 2024-06-15T01:06:53.699 INFO:tasks.workunit.client.1.smithi142.stdout:2/54: dread f4 [0,4194304] 0 2024-06-15T01:06:53.701 INFO:tasks.workunit.client.1.smithi142.stdout:5/51: getdents d2/de 0 2024-06-15T01:06:53.701 INFO:tasks.workunit.client.0.smithi137.stdout:4/68: dwrite fa [0,4194304] 0 2024-06-15T01:06:53.702 INFO:tasks.workunit.client.1.smithi142.stdout:6/60: dread f8 [0,4194304] 0 2024-06-15T01:06:53.703 INFO:tasks.workunit.client.0.smithi137.stdout:7/78: creat df/d10/f19 x:0 0 0 2024-06-15T01:06:53.707 INFO:tasks.workunit.client.1.smithi142.stdout:0/28: creat d3/f9 x:0 0 0 2024-06-15T01:06:53.707 INFO:tasks.workunit.client.0.smithi137.stdout:5/143: dwrite f3 [0,4194304] 0 2024-06-15T01:06:53.708 INFO:tasks.workunit.client.1.smithi142.stdout:0/29: chown d3/f9 10 1 2024-06-15T01:06:53.710 INFO:tasks.workunit.client.0.smithi137.stdout:4/69: chown dd/d15/f1a 1013118 1 2024-06-15T01:06:53.711 INFO:tasks.workunit.client.0.smithi137.stdout:7/79: mkdir df/d10/d1a 0 2024-06-15T01:06:53.713 INFO:tasks.workunit.client.0.smithi137.stdout:3/58: dwrite d0/d2/f5 [0,4194304] 0 2024-06-15T01:06:53.716 INFO:tasks.workunit.client.1.smithi142.stdout:9/49: write f5 [372642,12631] 0 2024-06-15T01:06:53.718 INFO:tasks.workunit.client.1.smithi142.stdout:8/71: link d0/d1/ld d0/l1a 0 2024-06-15T01:06:53.719 INFO:tasks.workunit.client.1.smithi142.stdout:6/61: mknod c16 0 2024-06-15T01:06:53.719 INFO:tasks.workunit.client.1.smithi142.stdout:8/72: chown d0/d17 46055 1 2024-06-15T01:06:53.720 INFO:tasks.workunit.client.0.smithi137.stdout:6/24: dwrite d0/f2 [0,4194304] 0 2024-06-15T01:06:53.723 INFO:tasks.workunit.client.0.smithi137.stdout:5/144: mkdir df/d11/d14/d3c 0 2024-06-15T01:06:53.725 INFO:tasks.workunit.client.1.smithi142.stdout:9/50: creat d6/dc/dd/f18 x:0 0 0 2024-06-15T01:06:53.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:53 smithi137 ceph-mon[29354]: pgmap v12: 65 pgs: 65 active+clean; 351 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 225 KiB/s rd, 21 MiB/s wr, 404 op/s 2024-06-15T01:06:53.728 INFO:tasks.workunit.client.1.smithi142.stdout:6/62: creat f17 x:0 0 0 2024-06-15T01:06:53.728 INFO:tasks.workunit.client.0.smithi137.stdout:0/65: dwrite d1/d2/f4 [0,4194304] 0 2024-06-15T01:06:53.728 INFO:tasks.workunit.client.0.smithi137.stdout:6/25: mkdir d0/d5 0 2024-06-15T01:06:53.732 INFO:tasks.workunit.client.0.smithi137.stdout:7/80: write f5 [1646477,39657] 0 2024-06-15T01:06:53.732 INFO:tasks.workunit.client.1.smithi142.stdout:8/73: unlink d0/ce 0 2024-06-15T01:06:53.734 INFO:tasks.workunit.client.1.smithi142.stdout:4/193: link d5/df/d12/l19 d5/d1c/d2a/d30/l4a 0 2024-06-15T01:06:53.734 INFO:tasks.workunit.client.1.smithi142.stdout:8/74: fdatasync d0/d1/f11 0 2024-06-15T01:06:53.735 INFO:tasks.workunit.client.0.smithi137.stdout:6/26: creat d0/f6 x:0 0 0 2024-06-15T01:06:53.735 INFO:tasks.workunit.client.1.smithi142.stdout:4/194: readlink d5/d1c/l27 0 2024-06-15T01:06:53.735 INFO:tasks.workunit.client.1.smithi142.stdout:8/75: chown d0/c9 100613594 1 2024-06-15T01:06:53.737 INFO:tasks.workunit.client.1.smithi142.stdout:0/30: truncate f1 4139566 0 2024-06-15T01:06:53.737 INFO:tasks.workunit.client.1.smithi142.stdout:5/52: link d2/l10 d2/de/l13 0 2024-06-15T01:06:53.738 INFO:tasks.workunit.client.1.smithi142.stdout:5/53: dread - d2/ff zero size 2024-06-15T01:06:53.738 INFO:tasks.workunit.client.1.smithi142.stdout:9/51: truncate d6/f9 34783 0 2024-06-15T01:06:53.740 INFO:tasks.workunit.client.1.smithi142.stdout:6/63: rename f10 to f18 0 2024-06-15T01:06:53.746 INFO:tasks.workunit.client.0.smithi137.stdout:5/145: mkdir df/d3d 0 2024-06-15T01:06:53.747 INFO:tasks.workunit.client.1.smithi142.stdout:4/195: chown d5/df/d15 3014 1 2024-06-15T01:06:53.749 INFO:tasks.workunit.client.0.smithi137.stdout:0/66: mkdir d1/d2/dc/d10/d14 0 2024-06-15T01:06:53.750 INFO:tasks.workunit.client.1.smithi142.stdout:0/31: symlink d3/la 0 2024-06-15T01:06:53.751 INFO:tasks.workunit.client.1.smithi142.stdout:0/32: write d3/f9 [68905,35007] 0 2024-06-15T01:06:53.752 INFO:tasks.workunit.client.1.smithi142.stdout:4/196: dwrite d5/f6 [0,4194304] 0 2024-06-15T01:06:53.753 INFO:tasks.workunit.client.1.smithi142.stdout:1/109: sync 2024-06-15T01:06:53.753 INFO:tasks.workunit.client.1.smithi142.stdout:3/105: sync 2024-06-15T01:06:53.753 INFO:tasks.workunit.client.0.smithi137.stdout:4/70: dwrite dd/f13 [0,4194304] 0 2024-06-15T01:06:53.753 INFO:tasks.workunit.client.0.smithi137.stdout:6/27: dread - d0/f6 zero size 2024-06-15T01:06:53.754 INFO:tasks.workunit.client.1.smithi142.stdout:3/106: chown d0/d3/f1a 24853 1 2024-06-15T01:06:53.754 INFO:tasks.workunit.client.0.smithi137.stdout:7/81: write df/d10/f19 [199701,51960] 0 2024-06-15T01:06:53.754 INFO:tasks.workunit.client.0.smithi137.stdout:0/67: stat l0 0 2024-06-15T01:06:53.756 INFO:tasks.workunit.client.1.smithi142.stdout:5/54: creat d2/f14 x:0 0 0 2024-06-15T01:06:53.758 INFO:tasks.workunit.client.0.smithi137.stdout:0/68: read d1/d2/f4 [3156688,79104] 0 2024-06-15T01:06:53.758 INFO:tasks.workunit.client.0.smithi137.stdout:5/146: write f7 [409918,32473] 0 2024-06-15T01:06:53.760 INFO:tasks.workunit.client.0.smithi137.stdout:6/28: chown d0/d5 1820599621 1 2024-06-15T01:06:53.761 INFO:tasks.workunit.client.0.smithi137.stdout:4/71: mknod dd/d15/c1d 0 2024-06-15T01:06:53.761 INFO:tasks.workunit.client.0.smithi137.stdout:3/59: dwrite d0/d2/fc [0,4194304] 0 2024-06-15T01:06:53.763 INFO:tasks.workunit.client.1.smithi142.stdout:9/52: chown d6/f9 42 1 2024-06-15T01:06:53.767 INFO:tasks.workunit.client.0.smithi137.stdout:0/69: write d1/f9 [3773286,7510] 0 2024-06-15T01:06:53.770 INFO:tasks.workunit.client.1.smithi142.stdout:2/55: truncate d8/f9 4560322 0 2024-06-15T01:06:53.770 INFO:tasks.workunit.client.1.smithi142.stdout:0/33: creat d3/d5/fb x:0 0 0 2024-06-15T01:06:53.771 INFO:tasks.workunit.client.1.smithi142.stdout:2/56: dread - d8/fa zero size 2024-06-15T01:06:53.771 INFO:tasks.workunit.client.1.smithi142.stdout:9/53: read d6/f9 [5098,129310] 0 2024-06-15T01:06:53.773 INFO:tasks.workunit.client.1.smithi142.stdout:1/110: truncate d6/f13 3161256 0 2024-06-15T01:06:53.775 INFO:tasks.workunit.client.1.smithi142.stdout:4/197: symlink d5/d9/l4b 0 2024-06-15T01:06:53.775 INFO:tasks.workunit.client.0.smithi137.stdout:3/60: dread d0/fd [0,4194304] 0 2024-06-15T01:06:53.776 INFO:tasks.workunit.client.1.smithi142.stdout:6/64: link c3 c19 0 2024-06-15T01:06:53.777 INFO:tasks.workunit.client.0.smithi137.stdout:7/82: rename c0 to df/d10/c1b 0 2024-06-15T01:06:53.777 INFO:tasks.workunit.client.1.smithi142.stdout:8/76: getdents d0 0 2024-06-15T01:06:53.778 INFO:tasks.workunit.client.1.smithi142.stdout:4/198: write d5/d9/fd [1664121,108683] 0 2024-06-15T01:06:53.778 INFO:tasks.workunit.client.1.smithi142.stdout:2/57: creat d8/fc x:0 0 0 2024-06-15T01:06:53.779 INFO:tasks.workunit.client.0.smithi137.stdout:7/83: chown f5 554291 1 2024-06-15T01:06:53.779 INFO:tasks.workunit.client.0.smithi137.stdout:0/70: dread d1/f9 [0,4194304] 0 2024-06-15T01:06:53.780 INFO:tasks.workunit.client.0.smithi137.stdout:5/147: creat df/d11/d14/d3c/f3e x:0 0 0 2024-06-15T01:06:53.780 INFO:tasks.workunit.client.1.smithi142.stdout:8/77: write d0/f5 [3753008,49935] 0 2024-06-15T01:06:53.780 INFO:tasks.workunit.client.1.smithi142.stdout:9/54: rename d6/dc/dd/f16 to d6/dc/f19 0 2024-06-15T01:06:53.782 INFO:tasks.workunit.client.1.smithi142.stdout:9/55: chown d6/dc/dd/c11 0 1 2024-06-15T01:06:53.782 INFO:tasks.workunit.client.1.smithi142.stdout:3/107: creat d0/f22 x:0 0 0 2024-06-15T01:06:53.784 INFO:tasks.workunit.client.1.smithi142.stdout:1/111: dwrite f5 [8388608,4194304] 0 2024-06-15T01:06:53.784 INFO:tasks.workunit.client.1.smithi142.stdout:6/65: fsync f9 0 2024-06-15T01:06:53.784 INFO:tasks.workunit.client.1.smithi142.stdout:9/56: write f2 [96305,45470] 0 2024-06-15T01:06:53.785 INFO:tasks.workunit.client.0.smithi137.stdout:6/29: creat d0/d5/f7 x:0 0 0 2024-06-15T01:06:53.787 INFO:tasks.workunit.client.1.smithi142.stdout:9/57: truncate f2 5040032 0 2024-06-15T01:06:53.788 INFO:tasks.workunit.client.1.smithi142.stdout:3/108: mknod d0/d1b/c23 0 2024-06-15T01:06:53.789 INFO:tasks.workunit.client.1.smithi142.stdout:9/58: dread d6/f9 [0,4194304] 0 2024-06-15T01:06:53.790 INFO:tasks.workunit.client.1.smithi142.stdout:4/199: dread d5/df/d2d/f35 [0,4194304] 0 2024-06-15T01:06:53.791 INFO:tasks.workunit.client.1.smithi142.stdout:6/66: mkdir d1a 0 2024-06-15T01:06:53.796 INFO:tasks.workunit.client.0.smithi137.stdout:4/72: write f8 [682856,90117] 0 2024-06-15T01:06:53.797 INFO:tasks.workunit.client.1.smithi142.stdout:1/112: creat d6/f26 x:0 0 0 2024-06-15T01:06:53.799 INFO:tasks.workunit.client.1.smithi142.stdout:5/55: dwrite d2/f7 [0,4194304] 0 2024-06-15T01:06:53.799 INFO:tasks.workunit.client.0.smithi137.stdout:7/84: dwrite fd [0,4194304] 0 2024-06-15T01:06:53.802 INFO:tasks.workunit.client.0.smithi137.stdout:0/71: dread d1/f9 [0,4194304] 0 2024-06-15T01:06:53.803 INFO:tasks.workunit.client.1.smithi142.stdout:1/113: dread d6/f1d [0,4194304] 0 2024-06-15T01:06:53.804 INFO:tasks.workunit.client.0.smithi137.stdout:5/148: rmdir df/d11/d14/d3c 39 2024-06-15T01:06:53.804 INFO:tasks.workunit.client.1.smithi142.stdout:3/109: symlink d0/d1b/l24 0 2024-06-15T01:06:53.807 INFO:tasks.workunit.client.0.smithi137.stdout:3/61: dwrite d0/d2/d8/fb [0,4194304] 0 2024-06-15T01:06:53.807 INFO:tasks.workunit.client.1.smithi142.stdout:4/200: rmdir d5/d9/d1a 39 2024-06-15T01:06:53.809 INFO:tasks.workunit.client.0.smithi137.stdout:7/85: unlink df/l11 0 2024-06-15T01:06:53.811 INFO:tasks.workunit.client.1.smithi142.stdout:2/58: link l1 d8/ld 0 2024-06-15T01:06:53.812 INFO:tasks.workunit.client.1.smithi142.stdout:8/78: write d0/f18 [1017049,60152] 0 2024-06-15T01:06:53.816 INFO:tasks.workunit.client.1.smithi142.stdout:5/56: rmdir d2 39 2024-06-15T01:06:53.816 INFO:tasks.workunit.client.0.smithi137.stdout:0/72: mkdir d1/d2/dc/d10/d14/d15 0 2024-06-15T01:06:53.816 INFO:tasks.workunit.client.0.smithi137.stdout:6/30: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:53.817 INFO:tasks.workunit.client.0.smithi137.stdout:3/62: mkdir d0/d13 0 2024-06-15T01:06:53.817 INFO:tasks.workunit.client.0.smithi137.stdout:7/86: write f1 [1327906,53714] 0 2024-06-15T01:06:53.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:53 smithi142 ceph-mon[37379]: pgmap v12: 65 pgs: 65 active+clean; 351 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 225 KiB/s rd, 21 MiB/s wr, 404 op/s 2024-06-15T01:06:53.819 INFO:tasks.workunit.client.1.smithi142.stdout:9/59: dwrite f5 [0,4194304] 0 2024-06-15T01:06:53.820 INFO:tasks.workunit.client.0.smithi137.stdout:5/149: dwrite fc [0,4194304] 0 2024-06-15T01:06:53.824 INFO:tasks.workunit.client.1.smithi142.stdout:6/67: dwrite f18 [0,4194304] 0 2024-06-15T01:06:53.825 INFO:tasks.workunit.client.0.smithi137.stdout:6/31: write d0/f4 [334415,78501] 0 2024-06-15T01:06:53.825 INFO:tasks.workunit.client.0.smithi137.stdout:3/63: chown d0/d2/fc 24 1 2024-06-15T01:06:53.828 INFO:tasks.workunit.client.0.smithi137.stdout:0/73: creat d1/d2/dc/f16 x:0 0 0 2024-06-15T01:06:53.828 INFO:tasks.workunit.client.1.smithi142.stdout:9/60: mknod d6/dc/c1a 0 2024-06-15T01:06:53.829 INFO:tasks.workunit.client.1.smithi142.stdout:9/61: readlink d6/dc/l13 0 2024-06-15T01:06:53.829 INFO:tasks.workunit.client.0.smithi137.stdout:2/48: sync 2024-06-15T01:06:53.830 INFO:tasks.workunit.client.0.smithi137.stdout:1/166: sync 2024-06-15T01:06:53.830 INFO:tasks.workunit.client.1.smithi142.stdout:8/79: dread d0/f8 [0,4194304] 0 2024-06-15T01:06:53.831 INFO:tasks.workunit.client.1.smithi142.stdout:5/57: readlink d2/de/l13 0 2024-06-15T01:06:53.832 INFO:tasks.workunit.client.1.smithi142.stdout:2/59: dread d8/f9 [0,4194304] 0 2024-06-15T01:06:53.833 INFO:tasks.workunit.client.0.smithi137.stdout:6/32: creat d0/d5/f8 x:0 0 0 2024-06-15T01:06:53.833 INFO:tasks.workunit.client.0.smithi137.stdout:2/49: fdatasync d0/f8 0 2024-06-15T01:06:53.834 INFO:tasks.workunit.client.1.smithi142.stdout:2/60: fdatasync f6 0 2024-06-15T01:06:53.835 INFO:tasks.workunit.client.1.smithi142.stdout:8/80: chown d0/f5 15 1 2024-06-15T01:06:53.836 INFO:tasks.workunit.client.1.smithi142.stdout:9/62: unlink f5 0 2024-06-15T01:06:53.836 INFO:tasks.workunit.client.0.smithi137.stdout:7/87: mkdir df/d18/d1c 0 2024-06-15T01:06:53.836 INFO:tasks.workunit.client.1.smithi142.stdout:2/61: fdatasync d8/fc 0 2024-06-15T01:06:53.838 INFO:tasks.workunit.client.1.smithi142.stdout:9/63: write d6/dc/dd/f18 [224527,44862] 0 2024-06-15T01:06:53.840 INFO:tasks.workunit.client.1.smithi142.stdout:2/62: write d8/fa [917119,82020] 0 2024-06-15T01:06:53.840 INFO:tasks.workunit.client.0.smithi137.stdout:4/73: dwrite dd/f16 [0,4194304] 0 2024-06-15T01:06:53.840 INFO:tasks.workunit.client.1.smithi142.stdout:6/68: link l11 d1a/l1b 0 2024-06-15T01:06:53.841 INFO:tasks.workunit.client.0.smithi137.stdout:1/167: truncate d0/d3/f36 230212 0 2024-06-15T01:06:53.841 INFO:tasks.workunit.client.1.smithi142.stdout:2/63: fsync f6 0 2024-06-15T01:06:53.841 INFO:tasks.workunit.client.1.smithi142.stdout:6/69: read f18 [3175307,91472] 0 2024-06-15T01:06:53.842 INFO:tasks.workunit.client.1.smithi142.stdout:5/58: dread d2/fd [0,4194304] 0 2024-06-15T01:06:53.844 INFO:tasks.workunit.client.0.smithi137.stdout:4/74: fdatasync dd/fe 0 2024-06-15T01:06:53.844 INFO:tasks.workunit.client.1.smithi142.stdout:6/70: readlink d1a/l1b 0 2024-06-15T01:06:53.845 INFO:tasks.workunit.client.0.smithi137.stdout:4/75: stat dd/d10/c11 0 2024-06-15T01:06:53.847 INFO:tasks.workunit.client.0.smithi137.stdout:4/76: write f5 [898015,18531] 0 2024-06-15T01:06:53.848 INFO:tasks.workunit.client.0.smithi137.stdout:7/88: mkdir df/d18/d1c/d1d 0 2024-06-15T01:06:53.850 INFO:tasks.workunit.client.1.smithi142.stdout:8/81: getdents d0/d1 0 2024-06-15T01:06:53.851 INFO:tasks.workunit.client.0.smithi137.stdout:6/33: unlink d0/f4 0 2024-06-15T01:06:53.852 INFO:tasks.workunit.client.0.smithi137.stdout:7/89: fsync f5 0 2024-06-15T01:06:53.852 INFO:tasks.workunit.client.0.smithi137.stdout:5/150: dwrite f9 [0,4194304] 0 2024-06-15T01:06:53.852 INFO:tasks.workunit.client.0.smithi137.stdout:1/168: mknod d0/d3/d1b/c42 0 2024-06-15T01:06:53.856 INFO:tasks.workunit.client.1.smithi142.stdout:5/59: mknod d2/de/c15 0 2024-06-15T01:06:53.857 INFO:tasks.workunit.client.1.smithi142.stdout:9/64: dread d6/dc/dd/f18 [0,4194304] 0 2024-06-15T01:06:53.858 INFO:tasks.workunit.client.1.smithi142.stdout:5/60: dread - d2/ff zero size 2024-06-15T01:06:53.861 INFO:tasks.workunit.client.1.smithi142.stdout:8/82: creat d0/d17/f1b x:0 0 0 2024-06-15T01:06:53.861 INFO:tasks.workunit.client.0.smithi137.stdout:1/169: chown d0/d3/f13 68455922 1 2024-06-15T01:06:53.863 INFO:tasks.workunit.client.0.smithi137.stdout:7/90: write fc [1846098,124286] 0 2024-06-15T01:06:53.864 INFO:tasks.workunit.client.1.smithi142.stdout:8/83: truncate d0/d17/f1b 961524 0 2024-06-15T01:06:53.865 INFO:tasks.workunit.client.1.smithi142.stdout:6/71: dwrite f18 [0,4194304] 0 2024-06-15T01:06:53.865 INFO:tasks.workunit.client.1.smithi142.stdout:3/110: dwrite d0/d3/da/f11 [0,4194304] 0 2024-06-15T01:06:53.866 INFO:tasks.workunit.client.0.smithi137.stdout:3/64: dread d0/d2/d8/f11 [0,4194304] 0 2024-06-15T01:06:53.866 INFO:tasks.workunit.client.1.smithi142.stdout:5/61: mkdir d2/de/d16 0 2024-06-15T01:06:53.867 INFO:tasks.workunit.client.1.smithi142.stdout:6/72: chown f17 729 1 2024-06-15T01:06:53.867 INFO:tasks.workunit.client.0.smithi137.stdout:2/50: dread d0/f8 [0,4194304] 0 2024-06-15T01:06:53.868 INFO:tasks.workunit.client.0.smithi137.stdout:7/91: write fd [2568928,40210] 0 2024-06-15T01:06:53.868 INFO:tasks.workunit.client.1.smithi142.stdout:6/73: write f6 [2213032,2710] 0 2024-06-15T01:06:53.868 INFO:tasks.workunit.client.1.smithi142.stdout:5/62: chown d2/f5 299921 1 2024-06-15T01:06:53.869 INFO:tasks.workunit.client.0.smithi137.stdout:6/34: mkdir d0/d5/d9 0 2024-06-15T01:06:53.871 INFO:tasks.workunit.client.0.smithi137.stdout:8/59: sync 2024-06-15T01:06:53.871 INFO:tasks.workunit.client.0.smithi137.stdout:9/57: sync 2024-06-15T01:06:53.871 INFO:tasks.workunit.client.0.smithi137.stdout:4/77: dwrite dd/fe [0,4194304] 0 2024-06-15T01:06:53.871 INFO:tasks.workunit.client.0.smithi137.stdout:0/74: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:53.873 INFO:tasks.workunit.client.1.smithi142.stdout:8/84: dread d0/f18 [0,4194304] 0 2024-06-15T01:06:53.874 INFO:tasks.workunit.client.0.smithi137.stdout:1/170: symlink d0/d10/d1d/l43 0 2024-06-15T01:06:53.878 INFO:tasks.workunit.client.0.smithi137.stdout:2/51: read - d0/fb zero size 2024-06-15T01:06:53.879 INFO:tasks.workunit.client.1.smithi142.stdout:3/111: dread d0/d3/da/f11 [0,4194304] 0 2024-06-15T01:06:53.880 INFO:tasks.workunit.client.0.smithi137.stdout:5/151: rename df/f10 to df/d11/d34/d35/f3f 0 2024-06-15T01:06:53.880 INFO:tasks.workunit.client.0.smithi137.stdout:0/75: chown d1/ld 28435251 1 2024-06-15T01:06:53.881 INFO:tasks.workunit.client.1.smithi142.stdout:8/85: symlink d0/l1c 0 2024-06-15T01:06:53.881 INFO:tasks.workunit.client.0.smithi137.stdout:6/35: write d0/d5/f8 [133513,94567] 0 2024-06-15T01:06:53.882 INFO:tasks.workunit.client.0.smithi137.stdout:3/65: creat d0/d2/d8/f14 x:0 0 0 2024-06-15T01:06:53.882 INFO:tasks.workunit.client.1.smithi142.stdout:6/74: mknod d1a/c1c 0 2024-06-15T01:06:53.882 INFO:tasks.workunit.client.1.smithi142.stdout:5/63: dwrite d2/f7 [0,4194304] 0 2024-06-15T01:06:53.882 INFO:tasks.workunit.client.1.smithi142.stdout:6/75: fsync fc 0 2024-06-15T01:06:53.883 INFO:tasks.workunit.client.1.smithi142.stdout:9/65: mkdir d6/dc/d1b 0 2024-06-15T01:06:53.883 INFO:tasks.workunit.client.0.smithi137.stdout:7/92: mkdir df/d18/d1c/d1d/d1e 0 2024-06-15T01:06:53.885 INFO:tasks.workunit.client.0.smithi137.stdout:2/52: mknod d0/c12 0 2024-06-15T01:06:53.887 INFO:tasks.workunit.client.1.smithi142.stdout:3/112: symlink d0/d1f/d20/l25 0 2024-06-15T01:06:53.887 INFO:tasks.workunit.client.1.smithi142.stdout:8/86: dread d0/f14 [0,4194304] 0 2024-06-15T01:06:53.888 INFO:tasks.workunit.client.0.smithi137.stdout:7/93: unlink f9 0 2024-06-15T01:06:53.888 INFO:tasks.workunit.client.0.smithi137.stdout:6/36: write d0/d5/f7 [677011,130216] 0 2024-06-15T01:06:53.890 INFO:tasks.workunit.client.1.smithi142.stdout:3/113: write d0/d3/f1a [3474266,97272] 0 2024-06-15T01:06:53.891 INFO:tasks.workunit.client.0.smithi137.stdout:3/66: truncate d0/d2/d8/fb 4835752 0 2024-06-15T01:06:53.891 INFO:tasks.workunit.client.1.smithi142.stdout:6/76: symlink d1a/l1d 0 2024-06-15T01:06:53.891 INFO:tasks.workunit.client.1.smithi142.stdout:8/87: rmdir d0/d1 39 2024-06-15T01:06:53.892 INFO:tasks.workunit.client.0.smithi137.stdout:6/37: write d0/d5/f8 [137755,25563] 0 2024-06-15T01:06:53.893 INFO:tasks.workunit.client.0.smithi137.stdout:5/152: rmdir df/d11/d34 39 2024-06-15T01:06:53.893 INFO:tasks.workunit.client.1.smithi142.stdout:3/114: chown d0/d1f 23 1 2024-06-15T01:06:53.894 INFO:tasks.workunit.client.0.smithi137.stdout:9/58: getdents df 0 2024-06-15T01:06:53.894 INFO:tasks.workunit.client.0.smithi137.stdout:8/60: dread f7 [0,4194304] 0 2024-06-15T01:06:53.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/77: dread f4 [0,4194304] 0 2024-06-15T01:06:53.896 INFO:tasks.workunit.client.0.smithi137.stdout:7/94: rename fc to df/d18/d1c/f1f 0 2024-06-15T01:06:53.897 INFO:tasks.workunit.client.0.smithi137.stdout:4/78: getdents dd 0 2024-06-15T01:06:53.897 INFO:tasks.workunit.client.0.smithi137.stdout:1/171: link d0/d3/f22 d0/d3/d1b/f44 0 2024-06-15T01:06:53.898 INFO:tasks.workunit.client.0.smithi137.stdout:4/79: fsync f5 0 2024-06-15T01:06:53.898 INFO:tasks.workunit.client.1.smithi142.stdout:3/115: creat d0/d1f/d20/f26 x:0 0 0 2024-06-15T01:06:53.898 INFO:tasks.workunit.client.1.smithi142.stdout:5/64: dwrite d2/fd [4194304,4194304] 0 2024-06-15T01:06:53.898 INFO:tasks.workunit.client.1.smithi142.stdout:6/78: readlink d1a/l1d 0 2024-06-15T01:06:53.899 INFO:tasks.workunit.client.1.smithi142.stdout:6/79: fsync fc 0 2024-06-15T01:06:53.899 INFO:tasks.workunit.client.1.smithi142.stdout:3/116: chown d0/d3/d15 1333810 1 2024-06-15T01:06:53.901 INFO:tasks.workunit.client.1.smithi142.stdout:3/117: fdatasync d0/d1f/d20/f26 0 2024-06-15T01:06:53.902 INFO:tasks.workunit.client.1.smithi142.stdout:5/65: mkdir d2/d17 0 2024-06-15T01:06:53.904 INFO:tasks.workunit.client.1.smithi142.stdout:3/118: chown d0/l1d 28 1 2024-06-15T01:06:53.904 INFO:tasks.workunit.client.0.smithi137.stdout:0/76: mkdir d1/d2/dc/d10/d14/d15/d17 0 2024-06-15T01:06:53.904 INFO:tasks.workunit.client.1.smithi142.stdout:6/80: creat d1a/f1e x:0 0 0 2024-06-15T01:06:53.905 INFO:tasks.workunit.client.1.smithi142.stdout:8/88: dwrite d0/d17/f1b [0,4194304] 0 2024-06-15T01:06:53.906 INFO:tasks.workunit.client.1.smithi142.stdout:3/119: dread - d0/d1f/d20/f26 zero size 2024-06-15T01:06:53.907 INFO:tasks.workunit.client.1.smithi142.stdout:5/66: mkdir d2/d18 0 2024-06-15T01:06:53.911 INFO:tasks.workunit.client.1.smithi142.stdout:8/89: dread d0/f5 [0,4194304] 0 2024-06-15T01:06:53.912 INFO:tasks.workunit.client.0.smithi137.stdout:9/59: dwrite fc [0,4194304] 0 2024-06-15T01:06:53.912 INFO:tasks.workunit.client.1.smithi142.stdout:6/81: rename f13 to d1a/f1f 0 2024-06-15T01:06:53.913 INFO:tasks.workunit.client.0.smithi137.stdout:3/67: dread d0/d2/d8/fb [0,4194304] 0 2024-06-15T01:06:53.914 INFO:tasks.workunit.client.0.smithi137.stdout:7/95: write df/d18/d1c/f1f [4905928,85784] 0 2024-06-15T01:06:53.914 INFO:tasks.workunit.client.0.smithi137.stdout:4/80: rename lb to dd/l1e 0 2024-06-15T01:06:53.915 INFO:tasks.workunit.client.0.smithi137.stdout:1/172: dread d0/d3/d14/f19 [0,4194304] 0 2024-06-15T01:06:53.916 INFO:tasks.workunit.client.0.smithi137.stdout:6/38: link d0/c1 d0/ca 0 2024-06-15T01:06:53.916 INFO:tasks.workunit.client.0.smithi137.stdout:4/81: readlink l6 0 2024-06-15T01:06:53.918 INFO:tasks.workunit.client.1.smithi142.stdout:6/82: creat d1a/f20 x:0 0 0 2024-06-15T01:06:53.918 INFO:tasks.workunit.client.1.smithi142.stdout:8/90: creat d0/d17/f1d x:0 0 0 2024-06-15T01:06:53.920 INFO:tasks.workunit.client.1.smithi142.stdout:6/83: dread - f17 zero size 2024-06-15T01:06:53.921 INFO:tasks.workunit.client.1.smithi142.stdout:9/66: read d6/dc/dd/f18 [101058,28174] 0 2024-06-15T01:06:53.921 INFO:tasks.workunit.client.0.smithi137.stdout:9/60: unlink f0 0 2024-06-15T01:06:53.921 INFO:tasks.workunit.client.1.smithi142.stdout:8/91: symlink d0/d17/l1e 0 2024-06-15T01:06:53.922 INFO:tasks.workunit.client.0.smithi137.stdout:3/68: readlink d0/d2/l7 0 2024-06-15T01:06:53.923 INFO:tasks.workunit.client.0.smithi137.stdout:4/82: dread f8 [0,4194304] 0 2024-06-15T01:06:53.923 INFO:tasks.workunit.client.0.smithi137.stdout:7/96: creat df/d13/f20 x:0 0 0 2024-06-15T01:06:53.923 INFO:tasks.workunit.client.1.smithi142.stdout:5/67: rmdir d2/de/d16 0 2024-06-15T01:06:53.923 INFO:tasks.workunit.client.0.smithi137.stdout:8/61: getdents d9/de 0 2024-06-15T01:06:53.924 INFO:tasks.workunit.client.1.smithi142.stdout:5/68: truncate d2/f9 877517 0 2024-06-15T01:06:53.924 INFO:tasks.workunit.client.0.smithi137.stdout:3/69: chown d0/fd 0 1 2024-06-15T01:06:53.925 INFO:tasks.workunit.client.0.smithi137.stdout:6/39: dread d0/f2 [0,4194304] 0 2024-06-15T01:06:53.926 INFO:tasks.workunit.client.0.smithi137.stdout:0/77: creat d1/d2/dc/d10/d14/d15/d17/f18 x:0 0 0 2024-06-15T01:06:53.927 INFO:tasks.workunit.client.0.smithi137.stdout:7/97: read df/d10/f19 [193073,80779] 0 2024-06-15T01:06:53.927 INFO:tasks.workunit.client.0.smithi137.stdout:3/70: chown d0/d2/d8/f14 333 1 2024-06-15T01:06:53.929 INFO:tasks.workunit.client.1.smithi142.stdout:9/67: stat d6/dc/f19 0 2024-06-15T01:06:53.929 INFO:tasks.workunit.client.1.smithi142.stdout:8/92: dread d0/d17/f1b [0,4194304] 0 2024-06-15T01:06:53.929 INFO:tasks.workunit.client.0.smithi137.stdout:7/98: dread - df/d13/f20 zero size 2024-06-15T01:06:53.930 INFO:tasks.workunit.client.0.smithi137.stdout:1/173: creat d0/d3/d14/d34/f45 x:0 0 0 2024-06-15T01:06:53.930 INFO:tasks.workunit.client.1.smithi142.stdout:5/69: mkdir d2/de/d19 0 2024-06-15T01:06:53.930 INFO:tasks.workunit.client.0.smithi137.stdout:6/40: dread d0/d5/f8 [0,4194304] 0 2024-06-15T01:06:53.931 INFO:tasks.workunit.client.0.smithi137.stdout:4/83: mkdir dd/d10/d1f 0 2024-06-15T01:06:53.932 INFO:tasks.workunit.client.0.smithi137.stdout:1/174: stat d0/d10 0 2024-06-15T01:06:53.932 INFO:tasks.workunit.client.1.smithi142.stdout:0/34: sync 2024-06-15T01:06:53.932 INFO:tasks.workunit.client.1.smithi142.stdout:7/63: sync 2024-06-15T01:06:53.933 INFO:tasks.workunit.client.0.smithi137.stdout:7/99: stat df/d13/l16 0 2024-06-15T01:06:53.935 INFO:tasks.workunit.client.0.smithi137.stdout:7/100: dread - df/d13/f20 zero size 2024-06-15T01:06:53.936 INFO:tasks.workunit.client.0.smithi137.stdout:0/78: rename d1/d2/f4 to d1/d2/dc/d10/d14/d15/f19 0 2024-06-15T01:06:53.938 INFO:tasks.workunit.client.0.smithi137.stdout:7/101: fdatasync df/f14 0 2024-06-15T01:06:53.939 INFO:tasks.workunit.client.0.smithi137.stdout:4/84: unlink dd/d15/f1a 0 2024-06-15T01:06:53.939 INFO:tasks.workunit.client.1.smithi142.stdout:3/120: write d0/d3/da/d18/f21 [760515,42254] 0 2024-06-15T01:06:53.939 INFO:tasks.workunit.client.1.smithi142.stdout:5/70: dread d2/f12 [0,4194304] 0 2024-06-15T01:06:53.940 INFO:tasks.workunit.client.1.smithi142.stdout:9/68: dwrite d6/dc/dd/f18 [0,4194304] 0 2024-06-15T01:06:53.941 INFO:tasks.workunit.client.1.smithi142.stdout:3/121: write d0/f22 [89755,72096] 0 2024-06-15T01:06:53.942 INFO:tasks.workunit.client.0.smithi137.stdout:6/41: dwrite d0/d5/f7 [0,4194304] 0 2024-06-15T01:06:53.943 INFO:tasks.workunit.client.1.smithi142.stdout:3/122: chown d0/d3/da/c10 842962327 1 2024-06-15T01:06:53.944 INFO:tasks.workunit.client.0.smithi137.stdout:1/175: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:06:53.945 INFO:tasks.workunit.client.0.smithi137.stdout:4/85: write f9 [2359706,77769] 0 2024-06-15T01:06:53.945 INFO:tasks.workunit.client.0.smithi137.stdout:6/42: write d0/f6 [704892,89143] 0 2024-06-15T01:06:53.945 INFO:tasks.workunit.client.1.smithi142.stdout:0/35: rmdir d3/d7 0 2024-06-15T01:06:53.947 INFO:tasks.workunit.client.1.smithi142.stdout:0/36: stat d3/f9 0 2024-06-15T01:06:53.950 INFO:tasks.workunit.client.1.smithi142.stdout:8/93: truncate d0/f15 1915448 0 2024-06-15T01:06:53.952 INFO:tasks.workunit.client.1.smithi142.stdout:6/84: dwrite f0 [0,4194304] 0 2024-06-15T01:06:53.952 INFO:tasks.workunit.client.1.smithi142.stdout:7/64: write d2/d8/dd/d15/f16 [672395,118635] 0 2024-06-15T01:06:53.953 INFO:tasks.workunit.client.1.smithi142.stdout:3/123: creat d0/d1f/f27 x:0 0 0 2024-06-15T01:06:53.954 INFO:tasks.workunit.client.1.smithi142.stdout:3/124: chown d0/d3/da/c10 5 1 2024-06-15T01:06:53.955 INFO:tasks.workunit.client.1.smithi142.stdout:0/37: mknod d3/cc 0 2024-06-15T01:06:53.955 INFO:tasks.workunit.client.1.smithi142.stdout:3/125: readlink d0/d3/l5 0 2024-06-15T01:06:53.956 INFO:tasks.workunit.client.1.smithi142.stdout:0/38: write d3/d5/fb [879192,58590] 0 2024-06-15T01:06:53.956 INFO:tasks.workunit.client.1.smithi142.stdout:5/71: creat d2/de/d19/f1a x:0 0 0 2024-06-15T01:06:53.957 INFO:tasks.workunit.client.0.smithi137.stdout:7/102: rename cb to df/d10/c21 0 2024-06-15T01:06:53.959 INFO:tasks.workunit.client.0.smithi137.stdout:7/103: write f1 [1112862,13054] 0 2024-06-15T01:06:53.959 INFO:tasks.workunit.client.1.smithi142.stdout:0/39: truncate d3/d5/fb 1938648 0 2024-06-15T01:06:53.959 INFO:tasks.workunit.client.1.smithi142.stdout:6/85: mknod d1a/c21 0 2024-06-15T01:06:53.961 INFO:tasks.workunit.client.0.smithi137.stdout:8/62: rename d9/f11 to d9/f12 0 2024-06-15T01:06:53.962 INFO:tasks.workunit.client.1.smithi142.stdout:5/72: symlink d2/de/l1b 0 2024-06-15T01:06:53.962 INFO:tasks.workunit.client.1.smithi142.stdout:0/40: truncate d3/f9 143204 0 2024-06-15T01:06:53.963 INFO:tasks.workunit.client.1.smithi142.stdout:9/69: rename d6/dc/dd/f18 to d6/dc/f1c 0 2024-06-15T01:06:53.964 INFO:tasks.workunit.client.1.smithi142.stdout:7/65: mkdir d2/d8/da/d18 0 2024-06-15T01:06:53.965 INFO:tasks.workunit.client.0.smithi137.stdout:6/43: symlink d0/lb 0 2024-06-15T01:06:53.966 INFO:tasks.workunit.client.1.smithi142.stdout:7/66: chown d2/d6/c13 10976856 1 2024-06-15T01:06:53.966 INFO:tasks.workunit.client.1.smithi142.stdout:3/126: dwrite d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:53.968 INFO:tasks.workunit.client.1.smithi142.stdout:7/67: write d2/d8/f17 [847825,69791] 0 2024-06-15T01:06:53.970 INFO:tasks.workunit.client.0.smithi137.stdout:7/104: dwrite f5 [0,4194304] 0 2024-06-15T01:06:53.975 INFO:tasks.workunit.client.1.smithi142.stdout:0/41: symlink d3/d5/ld 0 2024-06-15T01:06:53.977 INFO:tasks.workunit.client.0.smithi137.stdout:8/63: unlink f7 0 2024-06-15T01:06:53.978 INFO:tasks.workunit.client.1.smithi142.stdout:9/70: write d6/dc/f15 [531453,6379] 0 2024-06-15T01:06:53.980 INFO:tasks.workunit.client.0.smithi137.stdout:7/105: dread f3 [0,4194304] 0 2024-06-15T01:06:53.980 INFO:tasks.workunit.client.1.smithi142.stdout:6/86: link f0 d1a/f22 0 2024-06-15T01:06:53.981 INFO:tasks.workunit.client.0.smithi137.stdout:7/106: fsync fe 0 2024-06-15T01:06:53.981 INFO:tasks.workunit.client.1.smithi142.stdout:6/87: write d1a/f20 [83211,13773] 0 2024-06-15T01:06:53.983 INFO:tasks.workunit.client.0.smithi137.stdout:7/107: write df/d18/d1c/f1f [3393332,66892] 0 2024-06-15T01:06:53.984 INFO:tasks.workunit.client.0.smithi137.stdout:0/79: dread d1/d2/dc/d10/d14/d15/f19 [0,4194304] 0 2024-06-15T01:06:53.985 INFO:tasks.workunit.client.1.smithi142.stdout:3/127: unlink d0/d3/l4 0 2024-06-15T01:06:53.985 INFO:tasks.workunit.client.0.smithi137.stdout:7/108: write df/d13/f20 [725409,31791] 0 2024-06-15T01:06:53.986 INFO:tasks.workunit.client.1.smithi142.stdout:7/68: symlink d2/d8/dd/l19 0 2024-06-15T01:06:53.987 INFO:tasks.workunit.client.0.smithi137.stdout:3/71: truncate d0/d2/f5 2987182 0 2024-06-15T01:06:53.987 INFO:tasks.workunit.client.0.smithi137.stdout:7/109: stat ca 0 2024-06-15T01:06:53.988 INFO:tasks.workunit.client.1.smithi142.stdout:5/73: creat d2/d17/f1c x:0 0 0 2024-06-15T01:06:53.989 INFO:tasks.workunit.client.1.smithi142.stdout:5/74: chown d2/de/l1b 10788 1 2024-06-15T01:06:53.991 INFO:tasks.workunit.client.1.smithi142.stdout:9/71: chown d6/dc/dd/c11 1529 1 2024-06-15T01:06:53.992 INFO:tasks.workunit.client.1.smithi142.stdout:9/72: write f2 [6059046,3910] 0 2024-06-15T01:06:53.993 INFO:tasks.workunit.client.0.smithi137.stdout:1/176: write d0/d3/f22 [672966,119232] 0 2024-06-15T01:06:53.995 INFO:tasks.workunit.client.1.smithi142.stdout:3/128: rename d0/d3/da/f11 to d0/d3/da/d18/f28 0 2024-06-15T01:06:53.995 INFO:tasks.workunit.client.0.smithi137.stdout:8/64: creat d9/de/f13 x:0 0 0 2024-06-15T01:06:53.996 INFO:tasks.workunit.client.1.smithi142.stdout:0/42: rmdir d3 39 2024-06-15T01:06:53.997 INFO:tasks.workunit.client.0.smithi137.stdout:1/177: fdatasync d0/d10/f23 0 2024-06-15T01:06:53.999 INFO:tasks.workunit.client.1.smithi142.stdout:7/69: creat d2/f1a x:0 0 0 2024-06-15T01:06:54.000 INFO:tasks.workunit.client.0.smithi137.stdout:7/110: dwrite fd [0,4194304] 0 2024-06-15T01:06:54.000 INFO:tasks.workunit.client.1.smithi142.stdout:6/88: rmdir d1a 39 2024-06-15T01:06:54.003 INFO:tasks.workunit.client.0.smithi137.stdout:9/61: dwrite fd [8388608,4194304] 0 2024-06-15T01:06:54.003 INFO:tasks.workunit.client.1.smithi142.stdout:5/75: creat d2/de/d19/f1d x:0 0 0 2024-06-15T01:06:54.004 INFO:tasks.workunit.client.0.smithi137.stdout:1/178: write d0/d3/f2f [527151,14192] 0 2024-06-15T01:06:54.004 INFO:tasks.workunit.client.1.smithi142.stdout:5/76: fsync d2/ff 0 2024-06-15T01:06:54.004 INFO:tasks.workunit.client.0.smithi137.stdout:4/86: getdents dd/d15 0 2024-06-15T01:06:54.009 INFO:tasks.workunit.client.1.smithi142.stdout:7/70: dread d2/d8/dd/d15/f16 [0,4194304] 0 2024-06-15T01:06:54.010 INFO:tasks.workunit.client.0.smithi137.stdout:8/65: dread f8 [0,4194304] 0 2024-06-15T01:06:54.012 INFO:tasks.workunit.client.1.smithi142.stdout:1/114: sync 2024-06-15T01:06:54.013 INFO:tasks.workunit.client.1.smithi142.stdout:5/77: chown d2/l8 186512060 1 2024-06-15T01:06:54.015 INFO:tasks.workunit.client.0.smithi137.stdout:4/87: mknod dd/c20 0 2024-06-15T01:06:54.015 INFO:tasks.workunit.client.1.smithi142.stdout:5/78: fdatasync d2/f7 0 2024-06-15T01:06:54.016 INFO:tasks.workunit.client.1.smithi142.stdout:7/71: truncate d2/f1a 325140 0 2024-06-15T01:06:54.016 INFO:tasks.workunit.client.1.smithi142.stdout:3/129: link d0/d3/da/c1e d0/d1b/c29 0 2024-06-15T01:06:54.017 INFO:tasks.workunit.client.1.smithi142.stdout:7/72: rename d2 to d2/d1b 22 2024-06-15T01:06:54.018 INFO:tasks.workunit.client.1.smithi142.stdout:6/89: dwrite f17 [0,4194304] 0 2024-06-15T01:06:54.018 INFO:tasks.workunit.client.1.smithi142.stdout:5/79: chown d2/l10 154 1 2024-06-15T01:06:54.018 INFO:tasks.workunit.client.1.smithi142.stdout:7/73: chown d2/d8 5058 1 2024-06-15T01:06:54.019 INFO:tasks.workunit.client.0.smithi137.stdout:1/179: rename d0/d10/d1d/c35 to d0/d3/d1b/c46 0 2024-06-15T01:06:54.019 INFO:tasks.workunit.client.1.smithi142.stdout:3/130: chown d0/d3/da/c10 222 1 2024-06-15T01:06:54.020 INFO:tasks.workunit.client.1.smithi142.stdout:0/43: dread d3/f4 [0,4194304] 0 2024-06-15T01:06:54.020 INFO:tasks.workunit.client.1.smithi142.stdout:9/73: truncate f2 82347 0 2024-06-15T01:06:54.020 INFO:tasks.workunit.client.0.smithi137.stdout:0/80: rename d1/l5 to d1/l1a 0 2024-06-15T01:06:54.021 INFO:tasks.workunit.client.1.smithi142.stdout:5/80: read - d2/de/d19/f1a zero size 2024-06-15T01:06:54.022 INFO:tasks.workunit.client.0.smithi137.stdout:8/66: mknod d9/de/c14 0 2024-06-15T01:06:54.022 INFO:tasks.workunit.client.1.smithi142.stdout:5/81: write d2/f14 [421904,53099] 0 2024-06-15T01:06:54.022 INFO:tasks.workunit.client.0.smithi137.stdout:6/44: getdents d0/d5 0 2024-06-15T01:06:54.023 INFO:tasks.workunit.client.1.smithi142.stdout:5/82: write d2/f9 [817178,119013] 0 2024-06-15T01:06:54.023 INFO:tasks.workunit.client.1.smithi142.stdout:7/74: creat d2/d6/f1c x:0 0 0 2024-06-15T01:06:54.024 INFO:tasks.workunit.client.0.smithi137.stdout:3/72: creat d0/d2/f15 x:0 0 0 2024-06-15T01:06:54.025 INFO:tasks.workunit.client.1.smithi142.stdout:3/131: creat d0/d1b/f2a x:0 0 0 2024-06-15T01:06:54.026 INFO:tasks.workunit.client.0.smithi137.stdout:9/62: stat df/c10 0 2024-06-15T01:06:54.026 INFO:tasks.workunit.client.1.smithi142.stdout:3/132: read d0/d3/f1a [2017196,105051] 0 2024-06-15T01:06:54.026 INFO:tasks.workunit.client.0.smithi137.stdout:4/88: mkdir dd/d15/d21 0 2024-06-15T01:06:54.028 INFO:tasks.workunit.client.1.smithi142.stdout:6/90: mknod d1a/c23 0 2024-06-15T01:06:54.028 INFO:tasks.workunit.client.1.smithi142.stdout:9/74: creat d6/dc/dd/f1d x:0 0 0 2024-06-15T01:06:54.029 INFO:tasks.workunit.client.0.smithi137.stdout:0/81: creat d1/d2/dc/d10/d14/f1b x:0 0 0 2024-06-15T01:06:54.029 INFO:tasks.workunit.client.1.smithi142.stdout:0/44: truncate d3/f4 1408860 0 2024-06-15T01:06:54.031 INFO:tasks.workunit.client.0.smithi137.stdout:8/67: rename d9/f12 to d9/de/f15 0 2024-06-15T01:06:54.033 INFO:tasks.workunit.client.1.smithi142.stdout:7/75: creat d2/d6/f1d x:0 0 0 2024-06-15T01:06:54.034 INFO:tasks.workunit.client.0.smithi137.stdout:6/45: write d0/f2 [2808681,102845] 0 2024-06-15T01:06:54.034 INFO:tasks.workunit.client.0.smithi137.stdout:9/63: readlink l4 0 2024-06-15T01:06:54.036 INFO:tasks.workunit.client.1.smithi142.stdout:6/91: dwrite d1a/f20 [0,4194304] 0 2024-06-15T01:06:54.036 INFO:tasks.workunit.client.1.smithi142.stdout:3/133: symlink d0/d3/l2b 0 2024-06-15T01:06:54.037 INFO:tasks.workunit.client.0.smithi137.stdout:0/82: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:54.037 INFO:tasks.workunit.client.1.smithi142.stdout:3/134: readlink d0/d1b/l24 0 2024-06-15T01:06:54.038 INFO:tasks.workunit.client.0.smithi137.stdout:4/89: dread f8 [0,4194304] 0 2024-06-15T01:06:54.038 INFO:tasks.workunit.client.1.smithi142.stdout:1/115: truncate d6/f10 2989072 0 2024-06-15T01:06:54.039 INFO:tasks.workunit.client.0.smithi137.stdout:1/180: mkdir d0/d3/d47 0 2024-06-15T01:06:54.039 INFO:tasks.workunit.client.0.smithi137.stdout:7/111: write df/d10/f19 [379151,91730] 0 2024-06-15T01:06:54.040 INFO:tasks.workunit.client.1.smithi142.stdout:4/201: sync 2024-06-15T01:06:54.040 INFO:tasks.workunit.client.1.smithi142.stdout:2/64: sync 2024-06-15T01:06:54.041 INFO:tasks.workunit.client.0.smithi137.stdout:1/181: stat d0/d3/d14/d34 0 2024-06-15T01:06:54.041 INFO:tasks.workunit.client.0.smithi137.stdout:7/112: rename df/d18 to df/d18/d1c/d1d/d1e/d22 22 2024-06-15T01:06:54.046 INFO:tasks.workunit.client.1.smithi142.stdout:7/76: mkdir d2/d6/d1e 0 2024-06-15T01:06:54.047 INFO:tasks.workunit.client.1.smithi142.stdout:2/65: dwrite d8/fc [0,4194304] 0 2024-06-15T01:06:54.050 INFO:tasks.workunit.client.0.smithi137.stdout:7/113: dread df/d13/f20 [0,4194304] 0 2024-06-15T01:06:54.051 INFO:tasks.workunit.client.1.smithi142.stdout:6/92: creat d1a/f24 x:0 0 0 2024-06-15T01:06:54.051 INFO:tasks.workunit.client.0.smithi137.stdout:4/90: creat dd/f22 x:0 0 0 2024-06-15T01:06:54.053 INFO:tasks.workunit.client.0.smithi137.stdout:0/83: dwrite d1/d2/dc/f16 [0,4194304] 0 2024-06-15T01:06:54.053 INFO:tasks.workunit.client.1.smithi142.stdout:6/93: dread f4 [0,4194304] 0 2024-06-15T01:06:54.055 INFO:tasks.workunit.client.1.smithi142.stdout:6/94: dread - d1a/f1f zero size 2024-06-15T01:06:54.055 INFO:tasks.workunit.client.1.smithi142.stdout:7/77: dwrite d2/ff [0,4194304] 0 2024-06-15T01:06:54.055 INFO:tasks.workunit.client.0.smithi137.stdout:8/68: rmdir d9/de 39 2024-06-15T01:06:54.055 INFO:tasks.workunit.client.0.smithi137.stdout:0/84: chown d1/d2 25 1 2024-06-15T01:06:54.056 INFO:tasks.workunit.client.0.smithi137.stdout:9/64: dread f5 [0,4194304] 0 2024-06-15T01:06:54.057 INFO:tasks.workunit.client.0.smithi137.stdout:3/73: write d0/d2/d8/fb [2824170,126095] 0 2024-06-15T01:06:54.059 INFO:tasks.workunit.client.1.smithi142.stdout:5/83: dwrite d2/f12 [0,4194304] 0 2024-06-15T01:06:54.059 INFO:tasks.workunit.client.1.smithi142.stdout:9/75: rename d6/lb to d6/dc/d1b/l1e 0 2024-06-15T01:06:54.062 INFO:tasks.workunit.client.0.smithi137.stdout:6/46: write d0/d5/f8 [1123286,62523] 0 2024-06-15T01:06:54.063 INFO:tasks.workunit.client.1.smithi142.stdout:2/66: creat d8/fe x:0 0 0 2024-06-15T01:06:54.065 INFO:tasks.workunit.client.0.smithi137.stdout:1/182: getdents d0/d3/d47 0 2024-06-15T01:06:54.071 INFO:tasks.workunit.client.0.smithi137.stdout:4/91: mknod dd/c23 0 2024-06-15T01:06:54.074 INFO:tasks.workunit.client.0.smithi137.stdout:6/47: dwrite d0/f2 [0,4194304] 0 2024-06-15T01:06:54.077 INFO:tasks.workunit.client.1.smithi142.stdout:9/76: rmdir d6/dc 39 2024-06-15T01:06:54.077 INFO:tasks.workunit.client.0.smithi137.stdout:1/183: symlink d0/d2d/l48 0 2024-06-15T01:06:54.077 INFO:tasks.workunit.client.0.smithi137.stdout:0/85: dread - d1/d2/dc/d10/d14/d15/d17/f18 zero size 2024-06-15T01:06:54.079 INFO:tasks.workunit.client.0.smithi137.stdout:8/69: dread - d9/de/f13 zero size 2024-06-15T01:06:54.079 INFO:tasks.workunit.client.1.smithi142.stdout:5/84: unlink d2/de/d19/f1d 0 2024-06-15T01:06:54.083 INFO:tasks.workunit.client.0.smithi137.stdout:7/114: dwrite f5 [0,4194304] 0 2024-06-15T01:06:54.084 INFO:tasks.workunit.client.0.smithi137.stdout:6/48: creat d0/fc x:0 0 0 2024-06-15T01:06:54.085 INFO:tasks.workunit.client.1.smithi142.stdout:4/202: rename d5/df/d2d/f35 to d5/d9/d41/f4c 0 2024-06-15T01:06:54.085 INFO:tasks.workunit.client.0.smithi137.stdout:4/92: mknod dd/d10/d1f/c24 0 2024-06-15T01:06:54.085 INFO:tasks.workunit.client.0.smithi137.stdout:8/70: write f1 [1035706,84221] 0 2024-06-15T01:06:54.087 INFO:tasks.workunit.client.1.smithi142.stdout:8/94: sync 2024-06-15T01:06:54.090 INFO:tasks.workunit.client.1.smithi142.stdout:3/135: creat d0/f2c x:0 0 0 2024-06-15T01:06:54.090 INFO:tasks.workunit.client.0.smithi137.stdout:6/49: write d0/f6 [1299542,44215] 0 2024-06-15T01:06:54.090 INFO:tasks.workunit.client.1.smithi142.stdout:1/116: link d6/l18 d6/d25/l27 0 2024-06-15T01:06:54.090 INFO:tasks.workunit.client.0.smithi137.stdout:0/86: write d1/d2/dc/d10/d14/d15/d17/f18 [21993,8385] 0 2024-06-15T01:06:54.091 INFO:tasks.workunit.client.1.smithi142.stdout:1/117: chown d6/d12/d1c 5233 1 2024-06-15T01:06:54.093 INFO:tasks.workunit.client.1.smithi142.stdout:6/95: link d1a/f20 d1a/f25 0 2024-06-15T01:06:54.093 INFO:tasks.workunit.client.0.smithi137.stdout:7/115: write df/d13/f20 [1531383,111752] 0 2024-06-15T01:06:54.094 INFO:tasks.workunit.client.0.smithi137.stdout:8/71: fdatasync d9/de/f15 0 2024-06-15T01:06:54.094 INFO:tasks.workunit.client.1.smithi142.stdout:0/45: truncate d3/d5/fb 1696721 0 2024-06-15T01:06:54.095 INFO:tasks.workunit.client.1.smithi142.stdout:0/46: write d3/f9 [472239,63937] 0 2024-06-15T01:06:54.096 INFO:tasks.workunit.client.0.smithi137.stdout:1/184: dread d0/d3/d1b/f44 [0,4194304] 0 2024-06-15T01:06:54.096 INFO:tasks.workunit.client.0.smithi137.stdout:4/93: chown dd/d10/l19 6062 1 2024-06-15T01:06:54.096 INFO:tasks.workunit.client.0.smithi137.stdout:3/74: dread d0/d2/d8/f11 [0,4194304] 0 2024-06-15T01:06:54.096 INFO:tasks.workunit.client.0.smithi137.stdout:9/65: dread f8 [0,4194304] 0 2024-06-15T01:06:54.097 INFO:tasks.workunit.client.1.smithi142.stdout:9/77: truncate d6/dc/dd/f1d 1039773 0 2024-06-15T01:06:54.098 INFO:tasks.workunit.client.1.smithi142.stdout:5/85: rename d2/f5 to d2/f1e 0 2024-06-15T01:06:54.099 INFO:tasks.workunit.client.0.smithi137.stdout:0/87: write d1/f9 [4138874,67224] 0 2024-06-15T01:06:54.099 INFO:tasks.workunit.client.1.smithi142.stdout:9/78: dread d6/dc/dd/f1d [0,4194304] 0 2024-06-15T01:06:54.101 INFO:tasks.workunit.client.0.smithi137.stdout:0/88: stat d1/d2/dc/d10 0 2024-06-15T01:06:54.101 INFO:tasks.workunit.client.0.smithi137.stdout:7/116: rename ca to df/d18/c23 0 2024-06-15T01:06:54.101 INFO:tasks.workunit.client.0.smithi137.stdout:1/185: chown d0/d3/d14/l1a 7 1 2024-06-15T01:06:54.102 INFO:tasks.workunit.client.1.smithi142.stdout:8/95: rename d0/d17/f1b to d0/d17/f1f 0 2024-06-15T01:06:54.102 INFO:tasks.workunit.client.1.smithi142.stdout:3/136: fsync d0/d3/da/d18/f28 0 2024-06-15T01:06:54.103 INFO:tasks.workunit.client.0.smithi137.stdout:9/66: stat fc 0 2024-06-15T01:06:54.103 INFO:tasks.workunit.client.0.smithi137.stdout:4/94: write dd/f18 [920576,115286] 0 2024-06-15T01:06:54.104 INFO:tasks.workunit.client.1.smithi142.stdout:3/137: read - d0/d1f/f27 zero size 2024-06-15T01:06:54.105 INFO:tasks.workunit.client.0.smithi137.stdout:7/117: chown df/d10/f19 537 1 2024-06-15T01:06:54.105 INFO:tasks.workunit.client.1.smithi142.stdout:6/96: fdatasync f4 0 2024-06-15T01:06:54.106 INFO:tasks.workunit.client.0.smithi137.stdout:0/89: chown d1/d2/f6 75663 1 2024-06-15T01:06:54.107 INFO:tasks.workunit.client.1.smithi142.stdout:5/86: dwrite d2/d17/f1c [0,4194304] 0 2024-06-15T01:06:54.107 INFO:tasks.workunit.client.1.smithi142.stdout:0/47: symlink d3/le 0 2024-06-15T01:06:54.109 INFO:tasks.workunit.client.1.smithi142.stdout:5/87: write d2/de/d19/f1a [77404,16281] 0 2024-06-15T01:06:54.111 INFO:tasks.workunit.client.0.smithi137.stdout:3/75: truncate d0/d2/d8/fb 4863824 0 2024-06-15T01:06:54.111 INFO:tasks.workunit.client.0.smithi137.stdout:1/186: mknod d0/d10/c49 0 2024-06-15T01:06:54.113 INFO:tasks.workunit.client.1.smithi142.stdout:9/79: mknod d6/dc/d1b/c1f 0 2024-06-15T01:06:54.113 INFO:tasks.workunit.client.0.smithi137.stdout:8/72: dwrite f6 [0,4194304] 0 2024-06-15T01:06:54.114 INFO:tasks.workunit.client.0.smithi137.stdout:9/67: symlink df/l11 0 2024-06-15T01:06:54.114 INFO:tasks.workunit.client.1.smithi142.stdout:7/78: write d2/d8/dd/f10 [575015,17965] 0 2024-06-15T01:06:54.115 INFO:tasks.workunit.client.1.smithi142.stdout:3/138: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:54.119 INFO:tasks.workunit.client.0.smithi137.stdout:6/50: dwrite d0/f6 [0,4194304] 0 2024-06-15T01:06:54.119 INFO:tasks.workunit.client.1.smithi142.stdout:2/67: write f4 [718867,36764] 0 2024-06-15T01:06:54.120 INFO:tasks.workunit.client.1.smithi142.stdout:2/68: chown d8/ld 19994 1 2024-06-15T01:06:54.125 INFO:tasks.workunit.client.1.smithi142.stdout:6/97: unlink f17 0 2024-06-15T01:06:54.125 INFO:tasks.workunit.client.0.smithi137.stdout:0/90: mkdir d1/d2/dc/d10/d14/d15/d1c 0 2024-06-15T01:06:54.127 INFO:tasks.workunit.client.0.smithi137.stdout:7/118: dwrite f5 [4194304,4194304] 0 2024-06-15T01:06:54.127 INFO:tasks.workunit.client.0.smithi137.stdout:8/73: rmdir d9/de 39 2024-06-15T01:06:54.128 INFO:tasks.workunit.client.0.smithi137.stdout:7/119: chown df/d18 28494089 1 2024-06-15T01:06:54.130 INFO:tasks.workunit.client.0.smithi137.stdout:4/95: link c7 dd/d15/c25 0 2024-06-15T01:06:54.131 INFO:tasks.workunit.client.0.smithi137.stdout:6/51: symlink d0/ld 0 2024-06-15T01:06:54.133 INFO:tasks.workunit.client.0.smithi137.stdout:3/76: rename d0/d2/f5 to d0/f16 0 2024-06-15T01:06:54.134 INFO:tasks.workunit.client.1.smithi142.stdout:5/88: unlink d2/de/l1b 0 2024-06-15T01:06:54.135 INFO:tasks.workunit.client.0.smithi137.stdout:7/120: mknod df/d18/c24 0 2024-06-15T01:06:54.136 INFO:tasks.workunit.client.0.smithi137.stdout:7/121: stat df/d10/f19 0 2024-06-15T01:06:54.139 INFO:tasks.workunit.client.0.smithi137.stdout:6/52: rmdir d0/d5 39 2024-06-15T01:06:54.140 INFO:tasks.workunit.client.0.smithi137.stdout:4/96: creat dd/d10/d1f/f26 x:0 0 0 2024-06-15T01:06:54.141 INFO:tasks.workunit.client.0.smithi137.stdout:1/187: link d0/d10/d1d/l43 d0/d10/l4a 0 2024-06-15T01:06:54.142 INFO:tasks.workunit.client.0.smithi137.stdout:6/53: chown d0/fc 61 1 2024-06-15T01:06:54.143 INFO:tasks.workunit.client.1.smithi142.stdout:4/203: dwrite d5/d9/d23/f29 [0,4194304] 0 2024-06-15T01:06:54.150 INFO:tasks.workunit.client.0.smithi137.stdout:7/122: dwrite f5 [0,4194304] 0 2024-06-15T01:06:54.150 INFO:tasks.workunit.client.1.smithi142.stdout:4/204: dwrite d5/f8 [0,4194304] 0 2024-06-15T01:06:54.153 INFO:tasks.workunit.client.0.smithi137.stdout:1/188: creat d0/d10/d41/f4b x:0 0 0 2024-06-15T01:06:54.156 INFO:tasks.workunit.client.1.smithi142.stdout:5/89: creat d2/de/f1f x:0 0 0 2024-06-15T01:06:54.156 INFO:tasks.workunit.client.0.smithi137.stdout:4/97: dwrite dd/f16 [0,4194304] 0 2024-06-15T01:06:54.157 INFO:tasks.workunit.client.0.smithi137.stdout:1/189: read d0/d10/f2e [7012344,48352] 0 2024-06-15T01:06:54.157 INFO:tasks.workunit.client.0.smithi137.stdout:3/77: rename d0/d2/d8/c10 to d0/d2/c17 0 2024-06-15T01:06:54.158 INFO:tasks.workunit.client.1.smithi142.stdout:8/96: creat d0/f20 x:0 0 0 2024-06-15T01:06:54.161 INFO:tasks.workunit.client.1.smithi142.stdout:3/139: write d0/d3/f1a [1005325,59235] 0 2024-06-15T01:06:54.163 INFO:tasks.workunit.client.1.smithi142.stdout:7/79: mknod d2/d8/dd/d15/c1f 0 2024-06-15T01:06:54.163 INFO:tasks.workunit.client.0.smithi137.stdout:7/123: rmdir df/d18/d1c 39 2024-06-15T01:06:54.166 INFO:tasks.workunit.client.0.smithi137.stdout:1/190: mkdir d0/d3/d1b/d4c 0 2024-06-15T01:06:54.167 INFO:tasks.workunit.client.1.smithi142.stdout:4/205: mknod d5/d1c/d2a/c4d 0 2024-06-15T01:06:54.168 INFO:tasks.workunit.client.0.smithi137.stdout:4/98: dwrite dd/f22 [0,4194304] 0 2024-06-15T01:06:54.168 INFO:tasks.workunit.client.0.smithi137.stdout:7/124: readlink l7 0 2024-06-15T01:06:54.170 INFO:tasks.workunit.client.1.smithi142.stdout:5/90: symlink d2/l20 0 2024-06-15T01:06:54.170 INFO:tasks.workunit.client.0.smithi137.stdout:6/54: getdents d0 0 2024-06-15T01:06:54.172 INFO:tasks.workunit.client.1.smithi142.stdout:2/69: dwrite d8/f9 [0,4194304] 0 2024-06-15T01:06:54.173 INFO:tasks.workunit.client.1.smithi142.stdout:2/70: chown l0 984 1 2024-06-15T01:06:54.174 INFO:tasks.workunit.client.1.smithi142.stdout:3/140: creat d0/d3/d15/f2d x:0 0 0 2024-06-15T01:06:54.175 INFO:tasks.workunit.client.1.smithi142.stdout:7/80: rmdir d2/d6 39 2024-06-15T01:06:54.176 INFO:tasks.workunit.client.1.smithi142.stdout:6/98: dwrite d1a/f20 [0,4194304] 0 2024-06-15T01:06:54.178 INFO:tasks.workunit.client.1.smithi142.stdout:7/81: write d2/d8/dd/f10 [924345,10846] 0 2024-06-15T01:06:54.179 INFO:tasks.workunit.client.0.smithi137.stdout:0/91: write d1/d2/dc/d10/d14/d15/f19 [1730212,56932] 0 2024-06-15T01:06:54.179 INFO:tasks.workunit.client.1.smithi142.stdout:4/206: rename d5/d1c/d2a/d30/f3f to d5/d9/d23/f4e 0 2024-06-15T01:06:54.179 INFO:tasks.workunit.client.0.smithi137.stdout:1/191: rmdir d0/d3/d14 39 2024-06-15T01:06:54.180 INFO:tasks.workunit.client.1.smithi142.stdout:5/91: creat d2/f21 x:0 0 0 2024-06-15T01:06:54.183 INFO:tasks.workunit.client.1.smithi142.stdout:1/118: dread d6/f10 [0,4194304] 0 2024-06-15T01:06:54.184 INFO:tasks.workunit.client.0.smithi137.stdout:3/78: dwrite d0/d2/d8/f11 [4194304,4194304] 0 2024-06-15T01:06:54.184 INFO:tasks.workunit.client.0.smithi137.stdout:6/55: creat d0/d5/d9/fe x:0 0 0 2024-06-15T01:06:54.184 INFO:tasks.workunit.client.1.smithi142.stdout:8/97: mknod d0/c21 0 2024-06-15T01:06:54.184 INFO:tasks.workunit.client.1.smithi142.stdout:0/48: write f1 [1749315,24813] 0 2024-06-15T01:06:54.185 INFO:tasks.workunit.client.1.smithi142.stdout:2/71: mkdir d8/df 0 2024-06-15T01:06:54.188 INFO:tasks.workunit.client.1.smithi142.stdout:9/80: truncate d6/dc/f1c 1803205 0 2024-06-15T01:06:54.189 INFO:tasks.workunit.client.1.smithi142.stdout:2/72: truncate d8/fe 757311 0 2024-06-15T01:06:54.190 INFO:tasks.workunit.client.0.smithi137.stdout:6/56: write d0/fc [457488,57714] 0 2024-06-15T01:06:54.190 INFO:tasks.workunit.client.1.smithi142.stdout:1/119: truncate d6/d12/d1c/d1e/f20 730920 0 2024-06-15T01:06:54.191 INFO:tasks.workunit.client.1.smithi142.stdout:6/99: symlink d1a/l26 0 2024-06-15T01:06:54.192 INFO:tasks.workunit.client.0.smithi137.stdout:4/99: dread dd/f22 [0,4194304] 0 2024-06-15T01:06:54.192 INFO:tasks.workunit.client.0.smithi137.stdout:7/125: dread df/d18/d1c/f1f [0,4194304] 0 2024-06-15T01:06:54.192 INFO:tasks.workunit.client.0.smithi137.stdout:1/192: creat d0/d1f/f4d x:0 0 0 2024-06-15T01:06:54.193 INFO:tasks.workunit.client.1.smithi142.stdout:3/141: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:54.195 INFO:tasks.workunit.client.1.smithi142.stdout:7/82: chown d2/d6/f1d 216285350 1 2024-06-15T01:06:54.197 INFO:tasks.workunit.client.0.smithi137.stdout:1/193: read d0/d3/f13 [3954137,98991] 0 2024-06-15T01:06:54.197 INFO:tasks.workunit.client.0.smithi137.stdout:6/57: write d0/f2 [3326712,43672] 0 2024-06-15T01:06:54.198 INFO:tasks.workunit.client.1.smithi142.stdout:1/120: mkdir d6/d12/d28 0 2024-06-15T01:06:54.201 INFO:tasks.workunit.client.1.smithi142.stdout:3/142: unlink d0/f22 0 2024-06-15T01:06:54.201 INFO:tasks.workunit.client.0.smithi137.stdout:4/100: truncate dd/d10/f1c 442170 0 2024-06-15T01:06:54.201 INFO:tasks.workunit.client.0.smithi137.stdout:7/126: truncate df/f14 1029671 0 2024-06-15T01:06:54.202 INFO:tasks.workunit.client.1.smithi142.stdout:5/92: dwrite d2/fd [0,4194304] 0 2024-06-15T01:06:54.202 INFO:tasks.workunit.client.1.smithi142.stdout:0/49: read d3/f4 [1140579,85312] 0 2024-06-15T01:06:54.203 INFO:tasks.workunit.client.0.smithi137.stdout:1/194: symlink d0/d10/l4e 0 2024-06-15T01:06:54.203 INFO:tasks.workunit.client.0.smithi137.stdout:6/58: creat d0/d5/ff x:0 0 0 2024-06-15T01:06:54.204 INFO:tasks.workunit.client.1.smithi142.stdout:3/143: read - d0/d1b/f2a zero size 2024-06-15T01:06:54.205 INFO:tasks.workunit.client.0.smithi137.stdout:4/101: mknod dd/c27 0 2024-06-15T01:06:54.205 INFO:tasks.workunit.client.1.smithi142.stdout:4/207: mkdir d5/d9/d1a/d4f 0 2024-06-15T01:06:54.206 INFO:tasks.workunit.client.1.smithi142.stdout:2/73: mkdir d8/df/d10 0 2024-06-15T01:06:54.206 INFO:tasks.workunit.client.1.smithi142.stdout:9/81: dread d6/dc/dd/f1d [0,4194304] 0 2024-06-15T01:06:54.206 INFO:tasks.workunit.client.1.smithi142.stdout:5/93: stat d2/d17 0 2024-06-15T01:06:54.207 INFO:tasks.workunit.client.1.smithi142.stdout:8/98: creat d0/f22 x:0 0 0 2024-06-15T01:06:54.208 INFO:tasks.workunit.client.1.smithi142.stdout:7/83: creat d2/d8/da/d18/f20 x:0 0 0 2024-06-15T01:06:54.209 INFO:tasks.workunit.client.1.smithi142.stdout:5/94: chown d2/ca 414808249 1 2024-06-15T01:06:54.209 INFO:tasks.workunit.client.1.smithi142.stdout:8/99: fsync d0/f20 0 2024-06-15T01:06:54.209 INFO:tasks.workunit.client.1.smithi142.stdout:0/50: creat d3/ff x:0 0 0 2024-06-15T01:06:54.210 INFO:tasks.workunit.client.0.smithi137.stdout:0/92: dread d1/d2/dc/d10/d14/d15/f19 [0,4194304] 0 2024-06-15T01:06:54.211 INFO:tasks.workunit.client.0.smithi137.stdout:3/79: dwrite d0/d2/d8/f11 [0,4194304] 0 2024-06-15T01:06:54.212 INFO:tasks.workunit.client.1.smithi142.stdout:1/121: dread d6/f10 [0,4194304] 0 2024-06-15T01:06:54.213 INFO:tasks.workunit.client.0.smithi137.stdout:6/59: creat d0/f10 x:0 0 0 2024-06-15T01:06:54.213 INFO:tasks.workunit.client.1.smithi142.stdout:7/84: symlink d2/d6/l21 0 2024-06-15T01:06:54.214 INFO:tasks.workunit.client.1.smithi142.stdout:6/100: dwrite d1a/f24 [0,4194304] 0 2024-06-15T01:06:54.214 INFO:tasks.workunit.client.1.smithi142.stdout:2/74: creat d8/df/d10/f11 x:0 0 0 2024-06-15T01:06:54.219 INFO:tasks.workunit.client.0.smithi137.stdout:0/93: chown d1/d2/l3 1742464 1 2024-06-15T01:06:54.219 INFO:tasks.workunit.client.0.smithi137.stdout:7/127: dread df/d10/f19 [0,4194304] 0 2024-06-15T01:06:54.220 INFO:tasks.workunit.client.1.smithi142.stdout:6/101: chown fc 1713269 1 2024-06-15T01:06:54.220 INFO:tasks.workunit.client.1.smithi142.stdout:5/95: truncate d2/f21 736317 0 2024-06-15T01:06:54.220 INFO:tasks.workunit.client.1.smithi142.stdout:3/144: dwrite d0/d1f/f27 [0,4194304] 0 2024-06-15T01:06:54.220 INFO:tasks.workunit.client.1.smithi142.stdout:7/85: mkdir d2/d6/d22 0 2024-06-15T01:06:54.222 INFO:tasks.workunit.client.1.smithi142.stdout:1/122: symlink d6/l29 0 2024-06-15T01:06:54.223 INFO:tasks.workunit.client.1.smithi142.stdout:3/145: write d0/d1f/f27 [682367,23220] 0 2024-06-15T01:06:54.223 INFO:tasks.workunit.client.1.smithi142.stdout:7/86: fsync d2/d8/dd/f12 0 2024-06-15T01:06:54.224 INFO:tasks.workunit.client.0.smithi137.stdout:3/80: write d0/fd [3216822,26306] 0 2024-06-15T01:06:54.225 INFO:tasks.workunit.client.1.smithi142.stdout:3/146: fsync d0/d3/d15/f2d 0 2024-06-15T01:06:54.226 INFO:tasks.workunit.client.1.smithi142.stdout:5/96: read d2/f9 [589223,29369] 0 2024-06-15T01:06:54.226 INFO:tasks.workunit.client.0.smithi137.stdout:1/195: dwrite d0/d3/f36 [0,4194304] 0 2024-06-15T01:06:54.226 INFO:tasks.workunit.client.0.smithi137.stdout:7/128: fdatasync df/d10/f19 0 2024-06-15T01:06:54.227 INFO:tasks.workunit.client.1.smithi142.stdout:5/97: dread - d2/ff zero size 2024-06-15T01:06:54.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/102: dwrite dd/f14 [0,4194304] 0 2024-06-15T01:06:54.230 INFO:tasks.workunit.client.0.smithi137.stdout:0/94: creat d1/d2/dc/d10/d14/d15/d1c/f1d x:0 0 0 2024-06-15T01:06:54.230 INFO:tasks.workunit.client.1.smithi142.stdout:9/82: mknod d6/dc/dd/c20 0 2024-06-15T01:06:54.232 INFO:tasks.workunit.client.1.smithi142.stdout:8/100: rename d0/f8 to d0/f23 0 2024-06-15T01:06:54.232 INFO:tasks.workunit.client.0.smithi137.stdout:0/95: stat d1/d2/lb 0 2024-06-15T01:06:54.234 INFO:tasks.workunit.client.0.smithi137.stdout:3/81: creat d0/d2/d8/f18 x:0 0 0 2024-06-15T01:06:54.234 INFO:tasks.workunit.client.1.smithi142.stdout:1/123: creat d6/f2a x:0 0 0 2024-06-15T01:06:54.236 INFO:tasks.workunit.client.1.smithi142.stdout:8/101: read d0/f23 [2446788,63374] 0 2024-06-15T01:06:54.237 INFO:tasks.workunit.client.0.smithi137.stdout:7/129: read df/d13/f20 [927632,117164] 0 2024-06-15T01:06:54.238 INFO:tasks.workunit.client.0.smithi137.stdout:4/103: creat dd/f28 x:0 0 0 2024-06-15T01:06:54.238 INFO:tasks.workunit.client.0.smithi137.stdout:0/96: creat d1/d2/dc/d10/d14/d15/f1e x:0 0 0 2024-06-15T01:06:54.238 INFO:tasks.workunit.client.1.smithi142.stdout:4/208: dread d5/d9/d18/f2f [0,4194304] 0 2024-06-15T01:06:54.240 INFO:tasks.workunit.client.0.smithi137.stdout:4/104: write dd/f13 [4988642,36946] 0 2024-06-15T01:06:54.241 INFO:tasks.workunit.client.1.smithi142.stdout:3/147: dwrite d0/d3/da/d18/f21 [0,4194304] 0 2024-06-15T01:06:54.243 INFO:tasks.workunit.client.1.smithi142.stdout:9/83: dread d6/dc/f15 [0,4194304] 0 2024-06-15T01:06:54.243 INFO:tasks.workunit.client.1.smithi142.stdout:3/148: read d0/d1f/d20/f26 [2173220,122929] 0 2024-06-15T01:06:54.246 INFO:tasks.workunit.client.1.smithi142.stdout:2/75: getdents d8 0 2024-06-15T01:06:54.250 INFO:tasks.workunit.client.1.smithi142.stdout:6/102: rmdir d1a 39 2024-06-15T01:06:54.251 INFO:tasks.workunit.client.0.smithi137.stdout:3/82: mkdir d0/d13/d19 0 2024-06-15T01:06:54.253 INFO:tasks.workunit.client.0.smithi137.stdout:3/83: chown d0/d2/f15 22197914 1 2024-06-15T01:06:54.253 INFO:tasks.workunit.client.0.smithi137.stdout:3/84: stat d0/d13 0 2024-06-15T01:06:54.253 INFO:tasks.workunit.client.0.smithi137.stdout:4/105: creat dd/d15/d21/f29 x:0 0 0 2024-06-15T01:06:54.254 INFO:tasks.workunit.client.1.smithi142.stdout:2/76: dwrite d8/fe [0,4194304] 0 2024-06-15T01:06:54.255 INFO:tasks.workunit.client.1.smithi142.stdout:5/98: dread d2/f9 [0,4194304] 0 2024-06-15T01:06:54.255 INFO:tasks.workunit.client.1.smithi142.stdout:2/77: write d8/fa [269189,26993] 0 2024-06-15T01:06:54.255 INFO:tasks.workunit.client.0.smithi137.stdout:4/106: write f5 [1915105,70635] 0 2024-06-15T01:06:54.257 INFO:tasks.workunit.client.1.smithi142.stdout:3/149: unlink d0/d3/da/cd 0 2024-06-15T01:06:54.259 INFO:tasks.workunit.client.1.smithi142.stdout:3/150: write d0/d3/da/d18/f21 [1635607,68579] 0 2024-06-15T01:06:54.260 INFO:tasks.workunit.client.0.smithi137.stdout:0/97: link d1/d2/l12 d1/d2/dc/d10/d14/d15/d17/l1f 0 2024-06-15T01:06:54.261 INFO:tasks.workunit.client.1.smithi142.stdout:0/51: rmdir d3 39 2024-06-15T01:06:54.262 INFO:tasks.workunit.client.1.smithi142.stdout:1/124: symlink d6/d12/d28/l2b 0 2024-06-15T01:06:54.263 INFO:tasks.workunit.client.1.smithi142.stdout:1/125: dread - d6/f26 zero size 2024-06-15T01:06:54.263 INFO:tasks.workunit.client.1.smithi142.stdout:8/102: mknod d0/c24 0 2024-06-15T01:06:54.264 INFO:tasks.workunit.client.0.smithi137.stdout:0/98: write d1/f9 [5115500,116211] 0 2024-06-15T01:06:54.265 INFO:tasks.workunit.client.0.smithi137.stdout:3/85: dwrite d0/fd [0,4194304] 0 2024-06-15T01:06:54.265 INFO:tasks.workunit.client.0.smithi137.stdout:0/99: readlink l0 0 2024-06-15T01:06:54.267 INFO:tasks.workunit.client.1.smithi142.stdout:4/209: mkdir d5/d1c/d2a/d30/d46/d50 0 2024-06-15T01:06:54.268 INFO:tasks.workunit.client.0.smithi137.stdout:3/86: write d0/d2/f15 [340620,43107] 0 2024-06-15T01:06:54.268 INFO:tasks.workunit.client.0.smithi137.stdout:4/107: dread f5 [0,4194304] 0 2024-06-15T01:06:54.269 INFO:tasks.workunit.client.0.smithi137.stdout:0/100: truncate d1/d2/dc/f16 4993553 0 2024-06-15T01:06:54.270 INFO:tasks.workunit.client.0.smithi137.stdout:3/87: creat d0/f1a x:0 0 0 2024-06-15T01:06:54.272 INFO:tasks.workunit.client.1.smithi142.stdout:5/99: creat d2/de/d19/f22 x:0 0 0 2024-06-15T01:06:54.272 INFO:tasks.workunit.client.0.smithi137.stdout:4/108: creat dd/d15/d21/f2a x:0 0 0 2024-06-15T01:06:54.273 INFO:tasks.workunit.client.1.smithi142.stdout:2/78: mknod d8/df/d10/c12 0 2024-06-15T01:06:54.274 INFO:tasks.workunit.client.1.smithi142.stdout:7/87: getdents d2/d6 0 2024-06-15T01:06:54.274 INFO:tasks.workunit.client.0.smithi137.stdout:4/109: read dd/fe [2849677,103206] 0 2024-06-15T01:06:54.275 INFO:tasks.workunit.client.0.smithi137.stdout:0/101: mknod d1/c20 0 2024-06-15T01:06:54.276 INFO:tasks.workunit.client.1.smithi142.stdout:1/126: truncate d6/fc 4017580 0 2024-06-15T01:06:54.279 INFO:tasks.workunit.client.1.smithi142.stdout:6/103: symlink d1a/l27 0 2024-06-15T01:06:54.279 INFO:tasks.workunit.client.0.smithi137.stdout:4/110: mknod dd/d10/c2b 0 2024-06-15T01:06:54.280 INFO:tasks.workunit.client.1.smithi142.stdout:6/104: write f4 [635488,5078] 0 2024-06-15T01:06:54.280 INFO:tasks.workunit.client.0.smithi137.stdout:3/88: dwrite d0/f1a [0,4194304] 0 2024-06-15T01:06:54.281 INFO:tasks.workunit.client.1.smithi142.stdout:6/105: truncate d1a/f24 5024023 0 2024-06-15T01:06:54.281 INFO:tasks.workunit.client.1.smithi142.stdout:6/106: fdatasync f18 0 2024-06-15T01:06:54.281 INFO:tasks.workunit.client.1.smithi142.stdout:9/84: creat d6/f21 x:0 0 0 2024-06-15T01:06:54.282 INFO:tasks.workunit.client.0.smithi137.stdout:4/111: fsync dd/d10/d1f/f26 0 2024-06-15T01:06:54.282 INFO:tasks.workunit.client.1.smithi142.stdout:6/107: dread - fc zero size 2024-06-15T01:06:54.283 INFO:tasks.workunit.client.0.smithi137.stdout:4/112: fdatasync dd/f28 0 2024-06-15T01:06:54.283 INFO:tasks.workunit.client.0.smithi137.stdout:0/102: creat d1/d2/f21 x:0 0 0 2024-06-15T01:06:54.283 INFO:tasks.workunit.client.0.smithi137.stdout:4/113: fdatasync fa 0 2024-06-15T01:06:54.284 INFO:tasks.workunit.client.1.smithi142.stdout:5/100: rmdir d2/d17 39 2024-06-15T01:06:54.284 INFO:tasks.workunit.client.1.smithi142.stdout:5/101: read - d2/fc zero size 2024-06-15T01:06:54.284 INFO:tasks.workunit.client.0.smithi137.stdout:0/103: dread - d1/d2/f21 zero size 2024-06-15T01:06:54.287 INFO:tasks.workunit.client.0.smithi137.stdout:0/104: readlink d1/l1a 0 2024-06-15T01:06:54.287 INFO:tasks.workunit.client.1.smithi142.stdout:2/79: mkdir d8/df/d13 0 2024-06-15T01:06:54.287 INFO:tasks.workunit.client.1.smithi142.stdout:6/108: dread d1a/f25 [0,4194304] 0 2024-06-15T01:06:54.287 INFO:tasks.workunit.client.1.smithi142.stdout:2/80: truncate f6 2040039 0 2024-06-15T01:06:54.288 INFO:tasks.workunit.client.1.smithi142.stdout:2/81: chown d8/ld 15 1 2024-06-15T01:06:54.289 INFO:tasks.workunit.client.0.smithi137.stdout:3/89: rmdir d0/d13/d19 0 2024-06-15T01:06:54.289 INFO:tasks.workunit.client.1.smithi142.stdout:0/52: symlink d3/d5/l10 0 2024-06-15T01:06:54.290 INFO:tasks.workunit.client.1.smithi142.stdout:3/151: truncate d0/d3/da/d18/f28 3970540 0 2024-06-15T01:06:54.290 INFO:tasks.workunit.client.1.smithi142.stdout:2/82: truncate d8/df/d10/f11 367057 0 2024-06-15T01:06:54.290 INFO:tasks.workunit.client.1.smithi142.stdout:1/127: mkdir d6/d12/d1c/d2c 0 2024-06-15T01:06:54.291 INFO:tasks.workunit.client.1.smithi142.stdout:2/83: write f4 [245865,98286] 0 2024-06-15T01:06:54.292 INFO:tasks.workunit.client.1.smithi142.stdout:8/103: mkdir d0/d25 0 2024-06-15T01:06:54.292 INFO:tasks.workunit.client.1.smithi142.stdout:2/84: write d8/df/d10/f11 [809705,73761] 0 2024-06-15T01:06:54.293 INFO:tasks.workunit.client.1.smithi142.stdout:8/104: fsync d0/f22 0 2024-06-15T01:06:54.294 INFO:tasks.workunit.client.1.smithi142.stdout:4/210: mkdir d5/d1c/d2a/d30/d46/d50/d51 0 2024-06-15T01:06:54.295 INFO:tasks.workunit.client.1.smithi142.stdout:4/211: read - d5/d9/da/f3c zero size 2024-06-15T01:06:54.295 INFO:tasks.workunit.client.1.smithi142.stdout:4/212: fdatasync d5/d9/d23/f29 0 2024-06-15T01:06:54.296 INFO:tasks.workunit.client.1.smithi142.stdout:9/85: mkdir d6/dc/dd/d22 0 2024-06-15T01:06:54.297 INFO:tasks.workunit.client.0.smithi137.stdout:3/90: dread d0/f16 [0,4194304] 0 2024-06-15T01:06:54.297 INFO:tasks.workunit.client.0.smithi137.stdout:7/130: dwrite f3 [4194304,4194304] 0 2024-06-15T01:06:54.300 INFO:tasks.workunit.client.1.smithi142.stdout:6/109: creat d1a/f28 x:0 0 0 2024-06-15T01:06:54.300 INFO:tasks.workunit.client.1.smithi142.stdout:6/110: chown fc 57195907 1 2024-06-15T01:06:54.302 INFO:tasks.workunit.client.0.smithi137.stdout:7/131: dread df/d13/f20 [0,4194304] 0 2024-06-15T01:06:54.302 INFO:tasks.workunit.client.1.smithi142.stdout:7/88: mknod d2/c23 0 2024-06-15T01:06:54.303 INFO:tasks.workunit.client.1.smithi142.stdout:7/89: readlink d2/l4 0 2024-06-15T01:06:54.304 INFO:tasks.workunit.client.1.smithi142.stdout:0/53: symlink d3/d5/l11 0 2024-06-15T01:06:54.304 INFO:tasks.workunit.client.0.smithi137.stdout:7/132: write fd [1989922,7084] 0 2024-06-15T01:06:54.306 INFO:tasks.workunit.client.1.smithi142.stdout:3/152: mknod d0/d1f/d20/c2e 0 2024-06-15T01:06:54.309 INFO:tasks.workunit.client.1.smithi142.stdout:6/111: dread f18 [0,4194304] 0 2024-06-15T01:06:54.309 INFO:tasks.workunit.client.0.smithi137.stdout:3/91: dwrite d0/d2/d8/f18 [0,4194304] 0 2024-06-15T01:06:54.309 INFO:tasks.workunit.client.0.smithi137.stdout:6/60: truncate d0/f2 1285530 0 2024-06-15T01:06:54.310 INFO:tasks.workunit.client.1.smithi142.stdout:1/128: rename d6/f1d to d6/d12/f2d 0 2024-06-15T01:06:54.310 INFO:tasks.workunit.client.1.smithi142.stdout:6/112: dread f18 [0,4194304] 0 2024-06-15T01:06:54.311 INFO:tasks.workunit.client.0.smithi137.stdout:7/133: mknod df/d10/d1a/c25 0 2024-06-15T01:06:54.311 INFO:tasks.workunit.client.1.smithi142.stdout:1/129: write f5 [10259202,52415] 0 2024-06-15T01:06:54.316 INFO:tasks.workunit.client.0.smithi137.stdout:6/61: rmdir d0/d5 39 2024-06-15T01:06:54.319 INFO:tasks.workunit.client.0.smithi137.stdout:3/92: dwrite d0/fd [0,4194304] 0 2024-06-15T01:06:54.320 INFO:tasks.workunit.client.1.smithi142.stdout:9/86: mkdir d6/dc/dd/d23 0 2024-06-15T01:06:54.320 INFO:tasks.workunit.client.0.smithi137.stdout:7/134: creat df/d10/d1a/f26 x:0 0 0 2024-06-15T01:06:54.322 INFO:tasks.workunit.client.0.smithi137.stdout:0/105: truncate d1/d2/dc/d10/d14/d15/f19 4006551 0 2024-06-15T01:06:54.324 INFO:tasks.workunit.client.0.smithi137.stdout:0/106: fdatasync d1/d2/dc/d10/d14/f1b 0 2024-06-15T01:06:54.324 INFO:tasks.workunit.client.0.smithi137.stdout:7/135: dread df/d10/f19 [0,4194304] 0 2024-06-15T01:06:54.325 INFO:tasks.workunit.client.1.smithi142.stdout:0/54: mkdir d3/d12 0 2024-06-15T01:06:54.326 INFO:tasks.workunit.client.1.smithi142.stdout:0/55: readlink d3/l6 0 2024-06-15T01:06:54.326 INFO:tasks.workunit.client.1.smithi142.stdout:0/56: dread - d3/ff zero size 2024-06-15T01:06:54.328 INFO:tasks.workunit.client.0.smithi137.stdout:6/62: mknod d0/d5/c11 0 2024-06-15T01:06:54.329 INFO:tasks.workunit.client.0.smithi137.stdout:3/93: creat d0/d2/f1b x:0 0 0 2024-06-15T01:06:54.331 INFO:tasks.workunit.client.0.smithi137.stdout:6/63: creat d0/d5/f12 x:0 0 0 2024-06-15T01:06:54.333 INFO:tasks.workunit.client.0.smithi137.stdout:6/64: write d0/d5/f7 [1262478,44096] 0 2024-06-15T01:06:54.333 INFO:tasks.workunit.client.1.smithi142.stdout:2/85: rename d8/lb to d8/df/l14 0 2024-06-15T01:06:54.335 INFO:tasks.workunit.client.1.smithi142.stdout:2/86: dread f4 [0,4194304] 0 2024-06-15T01:06:54.335 INFO:tasks.workunit.client.1.smithi142.stdout:6/113: symlink d1a/l29 0 2024-06-15T01:06:54.335 INFO:tasks.workunit.client.0.smithi137.stdout:3/94: creat d0/d2/d8/f1c x:0 0 0 2024-06-15T01:06:54.335 INFO:tasks.workunit.client.1.smithi142.stdout:2/87: readlink l0 0 2024-06-15T01:06:54.336 INFO:tasks.workunit.client.0.smithi137.stdout:3/95: write d0/d2/f1b [547624,87037] 0 2024-06-15T01:06:54.338 INFO:tasks.workunit.client.1.smithi142.stdout:8/105: symlink d0/d1/l26 0 2024-06-15T01:06:54.338 INFO:tasks.workunit.client.0.smithi137.stdout:7/136: getdents df/d10 0 2024-06-15T01:06:54.339 INFO:tasks.workunit.client.0.smithi137.stdout:3/96: creat d0/f1d x:0 0 0 2024-06-15T01:06:54.340 INFO:tasks.workunit.client.1.smithi142.stdout:2/88: dread d8/fc [0,4194304] 0 2024-06-15T01:06:54.344 INFO:tasks.workunit.client.1.smithi142.stdout:7/90: creat d2/d8/dd/d14/f24 x:0 0 0 2024-06-15T01:06:54.344 INFO:tasks.workunit.client.0.smithi137.stdout:7/137: mkdir df/d18/d1c/d1d/d27 0 2024-06-15T01:06:54.344 INFO:tasks.workunit.client.0.smithi137.stdout:7/138: chown l7 125066313 1 2024-06-15T01:06:54.345 INFO:tasks.workunit.client.1.smithi142.stdout:0/57: symlink d3/d5/l13 0 2024-06-15T01:06:54.347 INFO:tasks.workunit.client.0.smithi137.stdout:3/97: mkdir d0/d2/d1e 0 2024-06-15T01:06:54.347 INFO:tasks.workunit.client.1.smithi142.stdout:4/213: write f2 [3240256,27588] 0 2024-06-15T01:06:54.348 INFO:tasks.workunit.client.1.smithi142.stdout:7/91: dread d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:06:54.348 INFO:tasks.workunit.client.0.smithi137.stdout:0/107: truncate d1/f9 4090970 0 2024-06-15T01:06:54.348 INFO:tasks.workunit.client.0.smithi137.stdout:0/108: readlink d1/ld 0 2024-06-15T01:06:54.349 INFO:tasks.workunit.client.1.smithi142.stdout:7/92: chown d2/d8/dd/d15/f16 13 1 2024-06-15T01:06:54.349 INFO:tasks.workunit.client.0.smithi137.stdout:7/139: creat df/d18/d1c/d1d/f28 x:0 0 0 2024-06-15T01:06:54.350 INFO:tasks.workunit.client.0.smithi137.stdout:0/109: write d1/d2/dc/f16 [2870528,5160] 0 2024-06-15T01:06:54.351 INFO:tasks.workunit.client.0.smithi137.stdout:3/98: creat d0/f1f x:0 0 0 2024-06-15T01:06:54.351 INFO:tasks.workunit.client.0.smithi137.stdout:7/140: chown df/d18/d1c/d1d/d1e 15693971 1 2024-06-15T01:06:54.353 INFO:tasks.workunit.client.1.smithi142.stdout:8/106: unlink d0/d1/c12 0 2024-06-15T01:06:54.354 INFO:tasks.workunit.client.1.smithi142.stdout:9/87: symlink d6/dc/dd/d22/l24 0 2024-06-15T01:06:54.355 INFO:tasks.workunit.client.0.smithi137.stdout:3/99: creat d0/d13/f20 x:0 0 0 2024-06-15T01:06:54.356 INFO:tasks.workunit.client.1.smithi142.stdout:2/89: mknod d8/df/d10/c15 0 2024-06-15T01:06:54.356 INFO:tasks.workunit.client.0.smithi137.stdout:7/141: mknod df/d18/d1c/d1d/c29 0 2024-06-15T01:06:54.357 INFO:tasks.workunit.client.1.smithi142.stdout:5/102: getdents d2/de 0 2024-06-15T01:06:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:6/65: chown d0/f2 8962028 1 2024-06-15T01:06:54.358 INFO:tasks.workunit.client.1.smithi142.stdout:2/90: write d8/fa [1726539,23551] 0 2024-06-15T01:06:54.358 INFO:tasks.workunit.client.0.smithi137.stdout:6/66: fdatasync d0/d5/ff 0 2024-06-15T01:06:54.359 INFO:tasks.workunit.client.1.smithi142.stdout:5/103: write d2/de/d19/f1a [899055,12815] 0 2024-06-15T01:06:54.360 INFO:tasks.workunit.client.0.smithi137.stdout:0/110: unlink d1/d2/l12 0 2024-06-15T01:06:54.362 INFO:tasks.workunit.client.1.smithi142.stdout:3/153: dwrite d0/d1f/d20/f26 [4194304,4194304] 0 2024-06-15T01:06:54.364 INFO:tasks.workunit.client.0.smithi137.stdout:3/100: rename d0/d2/d8/f18 to d0/d13/f21 0 2024-06-15T01:06:54.365 INFO:tasks.workunit.client.1.smithi142.stdout:7/93: rename d2/d8/dd/d15 to d2/d8/da/d18/d25 0 2024-06-15T01:06:54.365 INFO:tasks.workunit.client.0.smithi137.stdout:3/101: read - d0/d2/d8/f14 zero size 2024-06-15T01:06:54.366 INFO:tasks.workunit.client.0.smithi137.stdout:0/111: mkdir d1/d2/dc/d10/d22 0 2024-06-15T01:06:54.366 INFO:tasks.workunit.client.1.smithi142.stdout:1/130: truncate d6/f10 4447454 0 2024-06-15T01:06:54.368 INFO:tasks.workunit.client.1.smithi142.stdout:9/88: mkdir d6/dc/d1b/d25 0 2024-06-15T01:06:54.369 INFO:tasks.workunit.client.1.smithi142.stdout:9/89: chown d6/f9 270 1 2024-06-15T01:06:54.370 INFO:tasks.workunit.client.0.smithi137.stdout:6/67: rename d0/f2 to d0/d5/f13 0 2024-06-15T01:06:54.371 INFO:tasks.workunit.client.1.smithi142.stdout:0/58: creat d3/d12/f14 x:0 0 0 2024-06-15T01:06:54.371 INFO:tasks.workunit.client.1.smithi142.stdout:0/59: readlink d3/l6 0 2024-06-15T01:06:54.371 INFO:tasks.workunit.client.1.smithi142.stdout:2/91: write f4 [700447,11124] 0 2024-06-15T01:06:54.373 INFO:tasks.workunit.client.0.smithi137.stdout:6/68: write d0/d5/f8 [47338,108369] 0 2024-06-15T01:06:54.375 INFO:tasks.workunit.client.1.smithi142.stdout:4/214: link f2 d5/d1c/d2a/d30/d46/f52 0 2024-06-15T01:06:54.377 INFO:tasks.workunit.client.0.smithi137.stdout:3/102: dwrite d0/d13/f20 [0,4194304] 0 2024-06-15T01:06:54.377 INFO:tasks.workunit.client.1.smithi142.stdout:5/104: symlink d2/de/d19/l23 0 2024-06-15T01:06:54.377 INFO:tasks.workunit.client.0.smithi137.stdout:0/112: symlink d1/d2/dc/d10/d14/l23 0 2024-06-15T01:06:54.377 INFO:tasks.workunit.client.1.smithi142.stdout:2/92: dwrite d8/fe [0,4194304] 0 2024-06-15T01:06:54.378 INFO:tasks.workunit.client.1.smithi142.stdout:5/105: write d2/fc [370116,109672] 0 2024-06-15T01:06:54.380 INFO:tasks.workunit.client.1.smithi142.stdout:7/94: unlink d2/d8/dd/l19 0 2024-06-15T01:06:54.380 INFO:tasks.workunit.client.0.smithi137.stdout:6/69: readlink d0/ld 0 2024-06-15T01:06:54.381 INFO:tasks.workunit.client.0.smithi137.stdout:6/70: fdatasync d0/f10 0 2024-06-15T01:06:54.383 INFO:tasks.workunit.client.1.smithi142.stdout:1/131: symlink d6/d12/d28/l2e 0 2024-06-15T01:06:54.384 INFO:tasks.workunit.client.1.smithi142.stdout:8/107: fsync d0/f18 0 2024-06-15T01:06:54.384 INFO:tasks.workunit.client.1.smithi142.stdout:7/95: write d2/d6/f1d [416920,107151] 0 2024-06-15T01:06:54.385 INFO:tasks.workunit.client.1.smithi142.stdout:9/90: creat d6/dc/dd/d22/f26 x:0 0 0 2024-06-15T01:06:54.387 INFO:tasks.workunit.client.1.smithi142.stdout:0/60: symlink d3/d5/l15 0 2024-06-15T01:06:54.389 INFO:tasks.workunit.client.0.smithi137.stdout:3/103: getdents d0 0 2024-06-15T01:06:54.390 INFO:tasks.workunit.client.0.smithi137.stdout:3/104: fdatasync d0/d2/d8/f11 0 2024-06-15T01:06:54.390 INFO:tasks.workunit.client.0.smithi137.stdout:6/71: rmdir d0/d5 39 2024-06-15T01:06:54.391 INFO:tasks.workunit.client.1.smithi142.stdout:4/215: fdatasync d5/d1c/d2a/d30/d46/f52 0 2024-06-15T01:06:54.392 INFO:tasks.workunit.client.1.smithi142.stdout:2/93: creat d8/f16 x:0 0 0 2024-06-15T01:06:54.392 INFO:tasks.workunit.client.0.smithi137.stdout:3/105: truncate d0/d2/d8/f1c 985944 0 2024-06-15T01:06:54.393 INFO:tasks.workunit.client.0.smithi137.stdout:3/106: fdatasync d0/d2/f15 0 2024-06-15T01:06:54.394 INFO:tasks.workunit.client.1.smithi142.stdout:5/106: dwrite d2/de/f1f [0,4194304] 0 2024-06-15T01:06:54.394 INFO:tasks.workunit.client.1.smithi142.stdout:2/94: chown c2 63224 1 2024-06-15T01:06:54.394 INFO:tasks.workunit.client.1.smithi142.stdout:3/154: rename d0/d3/da/c1e to d0/d3/da/c2f 0 2024-06-15T01:06:54.396 INFO:tasks.workunit.client.1.smithi142.stdout:3/155: chown d0/d1b/l24 59 1 2024-06-15T01:06:54.397 INFO:tasks.workunit.client.1.smithi142.stdout:8/108: dwrite d0/f10 [0,4194304] 0 2024-06-15T01:06:54.399 INFO:tasks.workunit.client.1.smithi142.stdout:8/109: read d0/f5 [3578382,42617] 0 2024-06-15T01:06:54.402 INFO:tasks.workunit.client.0.smithi137.stdout:3/107: mkdir d0/d2/d22 0 2024-06-15T01:06:54.406 INFO:tasks.workunit.client.0.smithi137.stdout:3/108: mkdir d0/d2/d23 0 2024-06-15T01:06:54.409 INFO:tasks.workunit.client.0.smithi137.stdout:3/109: mkdir d0/d2/d22/d24 0 2024-06-15T01:06:54.410 INFO:tasks.workunit.client.1.smithi142.stdout:2/95: unlink c7 0 2024-06-15T01:06:54.411 INFO:tasks.workunit.client.1.smithi142.stdout:5/107: rmdir d2/de 39 2024-06-15T01:06:54.412 INFO:tasks.workunit.client.1.smithi142.stdout:3/156: mknod d0/d3/c30 0 2024-06-15T01:06:54.413 INFO:tasks.workunit.client.0.smithi137.stdout:6/72: dwrite d0/d5/f13 [0,4194304] 0 2024-06-15T01:06:54.415 INFO:tasks.workunit.client.1.smithi142.stdout:8/110: creat d0/d17/f27 x:0 0 0 2024-06-15T01:06:54.416 INFO:tasks.workunit.client.1.smithi142.stdout:8/111: fsync d0/d1/f11 0 2024-06-15T01:06:54.420 INFO:tasks.workunit.client.0.smithi137.stdout:6/73: link d0/c1 d0/c14 0 2024-06-15T01:06:54.421 INFO:tasks.workunit.client.0.smithi137.stdout:6/74: fsync d0/d5/f8 0 2024-06-15T01:06:54.422 INFO:tasks.workunit.client.0.smithi137.stdout:6/75: write d0/f10 [773446,112319] 0 2024-06-15T01:06:54.423 INFO:tasks.workunit.client.1.smithi142.stdout:2/96: rename d8/df/d10/c15 to d8/df/c17 0 2024-06-15T01:06:54.423 INFO:tasks.workunit.client.0.smithi137.stdout:6/76: dread - d0/d5/ff zero size 2024-06-15T01:06:54.424 INFO:tasks.workunit.client.0.smithi137.stdout:3/110: truncate d0/fd 2601669 0 2024-06-15T01:06:54.426 INFO:tasks.workunit.client.0.smithi137.stdout:6/77: creat d0/d5/f15 x:0 0 0 2024-06-15T01:06:54.427 INFO:tasks.workunit.client.0.smithi137.stdout:3/111: symlink d0/d2/d22/d24/l25 0 2024-06-15T01:06:54.427 INFO:tasks.workunit.client.0.smithi137.stdout:3/112: readlink d0/l12 0 2024-06-15T01:06:54.428 INFO:tasks.workunit.client.0.smithi137.stdout:3/113: chown d0/d2/d8/f11 28513511 1 2024-06-15T01:06:54.428 INFO:tasks.workunit.client.1.smithi142.stdout:5/108: dread d2/de/d19/f1a [0,4194304] 0 2024-06-15T01:06:54.429 INFO:tasks.workunit.client.1.smithi142.stdout:1/132: dread d6/f10 [0,4194304] 0 2024-06-15T01:06:54.430 INFO:tasks.workunit.client.1.smithi142.stdout:1/133: chown d6 0 1 2024-06-15T01:06:54.431 INFO:tasks.workunit.client.1.smithi142.stdout:3/157: rename d0/d3/c30 to d0/d1b/c31 0 2024-06-15T01:06:54.431 INFO:tasks.workunit.client.1.smithi142.stdout:1/134: fdatasync d6/f9 0 2024-06-15T01:06:54.432 INFO:tasks.workunit.client.0.smithi137.stdout:3/114: rename d0/d2/d8/fb to d0/d13/f26 0 2024-06-15T01:06:54.432 INFO:tasks.workunit.client.1.smithi142.stdout:1/135: write d6/f26 [861287,99032] 0 2024-06-15T01:06:54.434 INFO:tasks.workunit.client.1.smithi142.stdout:9/91: link d6/dc/cf d6/dc/d1b/d25/c27 0 2024-06-15T01:06:54.434 INFO:tasks.workunit.client.1.smithi142.stdout:1/136: truncate d6/f2a 556634 0 2024-06-15T01:06:54.435 INFO:tasks.workunit.client.0.smithi137.stdout:6/78: fsync d0/d5/f15 0 2024-06-15T01:06:54.436 INFO:tasks.workunit.client.1.smithi142.stdout:7/96: truncate d2/ff 1838878 0 2024-06-15T01:06:54.436 INFO:tasks.workunit.client.1.smithi142.stdout:9/92: dread - d6/dc/dd/d22/f26 zero size 2024-06-15T01:06:54.436 INFO:tasks.workunit.client.1.smithi142.stdout:0/61: dwrite d3/f4 [0,4194304] 0 2024-06-15T01:06:54.437 INFO:tasks.workunit.client.1.smithi142.stdout:4/216: getdents d5/d9/d23/d32 0 2024-06-15T01:06:54.437 INFO:tasks.workunit.client.0.smithi137.stdout:6/79: truncate d0/d5/ff 568467 0 2024-06-15T01:06:54.441 INFO:tasks.workunit.client.1.smithi142.stdout:8/112: write d0/f14 [18536,42433] 0 2024-06-15T01:06:54.441 INFO:tasks.workunit.client.0.smithi137.stdout:3/115: dwrite d0/d13/f21 [0,4194304] 0 2024-06-15T01:06:54.443 INFO:tasks.workunit.client.1.smithi142.stdout:2/97: creat d8/df/d13/f18 x:0 0 0 2024-06-15T01:06:54.444 INFO:tasks.workunit.client.1.smithi142.stdout:2/98: chown d8/f16 1 1 2024-06-15T01:06:54.445 INFO:tasks.workunit.client.1.smithi142.stdout:9/93: dwrite d6/dc/dd/d22/f26 [0,4194304] 0 2024-06-15T01:06:54.445 INFO:tasks.workunit.client.1.smithi142.stdout:2/99: fdatasync d8/fe 0 2024-06-15T01:06:54.447 INFO:tasks.workunit.client.1.smithi142.stdout:1/137: creat d6/d12/f2f x:0 0 0 2024-06-15T01:06:54.450 INFO:tasks.workunit.client.1.smithi142.stdout:0/62: unlink d3/le 0 2024-06-15T01:06:54.451 INFO:tasks.workunit.client.0.smithi137.stdout:6/80: dwrite d0/d5/f12 [0,4194304] 0 2024-06-15T01:06:54.451 INFO:tasks.workunit.client.1.smithi142.stdout:0/63: truncate d3/d12/f14 796223 0 2024-06-15T01:06:54.451 INFO:tasks.workunit.client.1.smithi142.stdout:0/64: fsync d3/f9 0 2024-06-15T01:06:54.456 INFO:tasks.workunit.client.1.smithi142.stdout:5/109: rename d2/d17/f1c to d2/de/f24 0 2024-06-15T01:06:54.457 INFO:tasks.workunit.client.0.smithi137.stdout:6/81: creat d0/d5/f16 x:0 0 0 2024-06-15T01:06:54.460 INFO:tasks.workunit.client.0.smithi137.stdout:3/116: dwrite d0/d2/d8/f1c [0,4194304] 0 2024-06-15T01:06:54.460 INFO:tasks.workunit.client.0.smithi137.stdout:6/82: chown d0/f6 552811 1 2024-06-15T01:06:54.461 INFO:tasks.workunit.client.1.smithi142.stdout:2/100: truncate d8/fc 2575539 0 2024-06-15T01:06:54.461 INFO:tasks.workunit.client.1.smithi142.stdout:1/138: mkdir d6/d12/d30 0 2024-06-15T01:06:54.464 INFO:tasks.workunit.client.1.smithi142.stdout:1/139: truncate d6/f26 1045218 0 2024-06-15T01:06:54.465 INFO:tasks.workunit.client.0.smithi137.stdout:6/83: creat d0/d5/f17 x:0 0 0 2024-06-15T01:06:54.465 INFO:tasks.workunit.client.1.smithi142.stdout:4/217: dwrite d5/d9/d41/f4c [4194304,4194304] 0 2024-06-15T01:06:54.466 INFO:tasks.workunit.client.1.smithi142.stdout:9/94: symlink d6/l28 0 2024-06-15T01:06:54.467 INFO:tasks.workunit.client.0.smithi137.stdout:3/117: rename d0/d2/c17 to d0/d2/d1e/c27 0 2024-06-15T01:06:54.467 INFO:tasks.workunit.client.1.smithi142.stdout:2/101: mkdir d8/d19 0 2024-06-15T01:06:54.468 INFO:tasks.workunit.client.0.smithi137.stdout:6/84: creat d0/d5/f18 x:0 0 0 2024-06-15T01:06:54.470 INFO:tasks.workunit.client.1.smithi142.stdout:9/95: read d6/dc/dd/f1d [598801,50000] 0 2024-06-15T01:06:54.471 INFO:tasks.workunit.client.0.smithi137.stdout:6/85: symlink d0/d5/d9/l19 0 2024-06-15T01:06:54.471 INFO:tasks.workunit.client.1.smithi142.stdout:3/158: getdents d0/d1f 0 2024-06-15T01:06:54.472 INFO:tasks.workunit.client.1.smithi142.stdout:1/140: dread d6/d12/d1c/d1e/f20 [0,4194304] 0 2024-06-15T01:06:54.472 INFO:tasks.workunit.client.1.smithi142.stdout:2/102: mkdir d8/df/d13/d1a 0 2024-06-15T01:06:54.473 INFO:tasks.workunit.client.0.smithi137.stdout:3/118: dread d0/d13/f26 [0,4194304] 0 2024-06-15T01:06:54.473 INFO:tasks.workunit.client.1.smithi142.stdout:3/159: read - d0/d3/d15/f2d zero size 2024-06-15T01:06:54.473 INFO:tasks.workunit.client.1.smithi142.stdout:2/103: chown d8/ld 15531401 1 2024-06-15T01:06:54.475 INFO:tasks.workunit.client.1.smithi142.stdout:2/104: write f4 [1024031,17854] 0 2024-06-15T01:06:54.475 INFO:tasks.workunit.client.1.smithi142.stdout:3/160: write d0/f2c [1031868,122593] 0 2024-06-15T01:06:54.477 INFO:tasks.workunit.client.1.smithi142.stdout:5/110: creat d2/f25 x:0 0 0 2024-06-15T01:06:54.477 INFO:tasks.workunit.client.1.smithi142.stdout:3/161: stat d0/d1b 0 2024-06-15T01:06:54.477 INFO:tasks.workunit.client.0.smithi137.stdout:6/86: read d0/fc [435645,35218] 0 2024-06-15T01:06:54.478 INFO:tasks.workunit.client.0.smithi137.stdout:3/119: mkdir d0/d2/d22/d24/d28 0 2024-06-15T01:06:54.479 INFO:tasks.workunit.client.0.smithi137.stdout:3/120: chown d0/f1d 174977158 1 2024-06-15T01:06:54.479 INFO:tasks.workunit.client.0.smithi137.stdout:6/87: chown d0/d5/ff 201811199 1 2024-06-15T01:06:54.483 INFO:tasks.workunit.client.0.smithi137.stdout:3/121: fdatasync d0/d13/f26 0 2024-06-15T01:06:54.483 INFO:tasks.workunit.client.0.smithi137.stdout:3/122: chown d0/d13 360960 1 2024-06-15T01:06:54.484 INFO:tasks.workunit.client.1.smithi142.stdout:7/97: write d2/d8/da/d18/d25/f16 [1817017,71458] 0 2024-06-15T01:06:54.489 INFO:tasks.workunit.client.1.smithi142.stdout:1/141: dread d6/d12/f2d [0,4194304] 0 2024-06-15T01:06:54.491 INFO:tasks.workunit.client.1.smithi142.stdout:5/111: dwrite d2/de/d19/f22 [0,4194304] 0 2024-06-15T01:06:54.491 INFO:tasks.workunit.client.1.smithi142.stdout:2/105: symlink d8/d19/l1b 0 2024-06-15T01:06:54.491 INFO:tasks.workunit.client.0.smithi137.stdout:6/88: dwrite d0/fc [0,4194304] 0 2024-06-15T01:06:54.493 INFO:tasks.workunit.client.0.smithi137.stdout:6/89: chown d0/d5/f7 218150219 1 2024-06-15T01:06:54.493 INFO:tasks.workunit.client.0.smithi137.stdout:3/123: dwrite d0/d2/d8/f1c [0,4194304] 0 2024-06-15T01:06:54.494 INFO:tasks.workunit.client.1.smithi142.stdout:2/106: fsync d8/f9 0 2024-06-15T01:06:54.494 INFO:tasks.workunit.client.1.smithi142.stdout:3/162: dwrite d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:54.494 INFO:tasks.workunit.client.1.smithi142.stdout:5/112: chown d2/ff 3 1 2024-06-15T01:06:54.497 INFO:tasks.workunit.client.1.smithi142.stdout:3/163: chown d0/d1b/l24 13887399 1 2024-06-15T01:06:54.497 INFO:tasks.workunit.client.1.smithi142.stdout:4/218: link d5/df/d12/l19 d5/d9/d18/l53 0 2024-06-15T01:06:54.497 INFO:tasks.workunit.client.0.smithi137.stdout:3/124: mkdir d0/d2/d22/d24/d29 0 2024-06-15T01:06:54.498 INFO:tasks.workunit.client.0.smithi137.stdout:6/90: mkdir d0/d1a 0 2024-06-15T01:06:54.499 INFO:tasks.workunit.client.0.smithi137.stdout:6/91: readlink d0/ld 0 2024-06-15T01:06:54.500 INFO:tasks.workunit.client.0.smithi137.stdout:3/125: creat d0/d2/d1e/f2a x:0 0 0 2024-06-15T01:06:54.501 INFO:tasks.workunit.client.0.smithi137.stdout:6/92: write d0/fc [1141099,122407] 0 2024-06-15T01:06:54.502 INFO:tasks.workunit.client.0.smithi137.stdout:3/126: write d0/f1f [543146,44900] 0 2024-06-15T01:06:54.503 INFO:tasks.workunit.client.0.smithi137.stdout:6/93: write d0/d5/d9/fe [869254,50335] 0 2024-06-15T01:06:54.504 INFO:tasks.workunit.client.1.smithi142.stdout:8/113: truncate d0/f5 3712662 0 2024-06-15T01:06:54.504 INFO:tasks.workunit.client.1.smithi142.stdout:1/142: dwrite d6/f9 [0,4194304] 0 2024-06-15T01:06:54.510 INFO:tasks.workunit.client.0.smithi137.stdout:6/94: unlink d0/d5/d9/l19 0 2024-06-15T01:06:54.511 INFO:tasks.workunit.client.1.smithi142.stdout:3/164: rename d0/d3/da/d18/f21 to d0/d1f/d20/f32 0 2024-06-15T01:06:54.514 INFO:tasks.workunit.client.1.smithi142.stdout:7/98: rmdir d2/d6/d1e 0 2024-06-15T01:06:54.517 INFO:tasks.workunit.client.1.smithi142.stdout:8/114: unlink d0/f10 0 2024-06-15T01:06:54.520 INFO:tasks.workunit.client.1.smithi142.stdout:7/99: dread - d2/d8/dd/d14/f24 zero size 2024-06-15T01:06:54.521 INFO:tasks.workunit.client.0.smithi137.stdout:6/95: dwrite d0/d5/f13 [0,4194304] 0 2024-06-15T01:06:54.522 INFO:tasks.workunit.client.1.smithi142.stdout:2/107: truncate d8/fe 1678748 0 2024-06-15T01:06:54.526 INFO:tasks.workunit.client.1.smithi142.stdout:1/143: dwrite d6/f2a [0,4194304] 0 2024-06-15T01:06:54.526 INFO:tasks.workunit.client.1.smithi142.stdout:3/165: symlink d0/d3/da/d18/l33 0 2024-06-15T01:06:54.527 INFO:tasks.workunit.client.1.smithi142.stdout:5/113: dwrite d2/f9 [0,4194304] 0 2024-06-15T01:06:54.529 INFO:tasks.workunit.client.0.smithi137.stdout:6/96: dread d0/f10 [0,4194304] 0 2024-06-15T01:06:54.530 INFO:tasks.workunit.client.0.smithi137.stdout:6/97: dread - d0/d5/f18 zero size 2024-06-15T01:06:54.534 INFO:tasks.workunit.client.1.smithi142.stdout:4/219: link d5/d9/d41/f36 d5/d9/da/f54 0 2024-06-15T01:06:54.534 INFO:tasks.workunit.client.0.smithi137.stdout:6/98: creat d0/d5/d9/f1b x:0 0 0 2024-06-15T01:06:54.535 INFO:tasks.workunit.client.1.smithi142.stdout:8/115: rename d0/d17/f1d to d0/d17/f28 0 2024-06-15T01:06:54.535 INFO:tasks.workunit.client.1.smithi142.stdout:9/96: dwrite d6/dc/f1c [0,4194304] 0 2024-06-15T01:06:54.535 INFO:tasks.workunit.client.1.smithi142.stdout:5/114: mknod d2/de/c26 0 2024-06-15T01:06:54.536 INFO:tasks.workunit.client.0.smithi137.stdout:6/99: truncate d0/d5/f15 349558 0 2024-06-15T01:06:54.540 INFO:tasks.workunit.client.0.smithi137.stdout:6/100: mkdir d0/d1c 0 2024-06-15T01:06:54.541 INFO:tasks.workunit.client.1.smithi142.stdout:7/100: rename d2/d6/c13 to d2/d8/da/c26 0 2024-06-15T01:06:54.542 INFO:tasks.workunit.client.0.smithi137.stdout:6/101: readlink d0/lb 0 2024-06-15T01:06:54.543 INFO:tasks.workunit.client.0.smithi137.stdout:6/102: write d0/d5/ff [1444942,130372] 0 2024-06-15T01:06:54.546 INFO:tasks.workunit.client.1.smithi142.stdout:4/220: write d5/d9/d23/f4e [139626,29300] 0 2024-06-15T01:06:54.547 INFO:tasks.workunit.client.1.smithi142.stdout:3/166: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:54.549 INFO:tasks.workunit.client.1.smithi142.stdout:7/101: mkdir d2/d8/da/d18/d25/d27 0 2024-06-15T01:06:54.550 INFO:tasks.workunit.client.1.smithi142.stdout:8/116: link d0/f20 d0/d1/f29 0 2024-06-15T01:06:54.551 INFO:tasks.workunit.client.0.smithi137.stdout:3/127: dwrite d0/fd [0,4194304] 0 2024-06-15T01:06:54.554 INFO:tasks.workunit.client.0.smithi137.stdout:3/128: mknod d0/d2/d22/c2b 0 2024-06-15T01:06:54.554 INFO:tasks.workunit.client.1.smithi142.stdout:3/167: write d0/d1f/d20/f32 [4417642,54419] 0 2024-06-15T01:06:54.554 INFO:tasks.workunit.client.1.smithi142.stdout:4/221: write d5/d9/d41/f4c [9431882,33565] 0 2024-06-15T01:06:54.555 INFO:tasks.workunit.client.1.smithi142.stdout:3/168: chown d0/d1b 846 1 2024-06-15T01:06:54.556 INFO:tasks.workunit.client.1.smithi142.stdout:3/169: readlink d0/d3/da/d18/l33 0 2024-06-15T01:06:54.557 INFO:tasks.workunit.client.1.smithi142.stdout:5/115: dwrite d2/de/d19/f22 [0,4194304] 0 2024-06-15T01:06:54.559 INFO:tasks.workunit.client.0.smithi137.stdout:3/129: truncate d0/d13/f26 381127 0 2024-06-15T01:06:54.560 INFO:tasks.workunit.client.1.smithi142.stdout:5/116: write d2/f7 [1801975,35250] 0 2024-06-15T01:06:54.562 INFO:tasks.workunit.client.1.smithi142.stdout:7/102: rename d2/d8/dd/d14/f24 to d2/d6/d22/f28 0 2024-06-15T01:06:54.562 INFO:tasks.workunit.client.1.smithi142.stdout:8/117: mknod d0/c2a 0 2024-06-15T01:06:54.562 INFO:tasks.workunit.client.0.smithi137.stdout:3/130: creat d0/d2/d22/f2c x:0 0 0 2024-06-15T01:06:54.563 INFO:tasks.workunit.client.1.smithi142.stdout:7/103: chown d2/d8/da/d18/d25 4 1 2024-06-15T01:06:54.565 INFO:tasks.workunit.client.1.smithi142.stdout:4/222: symlink d5/d1c/l55 0 2024-06-15T01:06:54.565 INFO:tasks.workunit.client.0.smithi137.stdout:3/131: mkdir d0/d2/d22/d24/d2d 0 2024-06-15T01:06:54.566 INFO:tasks.workunit.client.0.smithi137.stdout:3/132: write d0/d2/d22/f2c [13017,50381] 0 2024-06-15T01:06:54.567 INFO:tasks.workunit.client.0.smithi137.stdout:3/133: stat d0/f1f 0 2024-06-15T01:06:54.575 INFO:tasks.workunit.client.1.smithi142.stdout:9/97: link d6/dc/d1b/l1e d6/dc/l29 0 2024-06-15T01:06:54.576 INFO:tasks.workunit.client.0.smithi137.stdout:3/134: dwrite d0/d2/f15 [0,4194304] 0 2024-06-15T01:06:54.577 INFO:tasks.workunit.client.0.smithi137.stdout:3/135: read d0/d2/fc [2179877,12553] 0 2024-06-15T01:06:54.578 INFO:tasks.workunit.client.1.smithi142.stdout:7/104: symlink d2/d8/dd/l29 0 2024-06-15T01:06:54.579 INFO:tasks.workunit.client.1.smithi142.stdout:7/105: chown d2/d8/fe 87478 1 2024-06-15T01:06:54.585 INFO:tasks.workunit.client.0.smithi137.stdout:3/136: dwrite d0/d2/f15 [0,4194304] 0 2024-06-15T01:06:54.585 INFO:tasks.workunit.client.1.smithi142.stdout:1/144: write d6/f10 [4648057,35580] 0 2024-06-15T01:06:54.587 INFO:tasks.workunit.client.1.smithi142.stdout:8/118: mknod d0/d25/c2b 0 2024-06-15T01:06:54.589 INFO:tasks.workunit.client.0.smithi137.stdout:3/137: getdents d0/d2/d8 0 2024-06-15T01:06:54.589 INFO:tasks.workunit.client.0.smithi137.stdout:6/103: truncate d0/d5/f13 3541582 0 2024-06-15T01:06:54.626 INFO:tasks.workunit.client.0.smithi137.stdout:3/138: read d0/f1f [300045,59876] 0 2024-06-15T01:06:54.626 INFO:tasks.workunit.client.0.smithi137.stdout:6/104: mkdir d0/d1c/d1d 0 2024-06-15T01:06:54.627 INFO:tasks.workunit.client.1.smithi142.stdout:7/106: rename d2/c11 to d2/d8/da/d18/c2a 0 2024-06-15T01:06:54.627 INFO:tasks.workunit.client.1.smithi142.stdout:7/107: dread - d2/d6/d22/f28 zero size 2024-06-15T01:06:54.631 INFO:tasks.workunit.client.0.smithi137.stdout:6/105: rename d0/d5/ff to d0/d1c/f1e 0 2024-06-15T01:06:54.632 INFO:tasks.workunit.client.1.smithi142.stdout:9/98: symlink d6/l2a 0 2024-06-15T01:06:54.633 INFO:tasks.workunit.client.1.smithi142.stdout:9/99: read - d6/f21 zero size 2024-06-15T01:06:54.635 INFO:tasks.workunit.client.0.smithi137.stdout:6/106: write d0/d5/d9/fe [1374462,35162] 0 2024-06-15T01:06:54.636 INFO:tasks.workunit.client.0.smithi137.stdout:3/139: dwrite d0/f1d [0,4194304] 0 2024-06-15T01:06:54.636 INFO:tasks.workunit.client.1.smithi142.stdout:8/119: write d0/f23 [53754,11242] 0 2024-06-15T01:06:54.637 INFO:tasks.workunit.client.1.smithi142.stdout:7/108: mknod d2/d8/c2b 0 2024-06-15T01:06:54.637 INFO:tasks.workunit.client.0.smithi137.stdout:6/107: chown d0/d5/f7 130495610 1 2024-06-15T01:06:54.638 INFO:tasks.workunit.client.1.smithi142.stdout:0/65: sync 2024-06-15T01:06:54.638 INFO:tasks.workunit.client.1.smithi142.stdout:6/114: sync 2024-06-15T01:06:54.639 INFO:tasks.workunit.client.1.smithi142.stdout:7/109: read d2/d8/f17 [463664,122830] 0 2024-06-15T01:06:54.639 INFO:tasks.workunit.client.1.smithi142.stdout:3/170: getdents d0/d1f/d20 0 2024-06-15T01:06:54.640 INFO:tasks.workunit.client.1.smithi142.stdout:3/171: chown d0/d1f/d20 1 1 2024-06-15T01:06:54.641 INFO:tasks.workunit.client.1.smithi142.stdout:0/66: dread d3/d12/f14 [0,4194304] 0 2024-06-15T01:06:54.641 INFO:tasks.workunit.client.1.smithi142.stdout:9/100: creat d6/dc/dd/d22/f2b x:0 0 0 2024-06-15T01:06:54.642 INFO:tasks.workunit.client.0.smithi137.stdout:3/140: unlink d0/d2/d8/f11 0 2024-06-15T01:06:54.644 INFO:tasks.workunit.client.0.smithi137.stdout:3/141: write d0/d2/d22/f2c [146216,130679] 0 2024-06-15T01:06:54.646 INFO:tasks.workunit.client.0.smithi137.stdout:6/108: creat d0/d1c/d1d/f1f x:0 0 0 2024-06-15T01:06:54.646 INFO:tasks.workunit.client.1.smithi142.stdout:1/145: rename d6/l18 to d6/d25/l31 0 2024-06-15T01:06:54.646 INFO:tasks.workunit.client.1.smithi142.stdout:5/117: write d2/de/d19/f1a [1912865,123303] 0 2024-06-15T01:06:54.649 INFO:tasks.workunit.client.1.smithi142.stdout:6/115: creat d1a/f2a x:0 0 0 2024-06-15T01:06:54.650 INFO:tasks.workunit.client.1.smithi142.stdout:2/108: write d8/fe [1836146,106539] 0 2024-06-15T01:06:54.651 INFO:tasks.workunit.client.0.smithi137.stdout:3/142: mkdir d0/d2/d22/d24/d28/d2e 0 2024-06-15T01:06:54.652 INFO:tasks.workunit.client.1.smithi142.stdout:7/110: mkdir d2/d8/dd/d14/d2c 0 2024-06-15T01:06:54.653 INFO:tasks.workunit.client.0.smithi137.stdout:6/109: dwrite d0/d5/f12 [0,4194304] 0 2024-06-15T01:06:54.654 INFO:tasks.workunit.client.1.smithi142.stdout:5/118: dwrite d2/f21 [0,4194304] 0 2024-06-15T01:06:54.654 INFO:tasks.workunit.client.1.smithi142.stdout:3/172: chown d0/d3/da/c2f 121 1 2024-06-15T01:06:54.654 INFO:tasks.workunit.client.0.smithi137.stdout:3/143: mkdir d0/d2f 0 2024-06-15T01:06:54.659 INFO:tasks.workunit.client.1.smithi142.stdout:4/223: write f3 [1800015,44890] 0 2024-06-15T01:06:54.659 INFO:tasks.workunit.client.0.smithi137.stdout:3/144: unlink d0/d2/fc 0 2024-06-15T01:06:54.663 INFO:tasks.workunit.client.0.smithi137.stdout:6/110: dwrite d0/d5/f8 [0,4194304] 0 2024-06-15T01:06:54.663 INFO:tasks.workunit.client.1.smithi142.stdout:4/224: chown d5/d1c/d2a/l2b 19690 1 2024-06-15T01:06:54.663 INFO:tasks.workunit.client.1.smithi142.stdout:1/146: creat d6/d12/d1c/f32 x:0 0 0 2024-06-15T01:06:54.665 INFO:tasks.workunit.client.1.smithi142.stdout:5/119: dwrite d2/fd [0,4194304] 0 2024-06-15T01:06:54.666 INFO:tasks.workunit.client.0.smithi137.stdout:6/111: truncate d0/f10 1339635 0 2024-06-15T01:06:54.667 INFO:tasks.workunit.client.0.smithi137.stdout:3/145: dread d0/fd [0,4194304] 0 2024-06-15T01:06:54.667 INFO:tasks.workunit.client.0.smithi137.stdout:6/112: write d0/d5/f18 [968316,25091] 0 2024-06-15T01:06:54.667 INFO:tasks.workunit.client.1.smithi142.stdout:1/147: write d6/f2a [1734540,115234] 0 2024-06-15T01:06:54.669 INFO:tasks.workunit.client.1.smithi142.stdout:8/120: write d0/f18 [2361519,50613] 0 2024-06-15T01:06:54.669 INFO:tasks.workunit.client.1.smithi142.stdout:3/173: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:54.670 INFO:tasks.workunit.client.1.smithi142.stdout:6/116: fsync d1a/f22 0 2024-06-15T01:06:54.673 INFO:tasks.workunit.client.0.smithi137.stdout:6/113: symlink d0/d5/d9/l20 0 2024-06-15T01:06:54.674 INFO:tasks.workunit.client.1.smithi142.stdout:2/109: mkdir d8/df/d10/d1c 0 2024-06-15T01:06:54.679 INFO:tasks.workunit.client.0.smithi137.stdout:3/146: dwrite d0/d2/d22/f2c [0,4194304] 0 2024-06-15T01:06:54.682 INFO:tasks.workunit.client.1.smithi142.stdout:9/101: mknod d6/dc/d1b/d25/c2c 0 2024-06-15T01:06:54.683 INFO:tasks.workunit.client.0.smithi137.stdout:3/147: truncate d0/d13/f26 1235260 0 2024-06-15T01:06:54.684 INFO:tasks.workunit.client.1.smithi142.stdout:9/102: write d6/dc/dd/d22/f26 [487094,18161] 0 2024-06-15T01:06:54.684 INFO:tasks.workunit.client.1.smithi142.stdout:5/120: mkdir d2/de/d27 0 2024-06-15T01:06:54.687 INFO:tasks.workunit.client.1.smithi142.stdout:8/121: creat d0/d1/f2c x:0 0 0 2024-06-15T01:06:54.688 INFO:tasks.workunit.client.1.smithi142.stdout:3/174: mknod d0/d1f/c34 0 2024-06-15T01:06:54.689 INFO:tasks.workunit.client.1.smithi142.stdout:3/175: chown d0/d3/da/d18/l33 118101392 1 2024-06-15T01:06:54.690 INFO:tasks.workunit.client.1.smithi142.stdout:7/111: unlink d2/ff 0 2024-06-15T01:06:54.690 INFO:tasks.workunit.client.1.smithi142.stdout:6/117: rmdir d1a 39 2024-06-15T01:06:54.692 INFO:tasks.workunit.client.1.smithi142.stdout:0/67: link d3/d5/ld d3/d12/l16 0 2024-06-15T01:06:54.693 INFO:tasks.workunit.client.1.smithi142.stdout:4/225: creat d5/d9/d1a/d4f/f56 x:0 0 0 2024-06-15T01:06:54.694 INFO:tasks.workunit.client.1.smithi142.stdout:7/112: dread d2/d8/da/d18/d25/f16 [0,4194304] 0 2024-06-15T01:06:54.696 INFO:tasks.workunit.client.1.smithi142.stdout:1/148: truncate d6/f13 311555 0 2024-06-15T01:06:54.696 INFO:tasks.workunit.client.1.smithi142.stdout:1/149: chown d6/d12/d1c/d1e 433 1 2024-06-15T01:06:54.696 INFO:tasks.workunit.client.1.smithi142.stdout:0/68: dread d3/f4 [0,4194304] 0 2024-06-15T01:06:54.698 INFO:tasks.workunit.client.1.smithi142.stdout:3/176: symlink d0/d1f/l35 0 2024-06-15T01:06:54.699 INFO:tasks.workunit.client.1.smithi142.stdout:2/110: creat d8/df/d13/d1a/f1d x:0 0 0 2024-06-15T01:06:54.700 INFO:tasks.workunit.client.1.smithi142.stdout:3/177: chown d0/d3/l2b 962685 1 2024-06-15T01:06:54.701 INFO:tasks.workunit.client.1.smithi142.stdout:4/226: truncate d5/d1c/f34 974327 0 2024-06-15T01:06:54.703 INFO:tasks.workunit.client.1.smithi142.stdout:1/150: creat d6/d12/d28/f33 x:0 0 0 2024-06-15T01:06:54.703 INFO:tasks.workunit.client.1.smithi142.stdout:6/118: symlink d1a/l2b 0 2024-06-15T01:06:54.704 INFO:tasks.workunit.client.1.smithi142.stdout:7/113: dwrite d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:06:54.705 INFO:tasks.workunit.client.1.smithi142.stdout:1/151: write d6/d12/d1c/f32 [863762,105467] 0 2024-06-15T01:06:54.705 INFO:tasks.workunit.client.1.smithi142.stdout:0/69: dread d3/d12/f14 [0,4194304] 0 2024-06-15T01:06:54.706 INFO:tasks.workunit.client.1.smithi142.stdout:5/121: creat d2/f28 x:0 0 0 2024-06-15T01:06:54.709 INFO:tasks.workunit.client.1.smithi142.stdout:5/122: stat d2/fd 0 2024-06-15T01:06:54.709 INFO:tasks.workunit.client.1.smithi142.stdout:9/103: link d6/dc/dd/c20 d6/dc/dd/c2d 0 2024-06-15T01:06:54.709 INFO:tasks.workunit.client.1.smithi142.stdout:0/70: write f1 [2434832,20013] 0 2024-06-15T01:06:54.710 INFO:tasks.workunit.client.0.smithi137.stdout:3/148: fsync d0/d13/f26 0 2024-06-15T01:06:54.711 INFO:tasks.workunit.client.1.smithi142.stdout:2/111: mknod d8/df/d10/d1c/c1e 0 2024-06-15T01:06:54.713 INFO:tasks.workunit.client.1.smithi142.stdout:5/123: fsync d2/de/d19/f22 0 2024-06-15T01:06:54.714 INFO:tasks.workunit.client.0.smithi137.stdout:3/149: read d0/d2/f1b [246919,90997] 0 2024-06-15T01:06:54.714 INFO:tasks.workunit.client.1.smithi142.stdout:9/104: read d6/dc/dd/d22/f26 [1046317,7059] 0 2024-06-15T01:06:54.714 INFO:tasks.workunit.client.1.smithi142.stdout:2/112: chown d8/ld 17 1 2024-06-15T01:06:54.714 INFO:tasks.workunit.client.1.smithi142.stdout:7/114: write d2/d8/fe [218820,66998] 0 2024-06-15T01:06:54.714 INFO:tasks.workunit.client.1.smithi142.stdout:2/113: fdatasync d8/df/d13/f18 0 2024-06-15T01:06:54.716 INFO:tasks.workunit.client.1.smithi142.stdout:5/124: dread d2/f14 [0,4194304] 0 2024-06-15T01:06:54.717 INFO:tasks.workunit.client.1.smithi142.stdout:8/122: link d0/l1a d0/d17/l2d 0 2024-06-15T01:06:54.718 INFO:tasks.workunit.client.1.smithi142.stdout:4/227: chown d5/d1c/f34 14 1 2024-06-15T01:06:54.720 INFO:tasks.workunit.client.0.smithi137.stdout:3/150: dwrite d0/f1a [0,4194304] 0 2024-06-15T01:06:54.721 INFO:tasks.workunit.client.1.smithi142.stdout:0/71: creat d3/d5/f17 x:0 0 0 2024-06-15T01:06:54.724 INFO:tasks.workunit.client.0.smithi137.stdout:3/151: dread d0/d2/d22/f2c [0,4194304] 0 2024-06-15T01:06:54.726 INFO:tasks.workunit.client.0.smithi137.stdout:3/152: symlink d0/d2/d1e/l30 0 2024-06-15T01:06:54.729 INFO:tasks.workunit.client.1.smithi142.stdout:7/115: unlink d2/d8/fe 0 2024-06-15T01:06:54.730 INFO:tasks.workunit.client.0.smithi137.stdout:3/153: dread d0/f1f [0,4194304] 0 2024-06-15T01:06:54.731 INFO:tasks.workunit.client.1.smithi142.stdout:6/119: link d1a/f2a d1a/f2c 0 2024-06-15T01:06:54.733 INFO:tasks.workunit.client.1.smithi142.stdout:8/123: chown d0/c13 287890 1 2024-06-15T01:06:54.734 INFO:tasks.workunit.client.1.smithi142.stdout:8/124: chown d0/l1c 1419 1 2024-06-15T01:06:54.735 INFO:tasks.workunit.client.0.smithi137.stdout:3/154: dread d0/d2/f15 [0,4194304] 0 2024-06-15T01:06:54.737 INFO:tasks.workunit.client.0.smithi137.stdout:3/155: symlink d0/d13/l31 0 2024-06-15T01:06:54.738 INFO:tasks.workunit.client.1.smithi142.stdout:1/152: link d6/d12/d28/f33 d6/d12/d1c/f34 0 2024-06-15T01:06:54.740 INFO:tasks.workunit.client.1.smithi142.stdout:8/125: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:06:54.740 INFO:tasks.workunit.client.1.smithi142.stdout:9/105: symlink d6/dc/l2e 0 2024-06-15T01:06:54.741 INFO:tasks.workunit.client.1.smithi142.stdout:9/106: fdatasync d6/dc/f1c 0 2024-06-15T01:06:54.741 INFO:tasks.workunit.client.1.smithi142.stdout:8/126: truncate d0/f22 1031110 0 2024-06-15T01:06:54.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/116: creat d2/d8/f2d x:0 0 0 2024-06-15T01:06:54.743 INFO:tasks.workunit.client.0.smithi137.stdout:3/156: dwrite d0/d2/d1e/f2a [0,4194304] 0 2024-06-15T01:06:54.744 INFO:tasks.workunit.client.1.smithi142.stdout:6/120: mkdir d1a/d2d 0 2024-06-15T01:06:54.746 INFO:tasks.workunit.client.1.smithi142.stdout:3/178: write d0/d3/f12 [392148,67848] 0 2024-06-15T01:06:54.746 INFO:tasks.workunit.client.0.smithi137.stdout:3/157: fdatasync d0/d2/f15 0 2024-06-15T01:06:54.746 INFO:tasks.workunit.client.1.smithi142.stdout:6/121: dread f8 [0,4194304] 0 2024-06-15T01:06:54.749 INFO:tasks.workunit.client.1.smithi142.stdout:4/228: symlink d5/d9/l57 0 2024-06-15T01:06:54.753 INFO:tasks.workunit.client.1.smithi142.stdout:1/153: rename d6/d12/c1b to d6/d12/d1c/d2c/c35 0 2024-06-15T01:06:54.753 INFO:tasks.workunit.client.0.smithi137.stdout:3/158: symlink d0/d2/l32 0 2024-06-15T01:06:54.756 INFO:tasks.workunit.client.1.smithi142.stdout:9/107: mknod d6/dc/dd/d22/c2f 0 2024-06-15T01:06:54.756 INFO:tasks.workunit.client.0.smithi137.stdout:3/159: symlink d0/d2/d22/d24/d28/d2e/l33 0 2024-06-15T01:06:54.756 INFO:tasks.workunit.client.0.smithi137.stdout:3/160: readlink d0/d13/l31 0 2024-06-15T01:06:54.758 INFO:tasks.workunit.client.1.smithi142.stdout:8/127: symlink d0/d1/l2e 0 2024-06-15T01:06:54.758 INFO:tasks.workunit.client.0.smithi137.stdout:3/161: truncate d0/f16 2921443 0 2024-06-15T01:06:54.760 INFO:tasks.workunit.client.1.smithi142.stdout:1/154: dwrite f5 [12582912,4194304] 0 2024-06-15T01:06:54.760 INFO:tasks.workunit.client.1.smithi142.stdout:5/125: link d2/lb d2/d18/l29 0 2024-06-15T01:06:54.763 INFO:tasks.workunit.client.0.smithi137.stdout:3/162: dread d0/f1a [0,4194304] 0 2024-06-15T01:06:54.767 INFO:tasks.workunit.client.1.smithi142.stdout:3/179: rename d0/d1f/d20/c2e to d0/d1b/c36 0 2024-06-15T01:06:54.768 INFO:tasks.workunit.client.0.smithi137.stdout:3/163: rename d0/d2/d22/f2c to d0/d2/d22/d24/d2d/f34 0 2024-06-15T01:06:54.768 INFO:tasks.workunit.client.1.smithi142.stdout:2/114: getdents d8/df/d10/d1c 0 2024-06-15T01:06:54.768 INFO:tasks.workunit.client.0.smithi137.stdout:3/164: readlink d0/d2/d22/d24/d28/d2e/l33 0 2024-06-15T01:06:54.769 INFO:tasks.workunit.client.1.smithi142.stdout:2/115: dread f4 [0,4194304] 0 2024-06-15T01:06:54.771 INFO:tasks.workunit.client.0.smithi137.stdout:3/165: creat d0/d2f/f35 x:0 0 0 2024-06-15T01:06:54.773 INFO:tasks.workunit.client.0.smithi137.stdout:3/166: truncate d0/d2/f15 4618747 0 2024-06-15T01:06:54.775 INFO:tasks.workunit.client.0.smithi137.stdout:3/167: mknod d0/d13/c36 0 2024-06-15T01:06:54.775 INFO:tasks.workunit.client.1.smithi142.stdout:6/122: mkdir d1a/d2d/d2e 0 2024-06-15T01:06:54.775 INFO:tasks.workunit.client.1.smithi142.stdout:6/123: chown f8 1 1 2024-06-15T01:06:54.776 INFO:tasks.workunit.client.1.smithi142.stdout:1/155: mknod d6/d12/c36 0 2024-06-15T01:06:54.776 INFO:tasks.workunit.client.0.smithi137.stdout:3/168: truncate d0/d2/d22/d24/d2d/f34 4584568 0 2024-06-15T01:06:54.777 INFO:tasks.workunit.client.1.smithi142.stdout:5/126: creat d2/de/d19/f2a x:0 0 0 2024-06-15T01:06:54.778 INFO:tasks.workunit.client.1.smithi142.stdout:4/229: symlink d5/df/l58 0 2024-06-15T01:06:54.782 INFO:tasks.workunit.client.1.smithi142.stdout:1/156: dwrite d6/f26 [0,4194304] 0 2024-06-15T01:06:54.784 INFO:tasks.workunit.client.1.smithi142.stdout:7/117: creat d2/f2e x:0 0 0 2024-06-15T01:06:54.784 INFO:tasks.workunit.client.1.smithi142.stdout:2/116: mkdir d8/df/d1f 0 2024-06-15T01:06:54.785 INFO:tasks.workunit.client.1.smithi142.stdout:2/117: fsync f6 0 2024-06-15T01:06:54.785 INFO:tasks.workunit.client.1.smithi142.stdout:2/118: readlink l1 0 2024-06-15T01:06:54.785 INFO:tasks.workunit.client.1.smithi142.stdout:8/128: fdatasync d0/f5 0 2024-06-15T01:06:54.786 INFO:tasks.workunit.client.1.smithi142.stdout:2/119: chown d8/df/d10/c12 3966 1 2024-06-15T01:06:54.789 INFO:tasks.workunit.client.1.smithi142.stdout:5/127: unlink d2/de/f24 0 2024-06-15T01:06:54.790 INFO:tasks.workunit.client.1.smithi142.stdout:5/128: write d2/de/d19/f22 [2929985,20577] 0 2024-06-15T01:06:54.792 INFO:tasks.workunit.client.1.smithi142.stdout:9/108: rename d6/l14 to d6/dc/dd/l30 0 2024-06-15T01:06:54.794 INFO:tasks.workunit.client.1.smithi142.stdout:5/129: dread d2/f7 [0,4194304] 0 2024-06-15T01:06:54.799 INFO:tasks.workunit.client.1.smithi142.stdout:9/109: dwrite d6/f21 [0,4194304] 0 2024-06-15T01:06:54.799 INFO:tasks.workunit.client.1.smithi142.stdout:2/120: mknod d8/c20 0 2024-06-15T01:06:54.800 INFO:tasks.workunit.client.1.smithi142.stdout:8/129: fsync d0/d1/f29 0 2024-06-15T01:06:54.802 INFO:tasks.workunit.client.1.smithi142.stdout:6/124: write d1a/f2c [419133,112179] 0 2024-06-15T01:06:54.805 INFO:tasks.workunit.client.1.smithi142.stdout:1/157: rename d6/d12/d28/l2b to d6/d12/l37 0 2024-06-15T01:06:54.805 INFO:tasks.workunit.client.1.smithi142.stdout:5/130: rename d2/de/d19 to d2/de/d19/d2b 22 2024-06-15T01:06:54.808 INFO:tasks.workunit.client.1.smithi142.stdout:9/110: dread d6/f21 [0,4194304] 0 2024-06-15T01:06:54.808 INFO:tasks.workunit.client.1.smithi142.stdout:2/121: dread d8/fa [0,4194304] 0 2024-06-15T01:06:54.809 INFO:tasks.workunit.client.1.smithi142.stdout:3/180: getdents d0/d1f 0 2024-06-15T01:06:54.810 INFO:tasks.workunit.client.1.smithi142.stdout:3/181: fdatasync d0/d3/f1a 0 2024-06-15T01:06:54.810 INFO:tasks.workunit.client.1.smithi142.stdout:2/122: write f6 [640810,5599] 0 2024-06-15T01:06:54.810 INFO:tasks.workunit.client.1.smithi142.stdout:2/123: stat d8/f16 0 2024-06-15T01:06:54.811 INFO:tasks.workunit.client.1.smithi142.stdout:1/158: dread d6/fc [0,4194304] 0 2024-06-15T01:06:54.813 INFO:tasks.workunit.client.1.smithi142.stdout:1/159: write d6/f10 [1090420,61051] 0 2024-06-15T01:06:54.816 INFO:tasks.workunit.client.1.smithi142.stdout:2/124: dread d8/f9 [0,4194304] 0 2024-06-15T01:06:54.817 INFO:tasks.workunit.client.1.smithi142.stdout:3/182: dwrite d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:54.820 INFO:tasks.workunit.client.0.smithi137.stdout:5/153: sync 2024-06-15T01:06:54.821 INFO:tasks.workunit.client.0.smithi137.stdout:2/53: sync 2024-06-15T01:06:54.821 INFO:tasks.workunit.client.0.smithi137.stdout:2/54: write d0/fb [373393,14487] 0 2024-06-15T01:06:54.822 INFO:tasks.workunit.client.0.smithi137.stdout:2/55: truncate d0/f8 5128724 0 2024-06-15T01:06:54.823 INFO:tasks.workunit.client.0.smithi137.stdout:5/154: symlink df/d11/d34/l40 0 2024-06-15T01:06:54.824 INFO:tasks.workunit.client.0.smithi137.stdout:5/155: write df/d11/d14/f32 [246300,74657] 0 2024-06-15T01:06:54.824 INFO:tasks.workunit.client.0.smithi137.stdout:5/156: readlink df/l1b 0 2024-06-15T01:06:54.825 INFO:tasks.workunit.client.0.smithi137.stdout:5/157: chown df/d11 187554 1 2024-06-15T01:06:54.853 INFO:tasks.workunit.client.1.smithi142.stdout:8/130: unlink d0/cc 0 2024-06-15T01:06:54.856 INFO:tasks.workunit.client.1.smithi142.stdout:6/125: creat d1a/d2d/f2f x:0 0 0 2024-06-15T01:06:54.858 INFO:tasks.workunit.client.0.smithi137.stdout:5/158: link df/d11/d14/f2e df/f41 0 2024-06-15T01:06:54.862 INFO:tasks.workunit.client.1.smithi142.stdout:4/230: getdents d5 0 2024-06-15T01:06:54.865 INFO:tasks.workunit.client.0.smithi137.stdout:5/159: dwrite f9 [0,4194304] 0 2024-06-15T01:06:54.866 INFO:tasks.workunit.client.1.smithi142.stdout:1/160: symlink d6/d25/l38 0 2024-06-15T01:06:54.866 INFO:tasks.workunit.client.1.smithi142.stdout:1/161: fsync d6/f2a 0 2024-06-15T01:06:54.870 INFO:tasks.workunit.client.1.smithi142.stdout:8/131: rename d0/d1/f11 to d0/d17/f2f 0 2024-06-15T01:06:54.870 INFO:tasks.workunit.client.1.smithi142.stdout:8/132: chown d0/f23 27 1 2024-06-15T01:06:54.871 INFO:tasks.workunit.client.0.smithi137.stdout:5/160: getdents df/d11/d34/d35 0 2024-06-15T01:06:54.872 INFO:tasks.workunit.client.0.smithi137.stdout:5/161: readlink l8 0 2024-06-15T01:06:54.874 INFO:tasks.workunit.client.0.smithi137.stdout:5/162: write fd [4788129,56158] 0 2024-06-15T01:06:54.875 INFO:tasks.workunit.client.0.smithi137.stdout:2/56: dwrite d0/f1 [0,4194304] 0 2024-06-15T01:06:54.875 INFO:tasks.workunit.client.1.smithi142.stdout:7/118: getdents d2/d8/dd/d14 0 2024-06-15T01:06:54.876 INFO:tasks.workunit.client.1.smithi142.stdout:7/119: fsync d2/d6/f1c 0 2024-06-15T01:06:54.878 INFO:tasks.workunit.client.1.smithi142.stdout:7/120: dread d2/f1a [0,4194304] 0 2024-06-15T01:06:54.878 INFO:tasks.workunit.client.1.smithi142.stdout:9/111: mkdir d6/d31 0 2024-06-15T01:06:54.881 INFO:tasks.workunit.client.1.smithi142.stdout:2/125: truncate d8/f9 3219159 0 2024-06-15T01:06:54.881 INFO:tasks.workunit.client.0.smithi137.stdout:2/57: dread d0/f1 [0,4194304] 0 2024-06-15T01:06:54.882 INFO:tasks.workunit.client.1.smithi142.stdout:2/126: fsync d8/f16 0 2024-06-15T01:06:54.882 INFO:tasks.workunit.client.1.smithi142.stdout:3/183: mknod d0/c37 0 2024-06-15T01:06:54.884 INFO:tasks.workunit.client.1.smithi142.stdout:1/162: creat d6/d12/d28/f39 x:0 0 0 2024-06-15T01:06:54.884 INFO:tasks.workunit.client.1.smithi142.stdout:7/121: dread d2/d6/f1d [0,4194304] 0 2024-06-15T01:06:54.884 INFO:tasks.workunit.client.1.smithi142.stdout:3/184: dread - d0/d3/d15/f2d zero size 2024-06-15T01:06:54.885 INFO:tasks.workunit.client.1.smithi142.stdout:7/122: dread - d2/d6/f1c zero size 2024-06-15T01:06:54.885 INFO:tasks.workunit.client.1.smithi142.stdout:3/185: dread - d0/d1b/f2a zero size 2024-06-15T01:06:54.886 INFO:tasks.workunit.client.0.smithi137.stdout:2/58: dread d0/f1 [0,4194304] 0 2024-06-15T01:06:54.887 INFO:tasks.workunit.client.1.smithi142.stdout:6/126: truncate f18 4136315 0 2024-06-15T01:06:54.890 INFO:tasks.workunit.client.0.smithi137.stdout:2/59: mknod d0/de/d10/c13 0 2024-06-15T01:06:54.891 INFO:tasks.workunit.client.1.smithi142.stdout:4/231: mkdir d5/df/d59 0 2024-06-15T01:06:54.891 INFO:tasks.workunit.client.1.smithi142.stdout:4/232: write f2 [210090,6189] 0 2024-06-15T01:06:54.891 INFO:tasks.workunit.client.1.smithi142.stdout:5/131: link d2/de/c15 d2/d18/c2c 0 2024-06-15T01:06:54.891 INFO:tasks.workunit.client.0.smithi137.stdout:2/60: rmdir d0/de 39 2024-06-15T01:06:54.892 INFO:tasks.workunit.client.1.smithi142.stdout:5/132: write d2/de/d19/f22 [3357508,113294] 0 2024-06-15T01:06:54.895 INFO:tasks.workunit.client.0.smithi137.stdout:2/61: link d0/c7 d0/dc/c14 0 2024-06-15T01:06:54.896 INFO:tasks.workunit.client.1.smithi142.stdout:2/127: mknod d8/df/d10/c21 0 2024-06-15T01:06:54.898 INFO:tasks.workunit.client.1.smithi142.stdout:5/133: dwrite d2/ff [0,4194304] 0 2024-06-15T01:06:54.899 INFO:tasks.workunit.client.1.smithi142.stdout:5/134: write d2/f25 [268850,73764] 0 2024-06-15T01:06:54.900 INFO:tasks.workunit.client.0.smithi137.stdout:2/62: getdents d0/da 0 2024-06-15T01:06:54.900 INFO:tasks.workunit.client.0.smithi137.stdout:5/163: dread f7 [0,4194304] 0 2024-06-15T01:06:54.901 INFO:tasks.workunit.client.1.smithi142.stdout:3/186: creat d0/d1f/f38 x:0 0 0 2024-06-15T01:06:54.902 INFO:tasks.workunit.client.0.smithi137.stdout:2/63: write d0/f3 [559310,31967] 0 2024-06-15T01:06:54.908 INFO:tasks.workunit.client.0.smithi137.stdout:5/164: dread f5 [0,4194304] 0 2024-06-15T01:06:54.911 INFO:tasks.workunit.client.0.smithi137.stdout:5/165: write df/d11/d14/f21 [3997932,115467] 0 2024-06-15T01:06:54.911 INFO:tasks.workunit.client.0.smithi137.stdout:2/64: dwrite d0/fb [0,4194304] 0 2024-06-15T01:06:54.913 INFO:tasks.workunit.client.0.smithi137.stdout:2/65: rename d0/da to d0/da/d15 22 2024-06-15T01:06:54.914 INFO:tasks.workunit.client.0.smithi137.stdout:2/66: write d0/f8 [4585357,68134] 0 2024-06-15T01:06:54.919 INFO:tasks.workunit.client.0.smithi137.stdout:5/166: dwrite df/f2a [0,4194304] 0 2024-06-15T01:06:54.920 INFO:tasks.workunit.client.0.smithi137.stdout:2/67: link d0/f1 d0/de/f16 0 2024-06-15T01:06:54.921 INFO:tasks.workunit.client.0.smithi137.stdout:2/68: write d0/f8 [857106,13655] 0 2024-06-15T01:06:54.924 INFO:tasks.workunit.client.0.smithi137.stdout:2/69: unlink d0/c12 0 2024-06-15T01:06:54.927 INFO:tasks.workunit.client.0.smithi137.stdout:5/167: dwrite f7 [0,4194304] 0 2024-06-15T01:06:54.927 INFO:tasks.workunit.client.0.smithi137.stdout:2/70: creat d0/da/f17 x:0 0 0 2024-06-15T01:06:54.930 INFO:tasks.workunit.client.0.smithi137.stdout:5/168: truncate df/d11/d14/f19 2711550 0 2024-06-15T01:06:54.933 INFO:tasks.workunit.client.0.smithi137.stdout:5/169: getdents df/d11/d34 0 2024-06-15T01:06:54.933 INFO:tasks.workunit.client.0.smithi137.stdout:5/170: dread - df/d11/f31 zero size 2024-06-15T01:06:54.934 INFO:tasks.workunit.client.0.smithi137.stdout:5/171: dread - df/d11/f31 zero size 2024-06-15T01:06:54.935 INFO:tasks.workunit.client.0.smithi137.stdout:5/172: creat df/d3d/f42 x:0 0 0 2024-06-15T01:06:54.937 INFO:tasks.workunit.client.0.smithi137.stdout:5/173: creat df/d11/d14/f43 x:0 0 0 2024-06-15T01:06:54.938 INFO:tasks.workunit.client.0.smithi137.stdout:5/174: chown df/d11/d14/l1c 215 1 2024-06-15T01:06:54.939 INFO:tasks.workunit.client.0.smithi137.stdout:5/175: mkdir df/d44 0 2024-06-15T01:06:54.943 INFO:tasks.workunit.client.0.smithi137.stdout:5/176: dread df/d11/d14/f21 [0,4194304] 0 2024-06-15T01:06:54.944 INFO:tasks.workunit.client.0.smithi137.stdout:5/177: symlink df/d11/d34/d3b/l45 0 2024-06-15T01:06:54.945 INFO:tasks.workunit.client.0.smithi137.stdout:5/178: symlink df/d44/l46 0 2024-06-15T01:06:54.946 INFO:tasks.workunit.client.0.smithi137.stdout:5/179: readlink df/d11/l29 0 2024-06-15T01:06:54.947 INFO:tasks.workunit.client.0.smithi137.stdout:5/180: write df/d11/d14/f43 [663886,38721] 0 2024-06-15T01:06:54.952 INFO:tasks.workunit.client.0.smithi137.stdout:5/181: dwrite df/d3d/f42 [0,4194304] 0 2024-06-15T01:06:54.953 INFO:tasks.workunit.client.0.smithi137.stdout:5/182: chown l6 240565 1 2024-06-15T01:06:54.955 INFO:tasks.workunit.client.0.smithi137.stdout:5/183: symlink df/d44/l47 0 2024-06-15T01:06:54.956 INFO:tasks.workunit.client.0.smithi137.stdout:5/184: mknod df/d11/d34/c48 0 2024-06-15T01:06:54.962 INFO:tasks.workunit.client.0.smithi137.stdout:5/185: dwrite df/d11/f24 [0,4194304] 0 2024-06-15T01:06:54.963 INFO:tasks.workunit.client.0.smithi137.stdout:5/186: mkdir df/d11/d34/d3b/d49 0 2024-06-15T01:06:54.966 INFO:tasks.workunit.client.0.smithi137.stdout:5/187: fsync df/d11/d14/f2e 0 2024-06-15T01:06:54.970 INFO:tasks.workunit.client.0.smithi137.stdout:5/188: rename df/d11/d14/f2e to df/f4a 0 2024-06-15T01:06:54.975 INFO:tasks.workunit.client.0.smithi137.stdout:5/189: dwrite f7 [0,4194304] 0 2024-06-15T01:06:54.980 INFO:tasks.workunit.client.0.smithi137.stdout:5/190: dwrite f9 [0,4194304] 0 2024-06-15T01:06:54.986 INFO:tasks.workunit.client.1.smithi142.stdout:0/72: sync 2024-06-15T01:06:54.994 INFO:tasks.workunit.client.1.smithi142.stdout:5/135: dwrite d2/de/d19/f1a [0,4194304] 0 2024-06-15T01:06:54.995 INFO:tasks.workunit.client.1.smithi142.stdout:0/73: fdatasync d3/d5/f17 0 2024-06-15T01:06:55.000 INFO:tasks.workunit.client.1.smithi142.stdout:2/128: write d8/fa [236155,115751] 0 2024-06-15T01:06:55.000 INFO:tasks.workunit.client.1.smithi142.stdout:6/127: write f8 [747465,93663] 0 2024-06-15T01:06:55.002 INFO:tasks.workunit.client.1.smithi142.stdout:6/128: write f7 [2778379,81775] 0 2024-06-15T01:06:55.003 INFO:tasks.workunit.client.0.smithi137.stdout:2/71: truncate d0/f6 336586 0 2024-06-15T01:06:55.006 INFO:tasks.workunit.client.1.smithi142.stdout:0/74: symlink d3/l18 0 2024-06-15T01:06:55.007 INFO:tasks.workunit.client.0.smithi137.stdout:2/72: fsync d0/f5 0 2024-06-15T01:06:55.008 INFO:tasks.workunit.client.1.smithi142.stdout:0/75: chown d3/d12 1637619 1 2024-06-15T01:06:55.008 INFO:tasks.workunit.client.1.smithi142.stdout:8/133: getdents d0/d17 0 2024-06-15T01:06:55.010 INFO:tasks.workunit.client.1.smithi142.stdout:5/136: rename d2/fd to d2/de/d19/f2d 0 2024-06-15T01:06:55.015 INFO:tasks.workunit.client.1.smithi142.stdout:3/187: creat d0/f39 x:0 0 0 2024-06-15T01:06:55.015 INFO:tasks.workunit.client.1.smithi142.stdout:1/163: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:06:55.015 INFO:tasks.workunit.client.1.smithi142.stdout:4/233: dread d5/d1c/f34 [0,4194304] 0 2024-06-15T01:06:55.016 INFO:tasks.workunit.client.1.smithi142.stdout:0/76: creat d3/d12/f19 x:0 0 0 2024-06-15T01:06:55.018 INFO:tasks.workunit.client.1.smithi142.stdout:8/134: creat d0/d17/f30 x:0 0 0 2024-06-15T01:06:55.018 INFO:tasks.workunit.client.1.smithi142.stdout:7/123: creat d2/d8/da/d18/d25/d27/f2f x:0 0 0 2024-06-15T01:06:55.018 INFO:tasks.workunit.client.1.smithi142.stdout:6/129: dread f9 [0,4194304] 0 2024-06-15T01:06:55.019 INFO:tasks.workunit.client.1.smithi142.stdout:2/129: getdents d8/df/d10/d1c 0 2024-06-15T01:06:55.021 INFO:tasks.workunit.client.1.smithi142.stdout:3/188: symlink d0/d1f/d20/l3a 0 2024-06-15T01:06:55.021 INFO:tasks.workunit.client.1.smithi142.stdout:1/164: dread - d6/d12/f2f zero size 2024-06-15T01:06:55.022 INFO:tasks.workunit.client.1.smithi142.stdout:0/77: fdatasync d3/d5/fb 0 2024-06-15T01:06:55.026 INFO:tasks.workunit.client.1.smithi142.stdout:6/130: fsync f0 0 2024-06-15T01:06:55.027 INFO:tasks.workunit.client.1.smithi142.stdout:3/189: read d0/f2c [890343,73282] 0 2024-06-15T01:06:55.028 INFO:tasks.workunit.client.1.smithi142.stdout:2/130: rmdir d8/df/d13/d1a 39 2024-06-15T01:06:55.029 INFO:tasks.workunit.client.1.smithi142.stdout:5/137: dwrite d2/fc [0,4194304] 0 2024-06-15T01:06:55.029 INFO:tasks.workunit.client.1.smithi142.stdout:2/131: chown d8/df 618710 1 2024-06-15T01:06:55.030 INFO:tasks.workunit.client.1.smithi142.stdout:3/190: write d0/d3/f1a [3692346,22693] 0 2024-06-15T01:06:55.034 INFO:tasks.workunit.client.1.smithi142.stdout:2/132: write f6 [2882507,6917] 0 2024-06-15T01:06:55.034 INFO:tasks.workunit.client.1.smithi142.stdout:0/78: mknod d3/d5/c1a 0 2024-06-15T01:06:55.035 INFO:tasks.workunit.client.1.smithi142.stdout:8/135: rename d0/d17/l2d to d0/d1/l31 0 2024-06-15T01:06:55.036 INFO:tasks.workunit.client.1.smithi142.stdout:6/131: rename d1a/f2a to d1a/f30 0 2024-06-15T01:06:55.037 INFO:tasks.workunit.client.1.smithi142.stdout:8/136: dread - d0/d17/f28 zero size 2024-06-15T01:06:55.039 INFO:tasks.workunit.client.1.smithi142.stdout:3/191: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:55.040 INFO:tasks.workunit.client.1.smithi142.stdout:3/192: chown d0/d3/f1a 115 1 2024-06-15T01:06:55.042 INFO:tasks.workunit.client.1.smithi142.stdout:0/79: truncate d3/f4 1349477 0 2024-06-15T01:06:55.042 INFO:tasks.workunit.client.1.smithi142.stdout:3/193: truncate d0/d3/d15/f2d 701513 0 2024-06-15T01:06:55.043 INFO:tasks.workunit.client.1.smithi142.stdout:5/138: mkdir d2/d18/d2e 0 2024-06-15T01:06:55.043 INFO:tasks.workunit.client.0.smithi137.stdout:2/73: write d0/de/f16 [3695673,50921] 0 2024-06-15T01:06:55.044 INFO:tasks.workunit.client.1.smithi142.stdout:8/137: mknod d0/d1/c32 0 2024-06-15T01:06:55.044 INFO:tasks.workunit.client.1.smithi142.stdout:3/194: fdatasync d0/f39 0 2024-06-15T01:06:55.044 INFO:tasks.workunit.client.1.smithi142.stdout:4/234: getdents d5/df/d15 0 2024-06-15T01:06:55.045 INFO:tasks.workunit.client.1.smithi142.stdout:6/132: dwrite d1a/d2d/f2f [0,4194304] 0 2024-06-15T01:06:55.046 INFO:tasks.workunit.client.1.smithi142.stdout:6/133: stat f7 0 2024-06-15T01:06:55.046 INFO:tasks.workunit.client.1.smithi142.stdout:2/133: mkdir d8/df/d13/d22 0 2024-06-15T01:06:55.048 INFO:tasks.workunit.client.1.smithi142.stdout:6/134: write d1a/f1e [819759,77713] 0 2024-06-15T01:06:55.049 INFO:tasks.workunit.client.1.smithi142.stdout:1/165: getdents d6/d12/d28 0 2024-06-15T01:06:55.050 INFO:tasks.workunit.client.1.smithi142.stdout:6/135: readlink d1a/l2b 0 2024-06-15T01:06:55.051 INFO:tasks.workunit.client.1.smithi142.stdout:6/136: chown f0 1691 1 2024-06-15T01:06:55.052 INFO:tasks.workunit.client.1.smithi142.stdout:5/139: dread d2/de/d19/f1a [0,4194304] 0 2024-06-15T01:06:55.053 INFO:tasks.workunit.client.0.smithi137.stdout:5/191: fdatasync df/d3d/f42 0 2024-06-15T01:06:55.053 INFO:tasks.workunit.client.1.smithi142.stdout:3/195: symlink d0/d1f/d20/l3b 0 2024-06-15T01:06:55.054 INFO:tasks.workunit.client.0.smithi137.stdout:5/192: rmdir df/d44 39 2024-06-15T01:06:55.055 INFO:tasks.workunit.client.0.smithi137.stdout:5/193: write fd [4678444,85216] 0 2024-06-15T01:06:55.058 INFO:tasks.workunit.client.0.smithi137.stdout:5/194: rmdir df/d11/d34/d35 39 2024-06-15T01:06:55.058 INFO:tasks.workunit.client.0.smithi137.stdout:5/195: chown df/d11 1 1 2024-06-15T01:06:55.058 INFO:tasks.workunit.client.1.smithi142.stdout:3/196: dread d0/d1f/d20/f32 [0,4194304] 0 2024-06-15T01:06:55.059 INFO:tasks.workunit.client.1.smithi142.stdout:1/166: mkdir d6/d3a 0 2024-06-15T01:06:55.062 INFO:tasks.workunit.client.1.smithi142.stdout:6/137: write f9 [1299998,16795] 0 2024-06-15T01:06:55.062 INFO:tasks.workunit.client.0.smithi137.stdout:5/196: dread df/d11/f24 [0,4194304] 0 2024-06-15T01:06:55.064 INFO:tasks.workunit.client.1.smithi142.stdout:6/138: chown d1a/f20 3069204 1 2024-06-15T01:06:55.066 INFO:tasks.workunit.client.1.smithi142.stdout:2/134: dread f6 [0,4194304] 0 2024-06-15T01:06:55.066 INFO:tasks.workunit.client.1.smithi142.stdout:3/197: dwrite d0/d1f/f27 [4194304,4194304] 0 2024-06-15T01:06:55.068 INFO:tasks.workunit.client.1.smithi142.stdout:2/135: write d8/fe [2302649,30870] 0 2024-06-15T01:06:55.068 INFO:tasks.workunit.client.0.smithi137.stdout:5/197: truncate df/d11/f20 274023 0 2024-06-15T01:06:55.069 INFO:tasks.workunit.client.0.smithi137.stdout:5/198: rmdir df/d11/d34 39 2024-06-15T01:06:55.070 INFO:tasks.workunit.client.0.smithi137.stdout:5/199: write df/d3d/f42 [2800824,56685] 0 2024-06-15T01:06:55.070 INFO:tasks.workunit.client.1.smithi142.stdout:1/167: symlink d6/d25/l3b 0 2024-06-15T01:06:55.073 INFO:tasks.workunit.client.1.smithi142.stdout:5/140: mkdir d2/d2f 0 2024-06-15T01:06:55.076 INFO:tasks.workunit.client.1.smithi142.stdout:0/80: dread d3/f4 [0,4194304] 0 2024-06-15T01:06:55.077 INFO:tasks.workunit.client.0.smithi137.stdout:5/200: rmdir df/d11 39 2024-06-15T01:06:55.078 INFO:tasks.workunit.client.1.smithi142.stdout:3/198: rmdir d0/d1f/d20 39 2024-06-15T01:06:55.079 INFO:tasks.workunit.client.1.smithi142.stdout:3/199: write d0/f39 [407421,107167] 0 2024-06-15T01:06:55.082 INFO:tasks.workunit.client.1.smithi142.stdout:1/168: rmdir d6/d12/d28 39 2024-06-15T01:06:55.083 INFO:tasks.workunit.client.1.smithi142.stdout:5/141: truncate d2/f7 2855258 0 2024-06-15T01:06:55.084 INFO:tasks.workunit.client.1.smithi142.stdout:6/139: creat d1a/d2d/d2e/f31 x:0 0 0 2024-06-15T01:06:55.085 INFO:tasks.workunit.client.1.smithi142.stdout:5/142: read d2/f25 [262053,124771] 0 2024-06-15T01:06:55.086 INFO:tasks.workunit.client.0.smithi137.stdout:5/201: dwrite df/d11/d14/f21 [4194304,4194304] 0 2024-06-15T01:06:55.087 INFO:tasks.workunit.client.1.smithi142.stdout:2/136: dwrite d8/df/d13/d1a/f1d [0,4194304] 0 2024-06-15T01:06:55.087 INFO:tasks.workunit.client.1.smithi142.stdout:5/143: write d2/f28 [706771,21207] 0 2024-06-15T01:06:55.088 INFO:tasks.workunit.client.1.smithi142.stdout:2/137: write d8/df/d13/d1a/f1d [3445161,121571] 0 2024-06-15T01:06:55.089 INFO:tasks.workunit.client.1.smithi142.stdout:5/144: truncate d2/de/d19/f1a 5063661 0 2024-06-15T01:06:55.090 INFO:tasks.workunit.client.0.smithi137.stdout:5/202: chown df/d11/d34/d35/f3f 4835 1 2024-06-15T01:06:55.091 INFO:tasks.workunit.client.1.smithi142.stdout:8/138: truncate d0/f14 2195566 0 2024-06-15T01:06:55.092 INFO:tasks.workunit.client.1.smithi142.stdout:8/139: read - d0/f20 zero size 2024-06-15T01:06:55.093 INFO:tasks.workunit.client.0.smithi137.stdout:5/203: mkdir df/d11/d34/d3b/d4b 0 2024-06-15T01:06:55.098 INFO:tasks.workunit.client.1.smithi142.stdout:5/145: dwrite d2/de/f1f [0,4194304] 0 2024-06-15T01:06:55.099 INFO:tasks.workunit.client.0.smithi137.stdout:5/204: rename f5 to df/d11/f4c 0 2024-06-15T01:06:55.099 INFO:tasks.workunit.client.1.smithi142.stdout:4/235: truncate d5/d9/da/f54 869171 0 2024-06-15T01:06:55.100 INFO:tasks.workunit.client.1.smithi142.stdout:4/236: readlink d5/df/d12/l14 0 2024-06-15T01:06:55.100 INFO:tasks.workunit.client.1.smithi142.stdout:6/140: rename f4 to d1a/d2d/f32 0 2024-06-15T01:06:55.101 INFO:tasks.workunit.client.1.smithi142.stdout:5/146: write d2/f9 [3439376,33034] 0 2024-06-15T01:06:55.101 INFO:tasks.workunit.client.0.smithi137.stdout:9/68: sync 2024-06-15T01:06:55.102 INFO:tasks.workunit.client.0.smithi137.stdout:5/205: creat df/d44/f4d x:0 0 0 2024-06-15T01:06:55.103 INFO:tasks.workunit.client.0.smithi137.stdout:9/69: readlink l6 0 2024-06-15T01:06:55.104 INFO:tasks.workunit.client.1.smithi142.stdout:3/200: rmdir d0/d1f/d20 39 2024-06-15T01:06:55.104 INFO:tasks.workunit.client.0.smithi137.stdout:5/206: truncate df/f41 110131 0 2024-06-15T01:06:55.105 INFO:tasks.workunit.client.1.smithi142.stdout:1/169: mkdir d6/d12/d3c 0 2024-06-15T01:06:55.105 INFO:tasks.workunit.client.1.smithi142.stdout:4/237: write d5/d9/d41/f4c [8496637,25561] 0 2024-06-15T01:06:55.106 INFO:tasks.workunit.client.1.smithi142.stdout:4/238: truncate d5/d9/d1a/d4f/f56 288211 0 2024-06-15T01:06:55.107 INFO:tasks.workunit.client.1.smithi142.stdout:2/138: mkdir d8/d19/d23 0 2024-06-15T01:06:55.107 INFO:tasks.workunit.client.1.smithi142.stdout:4/239: read d5/d1c/f34 [501460,16338] 0 2024-06-15T01:06:55.108 INFO:tasks.workunit.client.0.smithi137.stdout:9/70: getdents df 0 2024-06-15T01:06:55.108 INFO:tasks.workunit.client.1.smithi142.stdout:4/240: read - d5/df/d12/f43 zero size 2024-06-15T01:06:55.108 INFO:tasks.workunit.client.1.smithi142.stdout:4/241: fsync d5/f6 0 2024-06-15T01:06:55.109 INFO:tasks.workunit.client.0.smithi137.stdout:9/71: getdents df 0 2024-06-15T01:06:55.111 INFO:tasks.workunit.client.0.smithi137.stdout:9/72: mkdir df/d12 0 2024-06-15T01:06:55.113 INFO:tasks.workunit.client.0.smithi137.stdout:9/73: write fd [1150143,92960] 0 2024-06-15T01:06:55.114 INFO:tasks.workunit.client.1.smithi142.stdout:0/81: write d3/d12/f14 [285243,97472] 0 2024-06-15T01:06:55.115 INFO:tasks.workunit.client.1.smithi142.stdout:0/82: chown d3/d5/l13 369621 1 2024-06-15T01:06:55.115 INFO:tasks.workunit.client.1.smithi142.stdout:6/141: creat d1a/d2d/f33 x:0 0 0 2024-06-15T01:06:55.117 INFO:tasks.workunit.client.0.smithi137.stdout:5/207: fdatasync df/d11/f13 0 2024-06-15T01:06:55.119 INFO:tasks.workunit.client.0.smithi137.stdout:5/208: write df/d3d/f42 [2194181,23142] 0 2024-06-15T01:06:55.121 INFO:tasks.workunit.client.1.smithi142.stdout:1/170: creat d6/d25/f3d x:0 0 0 2024-06-15T01:06:55.121 INFO:tasks.workunit.client.0.smithi137.stdout:5/209: creat df/d44/f4e x:0 0 0 2024-06-15T01:06:55.122 INFO:tasks.workunit.client.0.smithi137.stdout:5/210: read - df/d11/d14/f16 zero size 2024-06-15T01:06:55.123 INFO:tasks.workunit.client.0.smithi137.stdout:5/211: truncate fc 5065948 0 2024-06-15T01:06:55.123 INFO:tasks.workunit.client.0.smithi137.stdout:5/212: chown fd 469 1 2024-06-15T01:06:55.125 INFO:tasks.workunit.client.1.smithi142.stdout:4/242: creat d5/d9/da/f5a x:0 0 0 2024-06-15T01:06:55.127 INFO:tasks.workunit.client.0.smithi137.stdout:5/213: getdents df/d11/d34 0 2024-06-15T01:06:55.128 INFO:tasks.workunit.client.1.smithi142.stdout:0/83: mkdir d3/d5/d1b 0 2024-06-15T01:06:55.129 INFO:tasks.workunit.client.0.smithi137.stdout:5/214: mkdir df/d11/d34/d3b/d4b/d4f 0 2024-06-15T01:06:55.130 INFO:tasks.workunit.client.1.smithi142.stdout:5/147: mknod d2/d18/d2e/c30 0 2024-06-15T01:06:55.135 INFO:tasks.workunit.client.1.smithi142.stdout:0/84: truncate d3/d5/fb 1428665 0 2024-06-15T01:06:55.136 INFO:tasks.workunit.client.0.smithi137.stdout:5/215: dwrite df/f2b [0,4194304] 0 2024-06-15T01:06:55.137 INFO:tasks.workunit.client.1.smithi142.stdout:3/201: creat d0/f3c x:0 0 0 2024-06-15T01:06:55.137 INFO:tasks.workunit.client.0.smithi137.stdout:5/216: write df/f2b [4968774,31834] 0 2024-06-15T01:06:55.138 INFO:tasks.workunit.client.1.smithi142.stdout:2/139: creat d8/df/d13/f24 x:0 0 0 2024-06-15T01:06:55.139 INFO:tasks.workunit.client.1.smithi142.stdout:2/140: truncate d8/f16 278762 0 2024-06-15T01:06:55.140 INFO:tasks.workunit.client.0.smithi137.stdout:5/217: mkdir df/d50 0 2024-06-15T01:06:55.140 INFO:tasks.workunit.client.1.smithi142.stdout:3/202: dread d0/f39 [0,4194304] 0 2024-06-15T01:06:55.140 INFO:tasks.workunit.client.1.smithi142.stdout:5/148: mkdir d2/d17/d31 0 2024-06-15T01:06:55.141 INFO:tasks.workunit.client.1.smithi142.stdout:2/141: read d8/df/d13/d1a/f1d [1138704,53285] 0 2024-06-15T01:06:55.142 INFO:tasks.workunit.client.0.smithi137.stdout:5/218: symlink df/d50/l51 0 2024-06-15T01:06:55.143 INFO:tasks.workunit.client.0.smithi137.stdout:5/219: write df/d11/d14/f21 [8211609,112889] 0 2024-06-15T01:06:55.144 INFO:tasks.workunit.client.1.smithi142.stdout:1/171: truncate d6/f2a 1833564 0 2024-06-15T01:06:55.146 INFO:tasks.workunit.client.0.smithi137.stdout:5/220: mknod df/c52 0 2024-06-15T01:06:55.147 INFO:tasks.workunit.client.1.smithi142.stdout:7/124: sync 2024-06-15T01:06:55.147 INFO:tasks.workunit.client.1.smithi142.stdout:9/112: sync 2024-06-15T01:06:55.147 INFO:tasks.workunit.client.0.smithi137.stdout:5/221: chown df/d11/d14/l25 595 1 2024-06-15T01:06:55.148 INFO:tasks.workunit.client.1.smithi142.stdout:1/172: dread d6/f26 [0,4194304] 0 2024-06-15T01:06:55.149 INFO:tasks.workunit.client.1.smithi142.stdout:1/173: chown d6/d12/d30 1 1 2024-06-15T01:06:55.150 INFO:tasks.workunit.client.0.smithi137.stdout:5/222: mkdir df/d11/d34/d35/d53 0 2024-06-15T01:06:55.153 INFO:tasks.workunit.client.1.smithi142.stdout:3/203: creat d0/d3/f3d x:0 0 0 2024-06-15T01:06:55.155 INFO:tasks.workunit.client.0.smithi137.stdout:5/223: dwrite f9 [0,4194304] 0 2024-06-15T01:06:55.156 INFO:tasks.workunit.client.0.smithi137.stdout:5/224: write f9 [7472671,46206] 0 2024-06-15T01:06:55.161 INFO:tasks.workunit.client.1.smithi142.stdout:6/142: getdents d1a/d2d/d2e 0 2024-06-15T01:06:55.161 INFO:tasks.workunit.client.1.smithi142.stdout:7/125: truncate d2/f1a 1339957 0 2024-06-15T01:06:55.163 INFO:tasks.workunit.client.1.smithi142.stdout:0/85: creat d3/d5/d1b/f1c x:0 0 0 2024-06-15T01:06:55.165 INFO:tasks.workunit.client.0.smithi137.stdout:5/225: rename df/d50 to df/d44/d54 0 2024-06-15T01:06:55.167 INFO:tasks.workunit.client.1.smithi142.stdout:4/243: getdents d5/d9/d1a/d4f 0 2024-06-15T01:06:55.167 INFO:tasks.workunit.client.1.smithi142.stdout:3/204: dread d0/f39 [0,4194304] 0 2024-06-15T01:06:55.168 INFO:tasks.workunit.client.1.smithi142.stdout:8/140: dread d0/f14 [0,4194304] 0 2024-06-15T01:06:55.168 INFO:tasks.workunit.client.1.smithi142.stdout:5/149: dwrite d2/de/d19/f2d [4194304,4194304] 0 2024-06-15T01:06:55.168 INFO:tasks.workunit.client.1.smithi142.stdout:9/113: creat d6/d31/f32 x:0 0 0 2024-06-15T01:06:55.168 INFO:tasks.workunit.client.1.smithi142.stdout:2/142: dread d8/f9 [0,4194304] 0 2024-06-15T01:06:55.169 INFO:tasks.workunit.client.1.smithi142.stdout:6/143: symlink d1a/d2d/l34 0 2024-06-15T01:06:55.169 INFO:tasks.workunit.client.0.smithi137.stdout:5/226: unlink df/d11/f20 0 2024-06-15T01:06:55.170 INFO:tasks.workunit.client.1.smithi142.stdout:3/205: readlink d0/d3/l8 0 2024-06-15T01:06:55.170 INFO:tasks.workunit.client.1.smithi142.stdout:4/244: chown d5/d9/l4b 512177 1 2024-06-15T01:06:55.170 INFO:tasks.workunit.client.1.smithi142.stdout:9/114: chown d6/dc/d1b 79694 1 2024-06-15T01:06:55.170 INFO:tasks.workunit.client.1.smithi142.stdout:0/86: creat d3/d5/f1d x:0 0 0 2024-06-15T01:06:55.170 INFO:tasks.workunit.client.1.smithi142.stdout:2/143: dread - d8/df/d13/f18 zero size 2024-06-15T01:06:55.171 INFO:tasks.workunit.client.0.smithi137.stdout:5/227: creat df/d11/d34/d35/f55 x:0 0 0 2024-06-15T01:06:55.174 INFO:tasks.workunit.client.1.smithi142.stdout:3/206: write d0/d3/f3d [437314,123134] 0 2024-06-15T01:06:55.174 INFO:tasks.workunit.client.1.smithi142.stdout:7/126: creat d2/d8/dd/d14/f30 x:0 0 0 2024-06-15T01:06:55.174 INFO:tasks.workunit.client.1.smithi142.stdout:5/150: dread d2/de/d19/f1a [4194304,4194304] 0 2024-06-15T01:06:55.175 INFO:tasks.workunit.client.0.smithi137.stdout:5/228: creat df/d11/d34/d3b/d4b/d4f/f56 x:0 0 0 2024-06-15T01:06:55.175 INFO:tasks.workunit.client.1.smithi142.stdout:1/174: creat d6/d12/f3e x:0 0 0 2024-06-15T01:06:55.176 INFO:tasks.workunit.client.1.smithi142.stdout:0/87: dread - d3/d5/d1b/f1c zero size 2024-06-15T01:06:55.176 INFO:tasks.workunit.client.1.smithi142.stdout:6/144: symlink d1a/d2d/l35 0 2024-06-15T01:06:55.176 INFO:tasks.workunit.client.0.smithi137.stdout:5/229: truncate df/d11/f31 681154 0 2024-06-15T01:06:55.177 INFO:tasks.workunit.client.1.smithi142.stdout:5/151: fdatasync d2/de/f1f 0 2024-06-15T01:06:55.177 INFO:tasks.workunit.client.1.smithi142.stdout:9/115: symlink d6/dc/dd/d22/l33 0 2024-06-15T01:06:55.177 INFO:tasks.workunit.client.1.smithi142.stdout:7/127: truncate d2/f2e 1014041 0 2024-06-15T01:06:55.179 INFO:tasks.workunit.client.1.smithi142.stdout:0/88: truncate d3/d5/f17 879107 0 2024-06-15T01:06:55.181 INFO:tasks.workunit.client.1.smithi142.stdout:4/245: rename d5/df to d5/d1c/d2a/d30/d46/d50/d51/d5b 0 2024-06-15T01:06:55.182 INFO:tasks.workunit.client.0.smithi137.stdout:5/230: dwrite df/d11/f13 [0,4194304] 0 2024-06-15T01:06:55.184 INFO:tasks.workunit.client.0.smithi137.stdout:5/231: chown df/d11/l27 51510795 1 2024-06-15T01:06:55.187 INFO:tasks.workunit.client.0.smithi137.stdout:5/232: readlink df/d11/d34/l39 0 2024-06-15T01:06:55.187 INFO:tasks.workunit.client.1.smithi142.stdout:1/175: rmdir d6/d12/d1c/d1e 39 2024-06-15T01:06:55.187 INFO:tasks.workunit.client.1.smithi142.stdout:1/176: write d6/f9 [3652778,4456] 0 2024-06-15T01:06:55.189 INFO:tasks.workunit.client.1.smithi142.stdout:6/145: symlink d1a/d2d/d2e/l36 0 2024-06-15T01:06:55.190 INFO:tasks.workunit.client.1.smithi142.stdout:5/152: write d2/de/d19/f1a [326483,99735] 0 2024-06-15T01:06:55.190 INFO:tasks.workunit.client.0.smithi137.stdout:5/233: mknod df/d11/d14/d3c/c57 0 2024-06-15T01:06:55.191 INFO:tasks.workunit.client.0.smithi137.stdout:5/234: stat df/d11/d14/l1c 0 2024-06-15T01:06:55.192 INFO:tasks.workunit.client.0.smithi137.stdout:5/235: write f7 [776505,75951] 0 2024-06-15T01:06:55.192 INFO:tasks.workunit.client.0.smithi137.stdout:5/236: chown df/d44/d54 3 1 2024-06-15T01:06:55.193 INFO:tasks.workunit.client.1.smithi142.stdout:0/89: mkdir d3/d5/d1e 0 2024-06-15T01:06:55.195 INFO:tasks.workunit.client.1.smithi142.stdout:0/90: chown c2 56 1 2024-06-15T01:06:55.197 INFO:tasks.workunit.client.1.smithi142.stdout:7/128: rename d2/c23 to d2/d8/dd/d14/c31 0 2024-06-15T01:06:55.198 INFO:tasks.workunit.client.0.smithi137.stdout:5/237: dread df/d11/d14/f21 [0,4194304] 0 2024-06-15T01:06:55.198 INFO:tasks.workunit.client.0.smithi137.stdout:5/238: readlink df/d11/d34/d3b/l45 0 2024-06-15T01:06:55.199 INFO:tasks.workunit.client.1.smithi142.stdout:5/153: dread d2/de/f1f [0,4194304] 0 2024-06-15T01:06:55.199 INFO:tasks.workunit.client.1.smithi142.stdout:2/144: write d8/f9 [1743912,58403] 0 2024-06-15T01:06:55.199 INFO:tasks.workunit.client.1.smithi142.stdout:8/141: dwrite d0/d1/f29 [0,4194304] 0 2024-06-15T01:06:55.200 INFO:tasks.workunit.client.0.smithi137.stdout:5/239: readlink df/d44/l46 0 2024-06-15T01:06:55.200 INFO:tasks.workunit.client.1.smithi142.stdout:5/154: stat d2/f25 0 2024-06-15T01:06:55.201 INFO:tasks.workunit.client.0.smithi137.stdout:5/240: mknod df/c58 0 2024-06-15T01:06:55.202 INFO:tasks.workunit.client.0.smithi137.stdout:5/241: fdatasync df/d11/d34/d35/f3f 0 2024-06-15T01:06:55.204 INFO:tasks.workunit.client.1.smithi142.stdout:0/91: symlink d3/d5/d1b/l1f 0 2024-06-15T01:06:55.205 INFO:tasks.workunit.client.0.smithi137.stdout:5/242: truncate df/d11/d14/d3c/f3e 972892 0 2024-06-15T01:06:55.207 INFO:tasks.workunit.client.1.smithi142.stdout:5/155: dwrite d2/de/d19/f2d [4194304,4194304] 0 2024-06-15T01:06:55.207 INFO:tasks.workunit.client.1.smithi142.stdout:1/177: rename c2 to d6/d12/d30/c3f 0 2024-06-15T01:06:55.207 INFO:tasks.workunit.client.0.smithi137.stdout:5/243: dread df/f41 [0,4194304] 0 2024-06-15T01:06:55.209 INFO:tasks.workunit.client.0.smithi137.stdout:5/244: write df/d3d/f42 [2243256,117360] 0 2024-06-15T01:06:55.209 INFO:tasks.workunit.client.1.smithi142.stdout:3/207: write d0/d3/da/d18/f28 [3008621,73075] 0 2024-06-15T01:06:55.209 INFO:tasks.workunit.client.0.smithi137.stdout:5/245: stat df/d44/d54/l51 0 2024-06-15T01:06:55.211 INFO:tasks.workunit.client.0.smithi137.stdout:5/246: write df/d11/f24 [2513759,68707] 0 2024-06-15T01:06:55.215 INFO:tasks.workunit.client.0.smithi137.stdout:5/247: creat df/d11/d34/d3b/f59 x:0 0 0 2024-06-15T01:06:55.216 INFO:tasks.workunit.client.1.smithi142.stdout:5/156: dread d2/ff [0,4194304] 0 2024-06-15T01:06:55.217 INFO:tasks.workunit.client.1.smithi142.stdout:9/116: dread d6/dc/f19 [0,4194304] 0 2024-06-15T01:06:55.217 INFO:tasks.workunit.client.1.smithi142.stdout:0/92: mknod d3/d5/d1e/c20 0 2024-06-15T01:06:55.218 INFO:tasks.workunit.client.1.smithi142.stdout:3/208: mkdir d0/d1f/d3e 0 2024-06-15T01:06:55.219 INFO:tasks.workunit.client.1.smithi142.stdout:3/209: dread - d0/d1f/f38 zero size 2024-06-15T01:06:55.220 INFO:tasks.workunit.client.1.smithi142.stdout:3/210: stat d0/d3/d15 0 2024-06-15T01:06:55.220 INFO:tasks.workunit.client.0.smithi137.stdout:5/248: getdents df/d44 0 2024-06-15T01:06:55.221 INFO:tasks.workunit.client.1.smithi142.stdout:8/142: rename d0/d17/f28 to d0/f33 0 2024-06-15T01:06:55.222 INFO:tasks.workunit.client.0.smithi137.stdout:5/249: mknod df/d3d/c5a 0 2024-06-15T01:06:55.223 INFO:tasks.workunit.client.1.smithi142.stdout:9/117: chown d6/dc/dd/c2d 0 1 2024-06-15T01:06:55.223 INFO:tasks.workunit.client.1.smithi142.stdout:5/157: mknod d2/de/d27/c32 0 2024-06-15T01:06:55.224 INFO:tasks.workunit.client.1.smithi142.stdout:0/93: creat d3/d12/f21 x:0 0 0 2024-06-15T01:06:55.225 INFO:tasks.workunit.client.1.smithi142.stdout:3/211: readlink d0/d1f/d20/l3a 0 2024-06-15T01:06:55.226 INFO:tasks.workunit.client.0.smithi137.stdout:5/250: link df/c52 df/d44/c5b 0 2024-06-15T01:06:55.227 INFO:tasks.workunit.client.1.smithi142.stdout:9/118: write d6/dc/f15 [565587,119446] 0 2024-06-15T01:06:55.227 INFO:tasks.workunit.client.1.smithi142.stdout:5/158: read - d2/f1e zero size 2024-06-15T01:06:55.227 INFO:tasks.workunit.client.0.smithi137.stdout:5/251: truncate df/d11/d14/f32 649719 0 2024-06-15T01:06:55.230 INFO:tasks.workunit.client.0.smithi137.stdout:5/252: mkdir df/d11/d34/d3b/d49/d5c 0 2024-06-15T01:06:55.231 INFO:tasks.workunit.client.1.smithi142.stdout:9/119: fsync d6/dc/dd/f1d 0 2024-06-15T01:06:55.231 INFO:tasks.workunit.client.0.smithi137.stdout:5/253: fsync df/d11/f4c 0 2024-06-15T01:06:55.232 INFO:tasks.workunit.client.1.smithi142.stdout:2/145: truncate f6 87832 0 2024-06-15T01:06:55.233 INFO:tasks.workunit.client.0.smithi137.stdout:5/254: getdents df/d11/d30 0 2024-06-15T01:06:55.234 INFO:tasks.workunit.client.1.smithi142.stdout:3/212: symlink d0/l3f 0 2024-06-15T01:06:55.234 INFO:tasks.workunit.client.0.smithi137.stdout:5/255: fsync df/d11/d14/d3c/f3e 0 2024-06-15T01:06:55.237 INFO:tasks.workunit.client.1.smithi142.stdout:4/246: write d5/d9/da/f54 [1254526,47307] 0 2024-06-15T01:06:55.238 INFO:tasks.workunit.client.0.smithi137.stdout:5/256: rename df/d11/c38 to df/c5d 0 2024-06-15T01:06:55.239 INFO:tasks.workunit.client.1.smithi142.stdout:5/159: rename d2/de/c15 to d2/d17/d31/c33 0 2024-06-15T01:06:55.241 INFO:tasks.workunit.client.1.smithi142.stdout:0/94: getdents d3 0 2024-06-15T01:06:55.241 INFO:tasks.workunit.client.0.smithi137.stdout:5/257: creat df/d11/d14/f5e x:0 0 0 2024-06-15T01:06:55.242 INFO:tasks.workunit.client.1.smithi142.stdout:1/178: dwrite d6/d12/d1c/f34 [0,4194304] 0 2024-06-15T01:06:55.243 INFO:tasks.workunit.client.1.smithi142.stdout:5/160: chown d2/de/d19/f1a 0 1 2024-06-15T01:06:55.244 INFO:tasks.workunit.client.1.smithi142.stdout:9/120: mkdir d6/d34 0 2024-06-15T01:06:55.244 INFO:tasks.workunit.client.1.smithi142.stdout:2/146: dwrite d8/f16 [0,4194304] 0 2024-06-15T01:06:55.246 INFO:tasks.workunit.client.1.smithi142.stdout:3/213: dread d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:55.249 INFO:tasks.workunit.client.0.smithi137.stdout:5/258: write df/d11/d14/f15 [3399281,95438] 0 2024-06-15T01:06:55.250 INFO:tasks.workunit.client.1.smithi142.stdout:0/95: mkdir d3/d12/d22 0 2024-06-15T01:06:55.250 INFO:tasks.workunit.client.1.smithi142.stdout:0/96: write f1 [2478064,6370] 0 2024-06-15T01:06:55.253 INFO:tasks.workunit.client.1.smithi142.stdout:1/179: dread d6/f26 [0,4194304] 0 2024-06-15T01:06:55.254 INFO:tasks.workunit.client.1.smithi142.stdout:9/121: readlink d6/dc/d1b/l1e 0 2024-06-15T01:06:55.259 INFO:tasks.workunit.client.1.smithi142.stdout:2/147: rename d8/df/d13 to d8/d19/d23/d25 0 2024-06-15T01:06:55.259 INFO:tasks.workunit.client.1.smithi142.stdout:4/247: mknod d5/d1c/d2a/d30/c5c 0 2024-06-15T01:06:55.261 INFO:tasks.workunit.client.1.smithi142.stdout:0/97: readlink d3/d12/l16 0 2024-06-15T01:06:55.271 INFO:tasks.workunit.client.0.smithi137.stdout:5/259: truncate df/d11/d14/f15 3900259 0 2024-06-15T01:06:55.271 INFO:tasks.workunit.client.0.smithi137.stdout:5/260: fsync df/d11/f24 0 2024-06-15T01:06:55.273 INFO:tasks.workunit.client.0.smithi137.stdout:5/261: getdents df/d11/d34/d3b/d49/d5c 0 2024-06-15T01:06:55.276 INFO:tasks.workunit.client.0.smithi137.stdout:5/262: getdents df/d11/d14 0 2024-06-15T01:06:55.277 INFO:tasks.workunit.client.0.smithi137.stdout:5/263: chown df/d11/d14/f43 3 1 2024-06-15T01:06:55.278 INFO:tasks.workunit.client.0.smithi137.stdout:5/264: readlink df/d11/l1d 0 2024-06-15T01:06:55.279 INFO:tasks.workunit.client.0.smithi137.stdout:5/265: chown df/d11/d14 7122513 1 2024-06-15T01:06:55.291 INFO:tasks.workunit.client.0.smithi137.stdout:5/266: dread df/d11/d34/d35/f3f [0,4194304] 0 2024-06-15T01:06:55.294 INFO:tasks.workunit.client.0.smithi137.stdout:5/267: rename f9 to df/d3d/f5f 0 2024-06-15T01:06:55.294 INFO:tasks.workunit.client.0.smithi137.stdout:5/268: readlink df/d11/l1d 0 2024-06-15T01:06:55.296 INFO:tasks.workunit.client.0.smithi137.stdout:5/269: rmdir df/d11/d34/d3b 39 2024-06-15T01:06:55.298 INFO:tasks.workunit.client.0.smithi137.stdout:5/270: mknod df/d11/d34/d35/c60 0 2024-06-15T01:06:55.299 INFO:tasks.workunit.client.1.smithi142.stdout:9/122: rmdir d6/dc/d1b 39 2024-06-15T01:06:55.300 INFO:tasks.workunit.client.1.smithi142.stdout:9/123: fsync d6/dc/f1c 0 2024-06-15T01:06:55.300 INFO:tasks.workunit.client.1.smithi142.stdout:3/214: symlink d0/d1f/l40 0 2024-06-15T01:06:55.301 INFO:tasks.workunit.client.0.smithi137.stdout:5/271: creat df/d11/d30/f61 x:0 0 0 2024-06-15T01:06:55.301 INFO:tasks.workunit.client.1.smithi142.stdout:3/215: fdatasync d0/d3/da/d18/f28 0 2024-06-15T01:06:55.301 INFO:tasks.workunit.client.1.smithi142.stdout:4/248: mknod d5/d9/d23/d32/c5d 0 2024-06-15T01:06:55.303 INFO:tasks.workunit.client.1.smithi142.stdout:5/161: rmdir d2/d2f 0 2024-06-15T01:06:55.303 INFO:tasks.workunit.client.1.smithi142.stdout:5/162: chown d2/d17/d31 7 1 2024-06-15T01:06:55.304 INFO:tasks.workunit.client.1.smithi142.stdout:1/180: creat d6/d12/d28/f40 x:0 0 0 2024-06-15T01:06:55.306 INFO:tasks.workunit.client.0.smithi137.stdout:5/272: dwrite df/f2b [4194304,4194304] 0 2024-06-15T01:06:55.306 INFO:tasks.workunit.client.1.smithi142.stdout:2/148: link d8/fe d8/d19/d23/f26 0 2024-06-15T01:06:55.309 INFO:tasks.workunit.client.1.smithi142.stdout:3/216: truncate d0/f2c 1325165 0 2024-06-15T01:06:55.309 INFO:tasks.workunit.client.0.smithi137.stdout:5/273: rename df/d11/d14/d3c/c57 to df/d44/d54/c62 0 2024-06-15T01:06:55.310 INFO:tasks.workunit.client.0.smithi137.stdout:5/274: dread - df/d44/f4d zero size 2024-06-15T01:06:55.311 INFO:tasks.workunit.client.1.smithi142.stdout:5/163: creat d2/d18/d2e/f34 x:0 0 0 2024-06-15T01:06:55.311 INFO:tasks.workunit.client.1.smithi142.stdout:0/98: write d3/f4 [2384502,85414] 0 2024-06-15T01:06:55.313 INFO:tasks.workunit.client.1.smithi142.stdout:1/181: rename d6/cb to d6/d12/d28/c41 0 2024-06-15T01:06:55.313 INFO:tasks.workunit.client.1.smithi142.stdout:0/99: stat d3/ff 0 2024-06-15T01:06:55.314 INFO:tasks.workunit.client.0.smithi137.stdout:5/275: creat df/d11/d34/d3b/d4b/f63 x:0 0 0 2024-06-15T01:06:55.314 INFO:tasks.workunit.client.1.smithi142.stdout:2/149: truncate f4 1399083 0 2024-06-15T01:06:55.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/217: creat d0/d1b/f41 x:0 0 0 2024-06-15T01:06:55.316 INFO:tasks.workunit.client.1.smithi142.stdout:9/124: dwrite d6/dc/f19 [0,4194304] 0 2024-06-15T01:06:55.316 INFO:tasks.workunit.client.1.smithi142.stdout:1/182: fsync d6/d12/f22 0 2024-06-15T01:06:55.317 INFO:tasks.workunit.client.0.smithi137.stdout:5/276: link df/d11/d34/l39 df/d11/d14/l64 0 2024-06-15T01:06:55.319 INFO:tasks.workunit.client.1.smithi142.stdout:9/125: chown d6/l2a 6219 1 2024-06-15T01:06:55.319 INFO:tasks.workunit.client.1.smithi142.stdout:0/100: write d3/d5/d1b/f1c [1002707,33000] 0 2024-06-15T01:06:55.320 INFO:tasks.workunit.client.0.smithi137.stdout:5/277: getdents df/d11/d34/d3b 0 2024-06-15T01:06:55.322 INFO:tasks.workunit.client.1.smithi142.stdout:2/150: link d8/fa d8/d19/d23/d25/f27 0 2024-06-15T01:06:55.324 INFO:tasks.workunit.client.1.smithi142.stdout:1/183: symlink d6/d12/l42 0 2024-06-15T01:06:55.324 INFO:tasks.workunit.client.1.smithi142.stdout:9/126: rmdir d6/dc 39 2024-06-15T01:06:55.325 INFO:tasks.workunit.client.1.smithi142.stdout:0/101: write d3/d12/f21 [677801,103762] 0 2024-06-15T01:06:55.328 INFO:tasks.workunit.client.1.smithi142.stdout:5/164: dwrite d2/d18/d2e/f34 [0,4194304] 0 2024-06-15T01:06:55.329 INFO:tasks.workunit.client.1.smithi142.stdout:0/102: creat d3/f23 x:0 0 0 2024-06-15T01:06:55.330 INFO:tasks.workunit.client.1.smithi142.stdout:1/184: creat d6/d3a/f43 x:0 0 0 2024-06-15T01:06:55.333 INFO:tasks.workunit.client.1.smithi142.stdout:1/185: write d6/d12/d28/f40 [694915,81481] 0 2024-06-15T01:06:55.334 INFO:tasks.workunit.client.1.smithi142.stdout:9/127: mkdir d6/dc/d1b/d25/d35 0 2024-06-15T01:06:55.335 INFO:tasks.workunit.client.0.smithi137.stdout:5/278: getdents df/d11/d14/d3c 0 2024-06-15T01:06:55.337 INFO:tasks.workunit.client.1.smithi142.stdout:2/151: dwrite d8/fe [0,4194304] 0 2024-06-15T01:06:55.338 INFO:tasks.workunit.client.1.smithi142.stdout:5/165: dread d2/f9 [0,4194304] 0 2024-06-15T01:06:55.338 INFO:tasks.workunit.client.1.smithi142.stdout:9/128: mknod d6/d34/c36 0 2024-06-15T01:06:55.339 INFO:tasks.workunit.client.0.smithi137.stdout:5/279: getdents df/d11/d34/d3b/d4b/d4f 0 2024-06-15T01:06:55.340 INFO:tasks.workunit.client.1.smithi142.stdout:1/186: stat d6/d12/d1c/d2c/c35 0 2024-06-15T01:06:55.341 INFO:tasks.workunit.client.1.smithi142.stdout:9/129: truncate d6/dc/dd/d22/f2b 603598 0 2024-06-15T01:06:55.341 INFO:tasks.workunit.client.1.smithi142.stdout:9/130: chown d6/dc/dd/c17 8 1 2024-06-15T01:06:55.342 INFO:tasks.workunit.client.0.smithi137.stdout:5/280: mknod df/d11/d34/d3b/d4b/c65 0 2024-06-15T01:06:55.342 INFO:tasks.workunit.client.0.smithi137.stdout:5/281: stat df/d11/l28 0 2024-06-15T01:06:55.346 INFO:tasks.workunit.client.0.smithi137.stdout:5/282: dread df/d11/d34/d35/f3f [0,4194304] 0 2024-06-15T01:06:55.350 INFO:tasks.workunit.client.0.smithi137.stdout:5/283: dread df/f2a [0,4194304] 0 2024-06-15T01:06:55.350 INFO:tasks.workunit.client.1.smithi142.stdout:2/152: dread f4 [0,4194304] 0 2024-06-15T01:06:55.351 INFO:tasks.workunit.client.0.smithi137.stdout:5/284: rename df/d11/d34/d3b to df/d11/d34/d3b/d4b/d66 22 2024-06-15T01:06:55.352 INFO:tasks.workunit.client.1.smithi142.stdout:9/131: getdents d6/dc/dd/d23 0 2024-06-15T01:06:55.352 INFO:tasks.workunit.client.0.smithi137.stdout:5/285: chown df/d11/d34/l40 811633445 1 2024-06-15T01:06:55.353 INFO:tasks.workunit.client.1.smithi142.stdout:1/187: dwrite d6/d12/d1c/d1e/f24 [0,4194304] 0 2024-06-15T01:06:55.355 INFO:tasks.workunit.client.0.smithi137.stdout:5/286: rename df/d11/d34/d3b/d4b/d4f/f56 to df/d11/d34/d3b/d4b/d4f/f67 0 2024-06-15T01:06:55.359 INFO:tasks.workunit.client.1.smithi142.stdout:9/132: creat d6/dc/f37 x:0 0 0 2024-06-15T01:06:55.361 INFO:tasks.workunit.client.1.smithi142.stdout:5/166: write d2/f1e [719397,55669] 0 2024-06-15T01:06:55.361 INFO:tasks.workunit.client.1.smithi142.stdout:3/218: dwrite d0/f39 [0,4194304] 0 2024-06-15T01:06:55.364 INFO:tasks.workunit.client.0.smithi137.stdout:5/287: dwrite df/d11/d34/d3b/d4b/d4f/f67 [0,4194304] 0 2024-06-15T01:06:55.365 INFO:tasks.workunit.client.0.smithi137.stdout:5/288: chown df/d44/d54 113926 1 2024-06-15T01:06:55.367 INFO:tasks.workunit.client.1.smithi142.stdout:9/133: mknod d6/c38 0 2024-06-15T01:06:55.370 INFO:tasks.workunit.client.1.smithi142.stdout:2/153: dwrite d8/f9 [0,4194304] 0 2024-06-15T01:06:55.372 INFO:tasks.workunit.client.1.smithi142.stdout:3/219: creat d0/d3/da/f42 x:0 0 0 2024-06-15T01:06:55.373 INFO:tasks.workunit.client.0.smithi137.stdout:5/289: dwrite df/d11/d14/d3c/f3e [0,4194304] 0 2024-06-15T01:06:55.374 INFO:tasks.workunit.client.1.smithi142.stdout:9/134: dread - d6/d31/f32 zero size 2024-06-15T01:06:55.376 INFO:tasks.workunit.client.1.smithi142.stdout:3/220: fdatasync d0/d1b/f2a 0 2024-06-15T01:06:55.376 INFO:tasks.workunit.client.1.smithi142.stdout:0/103: dwrite d3/d5/fb [0,4194304] 0 2024-06-15T01:06:55.377 INFO:tasks.workunit.client.1.smithi142.stdout:1/188: dwrite d6/d12/d28/f40 [0,4194304] 0 2024-06-15T01:06:55.378 INFO:tasks.workunit.client.1.smithi142.stdout:2/154: fsync d8/d19/d23/d25/f27 0 2024-06-15T01:06:55.379 INFO:tasks.workunit.client.0.smithi137.stdout:5/290: rmdir df/d11/d34/d35/d53 0 2024-06-15T01:06:55.380 INFO:tasks.workunit.client.1.smithi142.stdout:0/104: read - d3/f23 zero size 2024-06-15T01:06:55.380 INFO:tasks.workunit.client.1.smithi142.stdout:5/167: dread d2/de/f1f [0,4194304] 0 2024-06-15T01:06:55.382 INFO:tasks.workunit.client.1.smithi142.stdout:0/105: dread - d3/d5/f1d zero size 2024-06-15T01:06:55.382 INFO:tasks.workunit.client.0.smithi137.stdout:5/291: write fe [2702772,57315] 0 2024-06-15T01:06:55.383 INFO:tasks.workunit.client.1.smithi142.stdout:0/106: chown d3/d5 26043193 1 2024-06-15T01:06:55.383 INFO:tasks.workunit.client.0.smithi137.stdout:5/292: chown df/d11/d34/l40 61 1 2024-06-15T01:06:55.384 INFO:tasks.workunit.client.0.smithi137.stdout:5/293: read - df/d11/d30/f61 zero size 2024-06-15T01:06:55.385 INFO:tasks.workunit.client.1.smithi142.stdout:9/135: dread d6/dc/f1c [0,4194304] 0 2024-06-15T01:06:55.386 INFO:tasks.workunit.client.1.smithi142.stdout:0/107: fdatasync d3/d5/f1d 0 2024-06-15T01:06:55.386 INFO:tasks.workunit.client.1.smithi142.stdout:1/189: write d6/d12/d1c/d1e/f20 [657278,29589] 0 2024-06-15T01:06:55.387 INFO:tasks.workunit.client.1.smithi142.stdout:2/155: dread d8/f9 [0,4194304] 0 2024-06-15T01:06:55.388 INFO:tasks.workunit.client.0.smithi137.stdout:5/294: mknod df/d11/d34/c68 0 2024-06-15T01:06:55.389 INFO:tasks.workunit.client.1.smithi142.stdout:5/168: mknod d2/d17/c35 0 2024-06-15T01:06:55.390 INFO:tasks.workunit.client.1.smithi142.stdout:9/136: fdatasync d6/f9 0 2024-06-15T01:06:55.393 INFO:tasks.workunit.client.1.smithi142.stdout:2/156: symlink d8/l28 0 2024-06-15T01:06:55.394 INFO:tasks.workunit.client.1.smithi142.stdout:0/108: dwrite d3/f4 [0,4194304] 0 2024-06-15T01:06:55.395 INFO:tasks.workunit.client.1.smithi142.stdout:5/169: creat d2/de/d19/f36 x:0 0 0 2024-06-15T01:06:55.396 INFO:tasks.workunit.client.1.smithi142.stdout:2/157: write d8/df/d10/f11 [1600995,56867] 0 2024-06-15T01:06:55.397 INFO:tasks.workunit.client.1.smithi142.stdout:0/109: write d3/d5/d1b/f1c [691301,5506] 0 2024-06-15T01:06:55.400 INFO:tasks.workunit.client.0.smithi137.stdout:5/295: link df/d11/d14/l64 df/d11/d34/d3b/d49/d5c/l69 0 2024-06-15T01:06:55.400 INFO:tasks.workunit.client.0.smithi137.stdout:5/296: stat df/d11/c2d 0 2024-06-15T01:06:55.402 INFO:tasks.workunit.client.0.smithi137.stdout:5/297: chown df/d11/d14/f21 3 1 2024-06-15T01:06:55.406 INFO:tasks.workunit.client.0.smithi137.stdout:5/298: mknod df/d11/d14/c6a 0 2024-06-15T01:06:55.430 INFO:tasks.workunit.client.0.smithi137.stdout:5/299: truncate df/f2a 110689 0 2024-06-15T01:06:55.432 INFO:tasks.workunit.client.0.smithi137.stdout:5/300: mkdir df/d44/d54/d6b 0 2024-06-15T01:06:55.433 INFO:tasks.workunit.client.0.smithi137.stdout:5/301: write df/d11/d14/d3c/f3e [776839,56751] 0 2024-06-15T01:06:55.435 INFO:tasks.workunit.client.1.smithi142.stdout:5/170: unlink d2/f1e 0 2024-06-15T01:06:55.435 INFO:tasks.workunit.client.1.smithi142.stdout:2/158: truncate d8/fe 4358185 0 2024-06-15T01:06:55.435 INFO:tasks.workunit.client.1.smithi142.stdout:1/190: link d6/d12/f3e d6/d3a/f44 0 2024-06-15T01:06:55.436 INFO:tasks.workunit.client.0.smithi137.stdout:5/302: mkdir df/d11/d14/d6c 0 2024-06-15T01:06:55.436 INFO:tasks.workunit.client.1.smithi142.stdout:0/110: read - d3/d5/f1d zero size 2024-06-15T01:06:55.437 INFO:tasks.workunit.client.1.smithi142.stdout:0/111: fsync d3/f4 0 2024-06-15T01:06:55.438 INFO:tasks.workunit.client.1.smithi142.stdout:0/112: dread - d3/ff zero size 2024-06-15T01:06:55.440 INFO:tasks.workunit.client.1.smithi142.stdout:2/159: mknod d8/df/d1f/c29 0 2024-06-15T01:06:55.441 INFO:tasks.workunit.client.1.smithi142.stdout:0/113: rmdir d3/d5 39 2024-06-15T01:06:55.441 INFO:tasks.workunit.client.1.smithi142.stdout:0/114: write d3/d12/f19 [291187,88139] 0 2024-06-15T01:06:55.446 INFO:tasks.workunit.client.1.smithi142.stdout:1/191: link d6/d12/d1c/d2c/c35 d6/c45 0 2024-06-15T01:06:55.448 INFO:tasks.workunit.client.1.smithi142.stdout:0/115: creat d3/d12/d22/f24 x:0 0 0 2024-06-15T01:06:55.450 INFO:tasks.workunit.client.1.smithi142.stdout:5/171: write d2/f25 [57782,85420] 0 2024-06-15T01:06:55.450 INFO:tasks.workunit.client.1.smithi142.stdout:3/221: dread d0/d3/da/d18/f28 [0,4194304] 0 2024-06-15T01:06:55.451 INFO:tasks.workunit.client.1.smithi142.stdout:2/160: write d8/d19/d23/d25/f27 [2337425,97579] 0 2024-06-15T01:06:55.453 INFO:tasks.workunit.client.1.smithi142.stdout:5/172: fsync d2/ff 0 2024-06-15T01:06:55.456 INFO:tasks.workunit.client.1.smithi142.stdout:1/192: dwrite d6/d12/f22 [0,4194304] 0 2024-06-15T01:06:55.458 INFO:tasks.workunit.client.1.smithi142.stdout:3/222: getdents d0 0 2024-06-15T01:06:55.458 INFO:tasks.workunit.client.1.smithi142.stdout:2/161: write f6 [522347,30062] 0 2024-06-15T01:06:55.459 INFO:tasks.workunit.client.1.smithi142.stdout:1/193: mkdir d6/d46 0 2024-06-15T01:06:55.460 INFO:tasks.workunit.client.1.smithi142.stdout:0/116: dwrite d3/d12/f21 [0,4194304] 0 2024-06-15T01:06:55.462 INFO:tasks.workunit.client.1.smithi142.stdout:3/223: creat d0/d3/f43 x:0 0 0 2024-06-15T01:06:55.462 INFO:tasks.workunit.client.1.smithi142.stdout:1/194: read d6/d12/d28/f33 [2508144,59122] 0 2024-06-15T01:06:55.462 INFO:tasks.workunit.client.1.smithi142.stdout:5/173: dread d2/f7 [0,4194304] 0 2024-06-15T01:06:55.463 INFO:tasks.workunit.client.1.smithi142.stdout:2/162: dread f4 [0,4194304] 0 2024-06-15T01:06:55.463 INFO:tasks.workunit.client.1.smithi142.stdout:0/117: chown d3/d5/d1e 58868334 1 2024-06-15T01:06:55.465 INFO:tasks.workunit.client.1.smithi142.stdout:1/195: read d6/d12/d28/f40 [2481212,52960] 0 2024-06-15T01:06:55.465 INFO:tasks.workunit.client.1.smithi142.stdout:0/118: dread - d3/f23 zero size 2024-06-15T01:06:55.468 INFO:tasks.workunit.client.1.smithi142.stdout:5/174: creat d2/d18/f37 x:0 0 0 2024-06-15T01:06:55.469 INFO:tasks.workunit.client.1.smithi142.stdout:3/224: mknod d0/d1b/c44 0 2024-06-15T01:06:55.471 INFO:tasks.workunit.client.1.smithi142.stdout:2/163: mkdir d8/df/d2a 0 2024-06-15T01:06:55.472 INFO:tasks.workunit.client.1.smithi142.stdout:2/164: readlink d8/ld 0 2024-06-15T01:06:55.472 INFO:tasks.workunit.client.1.smithi142.stdout:2/165: chown d8/d19/d23 3 1 2024-06-15T01:06:55.472 INFO:tasks.workunit.client.1.smithi142.stdout:1/196: mkdir d6/d12/d1c/d2c/d47 0 2024-06-15T01:06:55.472 INFO:tasks.workunit.client.1.smithi142.stdout:2/166: fdatasync d8/fa 0 2024-06-15T01:06:55.473 INFO:tasks.workunit.client.1.smithi142.stdout:0/119: rename d3/d5/l10 to d3/d12/l25 0 2024-06-15T01:06:55.473 INFO:tasks.workunit.client.1.smithi142.stdout:0/120: chown d3/la 25 1 2024-06-15T01:06:55.476 INFO:tasks.workunit.client.1.smithi142.stdout:2/167: rmdir d8/df/d10/d1c 39 2024-06-15T01:06:55.477 INFO:tasks.workunit.client.1.smithi142.stdout:0/121: rename d3/d5/d1b/f1c to d3/d5/d1b/f26 0 2024-06-15T01:06:55.478 INFO:tasks.workunit.client.1.smithi142.stdout:0/122: readlink d3/d5/l11 0 2024-06-15T01:06:55.479 INFO:tasks.workunit.client.1.smithi142.stdout:1/197: fsync d6/d12/f22 0 2024-06-15T01:06:55.481 INFO:tasks.workunit.client.1.smithi142.stdout:0/123: symlink d3/d5/d1e/l27 0 2024-06-15T01:06:55.481 INFO:tasks.workunit.client.1.smithi142.stdout:5/175: dwrite d2/f9 [4194304,4194304] 0 2024-06-15T01:06:55.483 INFO:tasks.workunit.client.1.smithi142.stdout:2/168: dread d8/fe [0,4194304] 0 2024-06-15T01:06:55.483 INFO:tasks.workunit.client.1.smithi142.stdout:5/176: write d2/d18/f37 [858203,64363] 0 2024-06-15T01:06:55.484 INFO:tasks.workunit.client.1.smithi142.stdout:0/124: rmdir d3/d12 39 2024-06-15T01:06:55.488 INFO:tasks.workunit.client.1.smithi142.stdout:2/169: creat d8/df/d2a/f2b x:0 0 0 2024-06-15T01:06:55.488 INFO:tasks.workunit.client.1.smithi142.stdout:0/125: write d3/d12/d22/f24 [534583,72803] 0 2024-06-15T01:06:55.490 INFO:tasks.workunit.client.1.smithi142.stdout:0/126: chown c2 0 1 2024-06-15T01:06:55.491 INFO:tasks.workunit.client.1.smithi142.stdout:5/177: link d2/f12 d2/de/d27/f38 0 2024-06-15T01:06:55.491 INFO:tasks.workunit.client.0.smithi137.stdout:8/74: sync 2024-06-15T01:06:55.492 INFO:tasks.workunit.client.1.smithi142.stdout:3/225: dwrite d0/f2c [0,4194304] 0 2024-06-15T01:06:55.493 INFO:tasks.workunit.client.1.smithi142.stdout:2/170: rmdir d8/df/d10 39 2024-06-15T01:06:55.494 INFO:tasks.workunit.client.0.smithi137.stdout:5/303: readlink df/d11/l1a 0 2024-06-15T01:06:55.494 INFO:tasks.workunit.client.0.smithi137.stdout:5/304: chown fe 15928445 1 2024-06-15T01:06:55.495 INFO:tasks.workunit.client.0.smithi137.stdout:8/75: stat d9/de/f15 0 2024-06-15T01:06:55.495 INFO:tasks.workunit.client.1.smithi142.stdout:5/178: mknod d2/d18/c39 0 2024-06-15T01:06:55.496 INFO:tasks.workunit.client.1.smithi142.stdout:3/226: mknod d0/d3/da/c45 0 2024-06-15T01:06:55.497 INFO:tasks.workunit.client.1.smithi142.stdout:7/129: sync 2024-06-15T01:06:55.497 INFO:tasks.workunit.client.1.smithi142.stdout:6/146: sync 2024-06-15T01:06:55.497 INFO:tasks.workunit.client.1.smithi142.stdout:7/130: readlink l0 0 2024-06-15T01:06:55.499 INFO:tasks.workunit.client.1.smithi142.stdout:5/179: mknod d2/de/c3a 0 2024-06-15T01:06:55.501 INFO:tasks.workunit.client.1.smithi142.stdout:0/127: creat d3/f28 x:0 0 0 2024-06-15T01:06:55.504 INFO:tasks.workunit.client.1.smithi142.stdout:5/180: unlink d2/de/d27/c32 0 2024-06-15T01:06:55.505 INFO:tasks.workunit.client.1.smithi142.stdout:0/128: mknod d3/d5/c29 0 2024-06-15T01:06:55.507 INFO:tasks.workunit.client.1.smithi142.stdout:6/147: dwrite d1a/f1e [0,4194304] 0 2024-06-15T01:06:55.507 INFO:tasks.workunit.client.1.smithi142.stdout:5/181: mknod d2/d17/c3b 0 2024-06-15T01:06:55.508 INFO:tasks.workunit.client.1.smithi142.stdout:7/131: dwrite d2/f3 [4194304,4194304] 0 2024-06-15T01:06:55.508 INFO:tasks.workunit.client.1.smithi142.stdout:0/129: creat d3/f2a x:0 0 0 2024-06-15T01:06:55.509 INFO:tasks.workunit.client.1.smithi142.stdout:6/148: write f7 [244832,125705] 0 2024-06-15T01:06:55.509 INFO:tasks.workunit.client.1.smithi142.stdout:5/182: chown d2/f21 1 1 2024-06-15T01:06:55.513 INFO:tasks.workunit.client.1.smithi142.stdout:0/130: write d3/ff [787688,98553] 0 2024-06-15T01:06:55.513 INFO:tasks.workunit.client.1.smithi142.stdout:5/183: chown d2/d18/d2e/f34 426853 1 2024-06-15T01:06:55.515 INFO:tasks.workunit.client.1.smithi142.stdout:0/131: creat d3/d5/d1e/f2b x:0 0 0 2024-06-15T01:06:55.516 INFO:tasks.workunit.client.0.smithi137.stdout:8/76: mknod d9/de/c16 0 2024-06-15T01:06:55.516 INFO:tasks.workunit.client.1.smithi142.stdout:7/132: creat d2/f32 x:0 0 0 2024-06-15T01:06:55.518 INFO:tasks.workunit.client.1.smithi142.stdout:5/184: dread d2/ff [0,4194304] 0 2024-06-15T01:06:55.519 INFO:tasks.workunit.client.0.smithi137.stdout:5/305: dread df/d11/d14/f21 [4194304,4194304] 0 2024-06-15T01:06:55.520 INFO:tasks.workunit.client.0.smithi137.stdout:8/77: symlink d9/de/l17 0 2024-06-15T01:06:55.521 INFO:tasks.workunit.client.1.smithi142.stdout:5/185: rmdir d2/d17/d31 39 2024-06-15T01:06:55.523 INFO:tasks.workunit.client.1.smithi142.stdout:5/186: write d2/fc [1602965,88667] 0 2024-06-15T01:06:55.523 INFO:tasks.workunit.client.1.smithi142.stdout:0/132: dwrite d3/d12/d22/f24 [0,4194304] 0 2024-06-15T01:06:55.524 INFO:tasks.workunit.client.1.smithi142.stdout:6/149: dread d1a/d2d/f32 [0,4194304] 0 2024-06-15T01:06:55.525 INFO:tasks.workunit.client.1.smithi142.stdout:5/187: fdatasync d2/f25 0 2024-06-15T01:06:55.526 INFO:tasks.workunit.client.1.smithi142.stdout:5/188: truncate d2/f12 4399786 0 2024-06-15T01:06:55.527 INFO:tasks.workunit.client.1.smithi142.stdout:6/150: readlink l15 0 2024-06-15T01:06:55.528 INFO:tasks.workunit.client.0.smithi137.stdout:8/78: dwrite d9/de/f13 [0,4194304] 0 2024-06-15T01:06:55.529 INFO:tasks.workunit.client.0.smithi137.stdout:5/306: dwrite fc [0,4194304] 0 2024-06-15T01:06:55.530 INFO:tasks.workunit.client.1.smithi142.stdout:0/133: rmdir d3/d12/d22 39 2024-06-15T01:06:55.532 INFO:tasks.workunit.client.1.smithi142.stdout:0/134: chown d3/d5/d1e/f2b 923297057 1 2024-06-15T01:06:55.532 INFO:tasks.workunit.client.0.smithi137.stdout:5/307: fsync df/d11/d14/f21 0 2024-06-15T01:06:55.537 INFO:tasks.workunit.client.1.smithi142.stdout:6/151: dwrite f7 [0,4194304] 0 2024-06-15T01:06:55.537 INFO:tasks.workunit.client.1.smithi142.stdout:5/189: creat d2/d17/d31/f3c x:0 0 0 2024-06-15T01:06:55.538 INFO:tasks.workunit.client.0.smithi137.stdout:5/308: dread df/d3d/f42 [0,4194304] 0 2024-06-15T01:06:55.541 INFO:tasks.workunit.client.0.smithi137.stdout:5/309: write df/d3d/f42 [1391488,81513] 0 2024-06-15T01:06:55.544 INFO:tasks.workunit.client.1.smithi142.stdout:0/135: read d3/d12/d22/f24 [959585,16738] 0 2024-06-15T01:06:55.544 INFO:tasks.workunit.client.1.smithi142.stdout:6/152: write d1a/f1f [993049,109169] 0 2024-06-15T01:06:55.545 INFO:tasks.workunit.client.1.smithi142.stdout:6/153: dread - d1a/d2d/d2e/f31 zero size 2024-06-15T01:06:55.545 INFO:tasks.workunit.client.1.smithi142.stdout:5/190: chown d2/fc 1911498 1 2024-06-15T01:06:55.546 INFO:tasks.workunit.client.1.smithi142.stdout:0/136: rename d3/d5/l13 to d3/d12/d22/l2c 0 2024-06-15T01:06:55.549 INFO:tasks.workunit.client.1.smithi142.stdout:6/154: creat d1a/d2d/d2e/f37 x:0 0 0 2024-06-15T01:06:55.550 INFO:tasks.workunit.client.1.smithi142.stdout:7/133: dread d2/d6/f9 [0,4194304] 0 2024-06-15T01:06:55.551 INFO:tasks.workunit.client.1.smithi142.stdout:7/134: fdatasync d2/d6/f7 0 2024-06-15T01:06:55.551 INFO:tasks.workunit.client.1.smithi142.stdout:7/135: stat d2/d6/l21 0 2024-06-15T01:06:55.552 INFO:tasks.workunit.client.1.smithi142.stdout:7/136: fdatasync d2/fb 0 2024-06-15T01:06:55.553 INFO:tasks.workunit.client.0.smithi137.stdout:8/79: truncate f6 2148957 0 2024-06-15T01:06:55.553 INFO:tasks.workunit.client.1.smithi142.stdout:0/137: creat d3/d12/d22/f2d x:0 0 0 2024-06-15T01:06:55.553 INFO:tasks.workunit.client.0.smithi137.stdout:5/310: dwrite df/d11/d14/f2f [0,4194304] 0 2024-06-15T01:06:55.554 INFO:tasks.workunit.client.1.smithi142.stdout:6/155: creat d1a/f38 x:0 0 0 2024-06-15T01:06:55.555 INFO:tasks.workunit.client.0.smithi137.stdout:5/311: write fc [164436,48402] 0 2024-06-15T01:06:55.557 INFO:tasks.workunit.client.1.smithi142.stdout:0/138: dread d3/d12/f21 [0,4194304] 0 2024-06-15T01:06:55.557 INFO:tasks.workunit.client.1.smithi142.stdout:0/139: write d3/d5/f17 [1889493,108895] 0 2024-06-15T01:06:55.558 INFO:tasks.workunit.client.1.smithi142.stdout:0/140: write d3/d12/f14 [1749088,28380] 0 2024-06-15T01:06:55.559 INFO:tasks.workunit.client.0.smithi137.stdout:5/312: creat df/d11/f6d x:0 0 0 2024-06-15T01:06:55.562 INFO:tasks.workunit.client.0.smithi137.stdout:5/313: mkdir df/d11/d14/d3c/d6e 0 2024-06-15T01:06:55.563 INFO:tasks.workunit.client.0.smithi137.stdout:5/314: write fc [2528145,60207] 0 2024-06-15T01:06:55.565 INFO:tasks.workunit.client.0.smithi137.stdout:5/315: truncate df/d11/d34/d35/f55 701404 0 2024-06-15T01:06:55.566 INFO:tasks.workunit.client.1.smithi142.stdout:7/137: rename d2/d8/da/d18 to d2/d8/da/d33 0 2024-06-15T01:06:55.566 INFO:tasks.workunit.client.1.smithi142.stdout:7/138: readlink d2/l4 0 2024-06-15T01:06:55.567 INFO:tasks.workunit.client.1.smithi142.stdout:6/156: fdatasync d1a/f20 0 2024-06-15T01:06:55.567 INFO:tasks.workunit.client.0.smithi137.stdout:8/80: read f6 [1623979,96346] 0 2024-06-15T01:06:55.569 INFO:tasks.workunit.client.1.smithi142.stdout:5/191: rmdir d2/d17/d31 39 2024-06-15T01:06:55.571 INFO:tasks.workunit.client.0.smithi137.stdout:8/81: creat d9/de/f18 x:0 0 0 2024-06-15T01:06:55.573 INFO:tasks.workunit.client.0.smithi137.stdout:5/316: rmdir df/d11/d14/d3c/d6e 0 2024-06-15T01:06:55.574 INFO:tasks.workunit.client.0.smithi137.stdout:8/82: symlink d9/de/l19 0 2024-06-15T01:06:55.575 INFO:tasks.workunit.client.0.smithi137.stdout:8/83: write f1 [1497544,126879] 0 2024-06-15T01:06:55.577 INFO:tasks.workunit.client.0.smithi137.stdout:5/317: fsync df/d11/d14/f2f 0 2024-06-15T01:06:55.578 INFO:tasks.workunit.client.1.smithi142.stdout:6/157: mkdir d1a/d39 0 2024-06-15T01:06:55.582 INFO:tasks.workunit.client.1.smithi142.stdout:5/192: symlink d2/l3d 0 2024-06-15T01:06:55.582 INFO:tasks.workunit.client.1.smithi142.stdout:0/141: link d3/d5/c29 d3/d5/d1b/c2e 0 2024-06-15T01:06:55.583 INFO:tasks.workunit.client.1.smithi142.stdout:0/142: chown d3/la 16 1 2024-06-15T01:06:55.584 INFO:tasks.workunit.client.1.smithi142.stdout:6/158: mkdir d1a/d39/d3a 0 2024-06-15T01:06:55.586 INFO:tasks.workunit.client.1.smithi142.stdout:6/159: mkdir d1a/d2d/d3b 0 2024-06-15T01:06:55.588 INFO:tasks.workunit.client.1.smithi142.stdout:0/143: symlink d3/l2f 0 2024-06-15T01:06:55.590 INFO:tasks.workunit.client.1.smithi142.stdout:6/160: mkdir d1a/d3c 0 2024-06-15T01:06:55.591 INFO:tasks.workunit.client.1.smithi142.stdout:0/144: creat d3/d5/d1b/f30 x:0 0 0 2024-06-15T01:06:55.592 INFO:tasks.workunit.client.1.smithi142.stdout:0/145: stat d3/d5/d1b/l1f 0 2024-06-15T01:06:55.592 INFO:tasks.workunit.client.1.smithi142.stdout:6/161: symlink d1a/d3c/l3d 0 2024-06-15T01:06:55.592 INFO:tasks.workunit.client.1.smithi142.stdout:0/146: fsync d3/f28 0 2024-06-15T01:06:55.593 INFO:tasks.workunit.client.0.smithi137.stdout:5/318: dread fd [0,4194304] 0 2024-06-15T01:06:55.596 INFO:tasks.workunit.client.1.smithi142.stdout:6/162: fdatasync f18 0 2024-06-15T01:06:55.597 INFO:tasks.workunit.client.1.smithi142.stdout:7/139: dwrite d2/d8/f17 [0,4194304] 0 2024-06-15T01:06:55.597 INFO:tasks.workunit.client.1.smithi142.stdout:5/193: write d2/f12 [4943320,10335] 0 2024-06-15T01:06:55.598 INFO:tasks.workunit.client.0.smithi137.stdout:5/319: dread df/d11/d34/d3b/d4b/d4f/f67 [0,4194304] 0 2024-06-15T01:06:55.601 INFO:tasks.workunit.client.1.smithi142.stdout:6/163: creat d1a/f3e x:0 0 0 2024-06-15T01:06:55.602 INFO:tasks.workunit.client.1.smithi142.stdout:7/140: mkdir d2/d6/d22/d34 0 2024-06-15T01:06:55.604 INFO:tasks.workunit.client.1.smithi142.stdout:6/164: rmdir d1a/d3c 39 2024-06-15T01:06:55.604 INFO:tasks.workunit.client.0.smithi137.stdout:5/320: write df/f4a [764436,108806] 0 2024-06-15T01:06:55.606 INFO:tasks.workunit.client.1.smithi142.stdout:0/147: getdents d3 0 2024-06-15T01:06:55.606 INFO:tasks.workunit.client.1.smithi142.stdout:6/165: truncate d1a/f38 435350 0 2024-06-15T01:06:55.607 INFO:tasks.workunit.client.1.smithi142.stdout:5/194: dwrite d2/f12 [0,4194304] 0 2024-06-15T01:06:55.608 INFO:tasks.workunit.client.0.smithi137.stdout:5/321: creat df/d11/d34/f6f x:0 0 0 2024-06-15T01:06:55.608 INFO:tasks.workunit.client.1.smithi142.stdout:6/166: write f9 [383741,121790] 0 2024-06-15T01:06:55.608 INFO:tasks.workunit.client.1.smithi142.stdout:0/148: rmdir d3/d12 39 2024-06-15T01:06:55.609 INFO:tasks.workunit.client.1.smithi142.stdout:5/195: write d2/f21 [709857,98492] 0 2024-06-15T01:06:55.610 INFO:tasks.workunit.client.0.smithi137.stdout:5/322: rmdir df/d11/d34/d3b/d49/d5c 39 2024-06-15T01:06:55.614 INFO:tasks.workunit.client.1.smithi142.stdout:6/167: creat d1a/d39/f3f x:0 0 0 2024-06-15T01:06:55.614 INFO:tasks.workunit.client.0.smithi137.stdout:5/323: unlink df/d11/l23 0 2024-06-15T01:06:55.615 INFO:tasks.workunit.client.0.smithi137.stdout:5/324: fsync df/d44/f4d 0 2024-06-15T01:06:55.615 INFO:tasks.workunit.client.1.smithi142.stdout:7/141: dwrite d2/d8/da/d33/d25/f16 [0,4194304] 0 2024-06-15T01:06:55.616 INFO:tasks.workunit.client.0.smithi137.stdout:5/325: stat df/d11/d14/d3c 0 2024-06-15T01:06:55.616 INFO:tasks.workunit.client.1.smithi142.stdout:5/196: dread d2/f7 [0,4194304] 0 2024-06-15T01:06:55.619 INFO:tasks.workunit.client.0.smithi137.stdout:5/326: mkdir df/d11/d70 0 2024-06-15T01:06:55.623 INFO:tasks.workunit.client.1.smithi142.stdout:0/149: rename c2 to d3/d12/c31 0 2024-06-15T01:06:55.624 INFO:tasks.workunit.client.0.smithi137.stdout:5/327: truncate df/d11/f4c 2591621 0 2024-06-15T01:06:55.625 INFO:tasks.workunit.client.1.smithi142.stdout:7/142: dwrite d2/f3 [4194304,4194304] 0 2024-06-15T01:06:55.627 INFO:tasks.workunit.client.1.smithi142.stdout:0/150: dread - d3/d5/d1e/f2b zero size 2024-06-15T01:06:55.627 INFO:tasks.workunit.client.1.smithi142.stdout:5/197: dwrite d2/d18/f37 [0,4194304] 0 2024-06-15T01:06:55.627 INFO:tasks.workunit.client.1.smithi142.stdout:7/143: dread - d2/d6/d22/f28 zero size 2024-06-15T01:06:55.628 INFO:tasks.workunit.client.1.smithi142.stdout:8/143: sync 2024-06-15T01:06:55.629 INFO:tasks.workunit.client.1.smithi142.stdout:4/249: sync 2024-06-15T01:06:55.632 INFO:tasks.workunit.client.1.smithi142.stdout:8/144: dread - d0/d17/f30 zero size 2024-06-15T01:06:55.632 INFO:tasks.workunit.client.1.smithi142.stdout:0/151: unlink d3/d12/f21 0 2024-06-15T01:06:55.632 INFO:tasks.workunit.client.0.smithi137.stdout:5/328: dwrite df/d3d/f42 [0,4194304] 0 2024-06-15T01:06:55.633 INFO:tasks.workunit.client.1.smithi142.stdout:4/250: chown d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d 28000 1 2024-06-15T01:06:55.640 INFO:tasks.workunit.client.1.smithi142.stdout:5/198: dwrite d2/d18/d2e/f34 [0,4194304] 0 2024-06-15T01:06:55.642 INFO:tasks.workunit.client.1.smithi142.stdout:0/152: rename d3/d5/l11 to d3/d12/l32 0 2024-06-15T01:06:55.644 INFO:tasks.workunit.client.1.smithi142.stdout:5/199: write d2/fc [726452,106980] 0 2024-06-15T01:06:55.647 INFO:tasks.workunit.client.1.smithi142.stdout:0/153: creat d3/d5/f33 x:0 0 0 2024-06-15T01:06:55.647 INFO:tasks.workunit.client.1.smithi142.stdout:5/200: write d2/fc [619903,19778] 0 2024-06-15T01:06:55.648 INFO:tasks.workunit.client.1.smithi142.stdout:8/145: dwrite d0/d17/f30 [0,4194304] 0 2024-06-15T01:06:55.649 INFO:tasks.workunit.client.1.smithi142.stdout:0/154: dread - d3/d5/f1d zero size 2024-06-15T01:06:55.649 INFO:tasks.workunit.client.1.smithi142.stdout:8/146: fsync d0/d17/f27 0 2024-06-15T01:06:55.653 INFO:tasks.workunit.client.1.smithi142.stdout:8/147: mknod d0/c34 0 2024-06-15T01:06:55.656 INFO:tasks.workunit.client.1.smithi142.stdout:5/201: dread d2/f7 [0,4194304] 0 2024-06-15T01:06:55.659 INFO:tasks.workunit.client.1.smithi142.stdout:8/148: rename d0/d17/f30 to d0/d25/f35 0 2024-06-15T01:06:55.660 INFO:tasks.workunit.client.1.smithi142.stdout:8/149: write d0/d1/f29 [1061589,110688] 0 2024-06-15T01:06:55.660 INFO:tasks.workunit.client.1.smithi142.stdout:8/150: stat d0/f23 0 2024-06-15T01:06:55.666 INFO:tasks.workunit.client.0.smithi137.stdout:5/329: dwrite df/d11/d14/f43 [0,4194304] 0 2024-06-15T01:06:55.667 INFO:tasks.workunit.client.1.smithi142.stdout:7/144: truncate d2/d6/f7 4063387 0 2024-06-15T01:06:55.669 INFO:tasks.workunit.client.0.smithi137.stdout:5/330: mknod df/d11/d34/c71 0 2024-06-15T01:06:55.670 INFO:tasks.workunit.client.1.smithi142.stdout:5/202: rename d2/d17 to d2/d3e 0 2024-06-15T01:06:55.672 INFO:tasks.workunit.client.0.smithi137.stdout:5/331: symlink df/d11/d34/d3b/d49/d5c/l72 0 2024-06-15T01:06:55.672 INFO:tasks.workunit.client.1.smithi142.stdout:7/145: creat d2/d8/da/d33/f35 x:0 0 0 2024-06-15T01:06:55.674 INFO:tasks.workunit.client.1.smithi142.stdout:5/203: symlink d2/de/d27/l3f 0 2024-06-15T01:06:55.674 INFO:tasks.workunit.client.0.smithi137.stdout:5/332: mknod df/d11/d34/d3b/c73 0 2024-06-15T01:06:55.675 INFO:tasks.workunit.client.1.smithi142.stdout:7/146: chown d2/d8/dd/d14/c31 998250 1 2024-06-15T01:06:55.677 INFO:tasks.workunit.client.1.smithi142.stdout:5/204: symlink d2/de/d27/l40 0 2024-06-15T01:06:55.677 INFO:tasks.workunit.client.0.smithi137.stdout:5/333: mknod df/d44/c74 0 2024-06-15T01:06:55.679 INFO:tasks.workunit.client.1.smithi142.stdout:4/251: dread d5/d9/d23/f29 [0,4194304] 0 2024-06-15T01:06:55.679 INFO:tasks.workunit.client.1.smithi142.stdout:7/147: creat d2/d6/d22/d34/f36 x:0 0 0 2024-06-15T01:06:55.683 INFO:tasks.workunit.client.1.smithi142.stdout:5/205: symlink d2/d3e/l41 0 2024-06-15T01:06:55.683 INFO:tasks.workunit.client.1.smithi142.stdout:6/168: dread d1a/f24 [0,4194304] 0 2024-06-15T01:06:55.683 INFO:tasks.workunit.client.0.smithi137.stdout:5/334: dwrite f7 [0,4194304] 0 2024-06-15T01:06:55.684 INFO:tasks.workunit.client.1.smithi142.stdout:0/155: truncate d3/d12/d22/f24 3947224 0 2024-06-15T01:06:55.685 INFO:tasks.workunit.client.0.smithi137.stdout:5/335: read df/f2b [231276,51657] 0 2024-06-15T01:06:55.688 INFO:tasks.workunit.client.0.smithi137.stdout:5/336: unlink fc 0 2024-06-15T01:06:55.689 INFO:tasks.workunit.client.1.smithi142.stdout:5/206: unlink d2/de/d19/l23 0 2024-06-15T01:06:55.689 INFO:tasks.workunit.client.1.smithi142.stdout:6/169: dwrite f6 [0,4194304] 0 2024-06-15T01:06:55.690 INFO:tasks.workunit.client.1.smithi142.stdout:7/148: fsync d2/d6/f7 0 2024-06-15T01:06:55.691 INFO:tasks.workunit.client.1.smithi142.stdout:0/156: creat d3/d5/f34 x:0 0 0 2024-06-15T01:06:55.693 INFO:tasks.workunit.client.1.smithi142.stdout:4/252: dread d5/d9/d41/f36 [0,4194304] 0 2024-06-15T01:06:55.694 INFO:tasks.workunit.client.1.smithi142.stdout:8/151: truncate d0/f22 827703 0 2024-06-15T01:06:55.696 INFO:tasks.workunit.client.0.smithi137.stdout:5/337: dwrite df/d44/f4e [0,4194304] 0 2024-06-15T01:06:55.697 INFO:tasks.workunit.client.1.smithi142.stdout:9/137: sync 2024-06-15T01:06:55.697 INFO:tasks.workunit.client.1.smithi142.stdout:1/198: sync 2024-06-15T01:06:55.701 INFO:tasks.workunit.client.1.smithi142.stdout:5/207: dwrite d2/de/d19/f22 [0,4194304] 0 2024-06-15T01:06:55.703 INFO:tasks.workunit.client.1.smithi142.stdout:6/170: dread d1a/f24 [0,4194304] 0 2024-06-15T01:06:55.703 INFO:tasks.workunit.client.1.smithi142.stdout:7/149: mkdir d2/d8/dd/d14/d2c/d37 0 2024-06-15T01:06:55.704 INFO:tasks.workunit.client.1.smithi142.stdout:5/208: stat d2/l10 0 2024-06-15T01:06:55.705 INFO:tasks.workunit.client.1.smithi142.stdout:0/157: creat d3/d5/d1b/f35 x:0 0 0 2024-06-15T01:06:55.706 INFO:tasks.workunit.client.0.smithi137.stdout:5/338: dwrite df/d11/d14/f2f [0,4194304] 0 2024-06-15T01:06:55.711 INFO:tasks.workunit.client.1.smithi142.stdout:4/253: dwrite d5/d9/d23/f3b [0,4194304] 0 2024-06-15T01:06:55.713 INFO:tasks.workunit.client.1.smithi142.stdout:9/138: symlink d6/dc/l39 0 2024-06-15T01:06:55.716 INFO:tasks.workunit.client.0.smithi137.stdout:5/339: rmdir df 39 2024-06-15T01:06:55.718 INFO:tasks.workunit.client.1.smithi142.stdout:5/209: rename d2/de/d27/l40 to d2/d18/l42 0 2024-06-15T01:06:55.718 INFO:tasks.workunit.client.1.smithi142.stdout:9/139: dread d6/dc/f19 [0,4194304] 0 2024-06-15T01:06:55.720 INFO:tasks.workunit.client.0.smithi137.stdout:5/340: mknod df/d44/c75 0 2024-06-15T01:06:55.722 INFO:tasks.workunit.client.1.smithi142.stdout:0/158: creat d3/d12/d22/f36 x:0 0 0 2024-06-15T01:06:55.724 INFO:tasks.workunit.client.0.smithi137.stdout:5/341: truncate df/f2a 414249 0 2024-06-15T01:06:55.728 INFO:tasks.workunit.client.1.smithi142.stdout:9/140: dread d6/dc/dd/d22/f2b [0,4194304] 0 2024-06-15T01:06:55.729 INFO:tasks.workunit.client.0.smithi137.stdout:5/342: getdents df/d11/d34/d3b/d4b/d4f 0 2024-06-15T01:06:55.731 INFO:tasks.workunit.client.0.smithi137.stdout:5/343: mkdir df/d76 0 2024-06-15T01:06:55.732 INFO:tasks.workunit.client.1.smithi142.stdout:7/150: rename d2/fb to d2/d6/d22/d34/f38 0 2024-06-15T01:06:55.732 INFO:tasks.workunit.client.1.smithi142.stdout:1/199: write d6/f26 [1043005,65896] 0 2024-06-15T01:06:55.738 INFO:tasks.workunit.client.1.smithi142.stdout:7/151: symlink d2/d8/da/d33/d25/d27/l39 0 2024-06-15T01:06:55.739 INFO:tasks.workunit.client.1.smithi142.stdout:9/141: link d6/dc/f37 d6/d34/f3a 0 2024-06-15T01:06:55.740 INFO:tasks.workunit.client.1.smithi142.stdout:1/200: symlink d6/d12/l48 0 2024-06-15T01:06:55.740 INFO:tasks.workunit.client.1.smithi142.stdout:6/171: link d1a/d2d/d2e/l36 d1a/l40 0 2024-06-15T01:06:55.742 INFO:tasks.workunit.client.1.smithi142.stdout:0/159: creat d3/f37 x:0 0 0 2024-06-15T01:06:55.743 INFO:tasks.workunit.client.1.smithi142.stdout:9/142: mknod d6/d31/c3b 0 2024-06-15T01:06:55.744 INFO:tasks.workunit.client.1.smithi142.stdout:7/152: dwrite d2/d8/f2d [0,4194304] 0 2024-06-15T01:06:55.746 INFO:tasks.workunit.client.0.smithi137.stdout:5/344: dwrite df/d11/d14/f16 [0,4194304] 0 2024-06-15T01:06:55.751 INFO:tasks.workunit.client.0.smithi137.stdout:5/345: creat df/d11/d34/d3b/d49/f77 x:0 0 0 2024-06-15T01:06:55.751 INFO:tasks.workunit.client.0.smithi137.stdout:5/346: write f3 [205953,34134] 0 2024-06-15T01:06:55.752 INFO:tasks.workunit.client.1.smithi142.stdout:8/152: truncate d0/f22 951808 0 2024-06-15T01:06:55.752 INFO:tasks.workunit.client.1.smithi142.stdout:9/143: fdatasync d6/f9 0 2024-06-15T01:06:55.753 INFO:tasks.workunit.client.1.smithi142.stdout:8/153: chown d0/d1/f2c 4403 1 2024-06-15T01:06:55.754 INFO:tasks.workunit.client.0.smithi137.stdout:5/347: symlink df/d11/d34/d3b/d4b/d4f/l78 0 2024-06-15T01:06:55.755 INFO:tasks.workunit.client.1.smithi142.stdout:6/172: creat d1a/d2d/d3b/f41 x:0 0 0 2024-06-15T01:06:55.755 INFO:tasks.workunit.client.0.smithi137.stdout:5/348: stat df/d11/d34/d3b/d4b/d4f/f67 0 2024-06-15T01:06:55.756 INFO:tasks.workunit.client.1.smithi142.stdout:4/254: write d5/d1c/f34 [496593,15570] 0 2024-06-15T01:06:55.756 INFO:tasks.workunit.client.1.smithi142.stdout:0/160: dwrite d3/f37 [0,4194304] 0 2024-06-15T01:06:55.756 INFO:tasks.workunit.client.1.smithi142.stdout:7/153: creat d2/d8/da/d33/f3a x:0 0 0 2024-06-15T01:06:55.758 INFO:tasks.workunit.client.1.smithi142.stdout:8/154: mknod d0/d25/c36 0 2024-06-15T01:06:55.761 INFO:tasks.workunit.client.1.smithi142.stdout:5/210: dread d2/f21 [0,4194304] 0 2024-06-15T01:06:55.762 INFO:tasks.workunit.client.1.smithi142.stdout:8/155: creat d0/d1/f37 x:0 0 0 2024-06-15T01:06:55.762 INFO:tasks.workunit.client.1.smithi142.stdout:6/173: getdents d1a/d39/d3a 0 2024-06-15T01:06:55.764 INFO:tasks.workunit.client.1.smithi142.stdout:7/154: truncate d2/d8/da/d33/f20 413123 0 2024-06-15T01:06:55.764 INFO:tasks.workunit.client.1.smithi142.stdout:5/211: write d2/de/f1f [205797,111920] 0 2024-06-15T01:06:55.766 INFO:tasks.workunit.client.1.smithi142.stdout:4/255: rename d5/d9/d23/f29 to d5/d1c/d2a/d30/d46/d50/f5e 0 2024-06-15T01:06:55.768 INFO:tasks.workunit.client.1.smithi142.stdout:7/155: write d2/d8/da/d33/d25/d27/f2f [510923,109872] 0 2024-06-15T01:06:55.769 INFO:tasks.workunit.client.1.smithi142.stdout:8/156: mknod d0/d1/c38 0 2024-06-15T01:06:55.770 INFO:tasks.workunit.client.1.smithi142.stdout:0/161: dwrite d3/d5/d1b/f35 [0,4194304] 0 2024-06-15T01:06:55.771 INFO:tasks.workunit.client.0.smithi137.stdout:5/349: read f3 [2955854,113801] 0 2024-06-15T01:06:55.772 INFO:tasks.workunit.client.1.smithi142.stdout:4/256: write d5/d1c/d2a/f3d [545443,117601] 0 2024-06-15T01:06:55.772 INFO:tasks.workunit.client.1.smithi142.stdout:8/157: fdatasync d0/f15 0 2024-06-15T01:06:55.772 INFO:tasks.workunit.client.1.smithi142.stdout:5/212: rename d2/de/l13 to d2/d18/l43 0 2024-06-15T01:06:55.775 INFO:tasks.workunit.client.0.smithi137.stdout:5/350: getdents df/d44 0 2024-06-15T01:06:55.777 INFO:tasks.workunit.client.1.smithi142.stdout:0/162: write d3/d5/fb [3297401,10570] 0 2024-06-15T01:06:55.777 INFO:tasks.workunit.client.0.smithi137.stdout:5/351: rmdir df/d11/d34/d3b/d4b 39 2024-06-15T01:06:55.780 INFO:tasks.workunit.client.1.smithi142.stdout:5/213: fsync d2/d3e/d31/f3c 0 2024-06-15T01:06:55.780 INFO:tasks.workunit.client.0.smithi137.stdout:5/352: dread df/d11/f31 [0,4194304] 0 2024-06-15T01:06:55.780 INFO:tasks.workunit.client.0.smithi137.stdout:5/353: stat df/l1b 0 2024-06-15T01:06:55.781 INFO:tasks.workunit.client.1.smithi142.stdout:9/144: write f2 [1110508,9920] 0 2024-06-15T01:06:55.781 INFO:tasks.workunit.client.0.smithi137.stdout:5/354: fsync df/d11/d14/f2f 0 2024-06-15T01:06:55.783 INFO:tasks.workunit.client.1.smithi142.stdout:0/163: fdatasync d3/f9 0 2024-06-15T01:06:55.784 INFO:tasks.workunit.client.1.smithi142.stdout:7/156: dwrite d2/f32 [0,4194304] 0 2024-06-15T01:06:55.784 INFO:tasks.workunit.client.0.smithi137.stdout:5/355: unlink df/d11/l28 0 2024-06-15T01:06:55.785 INFO:tasks.workunit.client.1.smithi142.stdout:0/164: chown d3 0 1 2024-06-15T01:06:55.786 INFO:tasks.workunit.client.0.smithi137.stdout:5/356: dread df/f41 [0,4194304] 0 2024-06-15T01:06:55.786 INFO:tasks.workunit.client.1.smithi142.stdout:5/214: write d2/d3e/d31/f3c [1024678,77516] 0 2024-06-15T01:06:55.790 INFO:tasks.workunit.client.1.smithi142.stdout:9/145: rename d6/l28 to d6/d31/l3c 0 2024-06-15T01:06:55.791 INFO:tasks.workunit.client.0.smithi137.stdout:5/357: link df/d44/l47 df/d11/l79 0 2024-06-15T01:06:55.791 INFO:tasks.workunit.client.1.smithi142.stdout:5/215: fsync d2/de/d19/f22 0 2024-06-15T01:06:55.793 INFO:tasks.workunit.client.0.smithi137.stdout:5/358: mkdir df/d7a 0 2024-06-15T01:06:55.794 INFO:tasks.workunit.client.1.smithi142.stdout:4/257: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f43 [0,4194304] 0 2024-06-15T01:06:55.795 INFO:tasks.workunit.client.1.smithi142.stdout:5/216: write d2/d18/d2e/f34 [2627762,5322] 0 2024-06-15T01:06:55.796 INFO:tasks.workunit.client.1.smithi142.stdout:7/157: dread d2/d8/f2d [0,4194304] 0 2024-06-15T01:06:55.797 INFO:tasks.workunit.client.0.smithi137.stdout:4/114: sync 2024-06-15T01:06:55.798 INFO:tasks.workunit.client.0.smithi137.stdout:1/196: sync 2024-06-15T01:06:55.799 INFO:tasks.workunit.client.1.smithi142.stdout:6/174: dwrite d1a/f25 [0,4194304] 0 2024-06-15T01:06:55.800 INFO:tasks.workunit.client.1.smithi142.stdout:5/217: stat d2/d18/l42 0 2024-06-15T01:06:55.801 INFO:tasks.workunit.client.0.smithi137.stdout:4/115: symlink dd/d10/d1f/l2c 0 2024-06-15T01:06:55.801 INFO:tasks.workunit.client.1.smithi142.stdout:6/175: chown f7 882 1 2024-06-15T01:06:55.803 INFO:tasks.workunit.client.0.smithi137.stdout:1/197: creat d0/d3/d1b/f4f x:0 0 0 2024-06-15T01:06:55.803 INFO:tasks.workunit.client.1.smithi142.stdout:8/158: write d0/f22 [427952,60142] 0 2024-06-15T01:06:55.803 INFO:tasks.workunit.client.1.smithi142.stdout:9/146: write d6/dc/dd/f1d [1201845,10471] 0 2024-06-15T01:06:55.804 INFO:tasks.workunit.client.0.smithi137.stdout:4/116: mknod dd/d15/c2d 0 2024-06-15T01:06:55.804 INFO:tasks.workunit.client.1.smithi142.stdout:5/218: chown d2/de/d19/f36 7541 1 2024-06-15T01:06:55.805 INFO:tasks.workunit.client.0.smithi137.stdout:5/359: getdents df/d11/d34/d3b/d49 0 2024-06-15T01:06:55.805 INFO:tasks.workunit.client.1.smithi142.stdout:6/176: mkdir d1a/d39/d42 0 2024-06-15T01:06:55.806 INFO:tasks.workunit.client.1.smithi142.stdout:2/171: sync 2024-06-15T01:06:55.806 INFO:tasks.workunit.client.1.smithi142.stdout:3/227: sync 2024-06-15T01:06:55.807 INFO:tasks.workunit.client.1.smithi142.stdout:5/219: fsync d2/de/d27/f38 0 2024-06-15T01:06:55.808 INFO:tasks.workunit.client.0.smithi137.stdout:1/198: rmdir d0/d1f 39 2024-06-15T01:06:55.809 INFO:tasks.workunit.client.1.smithi142.stdout:6/177: dread - fc zero size 2024-06-15T01:06:55.809 INFO:tasks.workunit.client.1.smithi142.stdout:5/220: readlink d2/l8 0 2024-06-15T01:06:55.809 INFO:tasks.workunit.client.1.smithi142.stdout:4/258: dwrite d5/d9/d23/f4e [0,4194304] 0 2024-06-15T01:06:55.810 INFO:tasks.workunit.client.1.smithi142.stdout:2/172: unlink c2 0 2024-06-15T01:06:55.813 INFO:tasks.workunit.client.0.smithi137.stdout:5/360: chown df/d11/d14/c6a 0 1 2024-06-15T01:06:55.814 INFO:tasks.workunit.client.1.smithi142.stdout:2/173: read f6 [265596,27615] 0 2024-06-15T01:06:55.814 INFO:tasks.workunit.client.1.smithi142.stdout:5/221: mknod d2/d3e/d31/c44 0 2024-06-15T01:06:55.815 INFO:tasks.workunit.client.0.smithi137.stdout:1/199: creat d0/d3/d1b/f50 x:0 0 0 2024-06-15T01:06:55.815 INFO:tasks.workunit.client.1.smithi142.stdout:9/147: getdents d6/dc/d1b/d25/d35 0 2024-06-15T01:06:55.815 INFO:tasks.workunit.client.1.smithi142.stdout:9/148: stat d6/dc/f15 0 2024-06-15T01:06:55.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:55 smithi142 ceph-mon[37379]: pgmap v13: 65 pgs: 65 active+clean; 351 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 208 KiB/s rd, 20 MiB/s wr, 233 op/s 2024-06-15T01:06:55.818 INFO:tasks.workunit.client.1.smithi142.stdout:6/178: mknod d1a/d39/d3a/c43 0 2024-06-15T01:06:55.818 INFO:tasks.workunit.client.0.smithi137.stdout:5/361: creat df/d11/d30/f7b x:0 0 0 2024-06-15T01:06:55.819 INFO:tasks.workunit.client.1.smithi142.stdout:3/228: rename d0/d3/da/f42 to d0/d1f/f46 0 2024-06-15T01:06:55.820 INFO:tasks.workunit.client.1.smithi142.stdout:5/222: creat d2/d18/f45 x:0 0 0 2024-06-15T01:06:55.821 INFO:tasks.workunit.client.1.smithi142.stdout:5/223: write d2/de/d19/f1a [5348116,40560] 0 2024-06-15T01:06:55.822 INFO:tasks.workunit.client.1.smithi142.stdout:4/259: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/c5f 0 2024-06-15T01:06:55.824 INFO:tasks.workunit.client.0.smithi137.stdout:1/200: mknod d0/d1f/c51 0 2024-06-15T01:06:55.824 INFO:tasks.workunit.client.0.smithi137.stdout:0/113: sync 2024-06-15T01:06:55.825 INFO:tasks.workunit.client.0.smithi137.stdout:7/142: sync 2024-06-15T01:06:55.825 INFO:tasks.workunit.client.0.smithi137.stdout:6/114: sync 2024-06-15T01:06:55.825 INFO:tasks.workunit.client.1.smithi142.stdout:6/179: rename d1a/d2d/d2e/f37 to d1a/d39/d42/f44 0 2024-06-15T01:06:55.826 INFO:tasks.workunit.client.1.smithi142.stdout:3/229: write d0/d1f/f46 [160015,75758] 0 2024-06-15T01:06:55.827 INFO:tasks.workunit.client.0.smithi137.stdout:0/114: write d1/d2/dc/f16 [4743486,125753] 0 2024-06-15T01:06:55.828 INFO:tasks.workunit.client.0.smithi137.stdout:1/201: write d0/d3/d14/d34/f45 [595682,65865] 0 2024-06-15T01:06:55.828 INFO:tasks.workunit.client.0.smithi137.stdout:7/143: rmdir df/d18 39 2024-06-15T01:06:55.830 INFO:tasks.workunit.client.0.smithi137.stdout:1/202: chown d0/d3/c24 1 1 2024-06-15T01:06:55.830 INFO:tasks.workunit.client.1.smithi142.stdout:7/158: write d2/f1a [1611958,27116] 0 2024-06-15T01:06:55.831 INFO:tasks.workunit.client.1.smithi142.stdout:7/159: dread - d2/d6/d22/d34/f36 zero size 2024-06-15T01:06:55.833 INFO:tasks.workunit.client.0.smithi137.stdout:6/115: rename d0/d5/d9/l20 to d0/d1a/l21 0 2024-06-15T01:06:55.834 INFO:tasks.workunit.client.1.smithi142.stdout:7/160: write d2/d6/f9 [2588182,103890] 0 2024-06-15T01:06:55.834 INFO:tasks.workunit.client.1.smithi142.stdout:4/260: fdatasync d5/d9/d23/f4e 0 2024-06-15T01:06:55.836 INFO:tasks.workunit.client.1.smithi142.stdout:6/180: unlink d1a/d39/d3a/c43 0 2024-06-15T01:06:55.837 INFO:tasks.workunit.client.0.smithi137.stdout:4/117: dread dd/f13 [0,4194304] 0 2024-06-15T01:06:55.838 INFO:tasks.workunit.client.1.smithi142.stdout:3/230: mknod d0/d3/c47 0 2024-06-15T01:06:55.839 INFO:tasks.workunit.client.1.smithi142.stdout:3/231: fdatasync d0/f39 0 2024-06-15T01:06:55.840 INFO:tasks.workunit.client.1.smithi142.stdout:2/174: creat d8/df/d10/d1c/f2c x:0 0 0 2024-06-15T01:06:55.841 INFO:tasks.workunit.client.1.smithi142.stdout:8/159: dwrite d0/f14 [0,4194304] 0 2024-06-15T01:06:55.842 INFO:tasks.workunit.client.0.smithi137.stdout:7/144: mknod df/d18/d1c/d1d/d27/c2a 0 2024-06-15T01:06:55.842 INFO:tasks.workunit.client.1.smithi142.stdout:2/175: write d8/fa [1049456,65979] 0 2024-06-15T01:06:55.843 INFO:tasks.workunit.client.0.smithi137.stdout:0/115: dwrite d1/d2/dc/d10/d14/f1b [0,4194304] 0 2024-06-15T01:06:55.843 INFO:tasks.workunit.client.1.smithi142.stdout:4/261: dread d5/d9/d41/f4c [8388608,4194304] 0 2024-06-15T01:06:55.844 INFO:tasks.workunit.client.0.smithi137.stdout:4/118: creat dd/d15/f2e x:0 0 0 2024-06-15T01:06:55.846 INFO:tasks.workunit.client.1.smithi142.stdout:2/176: chown d8/df/d1f/c29 99659 1 2024-06-15T01:06:55.849 INFO:tasks.workunit.client.0.smithi137.stdout:7/145: chown c8 390 1 2024-06-15T01:06:55.849 INFO:tasks.workunit.client.1.smithi142.stdout:7/161: symlink d2/d6/l3b 0 2024-06-15T01:06:55.850 INFO:tasks.workunit.client.0.smithi137.stdout:0/116: fdatasync d1/d2/dc/d10/d14/d15/f1e 0 2024-06-15T01:06:55.850 INFO:tasks.workunit.client.0.smithi137.stdout:5/362: dwrite df/d11/f13 [0,4194304] 0 2024-06-15T01:06:55.850 INFO:tasks.workunit.client.0.smithi137.stdout:6/116: dwrite d0/d5/d9/f1b [0,4194304] 0 2024-06-15T01:06:55.851 INFO:tasks.workunit.client.1.smithi142.stdout:3/232: dwrite d0/d3/d15/f2d [0,4194304] 0 2024-06-15T01:06:55.851 INFO:tasks.workunit.client.1.smithi142.stdout:6/181: unlink d1a/c21 0 2024-06-15T01:06:55.853 INFO:tasks.workunit.client.0.smithi137.stdout:6/117: chown d0/fc 549216207 1 2024-06-15T01:06:55.855 INFO:tasks.workunit.client.1.smithi142.stdout:9/149: getdents d6/d31 0 2024-06-15T01:06:55.856 INFO:tasks.workunit.client.0.smithi137.stdout:0/117: symlink d1/d2/dc/d10/d14/d15/l24 0 2024-06-15T01:06:55.861 INFO:tasks.workunit.client.1.smithi142.stdout:3/233: dwrite d0/d1b/f2a [0,4194304] 0 2024-06-15T01:06:55.862 INFO:tasks.workunit.client.1.smithi142.stdout:5/224: truncate d2/de/d19/f1a 2802521 0 2024-06-15T01:06:55.863 INFO:tasks.workunit.client.1.smithi142.stdout:9/150: dread d6/dc/f1c [0,4194304] 0 2024-06-15T01:06:55.865 INFO:tasks.workunit.client.0.smithi137.stdout:6/118: mknod d0/d1c/c22 0 2024-06-15T01:06:55.866 INFO:tasks.workunit.client.0.smithi137.stdout:4/119: link dd/l1e dd/d10/l2f 0 2024-06-15T01:06:55.866 INFO:tasks.workunit.client.1.smithi142.stdout:7/162: rmdir d2/d8/da/d33 39 2024-06-15T01:06:55.868 INFO:tasks.workunit.client.1.smithi142.stdout:6/182: creat d1a/d39/d42/f45 x:0 0 0 2024-06-15T01:06:55.869 INFO:tasks.workunit.client.0.smithi137.stdout:0/118: creat d1/d2/dc/d10/f25 x:0 0 0 2024-06-15T01:06:55.870 INFO:tasks.workunit.client.0.smithi137.stdout:6/119: mknod d0/d5/c23 0 2024-06-15T01:06:55.871 INFO:tasks.workunit.client.0.smithi137.stdout:2/74: sync 2024-06-15T01:06:55.871 INFO:tasks.workunit.client.0.smithi137.stdout:2/75: readlink - no filename 2024-06-15T01:06:55.871 INFO:tasks.workunit.client.1.smithi142.stdout:8/160: fdatasync d0/f14 0 2024-06-15T01:06:55.871 INFO:tasks.workunit.client.0.smithi137.stdout:3/169: sync 2024-06-15T01:06:55.871 INFO:tasks.workunit.client.0.smithi137.stdout:0/119: stat d1/d2/l3 0 2024-06-15T01:06:55.872 INFO:tasks.workunit.client.1.smithi142.stdout:5/225: creat d2/de/d27/f46 x:0 0 0 2024-06-15T01:06:55.874 INFO:tasks.workunit.client.0.smithi137.stdout:4/120: mkdir dd/d15/d30 0 2024-06-15T01:06:55.874 INFO:tasks.workunit.client.0.smithi137.stdout:5/363: dwrite df/d11/d30/f7b [0,4194304] 0 2024-06-15T01:06:55.874 INFO:tasks.workunit.client.1.smithi142.stdout:9/151: truncate d6/f9 312748 0 2024-06-15T01:06:55.875 INFO:tasks.workunit.client.1.smithi142.stdout:7/163: mkdir d2/d6/d22/d3c 0 2024-06-15T01:06:55.880 INFO:tasks.workunit.client.0.smithi137.stdout:3/170: mkdir d0/d13/d37 0 2024-06-15T01:06:55.881 INFO:tasks.workunit.client.0.smithi137.stdout:4/121: dread - dd/d10/d1f/f26 zero size 2024-06-15T01:06:55.882 INFO:tasks.workunit.client.0.smithi137.stdout:2/76: creat d0/da/f18 x:0 0 0 2024-06-15T01:06:55.882 INFO:tasks.workunit.client.0.smithi137.stdout:8/84: sync 2024-06-15T01:06:55.886 INFO:tasks.workunit.client.0.smithi137.stdout:9/74: sync 2024-06-15T01:06:55.888 INFO:tasks.workunit.client.0.smithi137.stdout:3/171: symlink d0/d2/l38 0 2024-06-15T01:06:55.888 INFO:tasks.workunit.client.0.smithi137.stdout:2/77: dread - d0/da/f11 zero size 2024-06-15T01:06:55.889 INFO:tasks.workunit.client.0.smithi137.stdout:8/85: mknod d9/c1a 0 2024-06-15T01:06:55.890 INFO:tasks.workunit.client.0.smithi137.stdout:4/122: symlink dd/d15/d30/l31 0 2024-06-15T01:06:55.891 INFO:tasks.workunit.client.0.smithi137.stdout:2/78: fdatasync d0/da/f17 0 2024-06-15T01:06:55.891 INFO:tasks.workunit.client.0.smithi137.stdout:9/75: mkdir df/d13 0 2024-06-15T01:06:55.894 INFO:tasks.workunit.client.0.smithi137.stdout:8/86: rmdir d9/de 39 2024-06-15T01:06:55.894 INFO:tasks.workunit.client.0.smithi137.stdout:0/120: dwrite d1/d2/dc/d10/d14/f1b [0,4194304] 0 2024-06-15T01:06:55.896 INFO:tasks.workunit.client.0.smithi137.stdout:6/120: dwrite d0/d1c/f1e [0,4194304] 0 2024-06-15T01:06:55.897 INFO:tasks.workunit.client.0.smithi137.stdout:5/364: getdents df/d11/d34/d3b/d4b 0 2024-06-15T01:06:55.898 INFO:tasks.workunit.client.0.smithi137.stdout:9/76: chown ce 10 1 2024-06-15T01:06:55.900 INFO:tasks.workunit.client.0.smithi137.stdout:3/172: dread d0/d13/f20 [0,4194304] 0 2024-06-15T01:06:55.901 INFO:tasks.workunit.client.0.smithi137.stdout:4/123: creat dd/d15/d21/f32 x:0 0 0 2024-06-15T01:06:55.903 INFO:tasks.workunit.client.0.smithi137.stdout:4/124: readlink dd/lf 0 2024-06-15T01:06:55.905 INFO:tasks.workunit.client.1.smithi142.stdout:7/164: mknod d2/d6/d22/d34/c3d 0 2024-06-15T01:06:55.907 INFO:tasks.workunit.client.1.smithi142.stdout:9/152: getdents d6/dc/d1b/d25/d35 0 2024-06-15T01:06:55.907 INFO:tasks.workunit.client.1.smithi142.stdout:6/183: dwrite d1a/f22 [0,4194304] 0 2024-06-15T01:06:55.908 INFO:tasks.workunit.client.1.smithi142.stdout:4/262: dread d5/d1c/d2a/d30/d46/f52 [0,4194304] 0 2024-06-15T01:06:55.909 INFO:tasks.workunit.client.1.smithi142.stdout:8/161: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:06:55.913 INFO:tasks.workunit.client.0.smithi137.stdout:6/121: dwrite d0/fc [0,4194304] 0 2024-06-15T01:06:55.913 INFO:tasks.workunit.client.0.smithi137.stdout:5/365: chown df/d11/d34/l39 7 1 2024-06-15T01:06:55.916 INFO:tasks.workunit.client.0.smithi137.stdout:0/121: dread d1/d2/dc/d10/d14/d15/f19 [0,4194304] 0 2024-06-15T01:06:55.917 INFO:tasks.workunit.client.0.smithi137.stdout:0/122: chown d1/d2/lb 9896 1 2024-06-15T01:06:55.919 INFO:tasks.workunit.client.0.smithi137.stdout:4/125: rename c0 to dd/c33 0 2024-06-15T01:06:55.921 INFO:tasks.workunit.client.1.smithi142.stdout:6/184: dwrite f8 [0,4194304] 0 2024-06-15T01:06:55.921 INFO:tasks.workunit.client.1.smithi142.stdout:5/226: rename d2/d3e/d31/c33 to d2/d3e/c47 0 2024-06-15T01:06:55.923 INFO:tasks.workunit.client.0.smithi137.stdout:9/77: dwrite fd [0,4194304] 0 2024-06-15T01:06:55.923 INFO:tasks.workunit.client.0.smithi137.stdout:3/173: mkdir d0/d2/d22/d24/d28/d2e/d39 0 2024-06-15T01:06:55.924 INFO:tasks.workunit.client.1.smithi142.stdout:3/234: truncate d0/f39 1563279 0 2024-06-15T01:06:55.932 INFO:tasks.workunit.client.0.smithi137.stdout:8/87: creat d9/f1b x:0 0 0 2024-06-15T01:06:55.936 INFO:tasks.workunit.client.0.smithi137.stdout:5/366: dwrite df/d11/f13 [0,4194304] 0 2024-06-15T01:06:55.939 INFO:tasks.workunit.client.1.smithi142.stdout:6/185: unlink d1a/f30 0 2024-06-15T01:06:55.939 INFO:tasks.workunit.client.1.smithi142.stdout:4/263: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/f60 x:0 0 0 2024-06-15T01:06:55.940 INFO:tasks.workunit.client.0.smithi137.stdout:2/79: dwrite d0/f6 [0,4194304] 0 2024-06-15T01:06:55.941 INFO:tasks.workunit.client.1.smithi142.stdout:4/264: write d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/f60 [36917,28238] 0 2024-06-15T01:06:55.942 INFO:tasks.workunit.client.1.smithi142.stdout:4/265: write d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f37 [1227645,7946] 0 2024-06-15T01:06:55.943 INFO:tasks.workunit.client.1.smithi142.stdout:4/266: stat d5/d9/da/l31 0 2024-06-15T01:06:55.945 INFO:tasks.workunit.client.0.smithi137.stdout:6/122: unlink d0/d1c/c22 0 2024-06-15T01:06:55.946 INFO:tasks.workunit.client.1.smithi142.stdout:9/153: mknod d6/dc/dd/d23/c3d 0 2024-06-15T01:06:55.946 INFO:tasks.workunit.client.1.smithi142.stdout:9/154: stat d6/f21 0 2024-06-15T01:06:55.947 INFO:tasks.workunit.client.0.smithi137.stdout:8/88: write d9/de/f13 [5137322,126213] 0 2024-06-15T01:06:55.948 INFO:tasks.workunit.client.1.smithi142.stdout:7/165: symlink d2/d8/da/d33/d25/l3e 0 2024-06-15T01:06:55.949 INFO:tasks.workunit.client.1.smithi142.stdout:7/166: write d2/d8/da/d33/f35 [95913,59517] 0 2024-06-15T01:06:55.950 INFO:tasks.workunit.client.1.smithi142.stdout:7/167: readlink l0 0 2024-06-15T01:06:55.950 INFO:tasks.workunit.client.0.smithi137.stdout:5/367: dread df/d11/f31 [0,4194304] 0 2024-06-15T01:06:55.952 INFO:tasks.workunit.client.1.smithi142.stdout:4/267: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f61 x:0 0 0 2024-06-15T01:06:55.952 INFO:tasks.workunit.client.0.smithi137.stdout:8/89: truncate d9/de/f18 407939 0 2024-06-15T01:06:55.953 INFO:tasks.workunit.client.0.smithi137.stdout:5/368: write df/d11/d14/f43 [1138940,54301] 0 2024-06-15T01:06:55.954 INFO:tasks.workunit.client.1.smithi142.stdout:7/168: dread d2/f32 [0,4194304] 0 2024-06-15T01:06:55.955 INFO:tasks.workunit.client.0.smithi137.stdout:3/174: symlink d0/d2/d23/l3a 0 2024-06-15T01:06:55.958 INFO:tasks.workunit.client.0.smithi137.stdout:3/175: fsync d0/d2/d1e/f2a 0 2024-06-15T01:06:55.959 INFO:tasks.workunit.client.0.smithi137.stdout:8/90: truncate d9/de/ff 646353 0 2024-06-15T01:06:55.960 INFO:tasks.workunit.client.1.smithi142.stdout:9/155: creat d6/d31/f3e x:0 0 0 2024-06-15T01:06:55.961 INFO:tasks.workunit.client.0.smithi137.stdout:2/80: creat d0/de/d10/f19 x:0 0 0 2024-06-15T01:06:55.961 INFO:tasks.workunit.client.0.smithi137.stdout:8/91: fdatasync d9/f1b 0 2024-06-15T01:06:55.962 INFO:tasks.workunit.client.1.smithi142.stdout:2/177: dread d8/fc [0,4194304] 0 2024-06-15T01:06:55.963 INFO:tasks.workunit.client.0.smithi137.stdout:2/81: fsync d0/de/f16 0 2024-06-15T01:06:55.963 INFO:tasks.workunit.client.0.smithi137.stdout:2/82: chown d0/dc 2706062 1 2024-06-15T01:06:55.963 INFO:tasks.workunit.client.1.smithi142.stdout:2/178: chown d8/d19/d23/f26 0 1 2024-06-15T01:06:55.964 INFO:tasks.workunit.client.0.smithi137.stdout:2/83: truncate d0/f9 994775 0 2024-06-15T01:06:55.966 INFO:tasks.workunit.client.0.smithi137.stdout:6/123: creat d0/d1c/f24 x:0 0 0 2024-06-15T01:06:55.970 INFO:tasks.workunit.client.1.smithi142.stdout:6/186: dwrite d1a/f2c [0,4194304] 0 2024-06-15T01:06:55.970 INFO:tasks.workunit.client.0.smithi137.stdout:0/123: symlink d1/d2/dc/d10/d14/d15/d17/l26 0 2024-06-15T01:06:55.972 INFO:tasks.workunit.client.0.smithi137.stdout:2/84: dwrite d0/da/f17 [0,4194304] 0 2024-06-15T01:06:55.972 INFO:tasks.workunit.client.0.smithi137.stdout:5/369: chown df/d11/d14/l1c 39521511 1 2024-06-15T01:06:55.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/179: dwrite d8/d19/d23/d25/f24 [0,4194304] 0 2024-06-15T01:06:55.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/180: write d8/f16 [55947,69378] 0 2024-06-15T01:06:55.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:55 smithi137 ceph-mon[29354]: pgmap v13: 65 pgs: 65 active+clean; 351 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 208 KiB/s rd, 20 MiB/s wr, 233 op/s 2024-06-15T01:06:55.976 INFO:tasks.workunit.client.0.smithi137.stdout:3/176: fdatasync d0/d13/f20 0 2024-06-15T01:06:55.978 INFO:tasks.workunit.client.0.smithi137.stdout:3/177: truncate d0/d2f/f35 689513 0 2024-06-15T01:06:55.983 INFO:tasks.workunit.client.1.smithi142.stdout:3/235: getdents d0/d1f 0 2024-06-15T01:06:55.984 INFO:tasks.workunit.client.0.smithi137.stdout:6/124: mknod d0/d5/d9/c25 0 2024-06-15T01:06:55.989 INFO:tasks.workunit.client.1.smithi142.stdout:7/169: dread d2/d8/dd/f10 [0,4194304] 0 2024-06-15T01:06:55.990 INFO:tasks.workunit.client.0.smithi137.stdout:0/124: symlink d1/d2/dc/d10/d14/l27 0 2024-06-15T01:06:55.990 INFO:tasks.workunit.client.0.smithi137.stdout:6/125: dread d0/d5/f15 [0,4194304] 0 2024-06-15T01:06:55.993 INFO:tasks.workunit.client.0.smithi137.stdout:3/178: truncate d0/d2/f1b 230320 0 2024-06-15T01:06:55.993 INFO:tasks.workunit.client.0.smithi137.stdout:8/92: mknod d9/c1c 0 2024-06-15T01:06:55.993 INFO:tasks.workunit.client.0.smithi137.stdout:3/179: chown d0/d2/d8 850 1 2024-06-15T01:06:55.995 INFO:tasks.workunit.client.0.smithi137.stdout:3/180: dread - d0/d2/d8/f14 zero size 2024-06-15T01:06:56.000 INFO:tasks.workunit.client.0.smithi137.stdout:9/78: dread fd [4194304,4194304] 0 2024-06-15T01:06:56.002 INFO:tasks.workunit.client.0.smithi137.stdout:4/126: dread dd/f12 [0,4194304] 0 2024-06-15T01:06:56.002 INFO:tasks.workunit.client.0.smithi137.stdout:8/93: dwrite d9/fc [0,4194304] 0 2024-06-15T01:06:56.003 INFO:tasks.workunit.client.1.smithi142.stdout:3/236: mkdir d0/d1f/d48 0 2024-06-15T01:06:56.003 INFO:tasks.workunit.client.1.smithi142.stdout:0/165: sync 2024-06-15T01:06:56.003 INFO:tasks.workunit.client.1.smithi142.stdout:1/201: sync 2024-06-15T01:06:56.003 INFO:tasks.workunit.client.0.smithi137.stdout:4/127: chown dd/c23 155 1 2024-06-15T01:06:56.004 INFO:tasks.workunit.client.1.smithi142.stdout:3/237: write d0/d3/d15/f2d [3006589,43271] 0 2024-06-15T01:06:56.004 INFO:tasks.workunit.client.1.smithi142.stdout:0/166: dread - d3/d5/f34 zero size 2024-06-15T01:06:56.004 INFO:tasks.workunit.client.1.smithi142.stdout:7/170: stat d2/d8/da/d33/d25/c1f 0 2024-06-15T01:06:56.005 INFO:tasks.workunit.client.0.smithi137.stdout:6/126: rename d0/d5/f18 to d0/d1a/f26 0 2024-06-15T01:06:56.005 INFO:tasks.workunit.client.0.smithi137.stdout:6/127: dread - d0/d1c/f24 zero size 2024-06-15T01:06:56.008 INFO:tasks.workunit.client.1.smithi142.stdout:8/162: write d0/f5 [4119313,49237] 0 2024-06-15T01:06:56.009 INFO:tasks.workunit.client.1.smithi142.stdout:8/163: write d0/d17/f27 [174729,129585] 0 2024-06-15T01:06:56.010 INFO:tasks.workunit.client.1.smithi142.stdout:8/164: chown d0/d25/f35 40359 1 2024-06-15T01:06:56.012 INFO:tasks.workunit.client.0.smithi137.stdout:2/85: fsync d0/da/f17 0 2024-06-15T01:06:56.012 INFO:tasks.workunit.client.0.smithi137.stdout:2/86: readlink - no filename 2024-06-15T01:06:56.014 INFO:tasks.workunit.client.0.smithi137.stdout:5/370: write df/f2a [910198,76744] 0 2024-06-15T01:06:56.014 INFO:tasks.workunit.client.0.smithi137.stdout:0/125: symlink d1/d2/l28 0 2024-06-15T01:06:56.016 INFO:tasks.workunit.client.0.smithi137.stdout:5/371: stat df/d11/d30 0 2024-06-15T01:06:56.017 INFO:tasks.workunit.client.0.smithi137.stdout:9/79: symlink df/l14 0 2024-06-15T01:06:56.021 INFO:tasks.workunit.client.1.smithi142.stdout:5/227: dwrite d2/f14 [0,4194304] 0 2024-06-15T01:06:56.022 INFO:tasks.workunit.client.1.smithi142.stdout:5/228: chown d2/d18/d2e/c30 13754640 1 2024-06-15T01:06:56.024 INFO:tasks.workunit.client.1.smithi142.stdout:4/268: fsync d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f61 0 2024-06-15T01:06:56.026 INFO:tasks.workunit.client.0.smithi137.stdout:0/126: dwrite d1/d2/dc/d10/d14/d15/d1c/f1d [0,4194304] 0 2024-06-15T01:06:56.028 INFO:tasks.workunit.client.1.smithi142.stdout:4/269: dread d5/d9/f1f [0,4194304] 0 2024-06-15T01:06:56.028 INFO:tasks.workunit.client.1.smithi142.stdout:9/156: truncate d6/dc/dd/f1d 109009 0 2024-06-15T01:06:56.028 INFO:tasks.workunit.client.0.smithi137.stdout:2/87: chown d0/c7 253821 1 2024-06-15T01:06:56.028 INFO:tasks.workunit.client.0.smithi137.stdout:9/80: write f5 [4858043,79250] 0 2024-06-15T01:06:56.029 INFO:tasks.workunit.client.1.smithi142.stdout:4/270: truncate d5/d9/da/f3c 130424 0 2024-06-15T01:06:56.030 INFO:tasks.workunit.client.1.smithi142.stdout:0/167: mkdir d3/d5/d1b/d38 0 2024-06-15T01:06:56.033 INFO:tasks.workunit.client.1.smithi142.stdout:1/202: dread d6/f13 [0,4194304] 0 2024-06-15T01:06:56.035 INFO:tasks.workunit.client.1.smithi142.stdout:6/187: creat d1a/f46 x:0 0 0 2024-06-15T01:06:56.039 INFO:tasks.workunit.client.1.smithi142.stdout:9/157: dwrite d6/d31/f3e [0,4194304] 0 2024-06-15T01:06:56.040 INFO:tasks.workunit.client.0.smithi137.stdout:5/372: mkdir df/d76/d7c 0 2024-06-15T01:06:56.040 INFO:tasks.workunit.client.0.smithi137.stdout:8/94: truncate f8 1698275 0 2024-06-15T01:06:56.042 INFO:tasks.workunit.client.0.smithi137.stdout:0/127: unlink d1/d2/dc/f16 0 2024-06-15T01:06:56.042 INFO:tasks.workunit.client.1.smithi142.stdout:5/229: unlink d2/d18/d2e/f34 0 2024-06-15T01:06:56.042 INFO:tasks.workunit.client.1.smithi142.stdout:5/230: stat d2/d18/d2e 0 2024-06-15T01:06:56.043 INFO:tasks.workunit.client.0.smithi137.stdout:3/181: dwrite d0/d2/f1b [0,4194304] 0 2024-06-15T01:06:56.043 INFO:tasks.workunit.client.0.smithi137.stdout:4/128: truncate f5 1444989 0 2024-06-15T01:06:56.044 INFO:tasks.workunit.client.1.smithi142.stdout:3/238: dread - d0/d1f/f38 zero size 2024-06-15T01:06:56.045 INFO:tasks.workunit.client.0.smithi137.stdout:4/129: write dd/d15/d21/f2a [1043750,105755] 0 2024-06-15T01:06:56.046 INFO:tasks.workunit.client.0.smithi137.stdout:8/95: write d9/fc [1101541,30662] 0 2024-06-15T01:06:56.047 INFO:tasks.workunit.client.1.smithi142.stdout:3/239: chown d0/d3/da/cb 2140046837 1 2024-06-15T01:06:56.048 INFO:tasks.workunit.client.1.smithi142.stdout:2/181: link d8/fc d8/df/f2d 0 2024-06-15T01:06:56.051 INFO:tasks.workunit.client.0.smithi137.stdout:3/182: dread d0/d2f/f35 [0,4194304] 0 2024-06-15T01:06:56.051 INFO:tasks.workunit.client.0.smithi137.stdout:0/128: fdatasync d1/d2/dc/d10/d14/f1b 0 2024-06-15T01:06:56.052 INFO:tasks.workunit.client.0.smithi137.stdout:8/96: chown d9/la 2998834 1 2024-06-15T01:06:56.053 INFO:tasks.workunit.client.1.smithi142.stdout:0/168: mkdir d3/d39 0 2024-06-15T01:06:56.054 INFO:tasks.workunit.client.1.smithi142.stdout:7/171: creat d2/d8/da/d33/d25/f3f x:0 0 0 2024-06-15T01:06:56.054 INFO:tasks.workunit.client.0.smithi137.stdout:6/128: truncate d0/d5/d9/fe 1352009 0 2024-06-15T01:06:56.055 INFO:tasks.workunit.client.1.smithi142.stdout:1/203: mknod d6/d12/d1c/d2c/d47/c49 0 2024-06-15T01:06:56.057 INFO:tasks.workunit.client.0.smithi137.stdout:0/129: rename d1/d2/dc/d10/d22 to d1/d2/dc/d10/d22/d29 22 2024-06-15T01:06:56.057 INFO:tasks.workunit.client.0.smithi137.stdout:4/130: write dd/f13 [1975725,58996] 0 2024-06-15T01:06:56.057 INFO:tasks.workunit.client.1.smithi142.stdout:2/182: creat d8/d19/d23/d25/d1a/f2e x:0 0 0 2024-06-15T01:06:56.059 INFO:tasks.workunit.client.0.smithi137.stdout:5/373: dwrite df/d11/f4c [0,4194304] 0 2024-06-15T01:06:56.060 INFO:tasks.workunit.client.0.smithi137.stdout:4/131: chown dd/d15/f2e 60226 1 2024-06-15T01:06:56.060 INFO:tasks.workunit.client.0.smithi137.stdout:3/183: creat d0/d2/d22/d24/d28/d2e/f3b x:0 0 0 2024-06-15T01:06:56.062 INFO:tasks.workunit.client.1.smithi142.stdout:4/271: rename d5/d9/c16 to d5/c62 0 2024-06-15T01:06:56.063 INFO:tasks.workunit.client.1.smithi142.stdout:0/169: dwrite d3/f23 [0,4194304] 0 2024-06-15T01:06:56.064 INFO:tasks.workunit.client.0.smithi137.stdout:9/81: mkdir df/d12/d15 0 2024-06-15T01:06:56.066 INFO:tasks.workunit.client.0.smithi137.stdout:3/184: stat d0/d2/d22/d24/d28/d2e/f3b 0 2024-06-15T01:06:56.067 INFO:tasks.workunit.client.1.smithi142.stdout:2/183: fsync f6 0 2024-06-15T01:06:56.067 INFO:tasks.workunit.client.0.smithi137.stdout:2/88: link d0/f6 d0/de/f1a 0 2024-06-15T01:06:56.069 INFO:tasks.workunit.client.1.smithi142.stdout:0/170: read d3/d12/f19 [34717,54765] 0 2024-06-15T01:06:56.071 INFO:tasks.workunit.client.0.smithi137.stdout:6/129: truncate d0/d5/f15 1391033 0 2024-06-15T01:06:56.072 INFO:tasks.workunit.client.0.smithi137.stdout:5/374: dwrite fe [0,4194304] 0 2024-06-15T01:06:56.074 INFO:tasks.workunit.client.1.smithi142.stdout:9/158: rename d6/dc/dd/d22/f26 to d6/dc/d1b/d25/f3f 0 2024-06-15T01:06:56.075 INFO:tasks.workunit.client.0.smithi137.stdout:8/97: creat d9/f1d x:0 0 0 2024-06-15T01:06:56.076 INFO:tasks.workunit.client.0.smithi137.stdout:4/132: mknod dd/d15/c34 0 2024-06-15T01:06:56.080 INFO:tasks.workunit.client.0.smithi137.stdout:9/82: dread f5 [4194304,4194304] 0 2024-06-15T01:06:56.081 INFO:tasks.workunit.client.0.smithi137.stdout:4/133: fdatasync dd/d15/d21/f32 0 2024-06-15T01:06:56.082 INFO:tasks.workunit.client.1.smithi142.stdout:8/165: truncate d0/d17/f1f 732739 0 2024-06-15T01:06:56.082 INFO:tasks.workunit.client.0.smithi137.stdout:2/89: mknod d0/c1b 0 2024-06-15T01:06:56.082 INFO:tasks.workunit.client.0.smithi137.stdout:8/98: mkdir d9/de/d1e 0 2024-06-15T01:06:56.083 INFO:tasks.workunit.client.1.smithi142.stdout:2/184: symlink d8/df/d10/d1c/l2f 0 2024-06-15T01:06:56.083 INFO:tasks.workunit.client.1.smithi142.stdout:5/231: write d2/f7 [3452321,116249] 0 2024-06-15T01:06:56.083 INFO:tasks.workunit.client.0.smithi137.stdout:5/375: rename df/d11/d34/d3b/c73 to df/d11/d14/d6c/c7d 0 2024-06-15T01:06:56.083 INFO:tasks.workunit.client.0.smithi137.stdout:8/99: write d9/f1b [112567,36026] 0 2024-06-15T01:06:56.085 INFO:tasks.workunit.client.1.smithi142.stdout:2/185: truncate d8/d19/d23/d25/d1a/f2e 256641 0 2024-06-15T01:06:56.085 INFO:tasks.workunit.client.0.smithi137.stdout:2/90: rmdir d0/dc 39 2024-06-15T01:06:56.085 INFO:tasks.workunit.client.1.smithi142.stdout:2/186: chown f4 4 1 2024-06-15T01:06:56.086 INFO:tasks.workunit.client.0.smithi137.stdout:2/91: truncate d0/f9 1317716 0 2024-06-15T01:06:56.086 INFO:tasks.workunit.client.0.smithi137.stdout:4/134: mknod dd/d15/c35 0 2024-06-15T01:06:56.088 INFO:tasks.workunit.client.0.smithi137.stdout:4/135: chown dd/d10/c11 2 1 2024-06-15T01:06:56.089 INFO:tasks.workunit.client.0.smithi137.stdout:9/83: dread fa [0,4194304] 0 2024-06-15T01:06:56.089 INFO:tasks.workunit.client.0.smithi137.stdout:4/136: dread - dd/d15/d21/f32 zero size 2024-06-15T01:06:56.090 INFO:tasks.workunit.client.1.smithi142.stdout:4/272: readlink d5/d1c/d2a/d30/l4a 0 2024-06-15T01:06:56.091 INFO:tasks.workunit.client.0.smithi137.stdout:9/84: chown l6 143350 1 2024-06-15T01:06:56.091 INFO:tasks.workunit.client.0.smithi137.stdout:5/376: mknod df/d11/d34/d3b/d4b/d4f/c7e 0 2024-06-15T01:06:56.093 INFO:tasks.workunit.client.0.smithi137.stdout:5/377: write df/d11/f6d [873565,39611] 0 2024-06-15T01:06:56.094 INFO:tasks.workunit.client.0.smithi137.stdout:5/378: read df/d11/d14/f43 [1382987,101753] 0 2024-06-15T01:06:56.094 INFO:tasks.workunit.client.1.smithi142.stdout:3/240: rename d0/d3/da/c14 to d0/d1b/c49 0 2024-06-15T01:06:56.095 INFO:tasks.workunit.client.0.smithi137.stdout:5/379: fdatasync df/d11/d14/f2f 0 2024-06-15T01:06:56.095 INFO:tasks.workunit.client.1.smithi142.stdout:3/241: stat d0/d1f 0 2024-06-15T01:06:56.096 INFO:tasks.workunit.client.1.smithi142.stdout:3/242: readlink d0/d3/l8 0 2024-06-15T01:06:56.098 INFO:tasks.workunit.client.0.smithi137.stdout:3/185: rmdir d0/d2/d22 39 2024-06-15T01:06:56.099 INFO:tasks.workunit.client.1.smithi142.stdout:2/187: chown d8/df/c17 28565315 1 2024-06-15T01:06:56.099 INFO:tasks.workunit.client.1.smithi142.stdout:1/204: dwrite d6/d12/d1c/f34 [4194304,4194304] 0 2024-06-15T01:06:56.099 INFO:tasks.workunit.client.1.smithi142.stdout:2/188: readlink l5 0 2024-06-15T01:06:56.101 INFO:tasks.workunit.client.1.smithi142.stdout:0/171: symlink d3/l3a 0 2024-06-15T01:06:56.104 INFO:tasks.workunit.client.1.smithi142.stdout:4/273: chown d5/d9/d23/d32/c5d 205282 1 2024-06-15T01:06:56.106 INFO:tasks.workunit.client.0.smithi137.stdout:3/186: dwrite d0/d2/d8/f1c [0,4194304] 0 2024-06-15T01:06:56.110 INFO:tasks.workunit.client.0.smithi137.stdout:8/100: rmdir d9 39 2024-06-15T01:06:56.113 INFO:tasks.workunit.client.0.smithi137.stdout:4/137: dread dd/d10/f1c [0,4194304] 0 2024-06-15T01:06:56.114 INFO:tasks.workunit.client.0.smithi137.stdout:2/92: truncate d0/da/f17 3741428 0 2024-06-15T01:06:56.114 INFO:tasks.workunit.client.1.smithi142.stdout:5/232: mknod d2/d3e/c48 0 2024-06-15T01:06:56.117 INFO:tasks.workunit.client.0.smithi137.stdout:0/130: dwrite d1/f9 [0,4194304] 0 2024-06-15T01:06:56.118 INFO:tasks.workunit.client.0.smithi137.stdout:9/85: mkdir df/d12/d15/d16 0 2024-06-15T01:06:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.122+0000 7fcfbccba700 1 -- 172.21.15.137:0/4134521160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb80ffcd0 msgr2=0x7fcfb8100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.123+0000 7fcfbccba700 1 --2- 172.21.15.137:0/4134521160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb8100150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009910 tx=0x7fcfac009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.124+0000 7fcfbccba700 1 -- 172.21.15.137:0/4134521160 shutdown_connections 2024-06-15T01:06:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.124+0000 7fcfbccba700 1 --2- 172.21.15.137:0/4134521160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb8100150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.124+0000 7fcfbccba700 1 --2- 172.21.15.137:0/4134521160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb8101070 0x7fcfb8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.124+0000 7fcfbccba700 1 -- 172.21.15.137:0/4134521160 >> 172.21.15.137:0/4134521160 conn(0x7fcfb80fa7c0 msgr2=0x7fcfb80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.124+0000 7fcfbccba700 1 -- 172.21.15.137:0/4134521160 shutdown_connections 2024-06-15T01:06:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.125+0000 7fcfbccba700 1 -- 172.21.15.137:0/4134521160 wait complete. 2024-06-15T01:06:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.126+0000 7fcfbccba700 1 Processor -- start 2024-06-15T01:06:56.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.126+0000 7fcfbccba700 1 -- start start 2024-06-15T01:06:56.123 INFO:tasks.workunit.client.1.smithi142.stdout:4/274: mknod d5/d9/d1a/d4f/c63 0 2024-06-15T01:06:56.123 INFO:tasks.workunit.client.1.smithi142.stdout:5/233: dwrite d2/de/d19/f22 [4194304,4194304] 0 2024-06-15T01:06:56.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.127+0000 7fcfbccba700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.127+0000 7fcfbccba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.127+0000 7fcfb5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.128+0000 7fcfb5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41892/0 (socket says 172.21.15.137:41892) 2024-06-15T01:06:56.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.128+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/432849468 learned_addr learned my addr 172.21.15.137:0/432849468 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:56.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.128+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819d3b0 con 0x7fcfb8101070 2024-06-15T01:06:56.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.128+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819d4f0 con 0x7fcfb80ffcd0 2024-06-15T01:06:56.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.128+0000 7fcfb659c700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.126 INFO:tasks.workunit.client.1.smithi142.stdout:0/172: dread d3/d12/d22/f24 [0,4194304] 0 2024-06-15T01:06:56.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.129+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffcd0 msgr2=0x7fcfb819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.129+0000 7fcfb5d9b700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.129+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0095f0 con 0x7fcfb8101070 2024-06-15T01:06:56.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.130+0000 7fcfb5d9b700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00b5c0 tx=0x7fcfac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:56.127 INFO:tasks.workunit.client.1.smithi142.stdout:3/243: symlink d0/d1f/l4a 0 2024-06-15T01:06:56.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.131+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfac01d070 con 0x7fcfb8101070 2024-06-15T01:06:56.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.131+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcfac00f460 con 0x7fcfb8101070 2024-06-15T01:06:56.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.131+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfac021640 con 0x7fcfb8101070 2024-06-15T01:06:56.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.132+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb81a1f40 con 0x7fcfb8101070 2024-06-15T01:06:56.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.132+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb81a23c0 con 0x7fcfb8101070 2024-06-15T01:06:56.129 INFO:tasks.workunit.client.0.smithi137.stdout:5/380: dwrite df/d11/d14/f32 [0,4194304] 0 2024-06-15T01:06:56.130 INFO:tasks.workunit.client.0.smithi137.stdout:4/138: mkdir dd/d15/d36 0 2024-06-15T01:06:56.130 INFO:tasks.workunit.client.0.smithi137.stdout:0/131: rename d1 to d1/d2/d2a 22 2024-06-15T01:06:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.137+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb81084d0 con 0x7fcfb8101070 2024-06-15T01:06:56.134 INFO:tasks.workunit.client.0.smithi137.stdout:8/101: dwrite d9/f1d [0,4194304] 0 2024-06-15T01:06:56.136 INFO:tasks.workunit.client.1.smithi142.stdout:7/172: rename d2/d8/da/d33/f3a to d2/d8/da/d33/d25/f40 0 2024-06-15T01:06:56.137 INFO:tasks.workunit.client.0.smithi137.stdout:4/139: mknod dd/d10/d1f/c37 0 2024-06-15T01:06:56.137 INFO:tasks.workunit.client.0.smithi137.stdout:8/102: fdatasync d9/de/f18 0 2024-06-15T01:06:56.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.140+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7fcfac00fa00 con 0x7fcfb8101070 2024-06-15T01:06:56.138 INFO:tasks.workunit.client.0.smithi137.stdout:4/140: truncate dd/f28 181561 0 2024-06-15T01:06:56.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/141: write dd/d15/d21/f29 [113260,31336] 0 2024-06-15T01:06:56.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.140+0000 7fcfa77fe700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 0x7fcfa006e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.141+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcfac08c070 con 0x7fcfb8101070 2024-06-15T01:06:56.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.142+0000 7fcfb659c700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 0x7fcfa006e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.140 INFO:tasks.workunit.client.0.smithi137.stdout:9/86: link fd df/d12/f17 0 2024-06-15T01:06:56.141 INFO:tasks.workunit.client.0.smithi137.stdout:2/93: truncate d0/f1 66676 0 2024-06-15T01:06:56.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.144+0000 7fcfb659c700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 0x7fcfa006e070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005950 tx=0x7fcfa800b410 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:56.142 INFO:tasks.workunit.client.1.smithi142.stdout:8/166: getdents d0/d1 0 2024-06-15T01:06:56.142 INFO:tasks.workunit.client.1.smithi142.stdout:6/188: dwrite d1a/f24 [4194304,4194304] 0 2024-06-15T01:06:56.143 INFO:tasks.workunit.client.1.smithi142.stdout:6/189: write f9 [3486952,47273] 0 2024-06-15T01:06:56.143 INFO:tasks.workunit.client.1.smithi142.stdout:7/173: write d2/d8/da/d33/d25/f16 [3113183,33575] 0 2024-06-15T01:06:56.143 INFO:tasks.workunit.client.0.smithi137.stdout:9/87: dread fa [0,4194304] 0 2024-06-15T01:06:56.144 INFO:tasks.workunit.client.0.smithi137.stdout:5/381: mknod df/d11/c7f 0 2024-06-15T01:06:56.145 INFO:tasks.workunit.client.0.smithi137.stdout:3/187: rename d0/d2/d1e/f2a to d0/d2/d22/d24/d28/d2e/d39/f3c 0 2024-06-15T01:06:56.145 INFO:tasks.workunit.client.0.smithi137.stdout:5/382: readlink df/l1b 0 2024-06-15T01:06:56.150 INFO:tasks.workunit.client.0.smithi137.stdout:4/142: symlink dd/d15/d30/l38 0 2024-06-15T01:06:56.151 INFO:tasks.workunit.client.0.smithi137.stdout:0/132: dread d1/d2/dc/d10/d14/f1b [0,4194304] 0 2024-06-15T01:06:56.151 INFO:tasks.workunit.client.1.smithi142.stdout:0/173: mknod d3/d5/d1e/c3b 0 2024-06-15T01:06:56.151 INFO:tasks.workunit.client.1.smithi142.stdout:2/189: link d8/d19/d23/d25/d1a/f1d d8/d19/f30 0 2024-06-15T01:06:56.152 INFO:tasks.workunit.client.1.smithi142.stdout:5/234: rename d2 to d2/d3e/d49 22 2024-06-15T01:06:56.152 INFO:tasks.workunit.client.1.smithi142.stdout:1/205: write d6/fc [881108,61795] 0 2024-06-15T01:06:56.152 INFO:tasks.workunit.client.1.smithi142.stdout:0/174: read d3/d12/d22/f24 [1981080,57343] 0 2024-06-15T01:06:56.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.157+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fcfac05aab0 con 0x7fcfb8101070 2024-06-15T01:06:56.156 INFO:tasks.workunit.client.0.smithi137.stdout:9/88: creat df/f18 x:0 0 0 2024-06-15T01:06:56.159 INFO:tasks.workunit.client.0.smithi137.stdout:8/103: dread f8 [0,4194304] 0 2024-06-15T01:06:56.159 INFO:tasks.workunit.client.0.smithi137.stdout:9/89: chown df/d12/d15/d16 109 1 2024-06-15T01:06:56.159 INFO:tasks.workunit.client.0.smithi137.stdout:2/94: dwrite d0/f9 [0,4194304] 0 2024-06-15T01:06:56.159 INFO:tasks.workunit.client.0.smithi137.stdout:2/95: stat d0/c1b 0 2024-06-15T01:06:56.161 INFO:tasks.workunit.client.1.smithi142.stdout:5/235: dwrite d2/de/d27/f46 [0,4194304] 0 2024-06-15T01:06:56.162 INFO:tasks.workunit.client.1.smithi142.stdout:6/190: readlink d1a/d2d/l34 0 2024-06-15T01:06:56.163 INFO:tasks.workunit.client.1.smithi142.stdout:5/236: stat d2/de 0 2024-06-15T01:06:56.164 INFO:tasks.workunit.client.1.smithi142.stdout:3/244: creat d0/d1f/f4b x:0 0 0 2024-06-15T01:06:56.164 INFO:tasks.workunit.client.1.smithi142.stdout:5/237: stat d2/l3d 0 2024-06-15T01:06:56.165 INFO:tasks.workunit.client.0.smithi137.stdout:5/383: mkdir df/d44/d54/d80 0 2024-06-15T01:06:56.165 INFO:tasks.workunit.client.0.smithi137.stdout:0/133: rename d1 to d1/d2b 22 2024-06-15T01:06:56.165 INFO:tasks.workunit.client.1.smithi142.stdout:2/190: dread d8/df/f2d [0,4194304] 0 2024-06-15T01:06:56.168 INFO:tasks.workunit.client.0.smithi137.stdout:4/143: truncate dd/d10/f1c 708457 0 2024-06-15T01:06:56.169 INFO:tasks.workunit.client.0.smithi137.stdout:3/188: dread d0/f16 [0,4194304] 0 2024-06-15T01:06:56.171 INFO:tasks.workunit.client.1.smithi142.stdout:3/245: truncate d0/d1f/d20/f26 8426406 0 2024-06-15T01:06:56.172 INFO:tasks.workunit.client.1.smithi142.stdout:3/246: chown d0/d3/c16 103 1 2024-06-15T01:06:56.174 INFO:tasks.workunit.client.1.smithi142.stdout:3/247: write d0/f2c [1723602,54595] 0 2024-06-15T01:06:56.175 INFO:tasks.workunit.client.1.smithi142.stdout:4/275: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:56.177 INFO:tasks.workunit.client.0.smithi137.stdout:8/104: symlink d9/de/d1e/l1f 0 2024-06-15T01:06:56.177 INFO:tasks.workunit.client.1.smithi142.stdout:0/175: creat d3/d12/f3c x:0 0 0 2024-06-15T01:06:56.178 INFO:tasks.workunit.client.0.smithi137.stdout:2/96: mkdir d0/d1c 0 2024-06-15T01:06:56.178 INFO:tasks.workunit.client.1.smithi142.stdout:7/174: getdents d2/d8/dd/d14/d2c 0 2024-06-15T01:06:56.179 INFO:tasks.workunit.client.0.smithi137.stdout:4/144: creat dd/d10/f39 x:0 0 0 2024-06-15T01:06:56.179 INFO:tasks.workunit.client.0.smithi137.stdout:5/384: dwrite df/f4a [0,4194304] 0 2024-06-15T01:06:56.180 INFO:tasks.workunit.client.0.smithi137.stdout:3/189: creat d0/d2/d23/f3d x:0 0 0 2024-06-15T01:06:56.181 INFO:tasks.workunit.client.1.smithi142.stdout:5/238: mknod d2/d3e/d31/c4a 0 2024-06-15T01:06:56.185 INFO:tasks.workunit.client.0.smithi137.stdout:0/134: getdents d1/d2 0 2024-06-15T01:06:56.186 INFO:tasks.workunit.client.0.smithi137.stdout:2/97: creat d0/d1c/f1d x:0 0 0 2024-06-15T01:06:56.188 INFO:tasks.workunit.client.0.smithi137.stdout:4/145: mkdir dd/d15/d21/d3a 0 2024-06-15T01:06:56.189 INFO:tasks.workunit.client.0.smithi137.stdout:8/105: creat d9/f20 x:0 0 0 2024-06-15T01:06:56.190 INFO:tasks.workunit.client.1.smithi142.stdout:3/248: dwrite d0/d3/f3d [0,4194304] 0 2024-06-15T01:06:56.191 INFO:tasks.workunit.client.1.smithi142.stdout:0/176: fdatasync d3/f28 0 2024-06-15T01:06:56.191 INFO:tasks.workunit.client.1.smithi142.stdout:2/191: mknod d8/d19/d23/d25/c31 0 2024-06-15T01:06:56.191 INFO:tasks.workunit.client.1.smithi142.stdout:4/276: dwrite d5/d9/da/f3c [0,4194304] 0 2024-06-15T01:06:56.192 INFO:tasks.workunit.client.1.smithi142.stdout:1/206: link d6/d12/d30/c3f d6/d12/d30/c4a 0 2024-06-15T01:06:56.192 INFO:tasks.workunit.client.0.smithi137.stdout:3/190: dread d0/d2f/f35 [0,4194304] 0 2024-06-15T01:06:56.193 INFO:tasks.workunit.client.0.smithi137.stdout:2/98: mknod d0/de/c1e 0 2024-06-15T01:06:56.193 INFO:tasks.workunit.client.1.smithi142.stdout:4/277: truncate f3 1913133 0 2024-06-15T01:06:56.194 INFO:tasks.workunit.client.1.smithi142.stdout:3/249: symlink d0/d3/l4c 0 2024-06-15T01:06:56.195 INFO:tasks.workunit.client.0.smithi137.stdout:5/385: dread df/d11/d14/f21 [0,4194304] 0 2024-06-15T01:06:56.196 INFO:tasks.workunit.client.1.smithi142.stdout:3/250: chown d0/d1f/d20/l25 1 1 2024-06-15T01:06:56.196 INFO:tasks.workunit.client.1.smithi142.stdout:1/207: read f5 [12375079,87146] 0 2024-06-15T01:06:56.197 INFO:tasks.workunit.client.0.smithi137.stdout:8/106: unlink d9/de/f15 0 2024-06-15T01:06:56.200 INFO:tasks.workunit.client.0.smithi137.stdout:3/191: dread d0/f1a [0,4194304] 0 2024-06-15T01:06:56.201 INFO:tasks.workunit.client.1.smithi142.stdout:3/251: rmdir d0/d1f/d20 39 2024-06-15T01:06:56.203 INFO:tasks.workunit.client.1.smithi142.stdout:0/177: symlink d3/d12/l3d 0 2024-06-15T01:06:56.204 INFO:tasks.workunit.client.1.smithi142.stdout:3/252: dread - d0/d1f/f4b zero size 2024-06-15T01:06:56.205 INFO:tasks.workunit.client.1.smithi142.stdout:2/192: mknod d8/df/d10/c32 0 2024-06-15T01:06:56.205 INFO:tasks.workunit.client.0.smithi137.stdout:8/107: dread d9/fc [0,4194304] 0 2024-06-15T01:06:56.206 INFO:tasks.workunit.client.1.smithi142.stdout:1/208: creat d6/d12/d1c/d1e/f4b x:0 0 0 2024-06-15T01:06:56.206 INFO:tasks.workunit.client.1.smithi142.stdout:1/209: chown d6/d12/d28/f33 9502 1 2024-06-15T01:06:56.207 INFO:tasks.workunit.client.0.smithi137.stdout:8/108: truncate d9/f1b 507158 0 2024-06-15T01:06:56.209 INFO:tasks.workunit.client.1.smithi142.stdout:8/167: write d0/d17/f1f [1756813,85239] 0 2024-06-15T01:06:56.209 INFO:tasks.workunit.client.0.smithi137.stdout:0/135: rename d1/d2/dc/d10/d14/d15/f1e to d1/f2c 0 2024-06-15T01:06:56.212 INFO:tasks.workunit.client.0.smithi137.stdout:5/386: link df/d44/f4e df/d11/d34/d35/f81 0 2024-06-15T01:06:56.213 INFO:tasks.workunit.client.1.smithi142.stdout:7/175: getdents d2/d6/d22/d34 0 2024-06-15T01:06:56.213 INFO:tasks.workunit.client.1.smithi142.stdout:7/176: stat d2/d8/da/d33/d25/l3e 0 2024-06-15T01:06:56.213 INFO:tasks.workunit.client.0.smithi137.stdout:3/192: symlink d0/d2/d1e/l3e 0 2024-06-15T01:06:56.213 INFO:tasks.workunit.client.0.smithi137.stdout:8/109: mknod d9/de/d1e/c21 0 2024-06-15T01:06:56.214 INFO:tasks.workunit.client.0.smithi137.stdout:3/193: stat d0/d2/d22/d24 0 2024-06-15T01:06:56.215 INFO:tasks.workunit.client.0.smithi137.stdout:3/194: write d0/f1d [2441783,37519] 0 2024-06-15T01:06:56.216 INFO:tasks.workunit.client.0.smithi137.stdout:9/90: dwrite f3 [0,4194304] 0 2024-06-15T01:06:56.217 INFO:tasks.workunit.client.0.smithi137.stdout:5/387: creat df/d11/d30/f82 x:0 0 0 2024-06-15T01:06:56.217 INFO:tasks.workunit.client.1.smithi142.stdout:3/253: mkdir d0/d3/d15/d4d 0 2024-06-15T01:06:56.218 INFO:tasks.workunit.client.0.smithi137.stdout:3/195: chown d0/d2/d23 0 1 2024-06-15T01:06:56.221 INFO:tasks.workunit.client.0.smithi137.stdout:4/146: dread f9 [0,4194304] 0 2024-06-15T01:06:56.221 INFO:tasks.workunit.client.0.smithi137.stdout:3/196: readlink d0/l12 0 2024-06-15T01:06:56.223 INFO:tasks.workunit.client.1.smithi142.stdout:4/278: truncate d5/d9/fd 1452568 0 2024-06-15T01:06:56.224 INFO:tasks.workunit.client.1.smithi142.stdout:4/279: readlink d5/d1c/l27 0 2024-06-15T01:06:56.226 INFO:tasks.workunit.client.0.smithi137.stdout:0/136: link d1/c8 d1/d2/dc/d10/d14/d15/c2d 0 2024-06-15T01:06:56.226 INFO:tasks.workunit.client.1.smithi142.stdout:0/178: mkdir d3/d39/d3e 0 2024-06-15T01:06:56.227 INFO:tasks.workunit.client.0.smithi137.stdout:2/99: rmdir d0/d1c 39 2024-06-15T01:06:56.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/147: creat dd/f3b x:0 0 0 2024-06-15T01:06:56.230 INFO:tasks.workunit.client.1.smithi142.stdout:7/177: truncate d2/f3 1288085 0 2024-06-15T01:06:56.232 INFO:tasks.workunit.client.0.smithi137.stdout:8/110: mkdir d9/d22 0 2024-06-15T01:06:56.232 INFO:tasks.workunit.client.1.smithi142.stdout:2/193: dwrite d8/d19/d23/f26 [0,4194304] 0 2024-06-15T01:06:56.234 INFO:tasks.workunit.client.1.smithi142.stdout:2/194: write d8/df/d10/f11 [2129765,235] 0 2024-06-15T01:06:56.235 INFO:tasks.workunit.client.0.smithi137.stdout:0/137: dwrite d1/f9 [0,4194304] 0 2024-06-15T01:06:56.235 INFO:tasks.workunit.client.0.smithi137.stdout:8/111: read d9/de/ff [554466,86674] 0 2024-06-15T01:06:56.235 INFO:tasks.workunit.client.1.smithi142.stdout:0/179: rename d3/d5/l15 to d3/l3f 0 2024-06-15T01:06:56.236 INFO:tasks.workunit.client.0.smithi137.stdout:0/138: fdatasync d1/d2/dc/d10/d14/f1b 0 2024-06-15T01:06:56.237 INFO:tasks.workunit.client.1.smithi142.stdout:7/178: rmdir d2/d8/da/d33/d25/d27 39 2024-06-15T01:06:56.238 INFO:tasks.workunit.client.0.smithi137.stdout:5/388: getdents df/d11/d70 0 2024-06-15T01:06:56.240 INFO:tasks.workunit.client.0.smithi137.stdout:5/389: dread df/d11/f31 [0,4194304] 0 2024-06-15T01:06:56.241 INFO:tasks.workunit.client.0.smithi137.stdout:5/390: readlink df/d11/d34/l37 0 2024-06-15T01:06:56.243 INFO:tasks.workunit.client.1.smithi142.stdout:1/210: truncate d6/f26 3618811 0 2024-06-15T01:06:56.243 INFO:tasks.workunit.client.1.smithi142.stdout:8/168: truncate d0/d25/f35 3614819 0 2024-06-15T01:06:56.243 INFO:tasks.workunit.client.0.smithi137.stdout:4/148: rename dd/d15/c1d to dd/d15/d21/d3a/c3c 0 2024-06-15T01:06:56.245 INFO:tasks.workunit.client.1.smithi142.stdout:3/254: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:56.252 INFO:tasks.workunit.client.0.smithi137.stdout:1/203: sync 2024-06-15T01:06:56.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/139: mknod d1/d2/c2e 0 2024-06-15T01:06:56.253 INFO:tasks.workunit.client.1.smithi142.stdout:7/179: dwrite d2/d6/f1c [0,4194304] 0 2024-06-15T01:06:56.255 INFO:tasks.workunit.client.1.smithi142.stdout:0/180: dwrite d3/d5/d1b/f30 [0,4194304] 0 2024-06-15T01:06:56.255 INFO:tasks.workunit.client.1.smithi142.stdout:8/169: write d0/d17/f2f [4264861,24263] 0 2024-06-15T01:06:56.256 INFO:tasks.workunit.client.0.smithi137.stdout:4/149: symlink dd/d15/d30/l3d 0 2024-06-15T01:06:56.256 INFO:tasks.workunit.client.1.smithi142.stdout:8/170: stat d0/c21 0 2024-06-15T01:06:56.258 INFO:tasks.workunit.client.1.smithi142.stdout:8/171: fsync d0/d1/f29 0 2024-06-15T01:06:56.259 INFO:tasks.workunit.client.1.smithi142.stdout:0/181: creat d3/d5/d1b/f40 x:0 0 0 2024-06-15T01:06:56.260 INFO:tasks.workunit.client.1.smithi142.stdout:7/180: truncate d2/d8/da/d33/f35 324864 0 2024-06-15T01:06:56.260 INFO:tasks.workunit.client.1.smithi142.stdout:0/182: dread - d3/d5/f34 zero size 2024-06-15T01:06:56.261 INFO:tasks.workunit.client.0.smithi137.stdout:8/112: link d9/de/ff d9/d22/f23 0 2024-06-15T01:06:56.261 INFO:tasks.workunit.client.0.smithi137.stdout:9/91: dwrite fd [8388608,4194304] 0 2024-06-15T01:06:56.261 INFO:tasks.workunit.client.0.smithi137.stdout:2/100: creat d0/f1f x:0 0 0 2024-06-15T01:06:56.261 INFO:tasks.workunit.client.1.smithi142.stdout:1/211: rename d6/d12/l37 to d6/d46/l4c 0 2024-06-15T01:06:56.264 INFO:tasks.workunit.client.0.smithi137.stdout:5/391: mknod df/d44/d54/d80/c83 0 2024-06-15T01:06:56.266 INFO:tasks.workunit.client.1.smithi142.stdout:4/280: dread d5/d9/fd [0,4194304] 0 2024-06-15T01:06:56.267 INFO:tasks.workunit.client.0.smithi137.stdout:1/204: rename d0/d3/d1b/f44 to d0/d3/d14/d34/f52 0 2024-06-15T01:06:56.267 INFO:tasks.workunit.client.0.smithi137.stdout:3/197: getdents d0/d2/d22/d24 0 2024-06-15T01:06:56.269 INFO:tasks.workunit.client.0.smithi137.stdout:1/205: chown d0/d3/l3a 3 1 2024-06-15T01:06:56.271 INFO:tasks.workunit.client.0.smithi137.stdout:2/101: creat d0/dc/f20 x:0 0 0 2024-06-15T01:06:56.274 INFO:tasks.workunit.client.0.smithi137.stdout:3/198: dread d0/d2/f1b [0,4194304] 0 2024-06-15T01:06:56.274 INFO:tasks.workunit.client.1.smithi142.stdout:4/281: write d5/f8 [1377148,28965] 0 2024-06-15T01:06:56.276 INFO:tasks.workunit.client.0.smithi137.stdout:7/146: sync 2024-06-15T01:06:56.276 INFO:tasks.workunit.client.0.smithi137.stdout:6/130: sync 2024-06-15T01:06:56.279 INFO:tasks.workunit.client.0.smithi137.stdout:3/199: chown d0/d2/d22/d24/l25 9406017 1 2024-06-15T01:06:56.280 INFO:tasks.workunit.client.0.smithi137.stdout:7/147: chown df/d18/d1c/d1d/d1e 15569 1 2024-06-15T01:06:56.281 INFO:tasks.workunit.client.0.smithi137.stdout:5/392: creat df/d44/f84 x:0 0 0 2024-06-15T01:06:56.283 INFO:tasks.workunit.client.1.smithi142.stdout:0/183: rename d3/l3f to d3/l41 0 2024-06-15T01:06:56.283 INFO:tasks.workunit.client.0.smithi137.stdout:4/150: symlink dd/d15/d36/l3e 0 2024-06-15T01:06:56.285 INFO:tasks.workunit.client.0.smithi137.stdout:1/206: mknod d0/d10/d1d/c53 0 2024-06-15T01:06:56.285 INFO:tasks.workunit.client.0.smithi137.stdout:4/151: chown dd/d10/c2b 99873844 1 2024-06-15T01:06:56.286 INFO:tasks.workunit.client.0.smithi137.stdout:1/207: chown d0/d3/d14/f19 2966 1 2024-06-15T01:06:56.287 INFO:tasks.workunit.client.0.smithi137.stdout:6/131: mknod d0/d1c/d1d/c27 0 2024-06-15T01:06:56.290 INFO:tasks.workunit.client.0.smithi137.stdout:7/148: creat df/d10/f2b x:0 0 0 2024-06-15T01:06:56.292 INFO:tasks.workunit.client.0.smithi137.stdout:6/132: write d0/d1c/f1e [2261580,6860] 0 2024-06-15T01:06:56.293 INFO:tasks.workunit.client.0.smithi137.stdout:5/393: chown df/d11/d14/l25 6 1 2024-06-15T01:06:56.297 INFO:tasks.workunit.client.0.smithi137.stdout:8/113: dwrite d9/de/ff [0,4194304] 0 2024-06-15T01:06:56.297 INFO:tasks.workunit.client.0.smithi137.stdout:7/149: read df/d10/f19 [283349,43864] 0 2024-06-15T01:06:56.297 INFO:tasks.workunit.client.0.smithi137.stdout:6/133: dread - d0/d1c/f24 zero size 2024-06-15T01:06:56.299 INFO:tasks.workunit.client.0.smithi137.stdout:2/102: symlink d0/l21 0 2024-06-15T01:06:56.300 INFO:tasks.workunit.client.0.smithi137.stdout:4/152: mknod dd/d15/d21/c3f 0 2024-06-15T01:06:56.305 INFO:tasks.workunit.client.0.smithi137.stdout:1/208: dwrite d0/d3/f2f [0,4194304] 0 2024-06-15T01:06:56.305 INFO:tasks.workunit.client.0.smithi137.stdout:9/92: dread f8 [0,4194304] 0 2024-06-15T01:06:56.306 INFO:tasks.workunit.client.0.smithi137.stdout:7/150: creat df/d10/d1a/f2c x:0 0 0 2024-06-15T01:06:56.307 INFO:tasks.workunit.client.0.smithi137.stdout:7/151: chown df/l17 94430 1 2024-06-15T01:06:56.309 INFO:tasks.workunit.client.0.smithi137.stdout:0/140: creat d1/d2/f2f x:0 0 0 2024-06-15T01:06:56.312 INFO:tasks.workunit.client.0.smithi137.stdout:7/152: write f1 [7201322,76844] 0 2024-06-15T01:06:56.316 INFO:tasks.workunit.client.0.smithi137.stdout:7/153: write df/d10/f19 [1219726,100332] 0 2024-06-15T01:06:56.318 INFO:tasks.workunit.client.0.smithi137.stdout:8/114: symlink d9/l24 0 2024-06-15T01:06:56.319 INFO:tasks.workunit.client.0.smithi137.stdout:2/103: dwrite d0/f3 [0,4194304] 0 2024-06-15T01:06:56.320 INFO:tasks.workunit.client.0.smithi137.stdout:7/154: dread - df/d10/d1a/f2c zero size 2024-06-15T01:06:56.327 INFO:tasks.workunit.client.0.smithi137.stdout:9/93: write f8 [4048706,80887] 0 2024-06-15T01:06:56.327 INFO:tasks.workunit.client.1.smithi142.stdout:9/159: sync 2024-06-15T01:06:56.327 INFO:tasks.workunit.client.1.smithi142.stdout:4/282: write d5/d9/d18/f2f [4407144,106189] 0 2024-06-15T01:06:56.327 INFO:tasks.workunit.client.1.smithi142.stdout:1/212: write d6/f26 [461854,84154] 0 2024-06-15T01:06:56.328 INFO:tasks.workunit.client.1.smithi142.stdout:4/283: write d5/d9/d23/f4e [1606993,27868] 0 2024-06-15T01:06:56.328 INFO:tasks.workunit.client.0.smithi137.stdout:1/209: dwrite d0/d10/d41/f4b [0,4194304] 0 2024-06-15T01:06:56.328 INFO:tasks.workunit.client.1.smithi142.stdout:3/255: dread d0/d1f/f27 [4194304,4194304] 0 2024-06-15T01:06:56.329 INFO:tasks.workunit.client.1.smithi142.stdout:9/160: creat d6/dc/d1b/f40 x:0 0 0 2024-06-15T01:06:56.330 INFO:tasks.workunit.client.0.smithi137.stdout:8/115: dwrite d9/d22/f23 [0,4194304] 0 2024-06-15T01:06:56.333 INFO:tasks.workunit.client.0.smithi137.stdout:3/200: getdents d0/d2/d1e 0 2024-06-15T01:06:56.334 INFO:tasks.workunit.client.1.smithi142.stdout:8/172: dread d0/f15 [0,4194304] 0 2024-06-15T01:06:56.337 INFO:tasks.workunit.client.0.smithi137.stdout:3/201: write d0/d2/d8/f14 [221312,63812] 0 2024-06-15T01:06:56.339 INFO:tasks.workunit.client.0.smithi137.stdout:0/141: creat d1/d2/dc/d10/d14/d15/f30 x:0 0 0 2024-06-15T01:06:56.340 INFO:tasks.workunit.client.1.smithi142.stdout:8/173: unlink d0/d1/l26 0 2024-06-15T01:06:56.342 INFO:tasks.workunit.client.1.smithi142.stdout:4/284: dwrite d5/f7 [0,4194304] 0 2024-06-15T01:06:56.342 INFO:tasks.workunit.client.0.smithi137.stdout:3/202: readlink d0/d2/d1e/l30 0 2024-06-15T01:06:56.342 INFO:tasks.workunit.client.0.smithi137.stdout:5/394: truncate df/d11/d14/f16 1950648 0 2024-06-15T01:06:56.344 INFO:tasks.workunit.client.1.smithi142.stdout:9/161: dread d6/d31/f3e [0,4194304] 0 2024-06-15T01:06:56.344 INFO:tasks.workunit.client.1.smithi142.stdout:3/256: dread d0/d3/f3d [0,4194304] 0 2024-06-15T01:06:56.344 INFO:tasks.workunit.client.1.smithi142.stdout:1/213: dread d6/d12/d1c/d1e/f20 [0,4194304] 0 2024-06-15T01:06:56.346 INFO:tasks.workunit.client.0.smithi137.stdout:5/395: stat df/d44/c74 0 2024-06-15T01:06:56.347 INFO:tasks.workunit.client.1.smithi142.stdout:9/162: fdatasync d6/d31/f32 0 2024-06-15T01:06:56.348 INFO:tasks.workunit.client.0.smithi137.stdout:6/134: getdents d0/d1c 0 2024-06-15T01:06:56.349 INFO:tasks.workunit.client.1.smithi142.stdout:4/285: creat d5/d1c/d2a/d30/f64 x:0 0 0 2024-06-15T01:06:56.349 INFO:tasks.workunit.client.0.smithi137.stdout:9/94: dwrite f5 [0,4194304] 0 2024-06-15T01:06:56.351 INFO:tasks.workunit.client.1.smithi142.stdout:0/184: dread f1 [0,4194304] 0 2024-06-15T01:06:56.353 INFO:tasks.workunit.client.1.smithi142.stdout:3/257: symlink d0/d1b/l4e 0 2024-06-15T01:06:56.353 INFO:tasks.workunit.client.1.smithi142.stdout:8/174: rename d0/l1c to d0/l39 0 2024-06-15T01:06:56.353 INFO:tasks.workunit.client.1.smithi142.stdout:1/214: mkdir d6/d4d 0 2024-06-15T01:06:56.355 INFO:tasks.workunit.client.0.smithi137.stdout:6/135: mknod d0/d1c/c28 0 2024-06-15T01:06:56.356 INFO:tasks.workunit.client.1.smithi142.stdout:9/163: creat d6/dc/dd/d22/f41 x:0 0 0 2024-06-15T01:06:56.356 INFO:tasks.workunit.client.1.smithi142.stdout:4/286: fdatasync f3 0 2024-06-15T01:06:56.357 INFO:tasks.workunit.client.1.smithi142.stdout:8/175: mkdir d0/d17/d3a 0 2024-06-15T01:06:56.358 INFO:tasks.workunit.client.0.smithi137.stdout:7/155: truncate df/d18/d1c/f1f 1507691 0 2024-06-15T01:06:56.360 INFO:tasks.workunit.client.1.smithi142.stdout:1/215: unlink d6/c23 0 2024-06-15T01:06:56.360 INFO:tasks.workunit.client.0.smithi137.stdout:5/396: symlink df/d3d/l85 0 2024-06-15T01:06:56.361 INFO:tasks.workunit.client.0.smithi137.stdout:4/153: dwrite dd/d10/f1c [0,4194304] 0 2024-06-15T01:06:56.362 INFO:tasks.workunit.client.1.smithi142.stdout:4/287: creat d5/d9/d41/f65 x:0 0 0 2024-06-15T01:06:56.362 INFO:tasks.workunit.client.1.smithi142.stdout:0/185: dwrite d3/d5/f17 [0,4194304] 0 2024-06-15T01:06:56.364 INFO:tasks.workunit.client.0.smithi137.stdout:5/397: dread - df/d11/d30/f82 zero size 2024-06-15T01:06:56.366 INFO:tasks.workunit.client.1.smithi142.stdout:9/164: symlink d6/l42 0 2024-06-15T01:06:56.366 INFO:tasks.workunit.client.1.smithi142.stdout:0/186: chown d3/f37 732 1 2024-06-15T01:06:56.367 INFO:tasks.workunit.client.0.smithi137.stdout:8/116: getdents d9/de/d1e 0 2024-06-15T01:06:56.367 INFO:tasks.workunit.client.0.smithi137.stdout:6/136: mkdir d0/d5/d9/d29 0 2024-06-15T01:06:56.369 INFO:tasks.workunit.client.1.smithi142.stdout:8/176: rename d0/l1a to d0/d1/l3b 0 2024-06-15T01:06:56.370 INFO:tasks.workunit.client.1.smithi142.stdout:4/288: symlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/l66 0 2024-06-15T01:06:56.370 INFO:tasks.workunit.client.0.smithi137.stdout:4/154: creat dd/d15/d30/f40 x:0 0 0 2024-06-15T01:06:56.370 INFO:tasks.workunit.client.0.smithi137.stdout:7/156: rename df/d18/d1c/d1d/d1e to df/d18/d1c/d1d/d27/d2d 0 2024-06-15T01:06:56.371 INFO:tasks.workunit.client.1.smithi142.stdout:1/216: mknod d6/d12/d1c/c4e 0 2024-06-15T01:06:56.371 INFO:tasks.workunit.client.0.smithi137.stdout:6/137: rmdir d0/d1c 39 2024-06-15T01:06:56.371 INFO:tasks.workunit.client.0.smithi137.stdout:7/157: stat df/d18/d1c 0 2024-06-15T01:06:56.372 INFO:tasks.workunit.client.0.smithi137.stdout:4/155: dread - dd/d10/d1f/f26 zero size 2024-06-15T01:06:56.373 INFO:tasks.workunit.client.1.smithi142.stdout:8/177: symlink d0/d17/l3c 0 2024-06-15T01:06:56.374 INFO:tasks.workunit.client.1.smithi142.stdout:8/178: readlink d0/d17/l3c 0 2024-06-15T01:06:56.375 INFO:tasks.workunit.client.0.smithi137.stdout:9/95: truncate f5 973573 0 2024-06-15T01:06:56.376 INFO:tasks.workunit.client.1.smithi142.stdout:8/179: write d0/f23 [4703438,60575] 0 2024-06-15T01:06:56.381 INFO:tasks.workunit.client.1.smithi142.stdout:4/289: dwrite d5/d9/da/f3c [0,4194304] 0 2024-06-15T01:06:56.383 INFO:tasks.workunit.client.1.smithi142.stdout:9/165: rename d6/dc/dd/d23 to d6/dc/d43 0 2024-06-15T01:06:56.383 INFO:tasks.workunit.client.0.smithi137.stdout:4/156: creat dd/d15/d30/f41 x:0 0 0 2024-06-15T01:06:56.384 INFO:tasks.workunit.client.0.smithi137.stdout:8/117: creat d9/d22/f25 x:0 0 0 2024-06-15T01:06:56.385 INFO:tasks.workunit.client.0.smithi137.stdout:9/96: symlink df/d12/l19 0 2024-06-15T01:06:56.386 INFO:tasks.workunit.client.1.smithi142.stdout:4/290: symlink d5/d9/da/l67 0 2024-06-15T01:06:56.386 INFO:tasks.workunit.client.1.smithi142.stdout:9/166: dread - d6/dc/d1b/f40 zero size 2024-06-15T01:06:56.386 INFO:tasks.workunit.client.0.smithi137.stdout:0/142: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:56.387 INFO:tasks.workunit.client.0.smithi137.stdout:9/97: dread - df/f18 zero size 2024-06-15T01:06:56.388 INFO:tasks.workunit.client.0.smithi137.stdout:9/98: chown df/d12 4584156 1 2024-06-15T01:06:56.389 INFO:tasks.workunit.client.0.smithi137.stdout:5/398: rename df/d11/d14/f19 to df/d44/d54/d6b/f86 0 2024-06-15T01:06:56.390 INFO:tasks.workunit.client.1.smithi142.stdout:4/291: dread d5/d1c/d2a/f3d [0,4194304] 0 2024-06-15T01:06:56.394 INFO:tasks.workunit.client.0.smithi137.stdout:4/157: unlink dd/c20 0 2024-06-15T01:06:56.396 INFO:tasks.workunit.client.1.smithi142.stdout:1/217: getdents d6/d12/d28 0 2024-06-15T01:06:56.397 INFO:tasks.workunit.client.0.smithi137.stdout:0/143: symlink d1/d2/dc/d10/d14/d15/d1c/l31 0 2024-06-15T01:06:56.398 INFO:tasks.workunit.client.0.smithi137.stdout:5/399: creat df/d11/d30/f87 x:0 0 0 2024-06-15T01:06:56.400 INFO:tasks.workunit.client.0.smithi137.stdout:6/138: creat d0/f2a x:0 0 0 2024-06-15T01:06:56.400 INFO:tasks.workunit.client.1.smithi142.stdout:4/292: mkdir d5/d9/d23/d68 0 2024-06-15T01:06:56.402 INFO:tasks.workunit.client.0.smithi137.stdout:8/118: dread f1 [0,4194304] 0 2024-06-15T01:06:56.405 INFO:tasks.workunit.client.1.smithi142.stdout:1/218: mknod d6/d12/d30/c4f 0 2024-06-15T01:06:56.405 INFO:tasks.workunit.client.0.smithi137.stdout:7/158: rename df/d10 to df/d13/d2e 0 2024-06-15T01:06:56.405 INFO:tasks.workunit.client.1.smithi142.stdout:3/258: dread d0/f2c [0,4194304] 0 2024-06-15T01:06:56.406 INFO:tasks.workunit.client.1.smithi142.stdout:8/180: link d0/d1/c32 d0/c3d 0 2024-06-15T01:06:56.406 INFO:tasks.workunit.client.0.smithi137.stdout:4/158: dwrite dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:06:56.407 INFO:tasks.workunit.client.1.smithi142.stdout:8/181: write d0/f5 [3354821,15000] 0 2024-06-15T01:06:56.408 INFO:tasks.workunit.client.1.smithi142.stdout:4/293: creat d5/d9/d18/f69 x:0 0 0 2024-06-15T01:06:56.411 INFO:tasks.workunit.client.1.smithi142.stdout:3/259: write d0/d1f/f46 [305468,125352] 0 2024-06-15T01:06:56.412 INFO:tasks.workunit.client.0.smithi137.stdout:5/400: dwrite df/d11/d34/d3b/d49/f77 [0,4194304] 0 2024-06-15T01:06:56.414 INFO:tasks.workunit.client.0.smithi137.stdout:6/139: chown d0/ca 41200 1 2024-06-15T01:06:56.417 INFO:tasks.workunit.client.1.smithi142.stdout:4/294: creat d5/d9/d1a/f6a x:0 0 0 2024-06-15T01:06:56.419 INFO:tasks.workunit.client.0.smithi137.stdout:7/159: mknod df/d18/d1c/d1d/c2f 0 2024-06-15T01:06:56.420 INFO:tasks.workunit.client.1.smithi142.stdout:3/260: readlink d0/d1f/l40 0 2024-06-15T01:06:56.420 INFO:tasks.workunit.client.0.smithi137.stdout:7/160: readlink df/d13/l16 0 2024-06-15T01:06:56.421 INFO:tasks.workunit.client.1.smithi142.stdout:8/182: rmdir d0/d17/d3a 0 2024-06-15T01:06:56.421 INFO:tasks.workunit.client.0.smithi137.stdout:7/161: write fe [63656,82554] 0 2024-06-15T01:06:56.422 INFO:tasks.workunit.client.1.smithi142.stdout:1/219: dread d6/d12/d28/f40 [0,4194304] 0 2024-06-15T01:06:56.422 INFO:tasks.workunit.client.0.smithi137.stdout:4/159: rename dd/d15/d21/f2a to dd/d15/f42 0 2024-06-15T01:06:56.422 INFO:tasks.workunit.client.1.smithi142.stdout:3/261: mknod d0/d3/c4f 0 2024-06-15T01:06:56.425 INFO:tasks.workunit.client.1.smithi142.stdout:3/262: write d0/d3/da/d18/f28 [3159799,3847] 0 2024-06-15T01:06:56.425 INFO:tasks.workunit.client.0.smithi137.stdout:4/160: chown dd/d10/f1c 0 1 2024-06-15T01:06:56.425 INFO:tasks.workunit.client.0.smithi137.stdout:6/140: fsync d0/f2a 0 2024-06-15T01:06:56.426 INFO:tasks.workunit.client.0.smithi137.stdout:6/141: fdatasync d0/d5/f16 0 2024-06-15T01:06:56.427 INFO:tasks.workunit.client.1.smithi142.stdout:8/183: symlink d0/d17/l3e 0 2024-06-15T01:06:56.427 INFO:tasks.workunit.client.1.smithi142.stdout:9/167: dwrite d6/f9 [0,4194304] 0 2024-06-15T01:06:56.428 INFO:tasks.workunit.client.0.smithi137.stdout:6/142: chown d0/d5 41 1 2024-06-15T01:06:56.429 INFO:tasks.workunit.client.0.smithi137.stdout:8/119: symlink d9/l26 0 2024-06-15T01:06:56.429 INFO:tasks.workunit.client.1.smithi142.stdout:3/263: write d0/f3c [246140,121734] 0 2024-06-15T01:06:56.430 INFO:tasks.workunit.client.0.smithi137.stdout:9/99: truncate f3 1151715 0 2024-06-15T01:06:56.431 INFO:tasks.workunit.client.1.smithi142.stdout:3/264: chown d0/d3/l4c 1463682 1 2024-06-15T01:06:56.432 INFO:tasks.workunit.client.1.smithi142.stdout:9/168: creat d6/d34/f44 x:0 0 0 2024-06-15T01:06:56.433 INFO:tasks.workunit.client.0.smithi137.stdout:0/144: creat d1/f32 x:0 0 0 2024-06-15T01:06:56.434 INFO:tasks.workunit.client.0.smithi137.stdout:7/162: rename fd to df/d13/d2e/d1a/f30 0 2024-06-15T01:06:56.435 INFO:tasks.workunit.client.1.smithi142.stdout:1/220: truncate d6/f9 1664233 0 2024-06-15T01:06:56.435 INFO:tasks.workunit.client.1.smithi142.stdout:1/221: chown d6/d12/f2f 16055641 1 2024-06-15T01:06:56.436 INFO:tasks.workunit.client.1.smithi142.stdout:8/184: fdatasync d0/d25/f35 0 2024-06-15T01:06:56.437 INFO:tasks.workunit.client.1.smithi142.stdout:3/265: symlink d0/d1b/l50 0 2024-06-15T01:06:56.438 INFO:tasks.workunit.client.0.smithi137.stdout:5/401: symlink df/d11/d14/d3c/l88 0 2024-06-15T01:06:56.441 INFO:tasks.workunit.client.1.smithi142.stdout:3/266: chown d0/d1b/c36 2766 1 2024-06-15T01:06:56.442 INFO:tasks.workunit.client.0.smithi137.stdout:9/100: mkdir df/d1a 0 2024-06-15T01:06:56.443 INFO:tasks.workunit.client.1.smithi142.stdout:9/169: creat d6/f45 x:0 0 0 2024-06-15T01:06:56.444 INFO:tasks.workunit.client.0.smithi137.stdout:4/161: dwrite dd/d10/f1c [0,4194304] 0 2024-06-15T01:06:56.444 INFO:tasks.workunit.client.0.smithi137.stdout:8/120: rename d9/fc to d9/de/d1e/f27 0 2024-06-15T01:06:56.444 INFO:tasks.workunit.client.0.smithi137.stdout:7/163: read df/d13/d2e/d1a/f30 [535465,89599] 0 2024-06-15T01:06:56.444 INFO:tasks.workunit.client.1.smithi142.stdout:9/170: fdatasync d6/d31/f32 0 2024-06-15T01:06:56.445 INFO:tasks.workunit.client.1.smithi142.stdout:8/185: rename d0/d1/f4 to d0/f3f 0 2024-06-15T01:06:56.446 INFO:tasks.workunit.client.1.smithi142.stdout:8/186: chown d0/f20 12 1 2024-06-15T01:06:56.448 INFO:tasks.workunit.client.1.smithi142.stdout:9/171: creat d6/dc/dd/f46 x:0 0 0 2024-06-15T01:06:56.449 INFO:tasks.workunit.client.0.smithi137.stdout:0/145: dwrite d1/d2/f2f [0,4194304] 0 2024-06-15T01:06:56.449 INFO:tasks.workunit.client.0.smithi137.stdout:8/121: truncate d9/d22/f25 162510 0 2024-06-15T01:06:56.451 INFO:tasks.workunit.client.0.smithi137.stdout:4/162: symlink dd/d15/l43 0 2024-06-15T01:06:56.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/101: mkdir df/d12/d1b 0 2024-06-15T01:06:56.452 INFO:tasks.workunit.client.0.smithi137.stdout:6/143: dread d0/d5/f15 [0,4194304] 0 2024-06-15T01:06:56.453 INFO:tasks.workunit.client.0.smithi137.stdout:9/102: dread - df/f18 zero size 2024-06-15T01:06:56.453 INFO:tasks.workunit.client.0.smithi137.stdout:5/402: rename df/d3d/l85 to df/d11/d34/d35/l89 0 2024-06-15T01:06:56.456 INFO:tasks.workunit.client.1.smithi142.stdout:8/187: mkdir d0/d17/d40 0 2024-06-15T01:06:56.456 INFO:tasks.workunit.client.1.smithi142.stdout:4/295: truncate d5/d1c/d2a/f3d 1177095 0 2024-06-15T01:06:56.457 INFO:tasks.workunit.client.1.smithi142.stdout:9/172: write d6/d31/f3e [561919,48491] 0 2024-06-15T01:06:56.458 INFO:tasks.workunit.client.1.smithi142.stdout:3/267: link d0/d3/da/c10 d0/d3/d15/d4d/c51 0 2024-06-15T01:06:56.458 INFO:tasks.workunit.client.0.smithi137.stdout:7/164: creat df/d13/d2e/f31 x:0 0 0 2024-06-15T01:06:56.459 INFO:tasks.workunit.client.1.smithi142.stdout:3/268: fdatasync d0/f3c 0 2024-06-15T01:06:56.463 INFO:tasks.workunit.client.1.smithi142.stdout:3/269: chown d0/d1b/c31 6621500 1 2024-06-15T01:06:56.464 INFO:tasks.workunit.client.0.smithi137.stdout:8/122: creat d9/de/d1e/f28 x:0 0 0 2024-06-15T01:06:56.465 INFO:tasks.workunit.client.0.smithi137.stdout:8/123: chown d9/f1b 830249 1 2024-06-15T01:06:56.467 INFO:tasks.workunit.client.0.smithi137.stdout:6/144: write d0/d5/f13 [2286041,115495] 0 2024-06-15T01:06:56.468 INFO:tasks.workunit.client.0.smithi137.stdout:8/124: write d9/de/f13 [2215741,14937] 0 2024-06-15T01:06:56.468 INFO:tasks.workunit.client.0.smithi137.stdout:6/145: stat d0/d5/f8 0 2024-06-15T01:06:56.469 INFO:tasks.workunit.client.0.smithi137.stdout:8/125: rename d9 to d9/d22/d29 22 2024-06-15T01:06:56.470 INFO:tasks.workunit.client.1.smithi142.stdout:9/173: getdents d6/dc/d1b/d25/d35 0 2024-06-15T01:06:56.471 INFO:tasks.workunit.client.1.smithi142.stdout:3/270: dwrite d0/d3/d15/f2d [0,4194304] 0 2024-06-15T01:06:56.474 INFO:tasks.workunit.client.0.smithi137.stdout:5/403: getdents df/d11/d70 0 2024-06-15T01:06:56.475 INFO:tasks.workunit.client.1.smithi142.stdout:8/188: creat d0/f41 x:0 0 0 2024-06-15T01:06:56.478 INFO:tasks.workunit.client.0.smithi137.stdout:0/146: creat d1/d2/d13/f33 x:0 0 0 2024-06-15T01:06:56.478 INFO:tasks.workunit.client.1.smithi142.stdout:9/174: mknod d6/d34/c47 0 2024-06-15T01:06:56.479 INFO:tasks.workunit.client.1.smithi142.stdout:9/175: dread - d6/f45 zero size 2024-06-15T01:06:56.482 INFO:tasks.workunit.client.0.smithi137.stdout:5/404: mkdir df/d11/d14/d3c/d8a 0 2024-06-15T01:06:56.484 INFO:tasks.workunit.client.0.smithi137.stdout:6/146: symlink d0/d5/d9/d29/l2b 0 2024-06-15T01:06:56.485 INFO:tasks.workunit.client.1.smithi142.stdout:9/176: symlink d6/dc/d1b/l48 0 2024-06-15T01:06:56.485 INFO:tasks.workunit.client.0.smithi137.stdout:8/126: mkdir d9/d2a 0 2024-06-15T01:06:56.486 INFO:tasks.workunit.client.0.smithi137.stdout:0/147: truncate d1/d2/f21 894154 0 2024-06-15T01:06:56.486 INFO:tasks.workunit.client.1.smithi142.stdout:3/271: dwrite d0/d1f/f27 [8388608,4194304] 0 2024-06-15T01:06:56.487 INFO:tasks.workunit.client.1.smithi142.stdout:8/189: symlink d0/d17/d40/l42 0 2024-06-15T01:06:56.487 INFO:tasks.workunit.client.0.smithi137.stdout:7/165: creat df/f32 x:0 0 0 2024-06-15T01:06:56.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/127: write d9/f1b [1503346,100944] 0 2024-06-15T01:06:56.489 INFO:tasks.workunit.client.0.smithi137.stdout:5/405: rename df/d11/l1d to df/d3d/l8b 0 2024-06-15T01:06:56.490 INFO:tasks.workunit.client.0.smithi137.stdout:8/128: rename d9 to d9/de/d2b 22 2024-06-15T01:06:56.490 INFO:tasks.workunit.client.1.smithi142.stdout:4/296: fsync d5/d1c/d2a/f3d 0 2024-06-15T01:06:56.490 INFO:tasks.workunit.client.1.smithi142.stdout:9/177: creat d6/dc/dd/d22/f49 x:0 0 0 2024-06-15T01:06:56.490 INFO:tasks.workunit.client.0.smithi137.stdout:6/147: creat d0/d5/f2c x:0 0 0 2024-06-15T01:06:56.493 INFO:tasks.workunit.client.0.smithi137.stdout:8/129: chown d9/l26 826 1 2024-06-15T01:06:56.495 INFO:tasks.workunit.client.0.smithi137.stdout:6/148: symlink d0/d5/d9/l2d 0 2024-06-15T01:06:56.496 INFO:tasks.workunit.client.1.smithi142.stdout:9/178: write d6/d34/f44 [870129,15591] 0 2024-06-15T01:06:56.496 INFO:tasks.workunit.client.1.smithi142.stdout:8/190: truncate d0/d1/f29 4758304 0 2024-06-15T01:06:56.496 INFO:tasks.workunit.client.0.smithi137.stdout:9/103: dread f2 [0,4194304] 0 2024-06-15T01:06:56.497 INFO:tasks.workunit.client.0.smithi137.stdout:7/166: dwrite f1 [0,4194304] 0 2024-06-15T01:06:56.498 INFO:tasks.workunit.client.0.smithi137.stdout:9/104: dread - df/f18 zero size 2024-06-15T01:06:56.498 INFO:tasks.workunit.client.1.smithi142.stdout:4/297: dread d5/d9/fd [0,4194304] 0 2024-06-15T01:06:56.499 INFO:tasks.workunit.client.0.smithi137.stdout:0/148: getdents d1/d2/dc/d10 0 2024-06-15T01:06:56.499 INFO:tasks.workunit.client.0.smithi137.stdout:7/167: stat df/d18/d1c 0 2024-06-15T01:06:56.499 INFO:tasks.workunit.client.1.smithi142.stdout:9/179: dread - d6/dc/dd/f46 zero size 2024-06-15T01:06:56.500 INFO:tasks.workunit.client.0.smithi137.stdout:7/168: readlink df/l17 0 2024-06-15T01:06:56.500 INFO:tasks.workunit.client.1.smithi142.stdout:1/222: dwrite d6/d12/f22 [0,4194304] 0 2024-06-15T01:06:56.501 INFO:tasks.workunit.client.1.smithi142.stdout:4/298: write d5/d9/d23/f3b [2392825,10241] 0 2024-06-15T01:06:56.501 INFO:tasks.workunit.client.0.smithi137.stdout:7/169: write f3 [4699497,56169] 0 2024-06-15T01:06:56.502 INFO:tasks.workunit.client.1.smithi142.stdout:9/180: write d6/dc/dd/d22/f49 [146706,12590] 0 2024-06-15T01:06:56.503 INFO:tasks.workunit.client.0.smithi137.stdout:8/130: dread d9/f1d [0,4194304] 0 2024-06-15T01:06:56.504 INFO:tasks.workunit.client.0.smithi137.stdout:6/149: mkdir d0/d1c/d1d/d2e 0 2024-06-15T01:06:56.505 INFO:tasks.workunit.client.1.smithi142.stdout:1/223: fdatasync d6/d12/f3e 0 2024-06-15T01:06:56.508 INFO:tasks.workunit.client.0.smithi137.stdout:0/149: unlink d1/d2/dc/cf 0 2024-06-15T01:06:56.508 INFO:tasks.workunit.client.0.smithi137.stdout:7/170: mkdir df/d18/d1c/d1d/d27/d33 0 2024-06-15T01:06:56.510 INFO:tasks.workunit.client.0.smithi137.stdout:6/150: creat d0/d1a/f2f x:0 0 0 2024-06-15T01:06:56.510 INFO:tasks.workunit.client.1.smithi142.stdout:6/191: sync 2024-06-15T01:06:56.511 INFO:tasks.workunit.client.1.smithi142.stdout:1/224: truncate d6/d12/d1c/d1e/f20 1667207 0 2024-06-15T01:06:56.512 INFO:tasks.workunit.client.1.smithi142.stdout:1/225: fsync d6/fc 0 2024-06-15T01:06:56.512 INFO:tasks.workunit.client.1.smithi142.stdout:9/181: symlink d6/dc/d1b/l4a 0 2024-06-15T01:06:56.513 INFO:tasks.workunit.client.1.smithi142.stdout:4/299: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f6b x:0 0 0 2024-06-15T01:06:56.514 INFO:tasks.workunit.client.1.smithi142.stdout:9/182: fdatasync d6/dc/dd/d22/f41 0 2024-06-15T01:06:56.515 INFO:tasks.workunit.client.1.smithi142.stdout:1/226: mknod d6/d46/c50 0 2024-06-15T01:06:56.517 INFO:tasks.workunit.client.1.smithi142.stdout:9/183: creat d6/dc/d1b/f4b x:0 0 0 2024-06-15T01:06:56.517 INFO:tasks.workunit.client.0.smithi137.stdout:6/151: dwrite d0/f6 [0,4194304] 0 2024-06-15T01:06:56.520 INFO:tasks.workunit.client.1.smithi142.stdout:6/192: symlink d1a/d39/d42/l47 0 2024-06-15T01:06:56.520 INFO:tasks.workunit.client.1.smithi142.stdout:1/227: creat d6/d12/d28/f51 x:0 0 0 2024-06-15T01:06:56.521 INFO:tasks.workunit.client.1.smithi142.stdout:9/184: dread d6/dc/f1c [0,4194304] 0 2024-06-15T01:06:56.522 INFO:tasks.workunit.client.0.smithi137.stdout:6/152: symlink d0/d5/l30 0 2024-06-15T01:06:56.522 INFO:tasks.workunit.client.1.smithi142.stdout:9/185: chown d6/dc/d1b/l48 817982 1 2024-06-15T01:06:56.523 INFO:tasks.workunit.client.1.smithi142.stdout:4/300: getdents d5/d1c/d2a/d30/d46/d50/d51/d5b/d15 0 2024-06-15T01:06:56.524 INFO:tasks.workunit.client.1.smithi142.stdout:9/186: write d6/dc/dd/d22/f41 [693534,123994] 0 2024-06-15T01:06:56.526 INFO:tasks.workunit.client.0.smithi137.stdout:0/150: dwrite d1/d2/dc/d10/d14/d15/d17/f18 [0,4194304] 0 2024-06-15T01:06:56.526 INFO:tasks.workunit.client.1.smithi142.stdout:6/193: creat d1a/d3c/f48 x:0 0 0 2024-06-15T01:06:56.528 INFO:tasks.workunit.client.0.smithi137.stdout:7/171: getdents df/d18/d1c 0 2024-06-15T01:06:56.530 INFO:tasks.workunit.client.0.smithi137.stdout:7/172: chown df/d13/d2e/c21 3896014 1 2024-06-15T01:06:56.531 INFO:tasks.workunit.client.1.smithi142.stdout:4/301: fsync d5/d9/f1f 0 2024-06-15T01:06:56.532 INFO:tasks.workunit.client.0.smithi137.stdout:6/153: dread d0/f6 [0,4194304] 0 2024-06-15T01:06:56.534 INFO:tasks.workunit.client.0.smithi137.stdout:5/406: truncate f7 213883 0 2024-06-15T01:06:56.534 INFO:tasks.workunit.client.1.smithi142.stdout:9/187: symlink d6/d31/l4c 0 2024-06-15T01:06:56.535 INFO:tasks.workunit.client.0.smithi137.stdout:5/407: chown f3 11846 1 2024-06-15T01:06:56.535 INFO:tasks.workunit.client.1.smithi142.stdout:6/194: creat d1a/d2d/d2e/f49 x:0 0 0 2024-06-15T01:06:56.536 INFO:tasks.workunit.client.0.smithi137.stdout:5/408: write df/d11/d14/f2f [527106,1048] 0 2024-06-15T01:06:56.538 INFO:tasks.workunit.client.0.smithi137.stdout:0/151: mkdir d1/d2/d34 0 2024-06-15T01:06:56.540 INFO:tasks.workunit.client.0.smithi137.stdout:6/154: write d0/d1a/f26 [1704910,56463] 0 2024-06-15T01:06:56.542 INFO:tasks.workunit.client.0.smithi137.stdout:5/409: creat df/d44/d54/d6b/f8c x:0 0 0 2024-06-15T01:06:56.543 INFO:tasks.workunit.client.1.smithi142.stdout:6/195: unlink d1a/d39/d42/l47 0 2024-06-15T01:06:56.543 INFO:tasks.workunit.client.0.smithi137.stdout:6/155: chown d0/d5/f16 987 1 2024-06-15T01:06:56.544 INFO:tasks.workunit.client.1.smithi142.stdout:3/272: dread d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:56.544 INFO:tasks.workunit.client.0.smithi137.stdout:5/410: write df/d11/f6d [1485252,42611] 0 2024-06-15T01:06:56.544 INFO:tasks.workunit.client.1.smithi142.stdout:1/228: getdents d6/d12/d1c/d2c/d47 0 2024-06-15T01:06:56.545 INFO:tasks.workunit.client.1.smithi142.stdout:4/302: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:06:56.545 INFO:tasks.workunit.client.1.smithi142.stdout:3/273: write d0/d1f/f4b [634546,7062] 0 2024-06-15T01:06:56.547 INFO:tasks.workunit.client.1.smithi142.stdout:1/229: chown d6/d12/d28/c41 2605 1 2024-06-15T01:06:56.547 INFO:tasks.workunit.client.1.smithi142.stdout:3/274: readlink d0/d1b/l50 0 2024-06-15T01:06:56.548 INFO:tasks.workunit.client.0.smithi137.stdout:0/152: creat d1/d2/dc/d10/d14/d15/f35 x:0 0 0 2024-06-15T01:06:56.549 INFO:tasks.workunit.client.0.smithi137.stdout:6/156: stat d0/d1a/l21 0 2024-06-15T01:06:56.550 INFO:tasks.workunit.client.0.smithi137.stdout:6/157: fsync d0/d5/f2c 0 2024-06-15T01:06:56.553 INFO:tasks.workunit.client.0.smithi137.stdout:7/173: getdents df/d18/d1c/d1d/d27 0 2024-06-15T01:06:56.553 INFO:tasks.workunit.client.0.smithi137.stdout:7/174: readlink l7 0 2024-06-15T01:06:56.554 INFO:tasks.workunit.client.1.smithi142.stdout:6/196: fsync d1a/f20 0 2024-06-15T01:06:56.558 INFO:tasks.workunit.client.1.smithi142.stdout:3/275: rename d0/d3/l2b to d0/d3/d15/l52 0 2024-06-15T01:06:56.559 INFO:tasks.workunit.client.1.smithi142.stdout:3/276: chown d0/d1b/l50 49510009 1 2024-06-15T01:06:56.562 INFO:tasks.workunit.client.1.smithi142.stdout:1/230: mkdir d6/d12/d52 0 2024-06-15T01:06:56.566 INFO:tasks.workunit.client.1.smithi142.stdout:9/188: write d6/dc/dd/f1d [369718,30345] 0 2024-06-15T01:06:56.566 INFO:tasks.workunit.client.1.smithi142.stdout:6/197: write d1a/f25 [4666230,101845] 0 2024-06-15T01:06:56.567 INFO:tasks.workunit.client.0.smithi137.stdout:7/175: dwrite f3 [4194304,4194304] 0 2024-06-15T01:06:56.567 INFO:tasks.workunit.client.0.smithi137.stdout:5/411: dwrite df/d11/d14/f5e [0,4194304] 0 2024-06-15T01:06:56.567 INFO:tasks.workunit.client.1.smithi142.stdout:9/189: chown d6 1 1 2024-06-15T01:06:56.569 INFO:tasks.workunit.client.0.smithi137.stdout:6/158: getdents d0/d5/d9/d29 0 2024-06-15T01:06:56.569 INFO:tasks.workunit.client.0.smithi137.stdout:0/153: dwrite d1/d2/dc/d10/d14/d15/d17/f18 [0,4194304] 0 2024-06-15T01:06:56.570 INFO:tasks.workunit.client.0.smithi137.stdout:6/159: chown d0/d5/d9/d29 307381593 1 2024-06-15T01:06:56.570 INFO:tasks.workunit.client.0.smithi137.stdout:7/176: rename df/d18/d1c to df/d18/d1c/d1d/d27/d2d/d34 22 2024-06-15T01:06:56.571 INFO:tasks.workunit.client.1.smithi142.stdout:3/277: mknod d0/d1f/c53 0 2024-06-15T01:06:56.572 INFO:tasks.workunit.client.1.smithi142.stdout:1/231: dwrite d6/d12/d1c/d1e/f24 [0,4194304] 0 2024-06-15T01:06:56.574 INFO:tasks.workunit.client.1.smithi142.stdout:9/190: mknod d6/dc/d1b/c4d 0 2024-06-15T01:06:56.574 INFO:tasks.workunit.client.1.smithi142.stdout:1/232: fsync d6/d25/f3d 0 2024-06-15T01:06:56.575 INFO:tasks.workunit.client.1.smithi142.stdout:3/278: truncate d0/f3c 445532 0 2024-06-15T01:06:56.576 INFO:tasks.workunit.client.0.smithi137.stdout:0/154: mknod d1/d2/dc/d10/d14/d15/c36 0 2024-06-15T01:06:56.579 INFO:tasks.workunit.client.0.smithi137.stdout:6/160: unlink d0/d5/c23 0 2024-06-15T01:06:56.581 INFO:tasks.workunit.client.0.smithi137.stdout:0/155: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:56.582 INFO:tasks.workunit.client.0.smithi137.stdout:7/177: mknod df/d18/d1c/d1d/d27/d2d/c35 0 2024-06-15T01:06:56.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.586+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfb80ff2d0 con 0x7fcfa006bbb0 2024-06-15T01:06:56.584 INFO:tasks.workunit.client.1.smithi142.stdout:6/198: dwrite f0 [0,4194304] 0 2024-06-15T01:06:56.585 INFO:tasks.workunit.client.0.smithi137.stdout:5/412: rename df/d11/d34/d3b/l45 to df/d44/d54/l8d 0 2024-06-15T01:06:56.586 INFO:tasks.workunit.client.1.smithi142.stdout:9/191: stat d6/dc/dd/f46 0 2024-06-15T01:06:56.587 INFO:tasks.workunit.client.1.smithi142.stdout:3/279: write d0/d1b/f41 [404736,17689] 0 2024-06-15T01:06:56.587 INFO:tasks.workunit.client.0.smithi137.stdout:0/156: symlink d1/d2/dc/d10/d14/d15/l37 0 2024-06-15T01:06:56.588 INFO:tasks.workunit.client.0.smithi137.stdout:6/161: getdents d0/d1c/d1d/d2e 0 2024-06-15T01:06:56.589 INFO:tasks.workunit.client.0.smithi137.stdout:0/157: write d1/d2/f21 [366511,87321] 0 2024-06-15T01:06:56.590 INFO:tasks.workunit.client.1.smithi142.stdout:1/233: dread - d6/d12/d28/f39 zero size 2024-06-15T01:06:56.592 INFO:tasks.workunit.client.1.smithi142.stdout:9/192: unlink d6/dc/f37 0 2024-06-15T01:06:56.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.595+0000 7fcfa77fe700 1 -- 172.21.15.137:0/432849468 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcfb80ff2d0 con 0x7fcfa006bbb0 2024-06-15T01:06:56.593 INFO:tasks.workunit.client.0.smithi137.stdout:6/162: read d0/f10 [230179,23620] 0 2024-06-15T01:06:56.593 INFO:tasks.workunit.client.0.smithi137.stdout:0/158: symlink d1/d2/d34/l38 0 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 msgr2=0x7fcfa006e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 0x7fcfa006e070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005950 tx=0x7fcfa800b410 comp rx=0 tx=0).stop 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 msgr2=0x7fcfb819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00b5c0 tx=0x7fcfac00bd70 comp rx=0 tx=0).stop 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 shutdown_connections 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fcfa006bbb0 0x7fcfa006e070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffcd0 0x7fcfb819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 --2- 172.21.15.137:0/432849468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8101070 0x7fcfb819cd90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.600+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 >> 172.21.15.137:0/432849468 conn(0x7fcfb80fa7c0 msgr2=0x7fcfb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:56.598 INFO:tasks.workunit.client.1.smithi142.stdout:9/193: write d6/d34/f44 [1171885,16620] 0 2024-06-15T01:06:56.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.601+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 shutdown_connections 2024-06-15T01:06:56.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.601+0000 7fcfbccba700 1 -- 172.21.15.137:0/432849468 wait complete. 2024-06-15T01:06:56.599 INFO:tasks.workunit.client.0.smithi137.stdout:7/178: dread df/f14 [0,4194304] 0 2024-06-15T01:06:56.599 INFO:tasks.workunit.client.0.smithi137.stdout:6/163: symlink d0/d1c/d1d/d2e/l31 0 2024-06-15T01:06:56.600 INFO:tasks.workunit.client.0.smithi137.stdout:0/159: creat d1/d2/dc/d10/d14/f39 x:0 0 0 2024-06-15T01:06:56.600 INFO:tasks.workunit.client.0.smithi137.stdout:6/164: fdatasync d0/d5/f2c 0 2024-06-15T01:06:56.601 INFO:tasks.workunit.client.1.smithi142.stdout:1/234: creat d6/d12/d1c/f53 x:0 0 0 2024-06-15T01:06:56.602 INFO:tasks.workunit.client.0.smithi137.stdout:0/160: mknod d1/d2/d13/c3a 0 2024-06-15T01:06:56.603 INFO:tasks.workunit.client.1.smithi142.stdout:9/194: mkdir d6/dc/d4e 0 2024-06-15T01:06:56.603 INFO:tasks.workunit.client.1.smithi142.stdout:6/199: dwrite d1a/d39/d42/f45 [0,4194304] 0 2024-06-15T01:06:56.604 INFO:tasks.workunit.client.0.smithi137.stdout:7/179: symlink df/d18/d1c/d1d/d27/d2d/l36 0 2024-06-15T01:06:56.605 INFO:tasks.workunit.client.0.smithi137.stdout:7/180: write df/d13/d2e/f2b [784313,78021] 0 2024-06-15T01:06:56.606 INFO:tasks.workunit.client.1.smithi142.stdout:3/280: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:06:56.608 INFO:tasks.workunit.client.1.smithi142.stdout:6/200: mknod d1a/d2d/d2e/c4a 0 2024-06-15T01:06:56.611 INFO:tasks.workunit.client.1.smithi142.stdout:1/235: creat d6/d12/d3c/f54 x:0 0 0 2024-06-15T01:06:56.612 INFO:tasks.workunit.client.1.smithi142.stdout:3/281: creat d0/d3/f54 x:0 0 0 2024-06-15T01:06:56.613 INFO:tasks.workunit.client.1.smithi142.stdout:6/201: mknod d1a/d39/d42/c4b 0 2024-06-15T01:06:56.614 INFO:tasks.workunit.client.1.smithi142.stdout:9/195: dwrite d6/dc/dd/f1d [0,4194304] 0 2024-06-15T01:06:56.618 INFO:tasks.workunit.client.1.smithi142.stdout:5/239: sync 2024-06-15T01:06:56.618 INFO:tasks.workunit.client.1.smithi142.stdout:7/181: sync 2024-06-15T01:06:56.618 INFO:tasks.workunit.client.1.smithi142.stdout:2/195: sync 2024-06-15T01:06:56.618 INFO:tasks.workunit.client.1.smithi142.stdout:0/187: sync 2024-06-15T01:06:56.619 INFO:tasks.workunit.client.1.smithi142.stdout:4/303: truncate d5/d9/d23/f4e 3219613 0 2024-06-15T01:06:56.620 INFO:tasks.workunit.client.0.smithi137.stdout:0/161: dread d1/d2/f21 [0,4194304] 0 2024-06-15T01:06:56.620 INFO:tasks.workunit.client.1.smithi142.stdout:1/236: creat d6/d12/d1c/d2c/d47/f55 x:0 0 0 2024-06-15T01:06:56.622 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:06:56.622 INFO:tasks.workunit.client.0.smithi137.stdout:0/162: truncate d1/d2/dc/d10/d14/f39 465854 0 2024-06-15T01:06:56.624 INFO:tasks.workunit.client.0.smithi137.stdout:0/163: dread - d1/d2/dc/d10/d14/d15/f35 zero size 2024-06-15T01:06:56.624 INFO:tasks.workunit.client.1.smithi142.stdout:7/182: chown d2/d8/dd/d14/c31 4324270 1 2024-06-15T01:06:56.625 INFO:tasks.workunit.client.1.smithi142.stdout:6/202: mkdir d1a/d39/d42/d4c 0 2024-06-15T01:06:56.625 INFO:tasks.workunit.client.1.smithi142.stdout:9/196: creat d6/dc/dd/f4f x:0 0 0 2024-06-15T01:06:56.626 INFO:tasks.workunit.client.1.smithi142.stdout:5/240: symlink d2/de/l4b 0 2024-06-15T01:06:56.631 INFO:tasks.workunit.client.0.smithi137.stdout:6/165: getdents d0/d1c/d1d/d2e 0 2024-06-15T01:06:56.632 INFO:tasks.workunit.client.0.smithi137.stdout:0/164: unlink d1/d2/lb 0 2024-06-15T01:06:56.632 INFO:tasks.workunit.client.0.smithi137.stdout:5/413: dwrite df/f2b [0,4194304] 0 2024-06-15T01:06:56.632 INFO:tasks.workunit.client.1.smithi142.stdout:2/196: dwrite d8/d19/d23/d25/d1a/f2e [0,4194304] 0 2024-06-15T01:06:56.633 INFO:tasks.workunit.client.1.smithi142.stdout:2/197: chown d8/d19/d23/d25/f24 1886 1 2024-06-15T01:06:56.634 INFO:tasks.workunit.client.1.smithi142.stdout:3/282: rmdir d0/d1f/d48 0 2024-06-15T01:06:56.635 INFO:tasks.workunit.client.1.smithi142.stdout:9/197: chown d6/dc/c1a 41 1 2024-06-15T01:06:56.636 INFO:tasks.workunit.client.0.smithi137.stdout:6/166: mkdir d0/d1c/d32 0 2024-06-15T01:06:56.636 INFO:tasks.workunit.client.1.smithi142.stdout:3/283: truncate d0/d3/f12 4458511 0 2024-06-15T01:06:56.638 INFO:tasks.workunit.client.1.smithi142.stdout:4/304: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/c6c 0 2024-06-15T01:06:56.641 INFO:tasks.workunit.client.1.smithi142.stdout:0/188: rename d3/l18 to d3/d5/d1e/l42 0 2024-06-15T01:06:56.641 INFO:tasks.workunit.client.0.smithi137.stdout:6/167: creat d0/d5/d9/f33 x:0 0 0 2024-06-15T01:06:56.642 INFO:tasks.workunit.client.0.smithi137.stdout:7/181: truncate f5 3186590 0 2024-06-15T01:06:56.647 INFO:tasks.workunit.client.0.smithi137.stdout:0/165: dwrite d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:56.648 INFO:tasks.workunit.client.0.smithi137.stdout:7/182: creat df/d18/d1c/d1d/d27/d2d/f37 x:0 0 0 2024-06-15T01:06:56.651 INFO:tasks.workunit.client.1.smithi142.stdout:9/198: write d6/dc/d1b/d25/f3f [4593808,102910] 0 2024-06-15T01:06:56.652 INFO:tasks.workunit.client.1.smithi142.stdout:9/199: truncate d6/d31/f32 64687 0 2024-06-15T01:06:56.654 INFO:tasks.workunit.client.1.smithi142.stdout:4/305: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f6d x:0 0 0 2024-06-15T01:06:56.654 INFO:tasks.workunit.client.1.smithi142.stdout:4/306: stat d5/d9/d18/l53 0 2024-06-15T01:06:56.657 INFO:tasks.workunit.client.1.smithi142.stdout:0/189: symlink d3/d5/l43 0 2024-06-15T01:06:56.660 INFO:tasks.workunit.client.0.smithi137.stdout:0/166: dwrite d1/f9 [0,4194304] 0 2024-06-15T01:06:56.660 INFO:tasks.workunit.client.0.smithi137.stdout:7/183: dwrite df/d13/d2e/f31 [0,4194304] 0 2024-06-15T01:06:56.662 INFO:tasks.workunit.client.1.smithi142.stdout:3/284: mkdir d0/d3/d15/d4d/d55 0 2024-06-15T01:06:56.664 INFO:tasks.workunit.client.1.smithi142.stdout:9/200: rmdir d6/dc/d1b 39 2024-06-15T01:06:56.664 INFO:tasks.workunit.client.0.smithi137.stdout:0/167: creat d1/d2/dc/f3b x:0 0 0 2024-06-15T01:06:56.670 INFO:tasks.workunit.client.1.smithi142.stdout:1/237: getdents d6/d12/d1c 0 2024-06-15T01:06:56.670 INFO:tasks.workunit.client.1.smithi142.stdout:4/307: creat d5/d9/d18/f6e x:0 0 0 2024-06-15T01:06:56.671 INFO:tasks.workunit.client.0.smithi137.stdout:7/184: rename fe to df/f38 0 2024-06-15T01:06:56.672 INFO:tasks.workunit.client.1.smithi142.stdout:9/201: dread d6/dc/f15 [0,4194304] 0 2024-06-15T01:06:56.673 INFO:tasks.workunit.client.0.smithi137.stdout:7/185: rename df/d18/d1c/d1d/d27/d33 to df/d18/d1c/d1d/d27/d33/d39 22 2024-06-15T01:06:56.675 INFO:tasks.workunit.client.0.smithi137.stdout:0/168: mknod d1/d2/dc/d10/d14/d15/d1c/c3c 0 2024-06-15T01:06:56.677 INFO:tasks.workunit.client.1.smithi142.stdout:0/190: read d3/d5/fb [2024941,130343] 0 2024-06-15T01:06:56.677 INFO:tasks.workunit.client.1.smithi142.stdout:1/238: readlink d6/d46/l4c 0 2024-06-15T01:06:56.678 INFO:tasks.workunit.client.1.smithi142.stdout:2/198: getdents d8/d19/d23/d25 0 2024-06-15T01:06:56.679 INFO:tasks.workunit.client.0.smithi137.stdout:7/186: rename df/d18/d1c/d1d/d27/d2d to df/d18/d1c/d3a 0 2024-06-15T01:06:56.681 INFO:tasks.workunit.client.0.smithi137.stdout:0/169: mknod d1/d2/dc/d10/d14/d15/d1c/c3d 0 2024-06-15T01:06:56.682 INFO:tasks.workunit.client.1.smithi142.stdout:2/199: dread - d8/df/d10/d1c/f2c zero size 2024-06-15T01:06:56.682 INFO:tasks.workunit.client.1.smithi142.stdout:0/191: mkdir d3/d5/d44 0 2024-06-15T01:06:56.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/187: mkdir df/d18/d3b 0 2024-06-15T01:06:56.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/188: dread - df/f32 zero size 2024-06-15T01:06:56.685 INFO:tasks.workunit.client.0.smithi137.stdout:0/170: mkdir d1/d2/d3e 0 2024-06-15T01:06:56.685 INFO:tasks.workunit.client.1.smithi142.stdout:7/183: dwrite d2/f3 [0,4194304] 0 2024-06-15T01:06:56.686 INFO:tasks.workunit.client.1.smithi142.stdout:5/241: dwrite d2/de/d19/f1a [0,4194304] 0 2024-06-15T01:06:56.687 INFO:tasks.workunit.client.0.smithi137.stdout:7/189: mkdir df/d13/d3c 0 2024-06-15T01:06:56.687 INFO:tasks.workunit.client.0.smithi137.stdout:7/190: stat c8 0 2024-06-15T01:06:56.687 INFO:tasks.workunit.client.0.smithi137.stdout:7/191: readlink df/l17 0 2024-06-15T01:06:56.687 INFO:tasks.workunit.client.1.smithi142.stdout:1/239: truncate d6/d12/d1c/d1e/f4b 165346 0 2024-06-15T01:06:56.688 INFO:tasks.workunit.client.0.smithi137.stdout:7/192: chown df/c12 301990983 1 2024-06-15T01:06:56.688 INFO:tasks.workunit.client.1.smithi142.stdout:0/192: creat d3/d39/f45 x:0 0 0 2024-06-15T01:06:56.690 INFO:tasks.workunit.client.0.smithi137.stdout:0/171: dread d1/d2/dc/d10/d14/d15/f19 [0,4194304] 0 2024-06-15T01:06:56.691 INFO:tasks.workunit.client.0.smithi137.stdout:0/172: readlink d1/d2/dc/d10/d14/l23 0 2024-06-15T01:06:56.691 INFO:tasks.workunit.client.1.smithi142.stdout:2/200: mkdir d8/d19/d33 0 2024-06-15T01:06:56.691 INFO:tasks.workunit.client.1.smithi142.stdout:5/242: stat d2/ca 0 2024-06-15T01:06:56.691 INFO:tasks.workunit.client.1.smithi142.stdout:7/184: readlink d2/d6/l3b 0 2024-06-15T01:06:56.696 INFO:tasks.workunit.client.1.smithi142.stdout:5/243: mknod d2/d18/c4c 0 2024-06-15T01:06:56.696 INFO:tasks.workunit.client.0.smithi137.stdout:0/173: dread d1/d2/dc/d10/d14/d15/d1c/f1d [0,4194304] 0 2024-06-15T01:06:56.699 INFO:tasks.workunit.client.0.smithi137.stdout:7/193: unlink f5 0 2024-06-15T01:06:56.700 INFO:tasks.workunit.client.1.smithi142.stdout:2/201: write d8/d19/d23/d25/f27 [881236,24260] 0 2024-06-15T01:06:56.700 INFO:tasks.workunit.client.1.smithi142.stdout:4/308: dwrite d5/d9/d23/f4e [0,4194304] 0 2024-06-15T01:06:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:0/174: link d1/f32 d1/d2/d34/f3f 0 2024-06-15T01:06:56.704 INFO:tasks.workunit.client.1.smithi142.stdout:2/202: write f6 [290102,109313] 0 2024-06-15T01:06:56.705 INFO:tasks.workunit.client.1.smithi142.stdout:7/185: truncate d2/d6/d22/d34/f36 698011 0 2024-06-15T01:06:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:0/175: creat d1/d2/d13/f40 x:0 0 0 2024-06-15T01:06:56.709 INFO:tasks.workunit.client.0.smithi137.stdout:7/194: dwrite df/d18/d1c/d3a/f37 [0,4194304] 0 2024-06-15T01:06:56.709 INFO:tasks.workunit.client.0.smithi137.stdout:0/176: write d1/d2/dc/d10/d14/f1b [2524524,775] 0 2024-06-15T01:06:56.716 INFO:tasks.workunit.client.1.smithi142.stdout:1/240: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:06:56.719 INFO:tasks.workunit.client.1.smithi142.stdout:3/285: dwrite d0/f39 [0,4194304] 0 2024-06-15T01:06:56.719 INFO:tasks.workunit.client.0.smithi137.stdout:7/195: creat df/f3d x:0 0 0 2024-06-15T01:06:56.720 INFO:tasks.workunit.client.1.smithi142.stdout:0/193: dwrite d3/d5/f17 [0,4194304] 0 2024-06-15T01:06:56.723 INFO:tasks.workunit.client.1.smithi142.stdout:1/241: write d6/f13 [911906,17488] 0 2024-06-15T01:06:56.725 INFO:tasks.workunit.client.0.smithi137.stdout:7/196: rename df/l17 to df/d13/d2e/l3e 0 2024-06-15T01:06:56.725 INFO:tasks.workunit.client.1.smithi142.stdout:1/242: fdatasync d6/d12/d3c/f54 0 2024-06-15T01:06:56.727 INFO:tasks.workunit.client.1.smithi142.stdout:4/309: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:56.728 INFO:tasks.workunit.client.1.smithi142.stdout:8/191: sync 2024-06-15T01:06:56.730 INFO:tasks.workunit.client.1.smithi142.stdout:7/186: rename d2/d8/da/d33/d25/c1f to d2/d8/da/c41 0 2024-06-15T01:06:56.734 INFO:tasks.workunit.client.1.smithi142.stdout:1/243: unlink d6/d12/d1c/d2c/d47/f55 0 2024-06-15T01:06:56.736 INFO:tasks.workunit.client.1.smithi142.stdout:0/194: dread d3/f23 [0,4194304] 0 2024-06-15T01:06:56.738 INFO:tasks.workunit.client.0.smithi137.stdout:7/197: dread df/d13/d2e/f19 [0,4194304] 0 2024-06-15T01:06:56.740 INFO:tasks.workunit.client.0.smithi137.stdout:7/198: fdatasync df/f14 0 2024-06-15T01:06:56.741 INFO:tasks.workunit.client.1.smithi142.stdout:7/187: creat d2/d8/f42 x:0 0 0 2024-06-15T01:06:56.742 INFO:tasks.workunit.client.1.smithi142.stdout:3/286: symlink d0/d3/d15/d4d/d55/l56 0 2024-06-15T01:06:56.743 INFO:tasks.workunit.client.1.smithi142.stdout:3/287: chown d0/f3c 2320 1 2024-06-15T01:06:56.744 INFO:tasks.workunit.client.1.smithi142.stdout:4/310: mkdir d5/d1c/d6f 0 2024-06-15T01:06:56.745 INFO:tasks.workunit.client.1.smithi142.stdout:1/244: creat d6/d46/f56 x:0 0 0 2024-06-15T01:06:56.746 INFO:tasks.workunit.client.1.smithi142.stdout:5/244: dread d2/de/f1f [0,4194304] 0 2024-06-15T01:06:56.749 INFO:tasks.workunit.client.1.smithi142.stdout:0/195: symlink d3/l46 0 2024-06-15T01:06:56.749 INFO:tasks.workunit.client.1.smithi142.stdout:7/188: dread d2/d8/da/d33/f20 [0,4194304] 0 2024-06-15T01:06:56.750 INFO:tasks.workunit.client.1.smithi142.stdout:5/245: fdatasync d2/ff 0 2024-06-15T01:06:56.750 INFO:tasks.workunit.client.1.smithi142.stdout:4/311: chown d5/d1c/d2a/d30/d46/d50 263 1 2024-06-15T01:06:56.750 INFO:tasks.workunit.client.1.smithi142.stdout:3/288: creat d0/d3/d15/d4d/d55/f57 x:0 0 0 2024-06-15T01:06:56.752 INFO:tasks.workunit.client.1.smithi142.stdout:3/289: chown d0/f39 1874000 1 2024-06-15T01:06:56.753 INFO:tasks.workunit.client.1.smithi142.stdout:5/246: mknod d2/de/c4d 0 2024-06-15T01:06:56.753 INFO:tasks.workunit.client.1.smithi142.stdout:8/192: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:06:56.755 INFO:tasks.workunit.client.1.smithi142.stdout:0/196: symlink d3/d12/l47 0 2024-06-15T01:06:56.756 INFO:tasks.workunit.client.1.smithi142.stdout:8/193: stat d0/f5 0 2024-06-15T01:06:56.756 INFO:tasks.workunit.client.1.smithi142.stdout:4/312: mknod d5/d1c/c70 0 2024-06-15T01:06:56.756 INFO:tasks.workunit.client.1.smithi142.stdout:5/247: mknod d2/d3e/d31/c4e 0 2024-06-15T01:06:56.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 -- 172.21.15.137:0/280714941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 msgr2=0x7f5618103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 --2- 172.21.15.137:0/280714941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618103a90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5614009910 tx=0x7f5614009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:56.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 -- 172.21.15.137:0/280714941 shutdown_connections 2024-06-15T01:06:56.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 --2- 172.21.15.137:0/280714941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 0x7f56181063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 --2- 172.21.15.137:0/280714941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618103a90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 -- 172.21.15.137:0/280714941 >> 172.21.15.137:0/280714941 conn(0x7f56180fb030 msgr2=0x7f56180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:56.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 -- 172.21.15.137:0/280714941 shutdown_connections 2024-06-15T01:06:56.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.785+0000 7f5620518700 1 -- 172.21.15.137:0/280714941 wait complete. 2024-06-15T01:06:56.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 Processor -- start 2024-06-15T01:06:56.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 -- start start 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 0x7f5618196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5618196da0 con 0x7f56181016a0 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f5620518700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5618196ee0 con 0x7f5618103fd0 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41906/0 (socket says 172.21.15.137:41906) 2024-06-15T01:06:56.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 -- 172.21.15.137:0/4092315300 learned_addr learned my addr 172.21.15.137:0/4092315300 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561dab3700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 0x7f5618196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 -- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 msgr2=0x7f5618196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 0x7f5618196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 -- 172.21.15.137:0/4092315300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140095f0 con 0x7f56181016a0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.786+0000 7f561e2b4700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f561400bde0 tx=0x7f561400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.787+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561401d070 con 0x7f56181016a0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.787+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5614005dc0 con 0x7f56181016a0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.787+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561819b930 con 0x7f56181016a0 2024-06-15T01:06:56.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.787+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5614003b10 con 0x7f56181016a0 2024-06-15T01:06:56.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.787+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561819be40 con 0x7f56181016a0 2024-06-15T01:06:56.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.788+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618190400 con 0x7f56181016a0 2024-06-15T01:06:56.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.788+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f56140048e0 con 0x7f56181016a0 2024-06-15T01:06:56.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.789+0000 7f560f7fe700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 0x7f560406e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:56.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.789+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f561408c700 con 0x7f56181016a0 2024-06-15T01:06:56.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.789+0000 7f561dab3700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 0x7f560406e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:56.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.789+0000 7f561dab3700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 0x7f560406e030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5608009bf0 tx=0x7f5608009260 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:56.790 INFO:tasks.workunit.client.1.smithi142.stdout:4/313: chown d5/d9/d18/l53 1869111071 1 2024-06-15T01:06:56.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:56.793+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f561405b0b0 con 0x7f56181016a0 2024-06-15T01:06:56.792 INFO:tasks.workunit.client.1.smithi142.stdout:2/203: dread d8/f9 [0,4194304] 0 2024-06-15T01:06:56.792 INFO:tasks.workunit.client.1.smithi142.stdout:5/248: unlink d2/d3e/d31/c4e 0 2024-06-15T01:06:56.793 INFO:tasks.workunit.client.1.smithi142.stdout:4/314: readlink d5/d9/d23/l2c 0 2024-06-15T01:06:56.793 INFO:tasks.workunit.client.1.smithi142.stdout:0/197: creat d3/f48 x:0 0 0 2024-06-15T01:06:56.795 INFO:tasks.workunit.client.1.smithi142.stdout:5/249: readlink d2/de/l4b 0 2024-06-15T01:06:56.795 INFO:tasks.workunit.client.1.smithi142.stdout:3/290: dwrite d0/d1f/f4b [0,4194304] 0 2024-06-15T01:06:56.796 INFO:tasks.workunit.client.1.smithi142.stdout:0/198: fdatasync d3/f48 0 2024-06-15T01:06:56.797 INFO:tasks.workunit.client.1.smithi142.stdout:2/204: chown d8/df/d10/f11 20937268 1 2024-06-15T01:06:56.798 INFO:tasks.workunit.client.0.smithi137.stdout:0/177: dwrite d1/f32 [0,4194304] 0 2024-06-15T01:06:56.801 INFO:tasks.workunit.client.0.smithi137.stdout:7/199: mkdir df/d13/d2e/d1a/d3f 0 2024-06-15T01:06:56.802 INFO:tasks.workunit.client.1.smithi142.stdout:4/315: symlink d5/d9/l71 0 2024-06-15T01:06:56.802 INFO:tasks.workunit.client.1.smithi142.stdout:1/245: dwrite d6/d12/f2d [0,4194304] 0 2024-06-15T01:06:56.804 INFO:tasks.workunit.client.0.smithi137.stdout:0/178: rmdir d1/d2/dc 39 2024-06-15T01:06:56.805 INFO:tasks.workunit.client.1.smithi142.stdout:4/316: dread - d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f61 zero size 2024-06-15T01:06:56.807 INFO:tasks.workunit.client.0.smithi137.stdout:0/179: creat d1/d2/d34/f41 x:0 0 0 2024-06-15T01:06:56.808 INFO:tasks.workunit.client.1.smithi142.stdout:3/291: rename d0/d3/c47 to d0/d3/d15/c58 0 2024-06-15T01:06:56.809 INFO:tasks.workunit.client.0.smithi137.stdout:7/200: truncate f1 1269537 0 2024-06-15T01:06:56.810 INFO:tasks.workunit.client.1.smithi142.stdout:8/194: dread d0/f14 [0,4194304] 0 2024-06-15T01:06:56.810 INFO:tasks.workunit.client.1.smithi142.stdout:0/199: creat d3/d5/d44/f49 x:0 0 0 2024-06-15T01:06:56.811 INFO:tasks.workunit.client.1.smithi142.stdout:5/250: rmdir d2 39 2024-06-15T01:06:56.811 INFO:tasks.workunit.client.1.smithi142.stdout:0/200: dread - d3/d5/d1e/f2b zero size 2024-06-15T01:06:56.813 INFO:tasks.workunit.client.1.smithi142.stdout:8/195: write d0/f15 [316096,9926] 0 2024-06-15T01:06:56.813 INFO:tasks.workunit.client.1.smithi142.stdout:4/317: dread d5/f7 [0,4194304] 0 2024-06-15T01:06:56.814 INFO:tasks.workunit.client.1.smithi142.stdout:8/196: chown d0/d25/c2b 931700947 1 2024-06-15T01:06:56.814 INFO:tasks.workunit.client.0.smithi137.stdout:0/180: mknod d1/d2/dc/d10/c42 0 2024-06-15T01:06:56.815 INFO:tasks.workunit.client.1.smithi142.stdout:2/205: mkdir d8/d19/d23/d25/d22/d34 0 2024-06-15T01:06:56.817 INFO:tasks.workunit.client.0.smithi137.stdout:0/181: stat d1/d2/dc/d10/d14/d15/d1c/c3c 0 2024-06-15T01:06:56.818 INFO:tasks.workunit.client.1.smithi142.stdout:3/292: mkdir d0/d1f/d20/d59 0 2024-06-15T01:06:56.819 INFO:tasks.workunit.client.0.smithi137.stdout:7/201: dwrite df/d18/d1c/d1d/f28 [0,4194304] 0 2024-06-15T01:06:56.821 INFO:tasks.workunit.client.0.smithi137.stdout:0/182: mknod d1/d2/c43 0 2024-06-15T01:06:56.823 INFO:tasks.workunit.client.0.smithi137.stdout:7/202: creat df/d13/d2e/d1a/f40 x:0 0 0 2024-06-15T01:06:56.824 INFO:tasks.workunit.client.1.smithi142.stdout:8/197: dwrite d0/f20 [0,4194304] 0 2024-06-15T01:06:56.824 INFO:tasks.workunit.client.1.smithi142.stdout:3/293: mknod d0/d1f/d20/c5a 0 2024-06-15T01:06:56.831 INFO:tasks.workunit.client.0.smithi137.stdout:0/183: dwrite d1/d2/dc/d10/d14/f39 [0,4194304] 0 2024-06-15T01:06:56.833 INFO:tasks.workunit.client.0.smithi137.stdout:2/104: sync 2024-06-15T01:06:56.833 INFO:tasks.workunit.client.0.smithi137.stdout:0/184: read - d1/d2/dc/f3b zero size 2024-06-15T01:06:56.833 INFO:tasks.workunit.client.1.smithi142.stdout:3/294: creat d0/d1b/f5b x:0 0 0 2024-06-15T01:06:56.834 INFO:tasks.workunit.client.0.smithi137.stdout:0/185: write d1/d2/d34/f3f [3325705,72661] 0 2024-06-15T01:06:56.836 INFO:tasks.workunit.client.0.smithi137.stdout:7/203: rename df/d13/l16 to df/d13/d3c/l41 0 2024-06-15T01:06:56.838 INFO:tasks.workunit.client.1.smithi142.stdout:4/318: rename d5/d1c/d2a/l2b to d5/d1c/d2a/d30/d46/d50/d51/d5b/l72 0 2024-06-15T01:06:56.841 INFO:tasks.workunit.client.0.smithi137.stdout:2/105: mkdir d0/dc/d22 0 2024-06-15T01:06:56.841 INFO:tasks.workunit.client.1.smithi142.stdout:2/206: link d8/df/d2a/f2b d8/df/d10/f35 0 2024-06-15T01:06:56.841 INFO:tasks.workunit.client.1.smithi142.stdout:2/207: stat l0 0 2024-06-15T01:06:56.842 INFO:tasks.workunit.client.0.smithi137.stdout:2/106: chown d0 647 1 2024-06-15T01:06:56.842 INFO:tasks.workunit.client.0.smithi137.stdout:2/107: chown d0/dc 0 1 2024-06-15T01:06:56.843 INFO:tasks.workunit.client.0.smithi137.stdout:0/186: symlink d1/d2/dc/d10/l44 0 2024-06-15T01:06:56.844 INFO:tasks.workunit.client.1.smithi142.stdout:0/201: rmdir d3/d5 39 2024-06-15T01:06:56.844 INFO:tasks.workunit.client.1.smithi142.stdout:1/246: dread d6/d12/d1c/f32 [0,4194304] 0 2024-06-15T01:06:56.846 INFO:tasks.workunit.client.0.smithi137.stdout:7/204: mknod df/c42 0 2024-06-15T01:06:56.846 INFO:tasks.workunit.client.1.smithi142.stdout:3/295: rmdir d0/d1f/d20 39 2024-06-15T01:06:56.847 INFO:tasks.workunit.client.0.smithi137.stdout:2/108: truncate d0/da/f17 960933 0 2024-06-15T01:06:56.847 INFO:tasks.workunit.client.0.smithi137.stdout:2/109: chown d0 15671256 1 2024-06-15T01:06:56.848 INFO:tasks.workunit.client.1.smithi142.stdout:3/296: write d0/d3/f1a [3557147,120237] 0 2024-06-15T01:06:56.848 INFO:tasks.workunit.client.0.smithi137.stdout:0/187: mknod d1/d2/d3e/c45 0 2024-06-15T01:06:56.848 INFO:tasks.workunit.client.1.smithi142.stdout:4/319: symlink d5/d1c/l73 0 2024-06-15T01:06:56.851 INFO:tasks.workunit.client.0.smithi137.stdout:7/205: mkdir df/d18/d1c/d1d/d27/d33/d43 0 2024-06-15T01:06:56.851 INFO:tasks.workunit.client.1.smithi142.stdout:6/203: sync 2024-06-15T01:06:56.852 INFO:tasks.workunit.client.0.smithi137.stdout:2/110: mkdir d0/dc/d22/d23 0 2024-06-15T01:06:56.852 INFO:tasks.workunit.client.0.smithi137.stdout:7/206: dread - df/f3d zero size 2024-06-15T01:06:56.852 INFO:tasks.workunit.client.1.smithi142.stdout:6/204: dread - d1a/f46 zero size 2024-06-15T01:06:56.854 INFO:tasks.workunit.client.1.smithi142.stdout:1/247: unlink d6/d12/d1c/d1e/f24 0 2024-06-15T01:06:56.855 INFO:tasks.workunit.client.1.smithi142.stdout:8/198: creat d0/f43 x:0 0 0 2024-06-15T01:06:56.855 INFO:tasks.workunit.client.0.smithi137.stdout:0/188: link d1/d2/d34/f3f d1/d2/dc/d10/d14/d15/d1c/f46 0 2024-06-15T01:06:56.859 INFO:tasks.workunit.client.1.smithi142.stdout:4/320: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f74 x:0 0 0 2024-06-15T01:06:56.859 INFO:tasks.workunit.client.0.smithi137.stdout:7/207: rename df/d18/d1c/d3a/l36 to df/d13/l44 0 2024-06-15T01:06:56.860 INFO:tasks.workunit.client.0.smithi137.stdout:7/208: chown df/d13/d2e/f2b 213641 1 2024-06-15T01:06:56.861 INFO:tasks.workunit.client.0.smithi137.stdout:2/111: chown d0/da/f17 3931 1 2024-06-15T01:06:56.862 INFO:tasks.workunit.client.1.smithi142.stdout:8/199: dwrite d0/f15 [0,4194304] 0 2024-06-15T01:06:56.864 INFO:tasks.workunit.client.1.smithi142.stdout:0/202: creat d3/d5/d44/f4a x:0 0 0 2024-06-15T01:06:56.866 INFO:tasks.workunit.client.1.smithi142.stdout:5/251: truncate d2/f14 3212988 0 2024-06-15T01:06:56.867 INFO:tasks.workunit.client.1.smithi142.stdout:4/321: dwrite d5/d9/da/f54 [0,4194304] 0 2024-06-15T01:06:56.868 INFO:tasks.workunit.client.0.smithi137.stdout:0/189: creat d1/d2/f47 x:0 0 0 2024-06-15T01:06:56.868 INFO:tasks.workunit.client.1.smithi142.stdout:6/205: write d1a/f3e [928209,114998] 0 2024-06-15T01:06:56.870 INFO:tasks.workunit.client.1.smithi142.stdout:4/322: truncate d5/d9/d23/f3b 4215791 0 2024-06-15T01:06:56.871 INFO:tasks.workunit.client.0.smithi137.stdout:7/209: rename df/d13/d2e/c21 to df/c45 0 2024-06-15T01:06:56.872 INFO:tasks.workunit.client.1.smithi142.stdout:4/323: write d5/d9/d41/f40 [473513,12881] 0 2024-06-15T01:06:56.873 INFO:tasks.workunit.client.0.smithi137.stdout:0/190: symlink d1/d2/d34/l48 0 2024-06-15T01:06:56.874 INFO:tasks.workunit.client.1.smithi142.stdout:2/208: link d8/d19/l1b d8/d19/l36 0 2024-06-15T01:06:56.874 INFO:tasks.workunit.client.1.smithi142.stdout:2/209: rename d8 to d8/d37 22 2024-06-15T01:06:56.875 INFO:tasks.workunit.client.1.smithi142.stdout:0/203: creat d3/d5/d44/f4b x:0 0 0 2024-06-15T01:06:56.878 INFO:tasks.workunit.client.1.smithi142.stdout:6/206: unlink f8 0 2024-06-15T01:06:56.881 INFO:tasks.workunit.client.1.smithi142.stdout:1/248: link d6/d12/c36 d6/d3a/c57 0 2024-06-15T01:06:56.881 INFO:tasks.workunit.client.0.smithi137.stdout:0/191: dwrite d1/f9 [0,4194304] 0 2024-06-15T01:06:56.884 INFO:tasks.workunit.client.1.smithi142.stdout:2/210: mknod d8/df/c38 0 2024-06-15T01:06:56.885 INFO:tasks.workunit.client.1.smithi142.stdout:0/204: write d3/d12/d22/f2d [130490,79678] 0 2024-06-15T01:06:56.885 INFO:tasks.workunit.client.0.smithi137.stdout:2/112: dwrite d0/de/f1a [4194304,4194304] 0 2024-06-15T01:06:56.886 INFO:tasks.workunit.client.1.smithi142.stdout:8/200: getdents d0/d25 0 2024-06-15T01:06:56.891 INFO:tasks.workunit.client.0.smithi137.stdout:2/113: creat d0/de/f24 x:0 0 0 2024-06-15T01:06:56.892 INFO:tasks.workunit.client.0.smithi137.stdout:0/192: link d1/d2/dc/d10/d14/l27 d1/d2/l49 0 2024-06-15T01:06:56.892 INFO:tasks.workunit.client.1.smithi142.stdout:1/249: mknod d6/d46/c58 0 2024-06-15T01:06:56.892 INFO:tasks.workunit.client.1.smithi142.stdout:5/252: link d2/d3e/d31/c4a d2/c4f 0 2024-06-15T01:06:56.894 INFO:tasks.workunit.client.1.smithi142.stdout:0/205: dread - d3/d5/d44/f4a zero size 2024-06-15T01:06:56.895 INFO:tasks.workunit.client.1.smithi142.stdout:4/324: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:56.896 INFO:tasks.workunit.client.1.smithi142.stdout:8/201: rename d0/f41 to d0/d17/f44 0 2024-06-15T01:06:56.897 INFO:tasks.workunit.client.1.smithi142.stdout:0/206: readlink d3/d5/d1b/l1f 0 2024-06-15T01:06:56.899 INFO:tasks.workunit.client.1.smithi142.stdout:6/207: dwrite d1a/f46 [0,4194304] 0 2024-06-15T01:06:56.899 INFO:tasks.workunit.client.0.smithi137.stdout:2/114: dwrite d0/de/d10/f19 [0,4194304] 0 2024-06-15T01:06:56.900 INFO:tasks.workunit.client.0.smithi137.stdout:2/115: write d0/f1f [1044024,24465] 0 2024-06-15T01:06:56.901 INFO:tasks.workunit.client.1.smithi142.stdout:5/253: readlink d2/d18/l29 0 2024-06-15T01:06:56.902 INFO:tasks.workunit.client.1.smithi142.stdout:3/297: truncate d0/d3/da/d18/f28 3327385 0 2024-06-15T01:06:56.904 INFO:tasks.workunit.client.0.smithi137.stdout:2/116: rename d0/d1c/f1d to d0/dc/d22/f25 0 2024-06-15T01:06:56.905 INFO:tasks.workunit.client.0.smithi137.stdout:2/117: stat d0/dc/c14 0 2024-06-15T01:06:56.905 INFO:tasks.workunit.client.1.smithi142.stdout:3/298: write d0/d3/d15/d4d/d55/f57 [123720,54303] 0 2024-06-15T01:06:56.906 INFO:tasks.workunit.client.0.smithi137.stdout:2/118: dread - d0/da/f18 zero size 2024-06-15T01:06:56.906 INFO:tasks.workunit.client.1.smithi142.stdout:1/250: creat d6/d12/d1c/d2c/f59 x:0 0 0 2024-06-15T01:06:56.912 INFO:tasks.workunit.client.1.smithi142.stdout:0/207: stat d3/d12/c31 0 2024-06-15T01:06:56.914 INFO:tasks.workunit.client.1.smithi142.stdout:0/208: chown d3/d5/d44/f4b 422396 1 2024-06-15T01:06:56.915 INFO:tasks.workunit.client.1.smithi142.stdout:6/208: readlink l11 0 2024-06-15T01:06:56.916 INFO:tasks.workunit.client.1.smithi142.stdout:4/325: fsync d5/d1c/d2a/f3d 0 2024-06-15T01:06:56.919 INFO:tasks.workunit.client.1.smithi142.stdout:1/251: creat d6/d12/d1c/d2c/d47/f5a x:0 0 0 2024-06-15T01:06:56.919 INFO:tasks.workunit.client.1.smithi142.stdout:9/202: sync 2024-06-15T01:06:56.922 INFO:tasks.workunit.client.1.smithi142.stdout:6/209: fsync d1a/f1e 0 2024-06-15T01:06:56.922 INFO:tasks.workunit.client.1.smithi142.stdout:8/202: read d0/d25/f35 [1100744,1782] 0 2024-06-15T01:06:56.927 INFO:tasks.workunit.client.1.smithi142.stdout:6/210: chown d1a/f25 1 1 2024-06-15T01:06:56.929 INFO:tasks.workunit.client.1.smithi142.stdout:5/254: dread d2/de/d27/f46 [0,4194304] 0 2024-06-15T01:06:56.929 INFO:tasks.workunit.client.1.smithi142.stdout:3/299: dwrite d0/d1b/f5b [0,4194304] 0 2024-06-15T01:06:56.930 INFO:tasks.workunit.client.1.smithi142.stdout:1/252: chown d6/d12/d1c/d2c/d47/f5a 5725 1 2024-06-15T01:06:56.931 INFO:tasks.workunit.client.1.smithi142.stdout:8/203: chown d0/f5 472597 1 2024-06-15T01:06:56.932 INFO:tasks.workunit.client.1.smithi142.stdout:0/209: getdents d3/d12 0 2024-06-15T01:06:56.932 INFO:tasks.workunit.client.1.smithi142.stdout:6/211: write d1a/f46 [659933,129588] 0 2024-06-15T01:06:56.935 INFO:tasks.workunit.client.1.smithi142.stdout:9/203: mknod d6/d34/c50 0 2024-06-15T01:06:56.935 INFO:tasks.workunit.client.1.smithi142.stdout:5/255: creat d2/de/f50 x:0 0 0 2024-06-15T01:06:56.935 INFO:tasks.workunit.client.1.smithi142.stdout:0/210: readlink d3/d5/d1e/l27 0 2024-06-15T01:06:56.936 INFO:tasks.workunit.client.1.smithi142.stdout:2/211: dread d8/fa [0,4194304] 0 2024-06-15T01:06:56.938 INFO:tasks.workunit.client.1.smithi142.stdout:3/300: mknod d0/d3/d15/c5c 0 2024-06-15T01:06:56.939 INFO:tasks.workunit.client.1.smithi142.stdout:0/211: mkdir d3/d5/d4c 0 2024-06-15T01:06:56.939 INFO:tasks.workunit.client.1.smithi142.stdout:8/204: mknod d0/c45 0 2024-06-15T01:06:56.941 INFO:tasks.workunit.client.1.smithi142.stdout:3/301: write d0/f3c [595251,126181] 0 2024-06-15T01:06:56.941 INFO:tasks.workunit.client.1.smithi142.stdout:2/212: readlink d8/d19/l1b 0 2024-06-15T01:06:56.942 INFO:tasks.workunit.client.1.smithi142.stdout:3/302: read d0/d3/f1a [3888878,52318] 0 2024-06-15T01:06:56.943 INFO:tasks.workunit.client.1.smithi142.stdout:6/212: truncate d1a/f2c 2585332 0 2024-06-15T01:06:56.943 INFO:tasks.workunit.client.1.smithi142.stdout:3/303: stat d0/d1b/l24 0 2024-06-15T01:06:56.944 INFO:tasks.workunit.client.0.smithi137.stdout:0/193: truncate d1/d2/dc/d10/d14/f1b 2207488 0 2024-06-15T01:06:56.944 INFO:tasks.workunit.client.1.smithi142.stdout:0/212: readlink d3/l41 0 2024-06-15T01:06:56.945 INFO:tasks.workunit.client.0.smithi137.stdout:0/194: chown d1/d2/d13/f40 550864 1 2024-06-15T01:06:56.945 INFO:tasks.workunit.client.1.smithi142.stdout:0/213: dread - d3/d5/d1b/f40 zero size 2024-06-15T01:06:56.946 INFO:tasks.workunit.client.0.smithi137.stdout:0/195: chown d1/l1a 7603218 1 2024-06-15T01:06:56.946 INFO:tasks.workunit.client.0.smithi137.stdout:2/119: truncate d0/fb 688901 0 2024-06-15T01:06:56.947 INFO:tasks.workunit.client.1.smithi142.stdout:9/204: mkdir d6/dc/d51 0 2024-06-15T01:06:56.948 INFO:tasks.workunit.client.0.smithi137.stdout:2/120: symlink d0/da/l26 0 2024-06-15T01:06:56.949 INFO:tasks.workunit.client.1.smithi142.stdout:9/205: rename d6/dc/d51 to d6/dc/d51/d52 22 2024-06-15T01:06:56.952 INFO:tasks.workunit.client.0.smithi137.stdout:2/121: unlink d0/dc/f20 0 2024-06-15T01:06:56.952 INFO:tasks.workunit.client.1.smithi142.stdout:2/213: mknod d8/df/d1f/c39 0 2024-06-15T01:06:56.952 INFO:tasks.workunit.client.1.smithi142.stdout:2/214: read d8/f9 [1733923,96988] 0 2024-06-15T01:06:56.955 INFO:tasks.workunit.client.0.smithi137.stdout:0/196: dwrite d1/d2/d13/f33 [0,4194304] 0 2024-06-15T01:06:56.957 INFO:tasks.workunit.client.1.smithi142.stdout:0/214: symlink d3/d5/d44/l4d 0 2024-06-15T01:06:56.964 INFO:tasks.workunit.client.1.smithi142.stdout:2/215: mkdir d8/d19/d23/d3a 0 2024-06-15T01:06:56.966 INFO:tasks.workunit.client.1.smithi142.stdout:6/213: symlink d1a/d39/d42/d4c/l4d 0 2024-06-15T01:06:56.966 INFO:tasks.workunit.client.1.smithi142.stdout:4/326: write d5/d1c/d2a/f3d [1620124,96221] 0 2024-06-15T01:06:56.967 INFO:tasks.workunit.client.1.smithi142.stdout:3/304: mkdir d0/d1f/d20/d5d 0 2024-06-15T01:06:56.967 INFO:tasks.workunit.client.1.smithi142.stdout:7/189: sync 2024-06-15T01:06:56.970 INFO:tasks.workunit.client.1.smithi142.stdout:0/215: rename d3/l41 to d3/d5/d1b/l4e 0 2024-06-15T01:06:56.970 INFO:tasks.workunit.client.0.smithi137.stdout:0/197: truncate d1/d2/dc/d10/d14/d15/d1c/f46 2114649 0 2024-06-15T01:06:56.970 INFO:tasks.workunit.client.1.smithi142.stdout:7/190: write d2/d6/f7 [422642,34599] 0 2024-06-15T01:06:56.971 INFO:tasks.workunit.client.1.smithi142.stdout:6/214: chown f18 597717 1 2024-06-15T01:06:56.971 INFO:tasks.workunit.client.0.smithi137.stdout:0/198: read - d1/d2/dc/d10/f25 zero size 2024-06-15T01:06:56.972 INFO:tasks.workunit.client.0.smithi137.stdout:2/122: dwrite d0/f3 [4194304,4194304] 0 2024-06-15T01:06:56.972 INFO:tasks.workunit.client.0.smithi137.stdout:0/199: readlink d1/l1a 0 2024-06-15T01:06:56.972 INFO:tasks.workunit.client.1.smithi142.stdout:8/205: rename d0/f5 to d0/f46 0 2024-06-15T01:06:56.973 INFO:tasks.workunit.client.1.smithi142.stdout:9/206: mkdir d6/dc/d51/d53 0 2024-06-15T01:06:56.973 INFO:tasks.workunit.client.0.smithi137.stdout:2/123: fdatasync d0/f9 0 2024-06-15T01:06:56.974 INFO:tasks.workunit.client.1.smithi142.stdout:1/253: fsync d6/d12/d1c/d2c/d47/f5a 0 2024-06-15T01:06:56.977 INFO:tasks.workunit.client.0.smithi137.stdout:0/200: creat d1/d2/d34/f4a x:0 0 0 2024-06-15T01:06:56.977 INFO:tasks.workunit.client.1.smithi142.stdout:5/256: dread d2/f12 [0,4194304] 0 2024-06-15T01:06:56.978 INFO:tasks.workunit.client.1.smithi142.stdout:8/206: write d0/d17/f1f [2404922,13276] 0 2024-06-15T01:06:56.978 INFO:tasks.workunit.client.0.smithi137.stdout:0/201: truncate d1/d2/d34/f41 735778 0 2024-06-15T01:06:56.978 INFO:tasks.workunit.client.0.smithi137.stdout:2/124: unlink d0/c2 0 2024-06-15T01:06:56.983 INFO:tasks.workunit.client.0.smithi137.stdout:2/125: rename d0/dc/d22/f25 to d0/de/f27 0 2024-06-15T01:06:56.983 INFO:tasks.workunit.client.1.smithi142.stdout:7/191: rename d2/f32 to d2/d6/f43 0 2024-06-15T01:06:56.983 INFO:tasks.workunit.client.1.smithi142.stdout:3/305: dread d0/d3/f3d [0,4194304] 0 2024-06-15T01:06:56.983 INFO:tasks.workunit.client.0.smithi137.stdout:2/126: dread - d0/de/f27 zero size 2024-06-15T01:06:56.984 INFO:tasks.workunit.client.1.smithi142.stdout:4/327: dwrite d5/d9/da/f3c [0,4194304] 0 2024-06-15T01:06:56.985 INFO:tasks.workunit.client.0.smithi137.stdout:0/202: creat d1/d2/dc/d10/d14/d15/f4b x:0 0 0 2024-06-15T01:06:56.988 INFO:tasks.workunit.client.1.smithi142.stdout:3/306: chown d0/d3/da/d18 1845831 1 2024-06-15T01:06:56.991 INFO:tasks.workunit.client.0.smithi137.stdout:2/127: link d0/de/d10/c13 d0/c28 0 2024-06-15T01:06:56.992 INFO:tasks.workunit.client.0.smithi137.stdout:2/128: fsync d0/de/f24 0 2024-06-15T01:06:56.994 INFO:tasks.workunit.client.1.smithi142.stdout:8/207: dwrite d0/f15 [0,4194304] 0 2024-06-15T01:06:56.995 INFO:tasks.workunit.client.1.smithi142.stdout:4/328: chown d5/d9/d18/l53 134021962 1 2024-06-15T01:06:56.995 INFO:tasks.workunit.client.0.smithi137.stdout:2/129: fdatasync d0/de/f16 0 2024-06-15T01:06:56.996 INFO:tasks.workunit.client.1.smithi142.stdout:2/216: creat d8/df/d10/f3b x:0 0 0 2024-06-15T01:06:56.998 INFO:tasks.workunit.client.1.smithi142.stdout:7/192: rmdir d2/d8/da/d33/d25 39 2024-06-15T01:06:56.998 INFO:tasks.workunit.client.1.smithi142.stdout:7/193: fdatasync d2/f1a 0 2024-06-15T01:06:56.999 INFO:tasks.workunit.client.0.smithi137.stdout:2/130: mknod d0/dc/d22/c29 0 2024-06-15T01:06:56.999 INFO:tasks.workunit.client.1.smithi142.stdout:9/207: dread d6/dc/dd/d22/f2b [0,4194304] 0 2024-06-15T01:06:57.000 INFO:tasks.workunit.client.1.smithi142.stdout:1/254: mkdir d6/d12/d5b 0 2024-06-15T01:06:57.001 INFO:tasks.workunit.client.1.smithi142.stdout:4/329: chown d5/d1c/f34 4 1 2024-06-15T01:06:57.001 INFO:tasks.workunit.client.0.smithi137.stdout:2/131: creat d0/de/d10/f2a x:0 0 0 2024-06-15T01:06:57.003 INFO:tasks.workunit.client.1.smithi142.stdout:9/208: fdatasync d6/dc/dd/f46 0 2024-06-15T01:06:57.005 INFO:tasks.workunit.client.1.smithi142.stdout:0/216: dwrite d3/f2a [0,4194304] 0 2024-06-15T01:06:57.007 INFO:tasks.workunit.client.0.smithi137.stdout:0/203: write d1/d2/f21 [1717577,24285] 0 2024-06-15T01:06:57.008 INFO:tasks.workunit.client.1.smithi142.stdout:0/217: fsync d3/d5/f17 0 2024-06-15T01:06:57.010 INFO:tasks.workunit.client.1.smithi142.stdout:6/215: getdents d1a 0 2024-06-15T01:06:57.012 INFO:tasks.workunit.client.0.smithi137.stdout:0/204: creat d1/d2/dc/d10/d14/d15/d17/f4c x:0 0 0 2024-06-15T01:06:57.013 INFO:tasks.workunit.client.1.smithi142.stdout:1/255: chown d6/d12/d30/c4a 403772 1 2024-06-15T01:06:57.015 INFO:tasks.workunit.client.0.smithi137.stdout:2/132: truncate d0/f8 2467950 0 2024-06-15T01:06:57.016 INFO:tasks.workunit.client.1.smithi142.stdout:4/330: creat d5/d1c/d2a/d30/d46/d50/d51/f75 x:0 0 0 2024-06-15T01:06:57.017 INFO:tasks.workunit.client.1.smithi142.stdout:9/209: creat d6/dc/dd/d22/f54 x:0 0 0 2024-06-15T01:06:57.018 INFO:tasks.workunit.client.0.smithi137.stdout:2/133: mkdir d0/de/d2b 0 2024-06-15T01:06:57.018 INFO:tasks.workunit.client.1.smithi142.stdout:5/257: write d2/d18/f37 [4410445,117293] 0 2024-06-15T01:06:57.020 INFO:tasks.workunit.client.1.smithi142.stdout:8/208: creat d0/f47 x:0 0 0 2024-06-15T01:06:57.021 INFO:tasks.workunit.client.0.smithi137.stdout:0/205: mknod d1/d2/dc/d10/d14/d15/d17/c4d 0 2024-06-15T01:06:57.022 INFO:tasks.workunit.client.0.smithi137.stdout:2/134: unlink d0/de/f27 0 2024-06-15T01:06:57.023 INFO:tasks.workunit.client.1.smithi142.stdout:6/216: mknod d1a/d39/d3a/c4e 0 2024-06-15T01:06:57.023 INFO:tasks.workunit.client.0.smithi137.stdout:2/135: creat d0/dc/f2c x:0 0 0 2024-06-15T01:06:57.024 INFO:tasks.workunit.client.0.smithi137.stdout:1/210: sync 2024-06-15T01:06:57.025 INFO:tasks.workunit.client.0.smithi137.stdout:3/203: sync 2024-06-15T01:06:57.025 INFO:tasks.workunit.client.0.smithi137.stdout:0/206: rename d1/d2/d3e/c45 to d1/d2/dc/d10/d14/d15/d1c/c4e 0 2024-06-15T01:06:57.026 INFO:tasks.workunit.client.0.smithi137.stdout:1/211: chown d0/d3/d14/f19 770302493 1 2024-06-15T01:06:57.027 INFO:tasks.workunit.client.0.smithi137.stdout:0/207: chown d1/c8 1906032 1 2024-06-15T01:06:57.028 INFO:tasks.workunit.client.1.smithi142.stdout:4/331: write f2 [222469,118699] 0 2024-06-15T01:06:57.029 INFO:tasks.workunit.client.0.smithi137.stdout:3/204: creat d0/d2/d23/f3f x:0 0 0 2024-06-15T01:06:57.030 INFO:tasks.workunit.client.1.smithi142.stdout:2/217: dwrite d8/df/f2d [0,4194304] 0 2024-06-15T01:06:57.030 INFO:tasks.workunit.client.1.smithi142.stdout:2/218: chown d8/df/d10/d1c 180497 1 2024-06-15T01:06:57.031 INFO:tasks.workunit.client.0.smithi137.stdout:3/205: write d0/d2/d8/f14 [474604,92014] 0 2024-06-15T01:06:57.032 INFO:tasks.workunit.client.1.smithi142.stdout:8/209: creat d0/d1/f48 x:0 0 0 2024-06-15T01:06:57.035 INFO:tasks.workunit.client.1.smithi142.stdout:0/218: rename d3/d5/c29 to d3/d12/c4f 0 2024-06-15T01:06:57.035 INFO:tasks.workunit.client.0.smithi137.stdout:1/212: dwrite d0/d3/f36 [0,4194304] 0 2024-06-15T01:06:57.036 INFO:tasks.workunit.client.1.smithi142.stdout:6/217: mknod d1a/d3c/c4f 0 2024-06-15T01:06:57.039 INFO:tasks.workunit.client.1.smithi142.stdout:3/307: truncate d0/d3/da/d18/f28 2205708 0 2024-06-15T01:06:57.040 INFO:tasks.workunit.client.1.smithi142.stdout:7/194: truncate d2/d6/f9 1386872 0 2024-06-15T01:06:57.041 INFO:tasks.workunit.client.0.smithi137.stdout:3/206: dread d0/d2/f1b [0,4194304] 0 2024-06-15T01:06:57.043 INFO:tasks.workunit.client.1.smithi142.stdout:1/256: rename d6/d46/l4c to d6/d3a/l5c 0 2024-06-15T01:06:57.045 INFO:tasks.workunit.client.1.smithi142.stdout:0/219: mknod d3/d5/c50 0 2024-06-15T01:06:57.045 INFO:tasks.workunit.client.1.smithi142.stdout:3/308: rmdir d0/d1f/d20 39 2024-06-15T01:06:57.046 INFO:tasks.workunit.client.1.smithi142.stdout:1/257: read d6/fc [87044,115703] 0 2024-06-15T01:06:57.047 INFO:tasks.workunit.client.0.smithi137.stdout:0/208: truncate d1/d2/d34/f3f 2076232 0 2024-06-15T01:06:57.048 INFO:tasks.workunit.client.1.smithi142.stdout:7/195: chown d2/d8/da/d33/d25/f40 4115 1 2024-06-15T01:06:57.049 INFO:tasks.workunit.client.1.smithi142.stdout:8/210: dwrite d0/f46 [0,4194304] 0 2024-06-15T01:06:57.051 INFO:tasks.workunit.client.0.smithi137.stdout:2/136: dwrite d0/fb [0,4194304] 0 2024-06-15T01:06:57.051 INFO:tasks.workunit.client.0.smithi137.stdout:0/209: chown d1/d2/dc/d10/d14/d15/d1c/l31 3959867 1 2024-06-15T01:06:57.052 INFO:tasks.workunit.client.0.smithi137.stdout:3/207: symlink d0/d2/d22/d24/d2d/l40 0 2024-06-15T01:06:57.053 INFO:tasks.workunit.client.1.smithi142.stdout:5/258: rename d2/f7 to d2/de/d19/f51 0 2024-06-15T01:06:57.054 INFO:tasks.workunit.client.0.smithi137.stdout:2/137: readlink d0/da/l26 0 2024-06-15T01:06:57.056 INFO:tasks.workunit.client.0.smithi137.stdout:3/208: dread - d0/d2/d23/f3f zero size 2024-06-15T01:06:57.057 INFO:tasks.workunit.client.0.smithi137.stdout:2/138: symlink d0/dc/l2d 0 2024-06-15T01:06:57.058 INFO:tasks.workunit.client.0.smithi137.stdout:3/209: chown d0/d2/l38 112659 1 2024-06-15T01:06:57.058 INFO:tasks.workunit.client.1.smithi142.stdout:4/332: rename d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f61 to d5/d1c/d2a/d30/d46/f76 0 2024-06-15T01:06:57.059 INFO:tasks.workunit.client.1.smithi142.stdout:0/220: symlink d3/d5/d4c/l51 0 2024-06-15T01:06:57.059 INFO:tasks.workunit.client.0.smithi137.stdout:2/139: write d0/da/f11 [370034,101295] 0 2024-06-15T01:06:57.061 INFO:tasks.workunit.client.1.smithi142.stdout:8/211: dread d0/f18 [0,4194304] 0 2024-06-15T01:06:57.063 INFO:tasks.workunit.client.1.smithi142.stdout:4/333: read d5/d9/d23/f4e [1485681,128981] 0 2024-06-15T01:06:57.064 INFO:tasks.workunit.client.0.smithi137.stdout:0/210: dwrite d1/d2/f47 [0,4194304] 0 2024-06-15T01:06:57.066 INFO:tasks.workunit.client.1.smithi142.stdout:0/221: write d3/d5/d44/f4a [610837,98364] 0 2024-06-15T01:06:57.069 INFO:tasks.workunit.client.1.smithi142.stdout:3/309: mknod d0/d1f/d20/d59/c5e 0 2024-06-15T01:06:57.070 INFO:tasks.workunit.client.0.smithi137.stdout:3/210: creat d0/d2/d8/f41 x:0 0 0 2024-06-15T01:06:57.071 INFO:tasks.workunit.client.1.smithi142.stdout:5/259: dwrite d2/de/d19/f22 [8388608,4194304] 0 2024-06-15T01:06:57.071 INFO:tasks.workunit.client.1.smithi142.stdout:6/218: rename d1a/c23 to d1a/d2d/c50 0 2024-06-15T01:06:57.072 INFO:tasks.workunit.client.0.smithi137.stdout:2/140: mkdir d0/dc/d2e 0 2024-06-15T01:06:57.072 INFO:tasks.workunit.client.1.smithi142.stdout:5/260: chown d2/de/d27 154148 1 2024-06-15T01:06:57.074 INFO:tasks.workunit.client.0.smithi137.stdout:0/211: creat d1/d2/dc/d10/f4f x:0 0 0 2024-06-15T01:06:57.077 INFO:tasks.workunit.client.0.smithi137.stdout:2/141: rmdir d0/de 39 2024-06-15T01:06:57.079 INFO:tasks.workunit.client.1.smithi142.stdout:8/212: symlink d0/d1/l49 0 2024-06-15T01:06:57.079 INFO:tasks.workunit.client.0.smithi137.stdout:0/212: creat d1/d2/dc/d10/d14/d15/f50 x:0 0 0 2024-06-15T01:06:57.080 INFO:tasks.workunit.client.1.smithi142.stdout:0/222: mkdir d3/d5/d1b/d52 0 2024-06-15T01:06:57.082 INFO:tasks.workunit.client.1.smithi142.stdout:7/196: getdents d2 0 2024-06-15T01:06:57.084 INFO:tasks.workunit.client.1.smithi142.stdout:3/310: dread d0/d3/f3d [0,4194304] 0 2024-06-15T01:06:57.084 INFO:tasks.workunit.client.1.smithi142.stdout:4/334: dread d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:57.085 INFO:tasks.workunit.client.0.smithi137.stdout:2/142: unlink d0/de/c1e 0 2024-06-15T01:06:57.088 INFO:tasks.workunit.client.0.smithi137.stdout:0/213: dwrite d1/d2/dc/d10/d14/d15/f4b [0,4194304] 0 2024-06-15T01:06:57.091 INFO:tasks.workunit.client.1.smithi142.stdout:8/213: creat d0/d17/d40/f4a x:0 0 0 2024-06-15T01:06:57.092 INFO:tasks.workunit.client.0.smithi137.stdout:0/214: write d1/f2c [368511,73358] 0 2024-06-15T01:06:57.092 INFO:tasks.workunit.client.1.smithi142.stdout:8/214: write d0/f22 [1361242,59395] 0 2024-06-15T01:06:57.094 INFO:tasks.workunit.client.1.smithi142.stdout:2/219: truncate d8/d19/d23/f26 3983227 0 2024-06-15T01:06:57.097 INFO:tasks.workunit.client.1.smithi142.stdout:7/197: mknod d2/d6/c44 0 2024-06-15T01:06:57.099 INFO:tasks.workunit.client.1.smithi142.stdout:4/335: mknod d5/d1c/d2a/d30/c77 0 2024-06-15T01:06:57.100 INFO:tasks.workunit.client.1.smithi142.stdout:9/210: dwrite d6/d34/f3a [0,4194304] 0 2024-06-15T01:06:57.101 INFO:tasks.workunit.client.1.smithi142.stdout:0/223: chown d3/d12/c31 166158367 1 2024-06-15T01:06:57.104 INFO:tasks.workunit.client.0.smithi137.stdout:3/211: dwrite d0/d2/f1b [0,4194304] 0 2024-06-15T01:06:57.105 INFO:tasks.workunit.client.1.smithi142.stdout:8/215: symlink d0/d17/d40/l4b 0 2024-06-15T01:06:57.106 INFO:tasks.workunit.client.1.smithi142.stdout:8/216: fdatasync d0/d17/f2f 0 2024-06-15T01:06:57.107 INFO:tasks.workunit.client.1.smithi142.stdout:2/220: unlink d8/d19/l36 0 2024-06-15T01:06:57.108 INFO:tasks.workunit.client.0.smithi137.stdout:2/143: dwrite d0/f5 [4194304,4194304] 0 2024-06-15T01:06:57.109 INFO:tasks.workunit.client.0.smithi137.stdout:3/212: read d0/d2/d22/d24/d2d/f34 [4243397,56231] 0 2024-06-15T01:06:57.109 INFO:tasks.workunit.client.1.smithi142.stdout:1/258: truncate d6/f9 2112120 0 2024-06-15T01:06:57.113 INFO:tasks.workunit.client.0.smithi137.stdout:2/144: dread d0/da/f17 [0,4194304] 0 2024-06-15T01:06:57.116 INFO:tasks.workunit.client.1.smithi142.stdout:9/211: dwrite d6/dc/dd/d22/f41 [0,4194304] 0 2024-06-15T01:06:57.119 INFO:tasks.workunit.client.0.smithi137.stdout:2/145: truncate d0/da/f17 49248 0 2024-06-15T01:06:57.121 INFO:tasks.workunit.client.0.smithi137.stdout:3/213: dwrite d0/d2/d23/f3d [0,4194304] 0 2024-06-15T01:06:57.122 INFO:tasks.workunit.client.1.smithi142.stdout:8/217: dread d0/f15 [0,4194304] 0 2024-06-15T01:06:57.122 INFO:tasks.workunit.client.1.smithi142.stdout:7/198: write d2/d8/dd/f12 [2010917,38270] 0 2024-06-15T01:06:57.123 INFO:tasks.workunit.client.0.smithi137.stdout:2/146: mknod d0/c2f 0 2024-06-15T01:06:57.124 INFO:tasks.workunit.client.1.smithi142.stdout:3/311: creat d0/d1f/d20/d5d/f5f x:0 0 0 2024-06-15T01:06:57.125 INFO:tasks.workunit.client.0.smithi137.stdout:3/214: unlink d0/d2/f1b 0 2024-06-15T01:06:57.126 INFO:tasks.workunit.client.0.smithi137.stdout:4/163: sync 2024-06-15T01:06:57.127 INFO:tasks.workunit.client.0.smithi137.stdout:8/131: sync 2024-06-15T01:06:57.127 INFO:tasks.workunit.client.0.smithi137.stdout:9/105: sync 2024-06-15T01:06:57.128 INFO:tasks.workunit.client.1.smithi142.stdout:4/336: creat d5/d9/d41/f78 x:0 0 0 2024-06-15T01:06:57.129 INFO:tasks.workunit.client.0.smithi137.stdout:8/132: chown f1 31881 1 2024-06-15T01:06:57.129 INFO:tasks.workunit.client.1.smithi142.stdout:8/218: truncate d0/d1/f2c 984742 0 2024-06-15T01:06:57.130 INFO:tasks.workunit.client.0.smithi137.stdout:9/106: dread f5 [0,4194304] 0 2024-06-15T01:06:57.130 INFO:tasks.workunit.client.0.smithi137.stdout:3/215: mkdir d0/d42 0 2024-06-15T01:06:57.132 INFO:tasks.workunit.client.0.smithi137.stdout:4/164: rename dd/c27 to dd/d15/c44 0 2024-06-15T01:06:57.132 INFO:tasks.workunit.client.1.smithi142.stdout:0/224: rename d3/d12/c31 to d3/d5/d4c/c53 0 2024-06-15T01:06:57.136 INFO:tasks.workunit.client.0.smithi137.stdout:4/165: truncate dd/d15/f2e 280434 0 2024-06-15T01:06:57.136 INFO:tasks.workunit.client.0.smithi137.stdout:3/216: dread d0/d2/d8/f14 [0,4194304] 0 2024-06-15T01:06:57.136 INFO:tasks.workunit.client.0.smithi137.stdout:9/107: dread fd [4194304,4194304] 0 2024-06-15T01:06:57.137 INFO:tasks.workunit.client.1.smithi142.stdout:5/261: link d2/l20 d2/d3e/l52 0 2024-06-15T01:06:57.137 INFO:tasks.workunit.client.1.smithi142.stdout:1/259: unlink d6/d25/l38 0 2024-06-15T01:06:57.138 INFO:tasks.workunit.client.0.smithi137.stdout:9/108: chown fd 19053 1 2024-06-15T01:06:57.138 INFO:tasks.workunit.client.1.smithi142.stdout:2/221: dwrite d8/d19/d23/d25/f18 [0,4194304] 0 2024-06-15T01:06:57.139 INFO:tasks.workunit.client.0.smithi137.stdout:9/109: write fc [439838,39411] 0 2024-06-15T01:06:57.142 INFO:tasks.workunit.client.1.smithi142.stdout:2/222: dread - d8/df/d10/f35 zero size 2024-06-15T01:06:57.142 INFO:tasks.workunit.client.1.smithi142.stdout:8/219: dwrite d0/f20 [4194304,4194304] 0 2024-06-15T01:06:57.143 INFO:tasks.workunit.client.1.smithi142.stdout:7/199: mknod d2/d8/dd/d14/d2c/d37/c45 0 2024-06-15T01:06:57.144 INFO:tasks.workunit.client.0.smithi137.stdout:2/147: dread d0/f8 [0,4194304] 0 2024-06-15T01:06:57.145 INFO:tasks.workunit.client.1.smithi142.stdout:4/337: creat d5/d1c/d6f/f79 x:0 0 0 2024-06-15T01:06:57.146 INFO:tasks.workunit.client.0.smithi137.stdout:4/166: dwrite f9 [0,4194304] 0 2024-06-15T01:06:57.146 INFO:tasks.workunit.client.1.smithi142.stdout:1/260: creat d6/d12/d28/f5d x:0 0 0 2024-06-15T01:06:57.148 INFO:tasks.workunit.client.1.smithi142.stdout:5/262: mknod d2/c53 0 2024-06-15T01:06:57.149 INFO:tasks.workunit.client.1.smithi142.stdout:1/261: dread - d6/d25/f3d zero size 2024-06-15T01:06:57.153 INFO:tasks.workunit.client.1.smithi142.stdout:2/223: dwrite d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:06:57.158 INFO:tasks.workunit.client.0.smithi137.stdout:4/167: getdents dd/d10 0 2024-06-15T01:06:57.158 INFO:tasks.workunit.client.0.smithi137.stdout:4/168: chown dd/d10/c2b 43 1 2024-06-15T01:06:57.180 INFO:tasks.workunit.client.1.smithi142.stdout:7/200: unlink d2/d6/d22/d34/f38 0 2024-06-15T01:06:57.181 INFO:tasks.workunit.client.0.smithi137.stdout:8/133: truncate d9/de/ff 2373591 0 2024-06-15T01:06:57.182 INFO:tasks.workunit.client.1.smithi142.stdout:7/201: write d2/d8/da/d33/f35 [706859,16207] 0 2024-06-15T01:06:57.182 INFO:tasks.workunit.client.0.smithi137.stdout:2/148: write d0/f8 [1810538,50941] 0 2024-06-15T01:06:57.183 INFO:tasks.workunit.client.0.smithi137.stdout:8/134: write d9/f20 [773742,94423] 0 2024-06-15T01:06:57.185 INFO:tasks.workunit.client.0.smithi137.stdout:9/110: dread df/d12/f17 [0,4194304] 0 2024-06-15T01:06:57.186 INFO:tasks.workunit.client.0.smithi137.stdout:9/111: write fc [3477952,86186] 0 2024-06-15T01:06:57.186 INFO:tasks.workunit.client.1.smithi142.stdout:2/224: mknod d8/d19/d23/d25/c3c 0 2024-06-15T01:06:57.187 INFO:tasks.workunit.client.1.smithi142.stdout:2/225: chown d8/l28 3910833 1 2024-06-15T01:06:57.187 INFO:tasks.workunit.client.0.smithi137.stdout:8/135: write d9/de/d1e/f28 [737355,106196] 0 2024-06-15T01:06:57.188 INFO:tasks.workunit.client.0.smithi137.stdout:3/217: dread d0/d2/d22/d24/d2d/f34 [0,4194304] 0 2024-06-15T01:06:57.188 INFO:tasks.workunit.client.0.smithi137.stdout:2/149: truncate d0/da/f18 47700 0 2024-06-15T01:06:57.191 INFO:tasks.workunit.client.1.smithi142.stdout:1/262: fdatasync d6/f9 0 2024-06-15T01:06:57.191 INFO:tasks.workunit.client.0.smithi137.stdout:8/136: rename d9 to d9/d2c 22 2024-06-15T01:06:57.192 INFO:tasks.workunit.client.1.smithi142.stdout:1/263: write d6/f26 [4095242,130041] 0 2024-06-15T01:06:57.193 INFO:tasks.workunit.client.0.smithi137.stdout:9/112: fsync fa 0 2024-06-15T01:06:57.193 INFO:tasks.workunit.client.1.smithi142.stdout:8/220: creat d0/f4c x:0 0 0 2024-06-15T01:06:57.195 INFO:tasks.workunit.client.0.smithi137.stdout:8/137: fsync f6 0 2024-06-15T01:06:57.197 INFO:tasks.workunit.client.0.smithi137.stdout:3/218: dwrite d0/f1d [0,4194304] 0 2024-06-15T01:06:57.198 INFO:tasks.workunit.client.1.smithi142.stdout:4/338: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d59/f7a x:0 0 0 2024-06-15T01:06:57.198 INFO:tasks.workunit.client.0.smithi137.stdout:8/138: truncate d9/d22/f25 555847 0 2024-06-15T01:06:57.200 INFO:tasks.workunit.client.0.smithi137.stdout:8/139: readlink d9/la 0 2024-06-15T01:06:57.201 INFO:tasks.workunit.client.1.smithi142.stdout:8/221: dwrite d0/f46 [0,4194304] 0 2024-06-15T01:06:57.201 INFO:tasks.workunit.client.0.smithi137.stdout:9/113: mknod df/c1c 0 2024-06-15T01:06:57.202 INFO:tasks.workunit.client.1.smithi142.stdout:7/202: creat d2/d8/dd/d14/f46 x:0 0 0 2024-06-15T01:06:57.203 INFO:tasks.workunit.client.0.smithi137.stdout:3/219: dread - d0/d2/d8/f41 zero size 2024-06-15T01:06:57.203 INFO:tasks.workunit.client.1.smithi142.stdout:8/222: write d0/d1/f29 [9073656,81716] 0 2024-06-15T01:06:57.204 INFO:tasks.workunit.client.0.smithi137.stdout:4/169: dread dd/f22 [0,4194304] 0 2024-06-15T01:06:57.204 INFO:tasks.workunit.client.0.smithi137.stdout:2/150: link d0/da/l26 d0/l30 0 2024-06-15T01:06:57.204 INFO:tasks.workunit.client.1.smithi142.stdout:2/226: mknod d8/c3d 0 2024-06-15T01:06:57.205 INFO:tasks.workunit.client.1.smithi142.stdout:8/223: stat d0/d17/f1f 0 2024-06-15T01:06:57.205 INFO:tasks.workunit.client.0.smithi137.stdout:2/151: fsync d0/f3 0 2024-06-15T01:06:57.206 INFO:tasks.workunit.client.1.smithi142.stdout:0/225: getdents d3/d12/d22 0 2024-06-15T01:06:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.210+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56180611d0 con 0x7f560406bb70 2024-06-15T01:06:57.207 INFO:tasks.workunit.client.0.smithi137.stdout:2/152: chown d0/de/d10/f2a 245262216 1 2024-06-15T01:06:57.208 INFO:tasks.workunit.client.1.smithi142.stdout:1/264: creat d6/d12/d30/f5e x:0 0 0 2024-06-15T01:06:57.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.212+0000 7f560f7fe700 1 -- 172.21.15.137:0/4092315300 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f56180611d0 con 0x7f560406bb70 2024-06-15T01:06:57.209 INFO:tasks.workunit.client.1.smithi142.stdout:1/265: chown d6/d12/d5b 16498776 1 2024-06-15T01:06:57.213 INFO:tasks.workunit.client.1.smithi142.stdout:9/212: write d6/dc/f15 [1380651,120022] 0 2024-06-15T01:06:57.214 INFO:tasks.workunit.client.1.smithi142.stdout:4/339: unlink d5/d9/d41/f40 0 2024-06-15T01:06:57.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.217+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 msgr2=0x7f560406e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.217+0000 7f5620518700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 0x7f560406e030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5608009bf0 tx=0x7f5608009260 comp rx=0 tx=0).stop 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.217+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 msgr2=0x7f5618196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.217+0000 7f5620518700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f561400bde0 tx=0x7f561400bec0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 shutdown_connections 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f560406bb70 0x7f560406e030 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56181016a0 0x7f5618196240 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 --2- 172.21.15.137:0/4092315300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5618103fd0 0x7f5618196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 >> 172.21.15.137:0/4092315300 conn(0x7f56180fb030 msgr2=0x7f56180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:57.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 shutdown_connections 2024-06-15T01:06:57.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.219+0000 7f5620518700 1 -- 172.21.15.137:0/4092315300 wait complete. 2024-06-15T01:06:57.218 INFO:tasks.workunit.client.0.smithi137.stdout:4/170: mkdir dd/d15/d36/d45 0 2024-06-15T01:06:57.218 INFO:tasks.workunit.client.1.smithi142.stdout:7/203: mkdir d2/d6/d22/d47 0 2024-06-15T01:06:57.220 INFO:tasks.workunit.client.0.smithi137.stdout:8/140: creat d9/d2a/f2d x:0 0 0 2024-06-15T01:06:57.221 INFO:tasks.workunit.client.1.smithi142.stdout:8/224: creat d0/d1/f4d x:0 0 0 2024-06-15T01:06:57.223 INFO:tasks.workunit.client.1.smithi142.stdout:4/340: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/f60 [0,4194304] 0 2024-06-15T01:06:57.224 INFO:tasks.workunit.client.1.smithi142.stdout:5/263: getdents d2/d3e/d31 0 2024-06-15T01:06:57.225 INFO:tasks.workunit.client.0.smithi137.stdout:4/171: write dd/f12 [4923794,121574] 0 2024-06-15T01:06:57.225 INFO:tasks.workunit.client.0.smithi137.stdout:4/172: chown dd/d15/d36/d45 8891371 1 2024-06-15T01:06:57.228 INFO:tasks.workunit.client.1.smithi142.stdout:8/225: dread d0/f23 [0,4194304] 0 2024-06-15T01:06:57.229 INFO:tasks.workunit.client.0.smithi137.stdout:3/220: link d0/d2/d8/f1c d0/d13/d37/f43 0 2024-06-15T01:06:57.230 INFO:tasks.workunit.client.1.smithi142.stdout:7/204: mkdir d2/d6/d22/d48 0 2024-06-15T01:06:57.231 INFO:tasks.workunit.client.0.smithi137.stdout:9/114: link f3 df/d12/d15/f1d 0 2024-06-15T01:06:57.232 INFO:tasks.workunit.client.0.smithi137.stdout:9/115: rename df to df/d1a/d1e 22 2024-06-15T01:06:57.232 INFO:tasks.workunit.client.0.smithi137.stdout:2/153: symlink d0/dc/d22/d23/l31 0 2024-06-15T01:06:57.232 INFO:tasks.workunit.client.1.smithi142.stdout:7/205: write d2/d8/dd/f12 [2611388,38658] 0 2024-06-15T01:06:57.233 INFO:tasks.workunit.client.1.smithi142.stdout:0/226: mknod d3/d5/d1b/d52/c54 0 2024-06-15T01:06:57.234 INFO:tasks.workunit.client.0.smithi137.stdout:4/173: creat dd/d15/d30/f46 x:0 0 0 2024-06-15T01:06:57.236 INFO:tasks.workunit.client.0.smithi137.stdout:3/221: unlink d0/f1d 0 2024-06-15T01:06:57.236 INFO:tasks.workunit.client.1.smithi142.stdout:8/226: dwrite d0/d17/f1f [0,4194304] 0 2024-06-15T01:06:57.239 INFO:tasks.workunit.client.0.smithi137.stdout:3/222: write d0/d13/f20 [2845759,57812] 0 2024-06-15T01:06:57.240 INFO:tasks.workunit.client.0.smithi137.stdout:4/174: creat dd/f47 x:0 0 0 2024-06-15T01:06:57.241 INFO:tasks.workunit.client.0.smithi137.stdout:9/116: read df/d12/d15/f1d [127963,123080] 0 2024-06-15T01:06:57.243 INFO:tasks.workunit.client.1.smithi142.stdout:5/264: rename d2/de/d19/f2d to d2/de/d27/f54 0 2024-06-15T01:06:57.243 INFO:tasks.workunit.client.0.smithi137.stdout:2/154: dwrite d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:06:57.246 INFO:tasks.workunit.client.1.smithi142.stdout:4/341: write d5/d1c/d2a/d30/d46/f76 [192141,69448] 0 2024-06-15T01:06:57.246 INFO:tasks.workunit.client.1.smithi142.stdout:5/265: fsync d2/de/f50 0 2024-06-15T01:06:57.247 INFO:tasks.workunit.client.1.smithi142.stdout:7/206: dwrite d2/d8/da/d33/d25/f3f [0,4194304] 0 2024-06-15T01:06:57.248 INFO:tasks.workunit.client.0.smithi137.stdout:3/223: creat d0/d2/d23/f44 x:0 0 0 2024-06-15T01:06:57.249 INFO:tasks.workunit.client.1.smithi142.stdout:5/266: read d2/de/d27/f38 [799208,31351] 0 2024-06-15T01:06:57.249 INFO:tasks.workunit.client.1.smithi142.stdout:5/267: chown d2/d3e/d31/c44 34551 1 2024-06-15T01:06:57.250 INFO:tasks.workunit.client.1.smithi142.stdout:0/227: mknod d3/d5/c55 0 2024-06-15T01:06:57.251 INFO:tasks.workunit.client.0.smithi137.stdout:3/224: chown d0/d13/c36 221590993 1 2024-06-15T01:06:57.252 INFO:tasks.workunit.client.1.smithi142.stdout:3/312: write d0/d3/da/d18/f28 [393990,84625] 0 2024-06-15T01:06:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:4/175: dwrite dd/d10/f39 [0,4194304] 0 2024-06-15T01:06:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:9/117: link df/f18 df/d12/f1f 0 2024-06-15T01:06:57.256 INFO:tasks.workunit.client.0.smithi137.stdout:2/155: creat d0/de/d2b/f32 x:0 0 0 2024-06-15T01:06:57.258 INFO:tasks.workunit.client.1.smithi142.stdout:9/213: creat d6/f55 x:0 0 0 2024-06-15T01:06:57.259 INFO:tasks.workunit.client.0.smithi137.stdout:3/225: rename d0/l12 to d0/d2/d8/l45 0 2024-06-15T01:06:57.260 INFO:tasks.workunit.client.1.smithi142.stdout:9/214: stat d6/f21 0 2024-06-15T01:06:57.262 INFO:tasks.workunit.client.1.smithi142.stdout:9/215: dread - d6/dc/dd/f46 zero size 2024-06-15T01:06:57.262 INFO:tasks.workunit.client.0.smithi137.stdout:9/118: mknod df/d1a/c20 0 2024-06-15T01:06:57.263 INFO:tasks.workunit.client.0.smithi137.stdout:9/119: readlink df/l14 0 2024-06-15T01:06:57.264 INFO:tasks.workunit.client.0.smithi137.stdout:9/120: readlink df/d12/l19 0 2024-06-15T01:06:57.265 INFO:tasks.workunit.client.0.smithi137.stdout:2/156: dwrite d0/dc/f2c [0,4194304] 0 2024-06-15T01:06:57.267 INFO:tasks.workunit.client.1.smithi142.stdout:5/268: dwrite d2/f12 [4194304,4194304] 0 2024-06-15T01:06:57.267 INFO:tasks.workunit.client.0.smithi137.stdout:4/176: getdents dd/d10 0 2024-06-15T01:06:57.269 INFO:tasks.workunit.client.0.smithi137.stdout:9/121: write f5 [1289900,121284] 0 2024-06-15T01:06:57.272 INFO:tasks.workunit.client.1.smithi142.stdout:2/227: dwrite f4 [0,4194304] 0 2024-06-15T01:06:57.272 INFO:tasks.workunit.client.1.smithi142.stdout:3/313: mkdir d0/d1f/d20/d5d/d60 0 2024-06-15T01:06:57.272 INFO:tasks.workunit.client.1.smithi142.stdout:8/227: rename d0/c13 to d0/d1/c4e 0 2024-06-15T01:06:57.273 INFO:tasks.workunit.client.1.smithi142.stdout:6/219: sync 2024-06-15T01:06:57.274 INFO:tasks.workunit.client.0.smithi137.stdout:3/226: read d0/d2/d8/f1c [1600241,50119] 0 2024-06-15T01:06:57.276 INFO:tasks.workunit.client.1.smithi142.stdout:5/269: creat d2/d18/f55 x:0 0 0 2024-06-15T01:06:57.276 INFO:tasks.workunit.client.1.smithi142.stdout:2/228: chown d8/df/d10/f11 3372516 1 2024-06-15T01:06:57.277 INFO:tasks.workunit.client.1.smithi142.stdout:2/229: truncate f4 5086120 0 2024-06-15T01:06:57.277 INFO:tasks.workunit.client.1.smithi142.stdout:4/342: mknod d5/d9/c7b 0 2024-06-15T01:06:57.279 INFO:tasks.workunit.client.1.smithi142.stdout:0/228: symlink d3/l56 0 2024-06-15T01:06:57.280 INFO:tasks.workunit.client.1.smithi142.stdout:3/314: read - d0/d1f/f38 zero size 2024-06-15T01:06:57.281 INFO:tasks.workunit.client.0.smithi137.stdout:3/227: mkdir d0/d2/d23/d46 0 2024-06-15T01:06:57.281 INFO:tasks.workunit.client.1.smithi142.stdout:4/343: stat d5/d1c/d2a/d30/d46/d50/d51/d5b/l58 0 2024-06-15T01:06:57.282 INFO:tasks.workunit.client.1.smithi142.stdout:7/207: rename d2/d6/f7 to d2/f49 0 2024-06-15T01:06:57.282 INFO:tasks.workunit.client.0.smithi137.stdout:4/177: getdents dd/d15/d36 0 2024-06-15T01:06:57.283 INFO:tasks.workunit.client.1.smithi142.stdout:0/229: creat d3/d5/d44/f57 x:0 0 0 2024-06-15T01:06:57.283 INFO:tasks.workunit.client.1.smithi142.stdout:3/315: chown d0/f39 0 1 2024-06-15T01:06:57.284 INFO:tasks.workunit.client.0.smithi137.stdout:4/178: chown dd/d10/f39 2065661 1 2024-06-15T01:06:57.286 INFO:tasks.workunit.client.1.smithi142.stdout:3/316: readlink d0/d3/da/d18/l33 0 2024-06-15T01:06:57.294 INFO:tasks.workunit.client.0.smithi137.stdout:2/157: truncate d0/f6 313724 0 2024-06-15T01:06:57.295 INFO:tasks.workunit.client.0.smithi137.stdout:9/122: write df/d12/d15/f1d [9916,79728] 0 2024-06-15T01:06:57.295 INFO:tasks.workunit.client.0.smithi137.stdout:2/158: chown d0/dc/d22 0 1 2024-06-15T01:06:57.296 INFO:tasks.workunit.client.0.smithi137.stdout:3/228: truncate d0/d13/f20 3783917 0 2024-06-15T01:06:57.296 INFO:tasks.workunit.client.0.smithi137.stdout:4/179: write f5 [1179673,29740] 0 2024-06-15T01:06:57.297 INFO:tasks.workunit.client.0.smithi137.stdout:2/159: write d0/f1f [688813,84848] 0 2024-06-15T01:06:57.298 INFO:tasks.workunit.client.0.smithi137.stdout:4/180: dread - dd/d10/d1f/f26 zero size 2024-06-15T01:06:57.299 INFO:tasks.workunit.client.0.smithi137.stdout:2/160: dread - d0/de/f24 zero size 2024-06-15T01:06:57.300 INFO:tasks.workunit.client.0.smithi137.stdout:4/181: mkdir dd/d15/d48 0 2024-06-15T01:06:57.305 INFO:tasks.workunit.client.0.smithi137.stdout:2/161: creat d0/de/d10/f33 x:0 0 0 2024-06-15T01:06:57.306 INFO:tasks.workunit.client.0.smithi137.stdout:4/182: chown dd/d15/d21/d3a/c3c 951 1 2024-06-15T01:06:57.307 INFO:tasks.workunit.client.0.smithi137.stdout:4/183: fdatasync dd/d15/d30/f46 0 2024-06-15T01:06:57.310 INFO:tasks.workunit.client.0.smithi137.stdout:4/184: mkdir dd/d15/d30/d49 0 2024-06-15T01:06:57.311 INFO:tasks.workunit.client.0.smithi137.stdout:3/229: dwrite d0/d13/f26 [0,4194304] 0 2024-06-15T01:06:57.312 INFO:tasks.workunit.client.0.smithi137.stdout:3/230: chown d0/f16 3209 1 2024-06-15T01:06:57.313 INFO:tasks.workunit.client.0.smithi137.stdout:4/185: mknod dd/d15/d30/d49/c4a 0 2024-06-15T01:06:57.323 INFO:tasks.workunit.client.0.smithi137.stdout:3/231: dwrite d0/d2/d22/d24/d28/d2e/f3b [0,4194304] 0 2024-06-15T01:06:57.324 INFO:tasks.workunit.client.0.smithi137.stdout:4/186: dwrite dd/fe [0,4194304] 0 2024-06-15T01:06:57.326 INFO:tasks.workunit.client.0.smithi137.stdout:3/232: write d0/d13/f21 [2273189,117106] 0 2024-06-15T01:06:57.327 INFO:tasks.workunit.client.0.smithi137.stdout:3/233: chown d0/d13 1597 1 2024-06-15T01:06:57.329 INFO:tasks.workunit.client.0.smithi137.stdout:4/187: creat dd/d15/d36/d45/f4b x:0 0 0 2024-06-15T01:06:57.330 INFO:tasks.workunit.client.0.smithi137.stdout:3/234: creat d0/d2/d22/d24/f47 x:0 0 0 2024-06-15T01:06:57.332 INFO:tasks.workunit.client.0.smithi137.stdout:3/235: creat d0/d2/d23/f48 x:0 0 0 2024-06-15T01:06:57.337 INFO:tasks.workunit.client.0.smithi137.stdout:4/188: dwrite dd/f3b [0,4194304] 0 2024-06-15T01:06:57.340 INFO:tasks.workunit.client.0.smithi137.stdout:4/189: symlink dd/d15/d30/l4c 0 2024-06-15T01:06:57.342 INFO:tasks.workunit.client.0.smithi137.stdout:4/190: write dd/d10/f1c [456111,113780] 0 2024-06-15T01:06:57.343 INFO:tasks.workunit.client.0.smithi137.stdout:4/191: truncate dd/f13 5490543 0 2024-06-15T01:06:57.346 INFO:tasks.workunit.client.0.smithi137.stdout:4/192: rename dd/d15/d30/l4c to dd/d10/l4d 0 2024-06-15T01:06:57.347 INFO:tasks.workunit.client.0.smithi137.stdout:4/193: write dd/d15/d21/f29 [1056666,77551] 0 2024-06-15T01:06:57.352 INFO:tasks.workunit.client.1.smithi142.stdout:2/230: creat d8/df/d1f/f3e x:0 0 0 2024-06-15T01:06:57.353 INFO:tasks.workunit.client.1.smithi142.stdout:8/228: symlink d0/l4f 0 2024-06-15T01:06:57.353 INFO:tasks.workunit.client.1.smithi142.stdout:2/231: write d8/df/d10/f11 [684506,103576] 0 2024-06-15T01:06:57.354 INFO:tasks.workunit.client.0.smithi137.stdout:3/236: write d0/d2f/f35 [610519,67995] 0 2024-06-15T01:06:57.354 INFO:tasks.workunit.client.1.smithi142.stdout:8/229: dread - d0/f43 zero size 2024-06-15T01:06:57.355 INFO:tasks.workunit.client.1.smithi142.stdout:8/230: dread - d0/d17/f44 zero size 2024-06-15T01:06:57.356 INFO:tasks.workunit.client.0.smithi137.stdout:4/194: dwrite dd/f17 [0,4194304] 0 2024-06-15T01:06:57.359 INFO:tasks.workunit.client.0.smithi137.stdout:3/237: creat d0/d2/d22/d24/d28/f49 x:0 0 0 2024-06-15T01:06:57.360 INFO:tasks.workunit.client.1.smithi142.stdout:4/344: creat d5/d9/da/f7c x:0 0 0 2024-06-15T01:06:57.361 INFO:tasks.workunit.client.0.smithi137.stdout:3/238: chown d0/d13/f21 143727 1 2024-06-15T01:06:57.362 INFO:tasks.workunit.client.1.smithi142.stdout:6/220: dwrite d1a/f20 [0,4194304] 0 2024-06-15T01:06:57.366 INFO:tasks.workunit.client.1.smithi142.stdout:5/270: rename d2/fc to d2/de/d27/f56 0 2024-06-15T01:06:57.366 INFO:tasks.workunit.client.0.smithi137.stdout:3/239: symlink d0/d2/d22/d24/d28/l4a 0 2024-06-15T01:06:57.367 INFO:tasks.workunit.client.0.smithi137.stdout:4/195: dwrite dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:06:57.368 INFO:tasks.workunit.client.1.smithi142.stdout:3/317: write d0/f2c [546104,120351] 0 2024-06-15T01:06:57.368 INFO:tasks.workunit.client.1.smithi142.stdout:3/318: readlink d0/d1b/l50 0 2024-06-15T01:06:57.369 INFO:tasks.workunit.client.1.smithi142.stdout:3/319: fsync d0/d3/d15/f2d 0 2024-06-15T01:06:57.371 INFO:tasks.workunit.client.0.smithi137.stdout:3/240: creat d0/d2/d22/d24/f4b x:0 0 0 2024-06-15T01:06:57.372 INFO:tasks.workunit.client.1.smithi142.stdout:3/320: read d0/d3/d15/d4d/d55/f57 [111749,94368] 0 2024-06-15T01:06:57.374 INFO:tasks.workunit.client.0.smithi137.stdout:4/196: dread dd/f22 [0,4194304] 0 2024-06-15T01:06:57.381 INFO:tasks.workunit.client.1.smithi142.stdout:6/221: readlink d1a/l2b 0 2024-06-15T01:06:57.383 INFO:tasks.workunit.client.1.smithi142.stdout:4/345: symlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/l7d 0 2024-06-15T01:06:57.383 INFO:tasks.workunit.client.0.smithi137.stdout:3/241: dwrite d0/d2/d22/d24/d28/f49 [0,4194304] 0 2024-06-15T01:06:57.384 INFO:tasks.workunit.client.1.smithi142.stdout:4/346: chown d5/d9/d23/d32/c5d 2918 1 2024-06-15T01:06:57.384 INFO:tasks.workunit.client.1.smithi142.stdout:4/347: readlink d5/d9/l26 0 2024-06-15T01:06:57.385 INFO:tasks.workunit.client.0.smithi137.stdout:4/197: dwrite fa [0,4194304] 0 2024-06-15T01:06:57.387 INFO:tasks.workunit.client.0.smithi137.stdout:4/198: readlink dd/d15/l43 0 2024-06-15T01:06:57.388 INFO:tasks.workunit.client.0.smithi137.stdout:5/414: sync 2024-06-15T01:06:57.389 INFO:tasks.workunit.client.0.smithi137.stdout:4/199: write dd/d10/d1f/f26 [723494,75366] 0 2024-06-15T01:06:57.392 INFO:tasks.workunit.client.1.smithi142.stdout:1/266: sync 2024-06-15T01:06:57.393 INFO:tasks.workunit.client.1.smithi142.stdout:9/216: link d6/dc/d1b/l1e d6/dc/d51/d53/l56 0 2024-06-15T01:06:57.394 INFO:tasks.workunit.client.0.smithi137.stdout:5/415: rmdir df/d11/d34/d3b/d4b/d4f 39 2024-06-15T01:06:57.394 INFO:tasks.workunit.client.1.smithi142.stdout:9/217: stat d6/d34/f3a 0 2024-06-15T01:06:57.394 INFO:tasks.workunit.client.1.smithi142.stdout:9/218: chown d6/d34 70 1 2024-06-15T01:06:57.395 INFO:tasks.workunit.client.0.smithi137.stdout:3/242: rmdir d0/d2/d22/d24/d29 0 2024-06-15T01:06:57.396 INFO:tasks.workunit.client.1.smithi142.stdout:9/219: write d6/d31/f3e [1854107,76408] 0 2024-06-15T01:06:57.399 INFO:tasks.workunit.client.0.smithi137.stdout:4/200: dwrite dd/d10/f39 [0,4194304] 0 2024-06-15T01:06:57.399 INFO:tasks.workunit.client.0.smithi137.stdout:4/201: stat dd/d15/f42 0 2024-06-15T01:06:57.400 INFO:tasks.workunit.client.1.smithi142.stdout:3/321: mkdir d0/d1f/d20/d59/d61 0 2024-06-15T01:06:57.403 INFO:tasks.workunit.client.1.smithi142.stdout:8/231: symlink d0/l50 0 2024-06-15T01:06:57.407 INFO:tasks.workunit.client.1.smithi142.stdout:9/220: dwrite f2 [0,4194304] 0 2024-06-15T01:06:57.407 INFO:tasks.workunit.client.1.smithi142.stdout:8/232: write d0/d17/f27 [174224,4220] 0 2024-06-15T01:06:57.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.412+0000 7f911fa93700 1 -- 172.21.15.137:0/2944644596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91180ff240 msgr2=0x7f91180ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.412+0000 7f911fa93700 1 --2- 172.21.15.137:0/2944644596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91180ff240 0x7f91180ff660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9114009910 tx=0x7f9114009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:57.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 -- 172.21.15.137:0/2944644596 shutdown_connections 2024-06-15T01:06:57.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 --2- 172.21.15.137:0/2944644596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9118100440 0x7f91181008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 --2- 172.21.15.137:0/2944644596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91180ff240 0x7f91180ff660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 -- 172.21.15.137:0/2944644596 >> 172.21.15.137:0/2944644596 conn(0x7f91180fa7c0 msgr2=0x7f91180fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:57.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 -- 172.21.15.137:0/2944644596 shutdown_connections 2024-06-15T01:06:57.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.413+0000 7f911fa93700 1 -- 172.21.15.137:0/2944644596 wait complete. 2024-06-15T01:06:57.412 INFO:tasks.workunit.client.1.smithi142.stdout:8/233: chown d0/d17/f44 19598 1 2024-06-15T01:06:57.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 Processor -- start 2024-06-15T01:06:57.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 -- start start 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91180ff240 0x7f9118105890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118106420 con 0x7f9118100440 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911fa93700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181a4030 con 0x7f91180ff240 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911d02e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911d02e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41930/0 (socket says 172.21.15.137:41930) 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.414+0000 7f911d02e700 1 -- 172.21.15.137:0/3052017073 learned_addr learned my addr 172.21.15.137:0/3052017073 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f911d02e700 1 -- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91180ff240 msgr2=0x7f9118105890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f911d02e700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91180ff240 0x7f9118105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f911d02e700 1 -- 172.21.15.137:0/3052017073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140095f0 con 0x7f9118100440 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f911d02e700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f910800c8a0 tx=0x7f910800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f910800cea0 con 0x7f9118100440 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.415+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9108004830 con 0x7f9118100440 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.416+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9108005540 con 0x7f9118100440 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.416+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91181a4230 con 0x7f9118100440 2024-06-15T01:06:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.416+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181a4800 con 0x7f9118100440 2024-06-15T01:06:57.416 INFO:tasks.workunit.client.1.smithi142.stdout:0/230: sync 2024-06-15T01:06:57.416 INFO:tasks.workunit.client.1.smithi142.stdout:6/222: mkdir d1a/d39/d3a/d51 0 2024-06-15T01:06:57.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.417+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911806aec0 con 0x7f9118100440 2024-06-15T01:06:57.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.418+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f91080049a0 con 0x7f9118100440 2024-06-15T01:06:57.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.419+0000 7f910effd700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 0x7f910406e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:57.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.420+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f910808aeb0 con 0x7f9118100440 2024-06-15T01:06:57.417 INFO:tasks.workunit.client.0.smithi137.stdout:3/243: dread d0/f16 [0,4194304] 0 2024-06-15T01:06:57.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.421+0000 7f911d82f700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 0x7f910406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:57.418 INFO:tasks.workunit.client.0.smithi137.stdout:3/244: stat d0/d13/d37 0 2024-06-15T01:06:57.418 INFO:tasks.workunit.client.1.smithi142.stdout:2/232: fdatasync d8/d19/d23/d25/d1a/f1d 0 2024-06-15T01:06:57.419 INFO:tasks.workunit.client.1.smithi142.stdout:0/231: dread - d3/d5/d44/f57 zero size 2024-06-15T01:06:57.420 INFO:tasks.workunit.client.0.smithi137.stdout:3/245: mkdir d0/d4c 0 2024-06-15T01:06:57.421 INFO:tasks.workunit.client.0.smithi137.stdout:3/246: mknod d0/d42/c4d 0 2024-06-15T01:06:57.421 INFO:tasks.workunit.client.1.smithi142.stdout:7/208: getdents d2 0 2024-06-15T01:06:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.424+0000 7f911d82f700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 0x7f910406e060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f911400b5c0 tx=0x7f9114005960 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:57.422 INFO:tasks.workunit.client.1.smithi142.stdout:9/221: creat d6/dc/d4e/f57 x:0 0 0 2024-06-15T01:06:57.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.427+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f9108055e40 con 0x7f9118100440 2024-06-15T01:06:57.424 INFO:tasks.workunit.client.0.smithi137.stdout:5/416: write df/d11/f31 [1456619,118997] 0 2024-06-15T01:06:57.426 INFO:tasks.workunit.client.0.smithi137.stdout:3/247: mkdir d0/d2/d4e 0 2024-06-15T01:06:57.427 INFO:tasks.workunit.client.1.smithi142.stdout:4/348: dwrite d5/d1c/d2a/d30/d46/d50/f5e [0,4194304] 0 2024-06-15T01:06:57.427 INFO:tasks.workunit.client.0.smithi137.stdout:5/417: mkdir df/d76/d8e 0 2024-06-15T01:06:57.427 INFO:tasks.workunit.client.0.smithi137.stdout:5/418: fsync df/d11/d30/f82 0 2024-06-15T01:06:57.429 INFO:tasks.workunit.client.1.smithi142.stdout:9/222: readlink d6/dc/dd/le 0 2024-06-15T01:06:57.431 INFO:tasks.workunit.client.0.smithi137.stdout:5/419: truncate f7 1078713 0 2024-06-15T01:06:57.436 INFO:tasks.workunit.client.1.smithi142.stdout:6/223: link d1a/d39/d42/f45 d1a/d2d/f52 0 2024-06-15T01:06:57.436 INFO:tasks.workunit.client.1.smithi142.stdout:2/233: dwrite d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:06:57.437 INFO:tasks.workunit.client.1.smithi142.stdout:7/209: unlink d2/d8/da/d33/d25/l3e 0 2024-06-15T01:06:57.437 INFO:tasks.workunit.client.0.smithi137.stdout:5/420: dread df/d11/d34/d35/f55 [0,4194304] 0 2024-06-15T01:06:57.437 INFO:tasks.workunit.client.0.smithi137.stdout:3/248: dwrite d0/d2/d23/f3f [0,4194304] 0 2024-06-15T01:06:57.438 INFO:tasks.workunit.client.1.smithi142.stdout:2/234: stat d8/df/c38 0 2024-06-15T01:06:57.442 INFO:tasks.workunit.client.1.smithi142.stdout:1/267: dread f5 [8388608,4194304] 0 2024-06-15T01:06:57.442 INFO:tasks.workunit.client.1.smithi142.stdout:7/210: fdatasync d2/f2e 0 2024-06-15T01:06:57.446 INFO:tasks.workunit.client.0.smithi137.stdout:5/421: dwrite df/d11/d30/f7b [0,4194304] 0 2024-06-15T01:06:57.448 INFO:tasks.workunit.client.1.smithi142.stdout:0/232: dwrite d3/d5/f1d [0,4194304] 0 2024-06-15T01:06:57.450 INFO:tasks.workunit.client.1.smithi142.stdout:7/211: dread - d2/d8/dd/d14/f30 zero size 2024-06-15T01:06:57.451 INFO:tasks.workunit.client.1.smithi142.stdout:6/224: readlink d1a/d3c/l3d 0 2024-06-15T01:06:57.455 INFO:tasks.workunit.client.1.smithi142.stdout:5/271: truncate d2/de/d19/f1a 3918512 0 2024-06-15T01:06:57.457 INFO:tasks.workunit.client.0.smithi137.stdout:3/249: dwrite d0/f1f [0,4194304] 0 2024-06-15T01:06:57.457 INFO:tasks.workunit.client.1.smithi142.stdout:0/233: stat d3/d5/d4c/l51 0 2024-06-15T01:06:57.458 INFO:tasks.workunit.client.1.smithi142.stdout:4/349: rename d5/d9/d1a/d4f to d5/d9/da/d7e 0 2024-06-15T01:06:57.459 INFO:tasks.workunit.client.0.smithi137.stdout:3/250: write d0/d2/d23/f44 [264038,15708] 0 2024-06-15T01:06:57.461 INFO:tasks.workunit.client.0.smithi137.stdout:5/422: dwrite df/f12 [0,4194304] 0 2024-06-15T01:06:57.462 INFO:tasks.workunit.client.1.smithi142.stdout:4/350: truncate d5/d9/d1a/f6a 849320 0 2024-06-15T01:06:57.463 INFO:tasks.workunit.client.1.smithi142.stdout:7/212: readlink d2/d8/da/d33/d25/d27/l39 0 2024-06-15T01:06:57.464 INFO:tasks.workunit.client.1.smithi142.stdout:7/213: readlink d2/d6/l21 0 2024-06-15T01:06:57.464 INFO:tasks.workunit.client.1.smithi142.stdout:6/225: dread d1a/f46 [0,4194304] 0 2024-06-15T01:06:57.465 INFO:tasks.workunit.client.1.smithi142.stdout:3/322: truncate d0/d1f/f27 3427571 0 2024-06-15T01:06:57.475 INFO:tasks.workunit.client.0.smithi137.stdout:3/251: dwrite d0/d2/d22/d24/f47 [0,4194304] 0 2024-06-15T01:06:57.476 INFO:tasks.workunit.client.1.smithi142.stdout:0/234: symlink d3/d5/d4c/l58 0 2024-06-15T01:06:57.477 INFO:tasks.workunit.client.0.smithi137.stdout:5/423: link df/d11/d14/f43 df/d76/f8f 0 2024-06-15T01:06:57.477 INFO:tasks.workunit.client.0.smithi137.stdout:3/252: chown d0/d2/d22/d24/d2d 0 1 2024-06-15T01:06:57.479 INFO:tasks.workunit.client.0.smithi137.stdout:5/424: creat df/d11/d14/d6c/f90 x:0 0 0 2024-06-15T01:06:57.479 INFO:tasks.workunit.client.0.smithi137.stdout:5/425: chown df/d11/f6d 589 1 2024-06-15T01:06:57.480 INFO:tasks.workunit.client.0.smithi137.stdout:3/253: symlink d0/d2/d22/d24/d28/d2e/l4f 0 2024-06-15T01:06:57.481 INFO:tasks.workunit.client.0.smithi137.stdout:5/426: dread - df/d44/d54/d6b/f8c zero size 2024-06-15T01:06:57.481 INFO:tasks.workunit.client.0.smithi137.stdout:3/254: chown d0/d4c 279 1 2024-06-15T01:06:57.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/234: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:06:57.483 INFO:tasks.workunit.client.1.smithi142.stdout:5/272: rename d2/d18/d2e/c30 to d2/d3e/d31/c57 0 2024-06-15T01:06:57.484 INFO:tasks.workunit.client.1.smithi142.stdout:9/223: dread d6/f9 [0,4194304] 0 2024-06-15T01:06:57.485 INFO:tasks.workunit.client.0.smithi137.stdout:5/427: creat df/d11/d34/d3b/d49/d5c/f91 x:0 0 0 2024-06-15T01:06:57.487 INFO:tasks.workunit.client.0.smithi137.stdout:5/428: symlink df/d11/d14/d6c/l92 0 2024-06-15T01:06:57.487 INFO:tasks.workunit.client.1.smithi142.stdout:4/351: dwrite d5/d9/d1a/f6a [0,4194304] 0 2024-06-15T01:06:57.488 INFO:tasks.workunit.client.1.smithi142.stdout:4/352: chown f3 876 1 2024-06-15T01:06:57.490 INFO:tasks.workunit.client.0.smithi137.stdout:5/429: write df/d11/d14/f15 [1333560,101593] 0 2024-06-15T01:06:57.490 INFO:tasks.workunit.client.1.smithi142.stdout:3/323: dread d0/d1f/f4b [0,4194304] 0 2024-06-15T01:06:57.492 INFO:tasks.workunit.client.1.smithi142.stdout:6/226: unlink d1a/f24 0 2024-06-15T01:06:57.494 INFO:tasks.workunit.client.1.smithi142.stdout:5/273: chown d2/ff 0 1 2024-06-15T01:06:57.495 INFO:tasks.workunit.client.1.smithi142.stdout:7/214: mkdir d2/d6/d4a 0 2024-06-15T01:06:57.496 INFO:tasks.workunit.client.1.smithi142.stdout:0/235: mkdir d3/d12/d22/d59 0 2024-06-15T01:06:57.498 INFO:tasks.workunit.client.1.smithi142.stdout:9/224: mkdir d6/dc/d1b/d25/d58 0 2024-06-15T01:06:57.499 INFO:tasks.workunit.client.1.smithi142.stdout:1/268: dread d6/fc [0,4194304] 0 2024-06-15T01:06:57.500 INFO:tasks.workunit.client.1.smithi142.stdout:7/215: truncate d2/d8/f42 243660 0 2024-06-15T01:06:57.500 INFO:tasks.workunit.client.1.smithi142.stdout:7/216: stat d2/d8/dd 0 2024-06-15T01:06:57.502 INFO:tasks.workunit.client.1.smithi142.stdout:8/235: symlink d0/d17/l51 0 2024-06-15T01:06:57.506 INFO:tasks.workunit.client.1.smithi142.stdout:7/217: creat d2/d8/dd/d14/f4b x:0 0 0 2024-06-15T01:06:57.506 INFO:tasks.workunit.client.1.smithi142.stdout:3/324: read d0/d1f/d20/f26 [8417024,97422] 0 2024-06-15T01:06:57.509 INFO:tasks.workunit.client.1.smithi142.stdout:5/274: write d2/de/d19/f22 [835899,27162] 0 2024-06-15T01:06:57.514 INFO:tasks.workunit.client.1.smithi142.stdout:1/269: write d6/f26 [1688997,67735] 0 2024-06-15T01:06:57.515 INFO:tasks.workunit.client.1.smithi142.stdout:8/236: read d0/d17/f27 [171372,48152] 0 2024-06-15T01:06:57.519 INFO:tasks.workunit.client.1.smithi142.stdout:2/235: dread d8/d19/d23/f26 [0,4194304] 0 2024-06-15T01:06:57.521 INFO:tasks.workunit.client.1.smithi142.stdout:4/353: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f6d [0,4194304] 0 2024-06-15T01:06:57.521 INFO:tasks.workunit.client.1.smithi142.stdout:3/325: write d0/d1f/d20/f26 [3848419,37882] 0 2024-06-15T01:06:57.521 INFO:tasks.workunit.client.0.smithi137.stdout:3/255: truncate d0/fd 3750823 0 2024-06-15T01:06:57.523 INFO:tasks.workunit.client.0.smithi137.stdout:3/256: write d0/d2/d8/f41 [684341,45661] 0 2024-06-15T01:06:57.523 INFO:tasks.workunit.client.0.smithi137.stdout:5/430: truncate df/d11/f13 44572 0 2024-06-15T01:06:57.527 INFO:tasks.workunit.client.1.smithi142.stdout:0/236: rename d3/ff to d3/d5/d44/f5a 0 2024-06-15T01:06:57.528 INFO:tasks.workunit.client.1.smithi142.stdout:4/354: write d5/d9/d23/f3b [3943698,120558] 0 2024-06-15T01:06:57.529 INFO:tasks.workunit.client.1.smithi142.stdout:7/218: dwrite d2/d8/da/d33/d25/f3f [0,4194304] 0 2024-06-15T01:06:57.531 INFO:tasks.workunit.client.0.smithi137.stdout:3/257: rename d0/d2/d22/d24/d2d to d0/d2/d50 0 2024-06-15T01:06:57.534 INFO:tasks.workunit.client.1.smithi142.stdout:6/227: dread d1a/f46 [0,4194304] 0 2024-06-15T01:06:57.535 INFO:tasks.workunit.client.0.smithi137.stdout:3/258: mknod d0/d42/c51 0 2024-06-15T01:06:57.536 INFO:tasks.workunit.client.0.smithi137.stdout:5/431: link df/d11/d14/d6c/c7d df/d44/d54/c93 0 2024-06-15T01:06:57.538 INFO:tasks.workunit.client.0.smithi137.stdout:3/259: mknod d0/d2/d23/d46/c52 0 2024-06-15T01:06:57.539 INFO:tasks.workunit.client.1.smithi142.stdout:1/270: creat d6/d12/d1c/d2c/d47/f5f x:0 0 0 2024-06-15T01:06:57.539 INFO:tasks.workunit.client.0.smithi137.stdout:3/260: write d0/d2/d23/f48 [608921,20714] 0 2024-06-15T01:06:57.540 INFO:tasks.workunit.client.1.smithi142.stdout:6/228: chown d1a/d39/d3a 1874 1 2024-06-15T01:06:57.540 INFO:tasks.workunit.client.0.smithi137.stdout:3/261: read d0/f16 [2532805,123049] 0 2024-06-15T01:06:57.541 INFO:tasks.workunit.client.1.smithi142.stdout:9/225: rename d6/dc/c1a to d6/dc/d1b/d25/d35/c59 0 2024-06-15T01:06:57.541 INFO:tasks.workunit.client.0.smithi137.stdout:5/432: link df/d11/d14/f2f df/d11/d34/f94 0 2024-06-15T01:06:57.543 INFO:tasks.workunit.client.0.smithi137.stdout:3/262: creat d0/d2/d50/f53 x:0 0 0 2024-06-15T01:06:57.545 INFO:tasks.workunit.client.1.smithi142.stdout:1/271: read - d6/d12/d28/f5d zero size 2024-06-15T01:06:57.545 INFO:tasks.workunit.client.1.smithi142.stdout:3/326: dwrite d0/d1f/d20/d5d/f5f [0,4194304] 0 2024-06-15T01:06:57.545 INFO:tasks.workunit.client.1.smithi142.stdout:5/275: creat d2/f58 x:0 0 0 2024-06-15T01:06:57.546 INFO:tasks.workunit.client.0.smithi137.stdout:3/263: creat d0/d2/d22/d24/f54 x:0 0 0 2024-06-15T01:06:57.547 INFO:tasks.workunit.client.1.smithi142.stdout:2/236: creat d8/df/f3f x:0 0 0 2024-06-15T01:06:57.548 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:57 smithi142 ceph-mon[37379]: pgmap v14: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 94 MiB/s wr, 343 op/s 2024-06-15T01:06:57.548 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:57 smithi142 ceph-mon[37379]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:57.549 INFO:tasks.workunit.client.1.smithi142.stdout:8/237: dwrite d0/f4c [0,4194304] 0 2024-06-15T01:06:57.551 INFO:tasks.workunit.client.0.smithi137.stdout:3/264: unlink d0/d2/d50/f34 0 2024-06-15T01:06:57.551 INFO:tasks.workunit.client.1.smithi142.stdout:6/229: rename l11 to d1a/d39/d3a/d51/l53 0 2024-06-15T01:06:57.551 INFO:tasks.workunit.client.1.smithi142.stdout:5/276: write d2/f25 [1201720,15685] 0 2024-06-15T01:06:57.559 INFO:tasks.workunit.client.1.smithi142.stdout:9/226: dwrite d6/dc/f15 [0,4194304] 0 2024-06-15T01:06:57.562 INFO:tasks.workunit.client.1.smithi142.stdout:2/237: fsync d8/d19/f30 0 2024-06-15T01:06:57.566 INFO:tasks.workunit.client.0.smithi137.stdout:5/433: rename df/d11/d34 to df/d44/d54/d95 0 2024-06-15T01:06:57.567 INFO:tasks.workunit.client.1.smithi142.stdout:6/230: readlink d1a/l40 0 2024-06-15T01:06:57.573 INFO:tasks.workunit.client.1.smithi142.stdout:7/219: getdents d2/d6/d22/d34 0 2024-06-15T01:06:57.575 INFO:tasks.workunit.client.1.smithi142.stdout:9/227: rmdir d6/d31 39 2024-06-15T01:06:57.575 INFO:tasks.workunit.client.1.smithi142.stdout:3/327: read d0/d1f/f27 [3256509,59423] 0 2024-06-15T01:06:57.576 INFO:tasks.workunit.client.0.smithi137.stdout:3/265: dwrite d0/d13/d37/f43 [0,4194304] 0 2024-06-15T01:06:57.577 INFO:tasks.workunit.client.1.smithi142.stdout:5/277: dwrite d2/de/f1f [0,4194304] 0 2024-06-15T01:06:57.577 INFO:tasks.workunit.client.1.smithi142.stdout:5/278: fsync d2/f28 0 2024-06-15T01:06:57.580 INFO:tasks.workunit.client.1.smithi142.stdout:1/272: creat d6/d12/d5b/f60 x:0 0 0 2024-06-15T01:06:57.581 INFO:tasks.workunit.client.0.smithi137.stdout:3/266: mknod d0/d2/d22/d24/d28/c55 0 2024-06-15T01:06:57.581 INFO:tasks.workunit.client.0.smithi137.stdout:3/267: chown d0/d2/l38 227 1 2024-06-15T01:06:57.584 INFO:tasks.workunit.client.1.smithi142.stdout:2/238: creat d8/f40 x:0 0 0 2024-06-15T01:06:57.584 INFO:tasks.workunit.client.1.smithi142.stdout:0/237: truncate d3/f23 1021902 0 2024-06-15T01:06:57.585 INFO:tasks.workunit.client.0.smithi137.stdout:3/268: mkdir d0/d56 0 2024-06-15T01:06:57.585 INFO:tasks.workunit.client.0.smithi137.stdout:3/269: write d0/d2/d50/f53 [251958,31288] 0 2024-06-15T01:06:57.586 INFO:tasks.workunit.client.1.smithi142.stdout:0/238: stat d3/d39 0 2024-06-15T01:06:57.586 INFO:tasks.workunit.client.1.smithi142.stdout:6/231: rename d1a/d39/d3a/c4e to d1a/d39/d42/c54 0 2024-06-15T01:06:57.589 INFO:tasks.workunit.client.1.smithi142.stdout:6/232: write f0 [1172263,28849] 0 2024-06-15T01:06:57.589 INFO:tasks.workunit.client.0.smithi137.stdout:3/270: creat d0/d2/d23/d46/f57 x:0 0 0 2024-06-15T01:06:57.589 INFO:tasks.workunit.client.1.smithi142.stdout:4/355: getdents d5/d1c/d2a/d30/d46/d50/d51 0 2024-06-15T01:06:57.590 INFO:tasks.workunit.client.1.smithi142.stdout:0/239: chown d3/d5/d1b/d52/c54 45045195 1 2024-06-15T01:06:57.591 INFO:tasks.workunit.client.1.smithi142.stdout:7/220: fsync d2/d8/da/d33/d25/f40 0 2024-06-15T01:06:57.594 INFO:tasks.workunit.client.1.smithi142.stdout:5/279: chown d2/de/d19/f1a 15593 1 2024-06-15T01:06:57.597 INFO:tasks.workunit.client.1.smithi142.stdout:9/228: rename d6/dc/d1b/l48 to d6/dc/d1b/d25/d35/l5a 0 2024-06-15T01:06:57.598 INFO:tasks.workunit.client.1.smithi142.stdout:2/239: mknod d8/df/d1f/c41 0 2024-06-15T01:06:57.601 INFO:tasks.workunit.client.1.smithi142.stdout:9/229: stat d6/dc/dd/d22/c2f 0 2024-06-15T01:06:57.603 INFO:tasks.workunit.client.1.smithi142.stdout:7/221: mknod d2/d6/d22/d3c/c4c 0 2024-06-15T01:06:57.604 INFO:tasks.workunit.client.1.smithi142.stdout:8/238: getdents d0/d17 0 2024-06-15T01:06:57.604 INFO:tasks.workunit.client.0.smithi137.stdout:3/271: rmdir d0/d2/d22/d24 39 2024-06-15T01:06:57.605 INFO:tasks.workunit.client.1.smithi142.stdout:6/233: write d1a/f38 [902876,5941] 0 2024-06-15T01:06:57.608 INFO:tasks.workunit.client.0.smithi137.stdout:3/272: creat d0/d2/d22/f58 x:0 0 0 2024-06-15T01:06:57.608 INFO:tasks.workunit.client.1.smithi142.stdout:9/230: mknod d6/dc/d51/d53/c5b 0 2024-06-15T01:06:57.611 INFO:tasks.workunit.client.1.smithi142.stdout:0/240: dwrite d3/f37 [0,4194304] 0 2024-06-15T01:06:57.613 INFO:tasks.workunit.client.0.smithi137.stdout:3/273: dread d0/d13/d37/f43 [0,4194304] 0 2024-06-15T01:06:57.616 INFO:tasks.workunit.client.1.smithi142.stdout:4/356: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f6b [0,4194304] 0 2024-06-15T01:06:57.617 INFO:tasks.workunit.client.0.smithi137.stdout:3/274: mknod d0/d2/d23/c59 0 2024-06-15T01:06:57.618 INFO:tasks.workunit.client.1.smithi142.stdout:6/234: readlink d1a/d39/d3a/d51/l53 0 2024-06-15T01:06:57.618 INFO:tasks.workunit.client.1.smithi142.stdout:5/280: dwrite d2/de/d27/f38 [4194304,4194304] 0 2024-06-15T01:06:57.619 INFO:tasks.workunit.client.0.smithi137.stdout:3/275: symlink d0/d4c/l5a 0 2024-06-15T01:06:57.622 INFO:tasks.workunit.client.1.smithi142.stdout:6/235: dread - d1a/d39/d42/f44 zero size 2024-06-15T01:06:57.622 INFO:tasks.workunit.client.1.smithi142.stdout:3/328: truncate d0/f39 3418149 0 2024-06-15T01:06:57.622 INFO:tasks.workunit.client.0.smithi137.stdout:3/276: symlink d0/d2/d50/l5b 0 2024-06-15T01:06:57.624 INFO:tasks.workunit.client.0.smithi137.stdout:3/277: dread - d0/d2/d23/d46/f57 zero size 2024-06-15T01:06:57.626 INFO:tasks.workunit.client.1.smithi142.stdout:9/231: readlink d6/dc/d51/d53/l56 0 2024-06-15T01:06:57.628 INFO:tasks.workunit.client.1.smithi142.stdout:8/239: dread d0/f22 [0,4194304] 0 2024-06-15T01:06:57.628 INFO:tasks.workunit.client.1.smithi142.stdout:2/240: dwrite d8/d19/d23/d25/f24 [0,4194304] 0 2024-06-15T01:06:57.628 INFO:tasks.workunit.client.1.smithi142.stdout:0/241: rename d3/d5/d44/l4d to d3/d5/l5b 0 2024-06-15T01:06:57.630 INFO:tasks.workunit.client.1.smithi142.stdout:1/273: dread d6/d12/d28/f33 [0,4194304] 0 2024-06-15T01:06:57.631 INFO:tasks.workunit.client.0.smithi137.stdout:3/278: rename d0/d13/f21 to d0/d2/d1e/f5c 0 2024-06-15T01:06:57.631 INFO:tasks.workunit.client.1.smithi142.stdout:6/236: chown d1a/l1b 27 1 2024-06-15T01:06:57.632 INFO:tasks.workunit.client.1.smithi142.stdout:4/357: read d5/f6 [914360,20989] 0 2024-06-15T01:06:57.632 INFO:tasks.workunit.client.1.smithi142.stdout:3/329: creat d0/d3/da/f62 x:0 0 0 2024-06-15T01:06:57.634 INFO:tasks.workunit.client.0.smithi137.stdout:3/279: unlink d0/d2/d8/f1c 0 2024-06-15T01:06:57.637 INFO:tasks.workunit.client.1.smithi142.stdout:1/274: dread - d6/d12/d30/f5e zero size 2024-06-15T01:06:57.637 INFO:tasks.workunit.client.1.smithi142.stdout:5/281: write d2/de/d27/f54 [2298177,121103] 0 2024-06-15T01:06:57.638 INFO:tasks.workunit.client.1.smithi142.stdout:6/237: chown f0 3 1 2024-06-15T01:06:57.638 INFO:tasks.workunit.client.1.smithi142.stdout:7/222: write d2/d6/f43 [4777213,83786] 0 2024-06-15T01:06:57.643 INFO:tasks.workunit.client.1.smithi142.stdout:8/240: mknod d0/d17/d40/c52 0 2024-06-15T01:06:57.645 INFO:tasks.workunit.client.1.smithi142.stdout:4/358: rename d5/f6 to d5/d9/da/f7f 0 2024-06-15T01:06:57.645 INFO:tasks.workunit.client.1.smithi142.stdout:8/241: fsync d0/d17/f1f 0 2024-06-15T01:06:57.648 INFO:tasks.workunit.client.1.smithi142.stdout:1/275: mkdir d6/d46/d61 0 2024-06-15T01:06:57.649 INFO:tasks.workunit.client.1.smithi142.stdout:8/242: write d0/d1/f29 [2161022,68459] 0 2024-06-15T01:06:57.650 INFO:tasks.workunit.client.1.smithi142.stdout:7/223: creat d2/d6/d22/f4d x:0 0 0 2024-06-15T01:06:57.650 INFO:tasks.workunit.client.1.smithi142.stdout:6/238: truncate d1a/d2d/d2e/f49 395966 0 2024-06-15T01:06:57.652 INFO:tasks.workunit.client.1.smithi142.stdout:4/359: creat d5/d9/d18/f80 x:0 0 0 2024-06-15T01:06:57.655 INFO:tasks.workunit.client.0.smithi137.stdout:3/280: write d0/f16 [1227935,30943] 0 2024-06-15T01:06:57.656 INFO:tasks.workunit.client.1.smithi142.stdout:3/330: creat d0/d1f/d3e/f63 x:0 0 0 2024-06-15T01:06:57.656 INFO:tasks.workunit.client.1.smithi142.stdout:5/282: dwrite d2/f9 [4194304,4194304] 0 2024-06-15T01:06:57.656 INFO:tasks.workunit.client.0.smithi137.stdout:3/281: rename d0/d2 to d0/d2/d8/d5d 22 2024-06-15T01:06:57.656 INFO:tasks.workunit.client.1.smithi142.stdout:8/243: mkdir d0/d25/d53 0 2024-06-15T01:06:57.657 INFO:tasks.workunit.client.1.smithi142.stdout:3/331: stat d0/d1f 0 2024-06-15T01:06:57.657 INFO:tasks.workunit.client.1.smithi142.stdout:8/244: chown d0/f3f 474633 1 2024-06-15T01:06:57.658 INFO:tasks.workunit.client.1.smithi142.stdout:5/283: chown d2/f21 0 1 2024-06-15T01:06:57.660 INFO:tasks.workunit.client.1.smithi142.stdout:8/245: truncate d0/f23 5732894 0 2024-06-15T01:06:57.660 INFO:tasks.workunit.client.1.smithi142.stdout:2/241: getdents d8/d19 0 2024-06-15T01:06:57.660 INFO:tasks.workunit.client.1.smithi142.stdout:3/332: write d0/d3/f54 [28552,56103] 0 2024-06-15T01:06:57.661 INFO:tasks.workunit.client.1.smithi142.stdout:6/239: symlink d1a/l55 0 2024-06-15T01:06:57.661 INFO:tasks.workunit.client.1.smithi142.stdout:5/284: fsync d2/de/d19/f36 0 2024-06-15T01:06:57.662 INFO:tasks.workunit.client.0.smithi137.stdout:3/282: rename d0/d2/d23/d46 to d0/d2/d22/d24/d5e 0 2024-06-15T01:06:57.663 INFO:tasks.workunit.client.1.smithi142.stdout:0/242: write d3/d12/d22/f24 [2860740,83618] 0 2024-06-15T01:06:57.664 INFO:tasks.workunit.client.1.smithi142.stdout:3/333: dread - d0/d3/da/f62 zero size 2024-06-15T01:06:57.664 INFO:tasks.workunit.client.1.smithi142.stdout:6/240: stat c16 0 2024-06-15T01:06:57.666 INFO:tasks.workunit.client.1.smithi142.stdout:7/224: rename d2/d8/dd/d14/c31 to d2/d8/da/d33/d25/d27/c4e 0 2024-06-15T01:06:57.666 INFO:tasks.workunit.client.1.smithi142.stdout:9/232: dwrite d6/d34/f44 [0,4194304] 0 2024-06-15T01:06:57.668 INFO:tasks.workunit.client.1.smithi142.stdout:8/246: write d0/f47 [778191,33180] 0 2024-06-15T01:06:57.670 INFO:tasks.workunit.client.0.smithi137.stdout:3/283: rename d0/d2/d22/d24/d28/f49 to d0/d13/d37/f5f 0 2024-06-15T01:06:57.671 INFO:tasks.workunit.client.1.smithi142.stdout:3/334: chown d0/d1b/c1c 0 1 2024-06-15T01:06:57.671 INFO:tasks.workunit.client.1.smithi142.stdout:2/242: creat d8/d19/d33/f42 x:0 0 0 2024-06-15T01:06:57.672 INFO:tasks.workunit.client.1.smithi142.stdout:4/360: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/c81 0 2024-06-15T01:06:57.673 INFO:tasks.workunit.client.1.smithi142.stdout:5/285: chown d2/de/c4d 124858 1 2024-06-15T01:06:57.673 INFO:tasks.workunit.client.1.smithi142.stdout:0/243: symlink d3/d5/d4c/l5c 0 2024-06-15T01:06:57.676 INFO:tasks.workunit.client.1.smithi142.stdout:6/241: symlink d1a/d2d/l56 0 2024-06-15T01:06:57.679 INFO:tasks.workunit.client.1.smithi142.stdout:8/247: rmdir d0/d17/d40 39 2024-06-15T01:06:57.679 INFO:tasks.workunit.client.1.smithi142.stdout:1/276: dwrite d6/d12/f3e [0,4194304] 0 2024-06-15T01:06:57.679 INFO:tasks.workunit.client.0.smithi137.stdout:3/284: rmdir d0/d2 39 2024-06-15T01:06:57.682 INFO:tasks.workunit.client.1.smithi142.stdout:0/244: write d3/d5/f34 [713248,10538] 0 2024-06-15T01:06:57.684 INFO:tasks.workunit.client.1.smithi142.stdout:3/335: creat d0/d3/d15/f64 x:0 0 0 2024-06-15T01:06:57.685 INFO:tasks.workunit.client.0.smithi137.stdout:3/285: link d0/d13/d37/f43 d0/d42/f60 0 2024-06-15T01:06:57.685 INFO:tasks.workunit.client.1.smithi142.stdout:9/233: readlink d6/dc/d1b/d25/d35/l5a 0 2024-06-15T01:06:57.688 INFO:tasks.workunit.client.1.smithi142.stdout:3/336: write d0/d1b/f41 [75307,38246] 0 2024-06-15T01:06:57.688 INFO:tasks.workunit.client.0.smithi137.stdout:3/286: link d0/d2/d22/c2b d0/d56/c61 0 2024-06-15T01:06:57.689 INFO:tasks.workunit.client.0.smithi137.stdout:3/287: stat d0/d2/d23/c59 0 2024-06-15T01:06:57.689 INFO:tasks.workunit.client.1.smithi142.stdout:2/243: write d8/d19/d23/d25/f27 [2944009,84721] 0 2024-06-15T01:06:57.690 INFO:tasks.workunit.client.1.smithi142.stdout:5/286: mknod d2/d3e/c59 0 2024-06-15T01:06:57.691 INFO:tasks.workunit.client.0.smithi137.stdout:3/288: rename d0 to d0/d2/d62 22 2024-06-15T01:06:57.692 INFO:tasks.workunit.client.1.smithi142.stdout:4/361: dread d5/d9/d1a/f6a [0,4194304] 0 2024-06-15T01:06:57.692 INFO:tasks.workunit.client.1.smithi142.stdout:0/245: creat d3/d5/d1e/f5d x:0 0 0 2024-06-15T01:06:57.693 INFO:tasks.workunit.client.1.smithi142.stdout:1/277: creat d6/d12/d1c/d2c/f62 x:0 0 0 2024-06-15T01:06:57.694 INFO:tasks.workunit.client.1.smithi142.stdout:6/242: link d1a/d39/f3f d1a/d39/f57 0 2024-06-15T01:06:57.697 INFO:tasks.workunit.client.0.smithi137.stdout:3/289: dwrite d0/d2/f15 [4194304,4194304] 0 2024-06-15T01:06:57.698 INFO:tasks.workunit.client.1.smithi142.stdout:2/244: rmdir d8/d19/d23/d25 39 2024-06-15T01:06:57.698 INFO:tasks.workunit.client.1.smithi142.stdout:9/234: creat d6/dc/d1b/d25/d35/f5c x:0 0 0 2024-06-15T01:06:57.699 INFO:tasks.workunit.client.1.smithi142.stdout:8/248: link d0/d1/c2 d0/d25/d53/c54 0 2024-06-15T01:06:57.699 INFO:tasks.workunit.client.1.smithi142.stdout:2/245: fsync f6 0 2024-06-15T01:06:57.700 INFO:tasks.workunit.client.0.smithi137.stdout:3/290: mkdir d0/d2/d22/d63 0 2024-06-15T01:06:57.701 INFO:tasks.workunit.client.1.smithi142.stdout:0/246: write d3/d39/f45 [891082,78382] 0 2024-06-15T01:06:57.703 INFO:tasks.workunit.client.0.smithi137.stdout:3/291: rmdir d0/d2/d22/d24 39 2024-06-15T01:06:57.705 INFO:tasks.workunit.client.0.smithi137.stdout:3/292: creat d0/d2/d23/f64 x:0 0 0 2024-06-15T01:06:57.707 INFO:tasks.workunit.client.1.smithi142.stdout:8/249: chown d0/c45 136535 1 2024-06-15T01:06:57.708 INFO:tasks.workunit.client.1.smithi142.stdout:0/247: chown d3/d5/d1e/f2b 40 1 2024-06-15T01:06:57.709 INFO:tasks.workunit.client.0.smithi137.stdout:3/293: chown d0/d2/d22/d24/d28/c55 376627179 1 2024-06-15T01:06:57.710 INFO:tasks.workunit.client.1.smithi142.stdout:6/243: mkdir d1a/d39/d42/d4c/d58 0 2024-06-15T01:06:57.711 INFO:tasks.workunit.client.0.smithi137.stdout:3/294: creat d0/d2f/f65 x:0 0 0 2024-06-15T01:06:57.711 INFO:tasks.workunit.client.0.smithi137.stdout:3/295: write d0/d2f/f65 [296970,127201] 0 2024-06-15T01:06:57.711 INFO:tasks.workunit.client.0.smithi137.stdout:3/296: fsync d0/d2/d22/d24/f4b 0 2024-06-15T01:06:57.712 INFO:tasks.workunit.client.1.smithi142.stdout:3/337: dwrite d0/d1f/d20/d5d/f5f [0,4194304] 0 2024-06-15T01:06:57.714 INFO:tasks.workunit.client.0.smithi137.stdout:3/297: mkdir d0/d13/d37/d66 0 2024-06-15T01:06:57.714 INFO:tasks.workunit.client.1.smithi142.stdout:5/287: dwrite d2/de/d19/f2a [0,4194304] 0 2024-06-15T01:06:57.714 INFO:tasks.workunit.client.1.smithi142.stdout:7/225: dread d2/f49 [0,4194304] 0 2024-06-15T01:06:57.715 INFO:tasks.workunit.client.1.smithi142.stdout:6/244: stat d1a/d39/d3a/d51/l53 0 2024-06-15T01:06:57.715 INFO:tasks.workunit.client.0.smithi137.stdout:3/298: chown d0/d2/d1e/f5c 70758560 1 2024-06-15T01:06:57.717 INFO:tasks.workunit.client.1.smithi142.stdout:9/235: symlink d6/dc/d1b/d25/d35/l5d 0 2024-06-15T01:06:57.717 INFO:tasks.workunit.client.0.smithi137.stdout:3/299: getdents d0/d2/d50 0 2024-06-15T01:06:57.717 INFO:tasks.workunit.client.1.smithi142.stdout:5/288: stat d2/c53 0 2024-06-15T01:06:57.718 INFO:tasks.workunit.client.0.smithi137.stdout:3/300: dread - d0/d2/d22/f58 zero size 2024-06-15T01:06:57.720 INFO:tasks.workunit.client.1.smithi142.stdout:1/278: write d6/d12/d28/f40 [1656886,87413] 0 2024-06-15T01:06:57.721 INFO:tasks.workunit.client.1.smithi142.stdout:8/250: rename d0/d1/l49 to d0/d1/l55 0 2024-06-15T01:06:57.723 INFO:tasks.workunit.client.1.smithi142.stdout:8/251: chown d0/d1/l2e 3654603 1 2024-06-15T01:06:57.727 INFO:tasks.workunit.client.1.smithi142.stdout:3/338: dwrite d0/d1f/d20/f26 [4194304,4194304] 0 2024-06-15T01:06:57.733 INFO:tasks.workunit.client.0.smithi137.stdout:6/168: sync 2024-06-15T01:06:57.733 INFO:tasks.workunit.client.0.smithi137.stdout:3/301: getdents d0/d2f 0 2024-06-15T01:06:57.736 INFO:tasks.workunit.client.1.smithi142.stdout:5/289: creat d2/de/f5a x:0 0 0 2024-06-15T01:06:57.738 INFO:tasks.workunit.client.1.smithi142.stdout:1/279: mkdir d6/d12/d3c/d63 0 2024-06-15T01:06:57.740 INFO:tasks.workunit.client.1.smithi142.stdout:7/226: truncate d2/d8/da/d33/f20 1274595 0 2024-06-15T01:06:57.741 INFO:tasks.workunit.client.0.smithi137.stdout:6/169: dwrite d0/d5/f8 [4194304,4194304] 0 2024-06-15T01:06:57.741 INFO:tasks.workunit.client.1.smithi142.stdout:7/227: truncate d2/d6/d22/f28 460093 0 2024-06-15T01:06:57.743 INFO:tasks.workunit.client.1.smithi142.stdout:4/362: getdents d5/d9/d23/d32 0 2024-06-15T01:06:57.743 INFO:tasks.workunit.client.1.smithi142.stdout:7/228: chown d2/d6/d22/d3c/c4c 306 1 2024-06-15T01:06:57.743 INFO:tasks.workunit.client.1.smithi142.stdout:4/363: fsync d5/d1c/d2a/d30/d46/d50/f5e 0 2024-06-15T01:06:57.745 INFO:tasks.workunit.client.1.smithi142.stdout:4/364: readlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/l39 0 2024-06-15T01:06:57.745 INFO:tasks.workunit.client.0.smithi137.stdout:3/302: mkdir d0/d2/d8/d67 0 2024-06-15T01:06:57.745 INFO:tasks.workunit.client.1.smithi142.stdout:4/365: fdatasync d5/d1c/f34 0 2024-06-15T01:06:57.746 INFO:tasks.workunit.client.0.smithi137.stdout:6/170: fsync d0/d1c/d1d/f1f 0 2024-06-15T01:06:57.746 INFO:tasks.workunit.client.0.smithi137.stdout:6/171: dread - d0/f2a zero size 2024-06-15T01:06:57.747 INFO:tasks.workunit.client.0.smithi137.stdout:6/172: chown d0/fc 347 1 2024-06-15T01:06:57.752 INFO:tasks.workunit.client.0.smithi137.stdout:3/303: rename d0/d2/d22/d24/d28/d2e/d39 to d0/d2/d23/d68 0 2024-06-15T01:06:57.758 INFO:tasks.workunit.client.1.smithi142.stdout:2/246: mkdir d8/d19/d23/d25/d22/d43 0 2024-06-15T01:06:57.759 INFO:tasks.workunit.client.0.smithi137.stdout:6/173: dwrite d0/d5/f13 [0,4194304] 0 2024-06-15T01:06:57.760 INFO:tasks.workunit.client.1.smithi142.stdout:6/245: dread d1a/f38 [0,4194304] 0 2024-06-15T01:06:57.761 INFO:tasks.workunit.client.1.smithi142.stdout:9/236: creat d6/dc/d1b/d25/d58/f5e x:0 0 0 2024-06-15T01:06:57.762 INFO:tasks.workunit.client.1.smithi142.stdout:6/246: truncate d1a/d2d/d3b/f41 813362 0 2024-06-15T01:06:57.762 INFO:tasks.workunit.client.1.smithi142.stdout:5/290: symlink d2/d3e/d31/l5b 0 2024-06-15T01:06:57.763 INFO:tasks.workunit.client.1.smithi142.stdout:9/237: truncate d6/f45 1028507 0 2024-06-15T01:06:57.765 INFO:tasks.workunit.client.1.smithi142.stdout:1/280: creat d6/d12/d28/f64 x:0 0 0 2024-06-15T01:06:57.766 INFO:tasks.workunit.client.0.smithi137.stdout:3/304: write d0/d13/f20 [766335,9162] 0 2024-06-15T01:06:57.767 INFO:tasks.workunit.client.0.smithi137.stdout:3/305: rmdir d0/d2/d22/d24/d28/d2e 39 2024-06-15T01:06:57.770 INFO:tasks.workunit.client.1.smithi142.stdout:5/291: dwrite d2/de/d27/f38 [0,4194304] 0 2024-06-15T01:06:57.770 INFO:tasks.workunit.client.0.smithi137.stdout:3/306: creat d0/d2f/f69 x:0 0 0 2024-06-15T01:06:57.771 INFO:tasks.workunit.client.1.smithi142.stdout:2/247: symlink d8/df/d1f/l44 0 2024-06-15T01:06:57.773 INFO:tasks.workunit.client.0.smithi137.stdout:3/307: creat d0/d56/f6a x:0 0 0 2024-06-15T01:06:57.774 INFO:tasks.workunit.client.1.smithi142.stdout:0/248: link d3/d5/d1e/c20 d3/d12/c5e 0 2024-06-15T01:06:57.776 INFO:tasks.workunit.client.0.smithi137.stdout:3/308: unlink d0/d2/d22/d24/f4b 0 2024-06-15T01:06:57.777 INFO:tasks.workunit.client.1.smithi142.stdout:8/252: rename d0/f22 to d0/d17/f56 0 2024-06-15T01:06:57.777 INFO:tasks.workunit.client.1.smithi142.stdout:6/247: dwrite d1a/d2d/f32 [0,4194304] 0 2024-06-15T01:06:57.790 INFO:tasks.workunit.client.0.smithi137.stdout:3/309: dwrite d0/d42/f60 [0,4194304] 0 2024-06-15T01:06:57.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/310: readlink d0/d2/d1e/l3e 0 2024-06-15T01:06:57.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/311: stat d0/d2/d23/f3f 0 2024-06-15T01:06:57.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/312: stat d0/d2/d22/d24/d28 0 2024-06-15T01:06:57.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/313: fdatasync d0/d2/d22/d24/f47 0 2024-06-15T01:06:57.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/314: write d0/d2/d23/f3d [3422724,95204] 0 2024-06-15T01:06:57.797 INFO:tasks.workunit.client.0.smithi137.stdout:3/315: write d0/d2/d22/d24/d28/d2e/f3b [861841,33401] 0 2024-06-15T01:06:57.799 INFO:tasks.workunit.client.0.smithi137.stdout:3/316: fdatasync d0/fd 0 2024-06-15T01:06:57.800 INFO:tasks.workunit.client.1.smithi142.stdout:2/248: mknod d8/c45 0 2024-06-15T01:06:57.801 INFO:tasks.workunit.client.0.smithi137.stdout:3/317: stat d0/d2/d1e/f5c 0 2024-06-15T01:06:57.801 INFO:tasks.workunit.client.0.smithi137.stdout:3/318: chown d0/d2/d1e 611322 1 2024-06-15T01:06:57.801 INFO:tasks.workunit.client.1.smithi142.stdout:7/229: rmdir d2/d6/d22/d48 0 2024-06-15T01:06:57.802 INFO:tasks.workunit.client.1.smithi142.stdout:1/281: mkdir d6/d12/d3c/d63/d65 0 2024-06-15T01:06:57.803 INFO:tasks.workunit.client.0.smithi137.stdout:3/319: symlink d0/d2/d22/d63/l6b 0 2024-06-15T01:06:57.804 INFO:tasks.workunit.client.1.smithi142.stdout:9/238: creat d6/d34/f5f x:0 0 0 2024-06-15T01:06:57.804 INFO:tasks.workunit.client.1.smithi142.stdout:6/248: read d1a/d2d/f52 [926543,108135] 0 2024-06-15T01:06:57.805 INFO:tasks.workunit.client.1.smithi142.stdout:3/339: write d0/d1f/f4b [1504663,52501] 0 2024-06-15T01:06:57.806 INFO:tasks.workunit.client.1.smithi142.stdout:0/249: creat d3/d39/d3e/f5f x:0 0 0 2024-06-15T01:06:57.807 INFO:tasks.workunit.client.1.smithi142.stdout:1/282: rmdir d6/d12/d1c/d1e 39 2024-06-15T01:06:57.807 INFO:tasks.workunit.client.1.smithi142.stdout:9/239: write d6/dc/dd/f46 [350082,51645] 0 2024-06-15T01:06:57.811 INFO:tasks.workunit.client.1.smithi142.stdout:7/230: creat d2/d6/d22/d47/f4f x:0 0 0 2024-06-15T01:06:57.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.816+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9118104d90 con 0x7f910406bba0 2024-06-15T01:06:57.813 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:57 smithi137 ceph-mon[29354]: pgmap v14: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 94 MiB/s wr, 343 op/s 2024-06-15T01:06:57.813 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:57 smithi137 ceph-mon[29354]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:57.821 INFO:tasks.workunit.client.1.smithi142.stdout:0/250: getdents d3/d5/d1e 0 2024-06-15T01:06:57.821 INFO:tasks.workunit.client.1.smithi142.stdout:4/366: dread f2 [0,4194304] 0 2024-06-15T01:06:57.822 INFO:tasks.workunit.client.1.smithi142.stdout:1/283: write d6/d12/d1c/f32 [1429147,34561] 0 2024-06-15T01:06:57.828 INFO:tasks.workunit.client.1.smithi142.stdout:3/340: dwrite d0/d3/d15/f64 [0,4194304] 0 2024-06-15T01:06:57.829 INFO:tasks.workunit.client.1.smithi142.stdout:9/240: dwrite d6/dc/d4e/f57 [0,4194304] 0 2024-06-15T01:06:57.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.834+0000 7f910effd700 1 -- 172.21.15.137:0/3052017073 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9118104d90 con 0x7f910406bba0 2024-06-15T01:06:57.832 INFO:tasks.workunit.client.1.smithi142.stdout:8/253: truncate d0/f46 1142365 0 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (6m) 13s ago 8m 25.5M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (8m) 13s ago 8m 7746k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (7m) 13s ago 7m 7386k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (8m) 13s ago 8m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (7m) 13s ago 7m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (6m) 13s ago 7m 86.7M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (2m) 13s ago 2m 103M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:06:57.832 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (2m) 13s ago 2m 107M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (2m) 13s ago 2m 12.9M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (2m) 13s ago 2m 15.9M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:9283,8765,8443 running (9m) 13s ago 9m 111M - 18.2.1 5be31c24972a 61486e562eb9 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (34s) 13s ago 7m 462M - 18.2.2-1443-g6f4b0956 e7282a3bffba df5c0f028b61 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (9m) 13s ago 9m 49.7M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (6m) 13s ago 6m 40.4M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (8m) 13s ago 8m 16.4M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 13s ago 7m 16.4M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (6m) 13s ago 6m 111M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (5m) 13s ago 5m 121M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (5m) 13s ago 5m 102M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (4m) 13s ago 4m 124M 4970M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (4m) 13s ago 4m 116M 4970M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:06:57.833 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (4m) 13s ago 4m 104M 4970M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:06:57.834 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (6m) 13s ago 7m 58.5M - 2.43.0 a07b618ecd1d 904ef44cd0a5 2024-06-15T01:06:57.834 INFO:tasks.workunit.client.1.smithi142.stdout:9/241: chown d6/d34 2082871414 1 2024-06-15T01:06:57.834 INFO:tasks.workunit.client.1.smithi142.stdout:3/341: chown d0/d3/da 1 1 2024-06-15T01:06:57.834 INFO:tasks.workunit.client.1.smithi142.stdout:7/231: dwrite d2/d6/d22/f28 [0,4194304] 0 2024-06-15T01:06:57.835 INFO:tasks.workunit.client.1.smithi142.stdout:0/251: symlink d3/d5/d1b/d52/l60 0 2024-06-15T01:06:57.837 INFO:tasks.workunit.client.1.smithi142.stdout:2/249: dread d8/d19/f30 [0,4194304] 0 2024-06-15T01:06:57.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.841+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 msgr2=0x7f910406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.841+0000 7f911fa93700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 0x7f910406e060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f911400b5c0 tx=0x7f9114005960 comp rx=0 tx=0).stop 2024-06-15T01:06:57.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.841+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 msgr2=0x7f9118105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:57.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.841+0000 7f911fa93700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f910800c8a0 tx=0x7f910800cc60 comp rx=0 tx=0).stop 2024-06-15T01:06:57.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.842+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 shutdown_connections 2024-06-15T01:06:57.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.842+0000 7f911fa93700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f910406bba0 0x7f910406e060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.842+0000 7f911fa93700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91180ff240 0x7f9118105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.842+0000 7f911fa93700 1 --2- 172.21.15.137:0/3052017073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9118100440 0x7f9118105dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:57.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.842+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 >> 172.21.15.137:0/3052017073 conn(0x7f91180fa7c0 msgr2=0x7f9118103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:57.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.843+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 shutdown_connections 2024-06-15T01:06:57.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:57.843+0000 7f911fa93700 1 -- 172.21.15.137:0/3052017073 wait complete. 2024-06-15T01:06:57.842 INFO:tasks.workunit.client.1.smithi142.stdout:6/249: dwrite d1a/d39/f3f [0,4194304] 0 2024-06-15T01:06:57.842 INFO:tasks.workunit.client.1.smithi142.stdout:1/284: symlink d6/d12/l66 0 2024-06-15T01:06:57.842 INFO:tasks.workunit.client.1.smithi142.stdout:4/367: dread d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:57.843 INFO:tasks.workunit.client.1.smithi142.stdout:8/254: rename d0/c45 to d0/d25/d53/c57 0 2024-06-15T01:06:57.844 INFO:tasks.workunit.client.1.smithi142.stdout:7/232: creat d2/d6/d22/d47/f50 x:0 0 0 2024-06-15T01:06:57.846 INFO:tasks.workunit.client.1.smithi142.stdout:9/242: mknod d6/dc/c60 0 2024-06-15T01:06:57.850 INFO:tasks.workunit.client.1.smithi142.stdout:6/250: creat d1a/d39/d3a/f59 x:0 0 0 2024-06-15T01:06:57.853 INFO:tasks.workunit.client.1.smithi142.stdout:1/285: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:06:57.854 INFO:tasks.workunit.client.1.smithi142.stdout:4/368: creat d5/d9/d1a/f82 x:0 0 0 2024-06-15T01:06:57.854 INFO:tasks.workunit.client.1.smithi142.stdout:2/250: rename d8/fa to d8/d19/d23/d25/d22/d43/f46 0 2024-06-15T01:06:57.854 INFO:tasks.workunit.client.1.smithi142.stdout:8/255: dwrite d0/f14 [0,4194304] 0 2024-06-15T01:06:57.858 INFO:tasks.workunit.client.1.smithi142.stdout:2/251: chown d8/df/c17 4 1 2024-06-15T01:06:57.858 INFO:tasks.workunit.client.1.smithi142.stdout:7/233: symlink d2/d8/dd/l51 0 2024-06-15T01:06:57.859 INFO:tasks.workunit.client.1.smithi142.stdout:8/256: readlink d0/d17/l1e 0 2024-06-15T01:06:57.860 INFO:tasks.workunit.client.1.smithi142.stdout:4/369: symlink d5/d9/d18/l83 0 2024-06-15T01:06:57.861 INFO:tasks.workunit.client.1.smithi142.stdout:7/234: rmdir d2 39 2024-06-15T01:06:57.862 INFO:tasks.workunit.client.1.smithi142.stdout:6/251: creat d1a/d39/d3a/f5a x:0 0 0 2024-06-15T01:06:57.863 INFO:tasks.workunit.client.1.smithi142.stdout:4/370: truncate d5/d9/d18/f80 492192 0 2024-06-15T01:06:57.866 INFO:tasks.workunit.client.1.smithi142.stdout:1/286: mknod d6/d46/d61/c67 0 2024-06-15T01:06:57.866 INFO:tasks.workunit.client.1.smithi142.stdout:2/252: read d8/df/f2d [1636741,97788] 0 2024-06-15T01:06:57.867 INFO:tasks.workunit.client.1.smithi142.stdout:4/371: chown d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/l66 1019 1 2024-06-15T01:06:57.870 INFO:tasks.workunit.client.1.smithi142.stdout:6/252: rmdir d1a/d2d/d2e 39 2024-06-15T01:06:57.871 INFO:tasks.workunit.client.1.smithi142.stdout:9/243: creat d6/dc/f61 x:0 0 0 2024-06-15T01:06:57.876 INFO:tasks.workunit.client.1.smithi142.stdout:7/235: creat d2/d8/da/d33/f52 x:0 0 0 2024-06-15T01:06:57.876 INFO:tasks.workunit.client.1.smithi142.stdout:8/257: dwrite d0/f33 [0,4194304] 0 2024-06-15T01:06:57.879 INFO:tasks.workunit.client.1.smithi142.stdout:6/253: mknod d1a/d39/d3a/d51/c5b 0 2024-06-15T01:06:57.882 INFO:tasks.workunit.client.1.smithi142.stdout:9/244: mkdir d6/dc/d1b/d25/d58/d62 0 2024-06-15T01:06:57.885 INFO:tasks.workunit.client.1.smithi142.stdout:1/287: dread d6/f10 [0,4194304] 0 2024-06-15T01:06:57.885 INFO:tasks.workunit.client.1.smithi142.stdout:3/342: truncate d0/f2c 2573806 0 2024-06-15T01:06:57.887 INFO:tasks.workunit.client.1.smithi142.stdout:2/253: dwrite d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:06:57.887 INFO:tasks.workunit.client.1.smithi142.stdout:7/236: symlink d2/d8/da/d33/d25/d27/l53 0 2024-06-15T01:06:57.890 INFO:tasks.workunit.client.1.smithi142.stdout:0/252: truncate d3/d5/fb 1483466 0 2024-06-15T01:06:57.893 INFO:tasks.workunit.client.1.smithi142.stdout:4/372: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/f60 [0,4194304] 0 2024-06-15T01:06:57.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/254: creat d1a/d39/d3a/d51/f5c x:0 0 0 2024-06-15T01:06:57.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/255: stat fc 0 2024-06-15T01:06:57.897 INFO:tasks.workunit.client.1.smithi142.stdout:2/254: symlink d8/d19/d33/l47 0 2024-06-15T01:06:57.898 INFO:tasks.workunit.client.1.smithi142.stdout:0/253: mkdir d3/d5/d4c/d61 0 2024-06-15T01:06:57.898 INFO:tasks.workunit.client.1.smithi142.stdout:7/237: chown d2/d8/da/d33/d25/d27/l39 0 1 2024-06-15T01:06:57.899 INFO:tasks.workunit.client.1.smithi142.stdout:8/258: dwrite d0/d1/f2c [0,4194304] 0 2024-06-15T01:06:57.903 INFO:tasks.workunit.client.1.smithi142.stdout:1/288: symlink d6/d12/d1c/d2c/d47/l68 0 2024-06-15T01:06:57.905 INFO:tasks.workunit.client.1.smithi142.stdout:8/259: write d0/d17/f1f [732543,73422] 0 2024-06-15T01:06:57.908 INFO:tasks.workunit.client.1.smithi142.stdout:9/245: dwrite d6/d31/f3e [0,4194304] 0 2024-06-15T01:06:57.908 INFO:tasks.workunit.client.1.smithi142.stdout:1/289: write d6/d12/f3e [3871025,128439] 0 2024-06-15T01:06:57.911 INFO:tasks.workunit.client.1.smithi142.stdout:0/254: write d3/d5/d44/f5a [1092475,82577] 0 2024-06-15T01:06:57.911 INFO:tasks.workunit.client.1.smithi142.stdout:6/256: dread d1a/d2d/d2e/f49 [0,4194304] 0 2024-06-15T01:06:57.912 INFO:tasks.workunit.client.1.smithi142.stdout:7/238: mknod d2/d8/da/d33/d25/d27/c54 0 2024-06-15T01:06:57.917 INFO:tasks.workunit.client.1.smithi142.stdout:1/290: write d6/d12/d28/f51 [692875,485] 0 2024-06-15T01:06:57.921 INFO:tasks.workunit.client.1.smithi142.stdout:8/260: creat d0/d25/d53/f58 x:0 0 0 2024-06-15T01:06:57.922 INFO:tasks.workunit.client.1.smithi142.stdout:1/291: dread - d6/d3a/f43 zero size 2024-06-15T01:06:57.922 INFO:tasks.workunit.client.1.smithi142.stdout:2/255: creat d8/df/f48 x:0 0 0 2024-06-15T01:06:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:4/202: sync 2024-06-15T01:06:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:7/210: sync 2024-06-15T01:06:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:1/213: sync 2024-06-15T01:06:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:2/162: sync 2024-06-15T01:06:57.925 INFO:tasks.workunit.client.1.smithi142.stdout:4/373: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:06:57.927 INFO:tasks.workunit.client.0.smithi137.stdout:4/203: read dd/f17 [3242865,22500] 0 2024-06-15T01:06:57.927 INFO:tasks.workunit.client.1.smithi142.stdout:4/374: stat d5/f7 0 2024-06-15T01:06:57.928 INFO:tasks.workunit.client.0.smithi137.stdout:1/214: fdatasync d0/f3b 0 2024-06-15T01:06:57.929 INFO:tasks.workunit.client.0.smithi137.stdout:4/204: write dd/f14 [3541836,128375] 0 2024-06-15T01:06:57.930 INFO:tasks.workunit.client.0.smithi137.stdout:1/215: write d0/d1f/f4d [368316,53913] 0 2024-06-15T01:06:57.930 INFO:tasks.workunit.client.1.smithi142.stdout:2/256: stat d8/df/f48 0 2024-06-15T01:06:57.931 INFO:tasks.workunit.client.1.smithi142.stdout:9/246: dwrite d6/f45 [0,4194304] 0 2024-06-15T01:06:57.932 INFO:tasks.workunit.client.0.smithi137.stdout:7/211: mknod df/d13/d2e/d1a/c46 0 2024-06-15T01:06:57.933 INFO:tasks.workunit.client.0.smithi137.stdout:7/212: rename df to df/d13/d47 22 2024-06-15T01:06:57.934 INFO:tasks.workunit.client.1.smithi142.stdout:4/375: chown d5/d9/d18 0 1 2024-06-15T01:06:57.935 INFO:tasks.workunit.client.1.smithi142.stdout:0/255: mknod d3/d12/c62 0 2024-06-15T01:06:57.935 INFO:tasks.workunit.client.0.smithi137.stdout:2/163: dwrite d0/de/d10/f33 [0,4194304] 0 2024-06-15T01:06:57.938 INFO:tasks.workunit.client.1.smithi142.stdout:8/261: mknod d0/c59 0 2024-06-15T01:06:57.938 INFO:tasks.workunit.client.1.smithi142.stdout:7/239: dwrite d2/d8/dd/d14/f4b [0,4194304] 0 2024-06-15T01:06:57.939 INFO:tasks.workunit.client.0.smithi137.stdout:4/205: creat dd/d15/d36/d45/f4e x:0 0 0 2024-06-15T01:06:57.940 INFO:tasks.workunit.client.1.smithi142.stdout:3/343: dread d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:57.940 INFO:tasks.workunit.client.1.smithi142.stdout:8/262: chown d0/f47 424639214 1 2024-06-15T01:06:57.941 INFO:tasks.workunit.client.1.smithi142.stdout:0/256: truncate d3/d5/d44/f49 1033279 0 2024-06-15T01:06:57.941 INFO:tasks.workunit.client.1.smithi142.stdout:3/344: dread - d0/d1f/d3e/f63 zero size 2024-06-15T01:06:57.942 INFO:tasks.workunit.client.1.smithi142.stdout:1/292: mknod d6/d12/d1c/c69 0 2024-06-15T01:06:57.942 INFO:tasks.workunit.client.0.smithi137.stdout:1/216: mknod d0/d2d/c54 0 2024-06-15T01:06:57.944 INFO:tasks.workunit.client.1.smithi142.stdout:3/345: truncate d0/d1f/f38 980928 0 2024-06-15T01:06:57.945 INFO:tasks.workunit.client.1.smithi142.stdout:9/247: creat d6/dc/d1b/d25/d58/f63 x:0 0 0 2024-06-15T01:06:57.948 INFO:tasks.workunit.client.0.smithi137.stdout:7/213: readlink df/d13/l44 0 2024-06-15T01:06:57.949 INFO:tasks.workunit.client.0.smithi137.stdout:4/206: dwrite dd/d10/f1c [0,4194304] 0 2024-06-15T01:06:57.949 INFO:tasks.workunit.client.1.smithi142.stdout:6/257: write d1a/f2c [27913,125755] 0 2024-06-15T01:06:57.949 INFO:tasks.workunit.client.0.smithi137.stdout:7/214: chown df/f14 9805 1 2024-06-15T01:06:57.950 INFO:tasks.workunit.client.1.smithi142.stdout:8/263: fdatasync d0/d17/d40/f4a 0 2024-06-15T01:06:57.950 INFO:tasks.workunit.client.1.smithi142.stdout:6/258: write d1a/d2d/d2e/f31 [400122,38992] 0 2024-06-15T01:06:57.951 INFO:tasks.workunit.client.0.smithi137.stdout:2/164: symlink d0/d1c/l34 0 2024-06-15T01:06:57.951 INFO:tasks.workunit.client.1.smithi142.stdout:0/257: creat d3/d5/d1e/f63 x:0 0 0 2024-06-15T01:06:57.952 INFO:tasks.workunit.client.1.smithi142.stdout:4/376: mknod d5/d9/c84 0 2024-06-15T01:06:57.953 INFO:tasks.workunit.client.1.smithi142.stdout:4/377: readlink d5/d9/l57 0 2024-06-15T01:06:57.953 INFO:tasks.workunit.client.1.smithi142.stdout:4/378: stat d5/d9/d23/f4e 0 2024-06-15T01:06:57.953 INFO:tasks.workunit.client.1.smithi142.stdout:3/346: rmdir d0/d3/da/d18 39 2024-06-15T01:06:57.958 INFO:tasks.workunit.client.1.smithi142.stdout:9/248: rename d6/dc/d1b/d25/d58/f5e to d6/dc/d1b/d25/d58/f64 0 2024-06-15T01:06:57.958 INFO:tasks.workunit.client.1.smithi142.stdout:8/264: chown d0/d25/d53/c57 1828330 1 2024-06-15T01:06:57.959 INFO:tasks.workunit.client.0.smithi137.stdout:1/217: dread d0/d3/d14/f38 [0,4194304] 0 2024-06-15T01:06:57.959 INFO:tasks.workunit.client.1.smithi142.stdout:1/293: symlink d6/d12/d1c/l6a 0 2024-06-15T01:06:57.961 INFO:tasks.workunit.client.1.smithi142.stdout:2/257: rmdir d8/d19/d23/d25/d22/d34 0 2024-06-15T01:06:57.963 INFO:tasks.workunit.client.1.smithi142.stdout:4/379: symlink d5/d9/da/d7e/l85 0 2024-06-15T01:06:57.963 INFO:tasks.workunit.client.0.smithi137.stdout:7/215: rename df/d13/d2e/f31 to df/d18/d1c/d1d/d27/f48 0 2024-06-15T01:06:57.963 INFO:tasks.workunit.client.0.smithi137.stdout:7/216: chown df/d18/d1c/d1d/f28 4266 1 2024-06-15T01:06:57.964 INFO:tasks.workunit.client.0.smithi137.stdout:7/217: write df/f3d [795750,121802] 0 2024-06-15T01:06:57.965 INFO:tasks.workunit.client.1.smithi142.stdout:7/240: creat d2/d8/da/f55 x:0 0 0 2024-06-15T01:06:57.965 INFO:tasks.workunit.client.1.smithi142.stdout:9/249: readlink d6/dc/l13 0 2024-06-15T01:06:57.966 INFO:tasks.workunit.client.0.smithi137.stdout:7/218: write df/d13/d2e/d1a/f40 [478914,10351] 0 2024-06-15T01:06:57.966 INFO:tasks.workunit.client.1.smithi142.stdout:7/241: dread - d2/d8/da/d33/d25/f40 zero size 2024-06-15T01:06:57.968 INFO:tasks.workunit.client.1.smithi142.stdout:0/258: rename d3/d12/l32 to d3/d5/d1b/l64 0 2024-06-15T01:06:57.969 INFO:tasks.workunit.client.0.smithi137.stdout:1/218: creat d0/d3/d7/f55 x:0 0 0 2024-06-15T01:06:57.969 INFO:tasks.workunit.client.1.smithi142.stdout:2/258: fdatasync d8/d19/d23/d25/f18 0 2024-06-15T01:06:57.969 INFO:tasks.workunit.client.1.smithi142.stdout:0/259: write d3/d5/f34 [850773,32281] 0 2024-06-15T01:06:57.970 INFO:tasks.workunit.client.0.smithi137.stdout:1/219: write d0/d3/d14/d34/f39 [755096,49901] 0 2024-06-15T01:06:57.970 INFO:tasks.workunit.client.1.smithi142.stdout:0/260: readlink d3/l3a 0 2024-06-15T01:06:57.971 INFO:tasks.workunit.client.1.smithi142.stdout:4/380: mknod d5/d9/d41/c86 0 2024-06-15T01:06:57.972 INFO:tasks.workunit.client.0.smithi137.stdout:7/219: write df/d13/d2e/f19 [2001520,115179] 0 2024-06-15T01:06:57.973 INFO:tasks.workunit.client.1.smithi142.stdout:9/250: mkdir d6/dc/d4e/d65 0 2024-06-15T01:06:57.974 INFO:tasks.workunit.client.0.smithi137.stdout:1/220: unlink d0/d10/c49 0 2024-06-15T01:06:57.975 INFO:tasks.workunit.client.0.smithi137.stdout:4/207: link c7 dd/d10/c4f 0 2024-06-15T01:06:57.975 INFO:tasks.workunit.client.1.smithi142.stdout:7/242: creat d2/d6/d22/d3c/f56 x:0 0 0 2024-06-15T01:06:57.977 INFO:tasks.workunit.client.0.smithi137.stdout:4/208: write dd/d15/d21/f32 [341317,72283] 0 2024-06-15T01:06:57.977 INFO:tasks.workunit.client.0.smithi137.stdout:4/209: dread - dd/f47 zero size 2024-06-15T01:06:57.979 INFO:tasks.workunit.client.0.smithi137.stdout:7/220: dread df/d13/d2e/f19 [0,4194304] 0 2024-06-15T01:06:57.979 INFO:tasks.workunit.client.0.smithi137.stdout:4/210: write dd/d10/f39 [2110643,91617] 0 2024-06-15T01:06:57.980 INFO:tasks.workunit.client.1.smithi142.stdout:0/261: creat d3/d12/d22/f65 x:0 0 0 2024-06-15T01:06:57.980 INFO:tasks.workunit.client.0.smithi137.stdout:1/221: symlink d0/d2d/l56 0 2024-06-15T01:06:57.983 INFO:tasks.workunit.client.0.smithi137.stdout:7/221: dread df/f38 [0,4194304] 0 2024-06-15T01:06:57.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/259: getdents d1a 0 2024-06-15T01:06:57.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/260: fdatasync d1a/f28 0 2024-06-15T01:06:57.984 INFO:tasks.workunit.client.1.smithi142.stdout:4/381: mknod d5/d9/d1a/c87 0 2024-06-15T01:06:57.986 INFO:tasks.workunit.client.1.smithi142.stdout:4/382: chown d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f6d 1901 1 2024-06-15T01:06:57.986 INFO:tasks.workunit.client.0.smithi137.stdout:2/165: truncate d0/f1f 888858 0 2024-06-15T01:06:57.989 INFO:tasks.workunit.client.1.smithi142.stdout:1/294: rename d6/c45 to d6/d12/d30/c6b 0 2024-06-15T01:06:57.989 INFO:tasks.workunit.client.0.smithi137.stdout:4/211: dwrite dd/f28 [0,4194304] 0 2024-06-15T01:06:57.989 INFO:tasks.workunit.client.0.smithi137.stdout:1/222: mknod d0/d3/d1b/c57 0 2024-06-15T01:06:57.990 INFO:tasks.workunit.client.1.smithi142.stdout:1/295: write d6/d12/d1c/d2c/f59 [936643,30042] 0 2024-06-15T01:06:57.990 INFO:tasks.workunit.client.0.smithi137.stdout:1/223: fsync d0/d3/f2f 0 2024-06-15T01:06:57.992 INFO:tasks.workunit.client.0.smithi137.stdout:1/224: chown d0/d3/d1b/c42 28 1 2024-06-15T01:06:57.993 INFO:tasks.workunit.client.1.smithi142.stdout:7/243: rename d2/d6/d22/f28 to d2/d6/d22/f57 0 2024-06-15T01:06:57.996 INFO:tasks.workunit.client.0.smithi137.stdout:1/225: write d0/d10/f2e [5627717,130569] 0 2024-06-15T01:06:57.997 INFO:tasks.workunit.client.0.smithi137.stdout:4/212: unlink dd/d10/c4f 0 2024-06-15T01:06:57.998 INFO:tasks.workunit.client.0.smithi137.stdout:4/213: truncate dd/f47 257874 0 2024-06-15T01:06:57.999 INFO:tasks.workunit.client.1.smithi142.stdout:9/251: mknod d6/dc/d4e/d65/c66 0 2024-06-15T01:06:58.000 INFO:tasks.workunit.client.0.smithi137.stdout:4/214: truncate dd/d15/d36/d45/f4b 1038465 0 2024-06-15T01:06:58.000 INFO:tasks.workunit.client.1.smithi142.stdout:4/383: symlink d5/d9/d23/l88 0 2024-06-15T01:06:58.002 INFO:tasks.workunit.client.0.smithi137.stdout:1/226: rename d0/d3/d7 to d0/d2d/d3e/d58 0 2024-06-15T01:06:58.002 INFO:tasks.workunit.client.1.smithi142.stdout:3/347: truncate d0/d1b/f2a 978419 0 2024-06-15T01:06:58.005 INFO:tasks.workunit.client.0.smithi137.stdout:7/222: dread f3 [0,4194304] 0 2024-06-15T01:06:58.005 INFO:tasks.workunit.client.1.smithi142.stdout:8/265: rename d0/d1 to d0/d25/d5a 0 2024-06-15T01:06:58.007 INFO:tasks.workunit.client.0.smithi137.stdout:4/215: mknod dd/d15/d36/d45/c50 0 2024-06-15T01:06:58.008 INFO:tasks.workunit.client.0.smithi137.stdout:2/166: dread d0/f9 [0,4194304] 0 2024-06-15T01:06:58.009 INFO:tasks.workunit.client.0.smithi137.stdout:2/167: write d0/de/f24 [484352,106817] 0 2024-06-15T01:06:58.009 INFO:tasks.workunit.client.0.smithi137.stdout:1/227: mkdir d0/d3/d1b/d4c/d59 0 2024-06-15T01:06:58.010 INFO:tasks.workunit.client.0.smithi137.stdout:4/216: mkdir dd/d15/d21/d3a/d51 0 2024-06-15T01:06:58.012 INFO:tasks.workunit.client.0.smithi137.stdout:2/168: rename d0/c2f to d0/dc/c35 0 2024-06-15T01:06:58.013 INFO:tasks.workunit.client.1.smithi142.stdout:2/259: creat d8/d19/d23/d25/f49 x:0 0 0 2024-06-15T01:06:58.014 INFO:tasks.workunit.client.0.smithi137.stdout:4/217: write dd/fe [4887607,95532] 0 2024-06-15T01:06:58.015 INFO:tasks.workunit.client.0.smithi137.stdout:1/228: symlink d0/d3/d1b/l5a 0 2024-06-15T01:06:58.015 INFO:tasks.workunit.client.1.smithi142.stdout:0/262: creat d3/d5/d4c/d61/f66 x:0 0 0 2024-06-15T01:06:58.017 INFO:tasks.workunit.client.0.smithi137.stdout:7/223: dwrite df/d13/d2e/f19 [0,4194304] 0 2024-06-15T01:06:58.020 INFO:tasks.workunit.client.1.smithi142.stdout:6/261: rename f9 to d1a/d39/d42/d4c/d58/f5d 0 2024-06-15T01:06:58.022 INFO:tasks.workunit.client.1.smithi142.stdout:7/244: dread d2/d8/f17 [0,4194304] 0 2024-06-15T01:06:58.023 INFO:tasks.workunit.client.1.smithi142.stdout:0/263: dwrite d3/d5/f1d [0,4194304] 0 2024-06-15T01:06:58.023 INFO:tasks.workunit.client.1.smithi142.stdout:7/245: readlink l0 0 2024-06-15T01:06:58.023 INFO:tasks.workunit.client.0.smithi137.stdout:1/229: symlink d0/d2d/d3e/l5b 0 2024-06-15T01:06:58.024 INFO:tasks.workunit.client.1.smithi142.stdout:7/246: chown d2/d8/da 369089 1 2024-06-15T01:06:58.025 INFO:tasks.workunit.client.1.smithi142.stdout:8/266: unlink d0/f20 0 2024-06-15T01:06:58.026 INFO:tasks.workunit.client.1.smithi142.stdout:1/296: link d6/d3a/f44 d6/d4d/f6c 0 2024-06-15T01:06:58.027 INFO:tasks.workunit.client.0.smithi137.stdout:7/224: creat df/d18/d1c/d1d/d27/f49 x:0 0 0 2024-06-15T01:06:58.028 INFO:tasks.workunit.client.1.smithi142.stdout:1/297: write d6/f13 [3553575,6275] 0 2024-06-15T01:06:58.029 INFO:tasks.workunit.client.0.smithi137.stdout:7/225: truncate df/d13/d2e/d1a/f30 4314684 0 2024-06-15T01:06:58.031 INFO:tasks.workunit.client.0.smithi137.stdout:2/169: write d0/de/f1a [465333,66861] 0 2024-06-15T01:06:58.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 -- 172.21.15.137:0/3016318087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac100cb0 msgr2=0x7f47ac101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 --2- 172.21.15.137:0/3016318087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac100cb0 0x7f47ac101110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f479c009910 tx=0x7f479c009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:58.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 -- 172.21.15.137:0/3016318087 shutdown_connections 2024-06-15T01:06:58.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 --2- 172.21.15.137:0/3016318087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac100cb0 0x7f47ac101110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 --2- 172.21.15.137:0/3016318087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac0ffab0 0x7f47ac0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 -- 172.21.15.137:0/3016318087 >> 172.21.15.137:0/3016318087 conn(0x7f47ac0fb030 msgr2=0x7f47ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:58.034 INFO:tasks.workunit.client.0.smithi137.stdout:1/230: write d0/d3/f13 [4413826,33580] 0 2024-06-15T01:06:58.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 -- 172.21.15.137:0/3016318087 shutdown_connections 2024-06-15T01:06:58.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.037+0000 7f47b377c700 1 -- 172.21.15.137:0/3016318087 wait complete. 2024-06-15T01:06:58.035 INFO:tasks.workunit.client.0.smithi137.stdout:2/170: write d0/da/f11 [767683,90088] 0 2024-06-15T01:06:58.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 Processor -- start 2024-06-15T01:06:58.036 INFO:tasks.workunit.client.0.smithi137.stdout:4/218: creat dd/d15/d21/d3a/d51/f52 x:0 0 0 2024-06-15T01:06:58.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 -- start start 2024-06-15T01:06:58.036 INFO:tasks.workunit.client.0.smithi137.stdout:2/171: chown d0/de/f24 5443118 1 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 0x7f47ac193ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 0x7f47ac194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac194a20 con 0x7f47ac0ffab0 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.039+0000 7f47b377c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac194b60 con 0x7f47ac100cb0 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b0d17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 0x7f47ac194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b0d17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 0x7f47ac194400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:35738/0 (socket says 172.21.15.137:35738) 2024-06-15T01:06:58.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b0d17700 1 -- 172.21.15.137:0/3037160451 learned_addr learned my addr 172.21.15.137:0/3037160451 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:58.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b1518700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 0x7f47ac193ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b1518700 1 -- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 msgr2=0x7f47ac194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b1518700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 0x7f47ac194400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.038 INFO:tasks.workunit.client.0.smithi137.stdout:1/231: truncate d0/d10/f23 4400829 0 2024-06-15T01:06:58.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.040+0000 7f47b1518700 1 -- 172.21.15.137:0/3037160451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0095f0 con 0x7f47ac0ffab0 2024-06-15T01:06:58.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.041+0000 7f47b1518700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 0x7f47ac193ec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47a8010aa0 tx=0x7f47a8010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:58.039 INFO:tasks.workunit.client.0.smithi137.stdout:7/226: mkdir df/d18/d1c/d3a/d4a 0 2024-06-15T01:06:58.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.041+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a8004200 con 0x7f47ac0ffab0 2024-06-15T01:06:58.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.042+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f47a8011040 con 0x7f47ac0ffab0 2024-06-15T01:06:58.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.042+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a8003b10 con 0x7f47ac0ffab0 2024-06-15T01:06:58.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.042+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac199610 con 0x7f47ac0ffab0 2024-06-15T01:06:58.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.042+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac199b00 con 0x7f47ac0ffab0 2024-06-15T01:06:58.040 INFO:tasks.workunit.client.0.smithi137.stdout:2/172: write d0/de/d10/f33 [325609,7113] 0 2024-06-15T01:06:58.040 INFO:tasks.workunit.client.0.smithi137.stdout:7/227: stat df/c42 0 2024-06-15T01:06:58.040 INFO:tasks.workunit.client.0.smithi137.stdout:1/232: write d0/d10/f23 [1370233,124861] 0 2024-06-15T01:06:58.040 INFO:tasks.workunit.client.0.smithi137.stdout:4/219: write dd/f14 [3720231,39864] 0 2024-06-15T01:06:58.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.044+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac1073e0 con 0x7f47ac0ffab0 2024-06-15T01:06:58.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.046+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f47a8004360 con 0x7f47ac0ffab0 2024-06-15T01:06:58.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.047+0000 7f47a27fc700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 0x7f4798072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.047+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f47a808b1b0 con 0x7f47ac0ffab0 2024-06-15T01:06:58.044 INFO:tasks.workunit.client.0.smithi137.stdout:4/220: mknod dd/c53 0 2024-06-15T01:06:58.045 INFO:tasks.workunit.client.1.smithi142.stdout:6/262: chown c3 1829 1 2024-06-15T01:06:58.045 INFO:tasks.workunit.client.1.smithi142.stdout:8/267: fsync d0/f18 0 2024-06-15T01:06:58.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.048+0000 7f47b0d17700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 0x7f4798072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.048+0000 7f47b0d17700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 0x7f4798072540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c005dc0 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:58.046 INFO:tasks.workunit.client.1.smithi142.stdout:9/252: write d6/dc/dd/d22/f2b [1485820,2763] 0 2024-06-15T01:06:58.047 INFO:tasks.workunit.client.0.smithi137.stdout:1/233: rename d0/d3/f36 to d0/d3/d1b/d4c/f5c 0 2024-06-15T01:06:58.048 INFO:tasks.workunit.client.1.smithi142.stdout:6/263: dread d1a/f46 [0,4194304] 0 2024-06-15T01:06:58.048 INFO:tasks.workunit.client.1.smithi142.stdout:1/298: mkdir d6/d46/d6d 0 2024-06-15T01:06:58.050 INFO:tasks.workunit.client.1.smithi142.stdout:1/299: write d6/d25/f3d [991859,85766] 0 2024-06-15T01:06:58.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.054+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f47a8059bc0 con 0x7f47ac0ffab0 2024-06-15T01:06:58.051 INFO:tasks.workunit.client.1.smithi142.stdout:8/268: truncate d0/d25/f35 849527 0 2024-06-15T01:06:58.051 INFO:tasks.workunit.client.0.smithi137.stdout:1/234: chown d0/c15 34544 1 2024-06-15T01:06:58.052 INFO:tasks.workunit.client.1.smithi142.stdout:1/300: fdatasync d6/d12/d1c/d2c/f62 0 2024-06-15T01:06:58.052 INFO:tasks.workunit.client.0.smithi137.stdout:2/173: dwrite d0/f8 [0,4194304] 0 2024-06-15T01:06:58.053 INFO:tasks.workunit.client.1.smithi142.stdout:1/301: write d6/d12/d1c/d2c/d47/f5a [732538,16083] 0 2024-06-15T01:06:58.054 INFO:tasks.workunit.client.1.smithi142.stdout:1/302: readlink d6/d12/d1c/l6a 0 2024-06-15T01:06:58.054 INFO:tasks.workunit.client.0.smithi137.stdout:2/174: write d0/f3 [3681100,130529] 0 2024-06-15T01:06:58.054 INFO:tasks.workunit.client.0.smithi137.stdout:4/221: creat dd/d10/f54 x:0 0 0 2024-06-15T01:06:58.055 INFO:tasks.workunit.client.1.smithi142.stdout:6/264: dread d1a/f1f [0,4194304] 0 2024-06-15T01:06:58.056 INFO:tasks.workunit.client.1.smithi142.stdout:3/348: link d0/d1b/c49 d0/d3/d15/c65 0 2024-06-15T01:06:58.059 INFO:tasks.workunit.client.0.smithi137.stdout:4/222: mkdir dd/d15/d36/d45/d55 0 2024-06-15T01:06:58.061 INFO:tasks.workunit.client.1.smithi142.stdout:8/269: readlink d0/d25/d5a/ld 0 2024-06-15T01:06:58.062 INFO:tasks.workunit.client.0.smithi137.stdout:4/223: write dd/f16 [123663,7954] 0 2024-06-15T01:06:58.062 INFO:tasks.workunit.client.1.smithi142.stdout:6/265: mknod d1a/d39/d42/d4c/c5e 0 2024-06-15T01:06:58.063 INFO:tasks.workunit.client.0.smithi137.stdout:1/235: creat d0/d3/d47/f5d x:0 0 0 2024-06-15T01:06:58.065 INFO:tasks.workunit.client.0.smithi137.stdout:7/228: getdents df/d18/d1c/d1d 0 2024-06-15T01:06:58.066 INFO:tasks.workunit.client.0.smithi137.stdout:2/175: dwrite d0/dc/f2c [4194304,4194304] 0 2024-06-15T01:06:58.068 INFO:tasks.workunit.client.1.smithi142.stdout:0/264: getdents d3 0 2024-06-15T01:06:58.068 INFO:tasks.workunit.client.1.smithi142.stdout:9/253: link d6/dc/l2e d6/d31/l67 0 2024-06-15T01:06:58.069 INFO:tasks.workunit.client.0.smithi137.stdout:0/215: sync 2024-06-15T01:06:58.070 INFO:tasks.workunit.client.0.smithi137.stdout:1/236: write d0/d3/d14/d34/f39 [1739690,97175] 0 2024-06-15T01:06:58.071 INFO:tasks.workunit.client.0.smithi137.stdout:1/237: chown d0/d3/d14/d34 536891862 1 2024-06-15T01:06:58.075 INFO:tasks.workunit.client.1.smithi142.stdout:6/266: creat d1a/d39/d3a/f5f x:0 0 0 2024-06-15T01:06:58.078 INFO:tasks.workunit.client.1.smithi142.stdout:0/265: unlink d3/d12/d22/f24 0 2024-06-15T01:06:58.079 INFO:tasks.workunit.client.1.smithi142.stdout:0/266: chown d3/d12/d22 165303318 1 2024-06-15T01:06:58.080 INFO:tasks.workunit.client.0.smithi137.stdout:1/238: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:06:58.082 INFO:tasks.workunit.client.1.smithi142.stdout:9/254: rename d6/dc/c60 to d6/dc/dd/c68 0 2024-06-15T01:06:58.083 INFO:tasks.workunit.client.1.smithi142.stdout:9/255: dread - d6/f55 zero size 2024-06-15T01:06:58.083 INFO:tasks.workunit.client.0.smithi137.stdout:4/224: dwrite f9 [0,4194304] 0 2024-06-15T01:06:58.084 INFO:tasks.workunit.client.1.smithi142.stdout:9/256: chown d6/dc/l29 661559341 1 2024-06-15T01:06:58.085 INFO:tasks.workunit.client.1.smithi142.stdout:1/303: link d6/l19 d6/d12/d5b/l6e 0 2024-06-15T01:06:58.085 INFO:tasks.workunit.client.0.smithi137.stdout:2/176: dwrite d0/de/f1a [0,4194304] 0 2024-06-15T01:06:58.086 INFO:tasks.workunit.client.1.smithi142.stdout:9/257: truncate d6/dc/dd/d22/f2b 2112157 0 2024-06-15T01:06:58.096 INFO:tasks.workunit.client.0.smithi137.stdout:2/177: dwrite d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:06:58.096 INFO:tasks.workunit.client.0.smithi137.stdout:2/178: chown d0/f3 3066 1 2024-06-15T01:06:58.096 INFO:tasks.workunit.client.0.smithi137.stdout:0/216: mknod d1/d2/dc/d10/c51 0 2024-06-15T01:06:58.097 INFO:tasks.workunit.client.0.smithi137.stdout:0/217: read - d1/d2/dc/d10/d14/d15/f50 zero size 2024-06-15T01:06:58.101 INFO:tasks.workunit.client.1.smithi142.stdout:0/267: rename d3/la to d3/d5/d44/l67 0 2024-06-15T01:06:58.106 INFO:tasks.workunit.client.1.smithi142.stdout:3/349: getdents d0/d1f/d20/d59 0 2024-06-15T01:06:58.107 INFO:tasks.workunit.client.0.smithi137.stdout:4/225: creat dd/d10/d1f/f56 x:0 0 0 2024-06-15T01:06:58.107 INFO:tasks.workunit.client.1.smithi142.stdout:3/350: truncate d0/d3/f12 4558270 0 2024-06-15T01:06:58.108 INFO:tasks.workunit.client.1.smithi142.stdout:1/304: rename d6/d12/d1c/d1e/f20 to d6/d12/d3c/d63/f6f 0 2024-06-15T01:06:58.108 INFO:tasks.workunit.client.0.smithi137.stdout:4/226: truncate dd/d15/d30/f40 1590 0 2024-06-15T01:06:58.109 INFO:tasks.workunit.client.1.smithi142.stdout:1/305: write d6/d12/d1c/f32 [1138011,69913] 0 2024-06-15T01:06:58.111 INFO:tasks.workunit.client.1.smithi142.stdout:3/351: creat d0/f66 x:0 0 0 2024-06-15T01:06:58.111 INFO:tasks.workunit.client.1.smithi142.stdout:3/352: readlink d0/d3/l4c 0 2024-06-15T01:06:58.112 INFO:tasks.workunit.client.1.smithi142.stdout:8/270: getdents d0/d25/d5a 0 2024-06-15T01:06:58.113 INFO:tasks.workunit.client.1.smithi142.stdout:1/306: creat d6/d3a/f70 x:0 0 0 2024-06-15T01:06:58.113 INFO:tasks.workunit.client.0.smithi137.stdout:1/239: rename d0/d3/d14/d34/f39 to d0/d2d/f5e 0 2024-06-15T01:06:58.115 INFO:tasks.workunit.client.1.smithi142.stdout:1/307: symlink d6/d12/d28/l71 0 2024-06-15T01:06:58.115 INFO:tasks.workunit.client.0.smithi137.stdout:7/229: mknod df/d18/d1c/d1d/d27/d33/d43/c4b 0 2024-06-15T01:06:58.116 INFO:tasks.workunit.client.0.smithi137.stdout:4/227: readlink dd/d10/l2f 0 2024-06-15T01:06:58.116 INFO:tasks.workunit.client.1.smithi142.stdout:6/267: dread d1a/d2d/f52 [0,4194304] 0 2024-06-15T01:06:58.116 INFO:tasks.workunit.client.1.smithi142.stdout:3/353: unlink d0/d3/d15/c58 0 2024-06-15T01:06:58.117 INFO:tasks.workunit.client.1.smithi142.stdout:8/271: mknod d0/c5b 0 2024-06-15T01:06:58.117 INFO:tasks.workunit.client.1.smithi142.stdout:6/268: chown d1a/d39/d3a/f5f 383 1 2024-06-15T01:06:58.118 INFO:tasks.workunit.client.0.smithi137.stdout:7/230: dread df/f38 [0,4194304] 0 2024-06-15T01:06:58.118 INFO:tasks.workunit.client.1.smithi142.stdout:1/308: symlink d6/d25/l72 0 2024-06-15T01:06:58.118 INFO:tasks.workunit.client.1.smithi142.stdout:8/272: write d0/d25/d5a/f2c [4608654,40835] 0 2024-06-15T01:06:58.119 INFO:tasks.workunit.client.1.smithi142.stdout:1/309: chown d6/d12/d30/c4a 3093997 1 2024-06-15T01:06:58.120 INFO:tasks.workunit.client.0.smithi137.stdout:0/218: rename d1/d2/d3e to d1/d2/dc/d10/d14/d52 0 2024-06-15T01:06:58.120 INFO:tasks.workunit.client.1.smithi142.stdout:1/310: chown d6/d12/d1c/d2c/d47/f5f 14 1 2024-06-15T01:06:58.121 INFO:tasks.workunit.client.1.smithi142.stdout:3/354: dread d0/d3/d15/f64 [0,4194304] 0 2024-06-15T01:06:58.123 INFO:tasks.workunit.client.0.smithi137.stdout:0/219: truncate d1/d2/d34/f4a 617083 0 2024-06-15T01:06:58.124 INFO:tasks.workunit.client.1.smithi142.stdout:3/355: unlink d0/d1b/l24 0 2024-06-15T01:06:58.124 INFO:tasks.workunit.client.1.smithi142.stdout:3/356: read - d0/f66 zero size 2024-06-15T01:06:58.124 INFO:tasks.workunit.client.1.smithi142.stdout:1/311: creat d6/d12/d30/f73 x:0 0 0 2024-06-15T01:06:58.126 INFO:tasks.workunit.client.0.smithi137.stdout:4/228: unlink dd/d15/c25 0 2024-06-15T01:06:58.126 INFO:tasks.workunit.client.1.smithi142.stdout:1/312: read d6/d12/f22 [1467140,101224] 0 2024-06-15T01:06:58.127 INFO:tasks.workunit.client.1.smithi142.stdout:3/357: mknod d0/d3/d15/c67 0 2024-06-15T01:06:58.128 INFO:tasks.workunit.client.0.smithi137.stdout:1/240: dwrite d0/f2 [0,4194304] 0 2024-06-15T01:06:58.128 INFO:tasks.workunit.client.1.smithi142.stdout:3/358: truncate d0/d1f/f46 933259 0 2024-06-15T01:06:58.129 INFO:tasks.workunit.client.0.smithi137.stdout:2/179: getdents d0/de/d2b 0 2024-06-15T01:06:58.129 INFO:tasks.workunit.client.0.smithi137.stdout:2/180: write d0/f5 [4389616,130490] 0 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.0.smithi137.stdout:6/174: sync 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.0.smithi137.stdout:8/141: sync 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.0.smithi137.stdout:3/320: sync 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.0.smithi137.stdout:5/434: sync 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.0.smithi137.stdout:9/123: sync 2024-06-15T01:06:58.130 INFO:tasks.workunit.client.1.smithi142.stdout:1/313: symlink d6/d12/d1c/l74 0 2024-06-15T01:06:58.131 INFO:tasks.workunit.client.0.smithi137.stdout:9/124: read - df/f18 zero size 2024-06-15T01:06:58.132 INFO:tasks.workunit.client.0.smithi137.stdout:9/125: fdatasync f5 0 2024-06-15T01:06:58.132 INFO:tasks.workunit.client.0.smithi137.stdout:9/126: chown df/d12/f1f 3736679 1 2024-06-15T01:06:58.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/175: write d0/d5/f12 [4499990,87180] 0 2024-06-15T01:06:58.134 INFO:tasks.workunit.client.0.smithi137.stdout:0/220: mkdir d1/d2/dc/d10/d53 0 2024-06-15T01:06:58.134 INFO:tasks.workunit.client.0.smithi137.stdout:8/142: chown d9/de 0 1 2024-06-15T01:06:58.134 INFO:tasks.workunit.client.0.smithi137.stdout:9/127: chown df/c1c 800371 1 2024-06-15T01:06:58.137 INFO:tasks.workunit.client.0.smithi137.stdout:9/128: dread - df/d12/f1f zero size 2024-06-15T01:06:58.138 INFO:tasks.workunit.client.0.smithi137.stdout:1/241: write d0/d3/f2f [4880780,28838] 0 2024-06-15T01:06:58.138 INFO:tasks.workunit.client.0.smithi137.stdout:0/221: chown d1/d2/d13/f33 3385 1 2024-06-15T01:06:58.139 INFO:tasks.workunit.client.0.smithi137.stdout:6/176: write d0/d1c/d1d/f1f [1033627,26894] 0 2024-06-15T01:06:58.140 INFO:tasks.workunit.client.0.smithi137.stdout:4/229: chown dd/d15/c44 0 1 2024-06-15T01:06:58.141 INFO:tasks.workunit.client.0.smithi137.stdout:3/321: chown d0/d2/d22/c2b 143111 1 2024-06-15T01:06:58.141 INFO:tasks.workunit.client.0.smithi137.stdout:8/143: rmdir d9/d22 39 2024-06-15T01:06:58.142 INFO:tasks.workunit.client.0.smithi137.stdout:4/230: chown dd/d15/d36/d45 0 1 2024-06-15T01:06:58.143 INFO:tasks.workunit.client.0.smithi137.stdout:6/177: chown d0/d5/d9/c25 88112562 1 2024-06-15T01:06:58.143 INFO:tasks.workunit.client.0.smithi137.stdout:4/231: write f8 [5232857,115885] 0 2024-06-15T01:06:58.144 INFO:tasks.workunit.client.0.smithi137.stdout:1/242: mknod d0/d2d/c5f 0 2024-06-15T01:06:58.145 INFO:tasks.workunit.client.0.smithi137.stdout:8/144: mknod d9/de/c2e 0 2024-06-15T01:06:58.147 INFO:tasks.workunit.client.0.smithi137.stdout:1/243: truncate d0/d3/d1b/f4f 1041883 0 2024-06-15T01:06:58.147 INFO:tasks.workunit.client.0.smithi137.stdout:9/129: symlink df/d12/d1b/l21 0 2024-06-15T01:06:58.148 INFO:tasks.workunit.client.0.smithi137.stdout:3/322: mkdir d0/d2/d50/d6c 0 2024-06-15T01:06:58.149 INFO:tasks.workunit.client.0.smithi137.stdout:4/232: dread dd/f22 [0,4194304] 0 2024-06-15T01:06:58.150 INFO:tasks.workunit.client.0.smithi137.stdout:0/222: mkdir d1/d2/dc/d10/d14/d15/d17/d54 0 2024-06-15T01:06:58.152 INFO:tasks.workunit.client.0.smithi137.stdout:0/223: write d1/d2/d13/f40 [941876,55186] 0 2024-06-15T01:06:58.152 INFO:tasks.workunit.client.0.smithi137.stdout:6/178: symlink d0/d1c/d32/l34 0 2024-06-15T01:06:58.153 INFO:tasks.workunit.client.1.smithi142.stdout:3/359: dread d0/d3/da/d18/f28 [0,4194304] 0 2024-06-15T01:06:58.157 INFO:tasks.workunit.client.0.smithi137.stdout:4/233: mknod dd/d15/d36/d45/c57 0 2024-06-15T01:06:58.157 INFO:tasks.workunit.client.0.smithi137.stdout:9/130: creat df/d13/f22 x:0 0 0 2024-06-15T01:06:58.159 INFO:tasks.workunit.client.1.smithi142.stdout:3/360: symlink d0/d3/d15/d4d/d55/l68 0 2024-06-15T01:06:58.160 INFO:tasks.workunit.client.1.smithi142.stdout:5/292: sync 2024-06-15T01:06:58.161 INFO:tasks.workunit.client.0.smithi137.stdout:0/224: mkdir d1/d2/d55 0 2024-06-15T01:06:58.163 INFO:tasks.workunit.client.0.smithi137.stdout:1/244: dwrite d0/d10/f2e [0,4194304] 0 2024-06-15T01:06:58.163 INFO:tasks.workunit.client.0.smithi137.stdout:6/179: rename d0/d5/d9/f33 to d0/d1a/f35 0 2024-06-15T01:06:58.165 INFO:tasks.workunit.client.0.smithi137.stdout:4/234: unlink dd/d10/d1f/f26 0 2024-06-15T01:06:58.168 INFO:tasks.workunit.client.0.smithi137.stdout:3/323: dwrite d0/d2/f15 [4194304,4194304] 0 2024-06-15T01:06:58.168 INFO:tasks.workunit.client.0.smithi137.stdout:4/235: chown dd/f28 1621 1 2024-06-15T01:06:58.170 INFO:tasks.workunit.client.0.smithi137.stdout:6/180: rename d0/d5/f15 to d0/d1c/d1d/d2e/f36 0 2024-06-15T01:06:58.171 INFO:tasks.workunit.client.0.smithi137.stdout:1/245: write d0/d3/d14/f19 [5180716,81260] 0 2024-06-15T01:06:58.171 INFO:tasks.workunit.client.0.smithi137.stdout:3/324: chown d0/d2/d1e/l30 2106506 1 2024-06-15T01:06:58.174 INFO:tasks.workunit.client.0.smithi137.stdout:6/181: dread - d0/d5/f17 zero size 2024-06-15T01:06:58.176 INFO:tasks.workunit.client.0.smithi137.stdout:3/325: creat d0/d2/d22/d63/f6d x:0 0 0 2024-06-15T01:06:58.177 INFO:tasks.workunit.client.1.smithi142.stdout:5/293: dwrite d2/f21 [0,4194304] 0 2024-06-15T01:06:58.179 INFO:tasks.workunit.client.0.smithi137.stdout:3/326: creat d0/d2/d1e/f6e x:0 0 0 2024-06-15T01:06:58.179 INFO:tasks.workunit.client.0.smithi137.stdout:4/236: link dd/d10/d1f/c24 dd/d15/d21/c58 0 2024-06-15T01:06:58.181 INFO:tasks.workunit.client.0.smithi137.stdout:0/225: read d1/d2/dc/d10/d14/f1b [264960,117533] 0 2024-06-15T01:06:58.182 INFO:tasks.workunit.client.0.smithi137.stdout:4/237: chown dd/d15/d36/l3e 6 1 2024-06-15T01:06:58.182 INFO:tasks.workunit.client.0.smithi137.stdout:6/182: rename d0/d1c/d32/l34 to d0/l37 0 2024-06-15T01:06:58.185 INFO:tasks.workunit.client.0.smithi137.stdout:6/183: write d0/d5/f17 [786546,27800] 0 2024-06-15T01:06:58.186 INFO:tasks.workunit.client.0.smithi137.stdout:3/327: creat d0/d2/d23/f6f x:0 0 0 2024-06-15T01:06:58.192 INFO:tasks.workunit.client.0.smithi137.stdout:0/226: mkdir d1/d2/dc/d10/d14/d56 0 2024-06-15T01:06:58.194 INFO:tasks.workunit.client.0.smithi137.stdout:9/131: dread f2 [0,4194304] 0 2024-06-15T01:06:58.194 INFO:tasks.workunit.client.0.smithi137.stdout:4/238: dwrite dd/f18 [0,4194304] 0 2024-06-15T01:06:58.195 INFO:tasks.workunit.client.0.smithi137.stdout:6/184: creat d0/d1c/d1d/d2e/f38 x:0 0 0 2024-06-15T01:06:58.196 INFO:tasks.workunit.client.0.smithi137.stdout:0/227: readlink d1/d2/dc/d10/d14/d15/d17/l26 0 2024-06-15T01:06:58.204 INFO:tasks.workunit.client.0.smithi137.stdout:9/132: creat df/d12/f23 x:0 0 0 2024-06-15T01:06:58.205 INFO:tasks.workunit.client.0.smithi137.stdout:0/228: read d1/d2/f6 [3842645,45361] 0 2024-06-15T01:06:58.207 INFO:tasks.workunit.client.0.smithi137.stdout:4/239: write dd/f22 [4332659,125279] 0 2024-06-15T01:06:58.213 INFO:tasks.workunit.client.0.smithi137.stdout:0/229: unlink d1/d2/dc/c11 0 2024-06-15T01:06:58.214 INFO:tasks.workunit.client.0.smithi137.stdout:4/240: write dd/d15/f42 [2064365,87105] 0 2024-06-15T01:06:58.215 INFO:tasks.workunit.client.0.smithi137.stdout:0/230: chown d1/d2/dc/d10/d14/d15/l24 152 1 2024-06-15T01:06:58.217 INFO:tasks.workunit.client.0.smithi137.stdout:6/185: creat d0/f39 x:0 0 0 2024-06-15T01:06:58.219 INFO:tasks.workunit.client.0.smithi137.stdout:4/241: symlink dd/d15/l59 0 2024-06-15T01:06:58.220 INFO:tasks.workunit.client.0.smithi137.stdout:4/242: fdatasync dd/d15/d21/f32 0 2024-06-15T01:06:58.227 INFO:tasks.workunit.client.0.smithi137.stdout:4/243: link dd/d15/c34 dd/d10/c5a 0 2024-06-15T01:06:58.228 INFO:tasks.workunit.client.0.smithi137.stdout:7/231: truncate df/d18/d1c/d1d/f28 4192755 0 2024-06-15T01:06:58.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/244: mkdir dd/d15/d21/d3a/d51/d5b 0 2024-06-15T01:06:58.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/245: dread - dd/d10/d1f/f56 zero size 2024-06-15T01:06:58.231 INFO:tasks.workunit.client.0.smithi137.stdout:4/246: chown dd/d15/d21/d3a 127 1 2024-06-15T01:06:58.232 INFO:tasks.workunit.client.0.smithi137.stdout:7/232: write df/f38 [487628,104416] 0 2024-06-15T01:06:58.235 INFO:tasks.workunit.client.0.smithi137.stdout:7/233: creat df/f4c x:0 0 0 2024-06-15T01:06:58.238 INFO:tasks.workunit.client.0.smithi137.stdout:7/234: symlink df/d18/d1c/d3a/l4d 0 2024-06-15T01:06:58.238 INFO:tasks.workunit.client.0.smithi137.stdout:5/435: dwrite df/d11/d14/f43 [0,4194304] 0 2024-06-15T01:06:58.249 INFO:tasks.workunit.client.0.smithi137.stdout:5/436: dwrite df/d11/d14/f5e [0,4194304] 0 2024-06-15T01:06:58.249 INFO:tasks.workunit.client.0.smithi137.stdout:7/235: dwrite df/d13/f20 [0,4194304] 0 2024-06-15T01:06:58.251 INFO:tasks.workunit.client.0.smithi137.stdout:5/437: chown df/d11/d14/d6c 13 1 2024-06-15T01:06:58.254 INFO:tasks.workunit.client.0.smithi137.stdout:5/438: chown df/c52 7031198 1 2024-06-15T01:06:58.255 INFO:tasks.workunit.client.1.smithi142.stdout:1/314: rmdir d6/d12/d3c 39 2024-06-15T01:06:58.256 INFO:tasks.workunit.client.0.smithi137.stdout:5/439: write df/d11/d30/f61 [996519,90882] 0 2024-06-15T01:06:58.257 INFO:tasks.workunit.client.1.smithi142.stdout:8/273: getdents d0 0 2024-06-15T01:06:58.259 INFO:tasks.workunit.client.0.smithi137.stdout:5/440: readlink df/d44/l47 0 2024-06-15T01:06:58.260 INFO:tasks.workunit.client.1.smithi142.stdout:3/361: write d0/d1f/d20/f32 [2663069,126769] 0 2024-06-15T01:06:58.260 INFO:tasks.workunit.client.1.smithi142.stdout:5/294: creat d2/d3e/f5c x:0 0 0 2024-06-15T01:06:58.263 INFO:tasks.workunit.client.0.smithi137.stdout:5/441: creat df/d44/d54/d95/d3b/d49/f96 x:0 0 0 2024-06-15T01:06:58.263 INFO:tasks.workunit.client.1.smithi142.stdout:1/315: write d6/d12/d28/f33 [5502694,32929] 0 2024-06-15T01:06:58.265 INFO:tasks.workunit.client.1.smithi142.stdout:0/268: mknod d3/d5/c68 0 2024-06-15T01:06:58.266 INFO:tasks.workunit.client.1.smithi142.stdout:1/316: fdatasync d6/d12/d28/f64 0 2024-06-15T01:06:58.266 INFO:tasks.workunit.client.1.smithi142.stdout:3/362: rmdir d0/d3/da/d18 39 2024-06-15T01:06:58.267 INFO:tasks.workunit.client.0.smithi137.stdout:0/231: rename d1/f2c to d1/d2/dc/d10/d14/d15/f57 0 2024-06-15T01:06:58.269 INFO:tasks.workunit.client.1.smithi142.stdout:6/269: dwrite d1a/d39/d42/f45 [0,4194304] 0 2024-06-15T01:06:58.272 INFO:tasks.workunit.client.0.smithi137.stdout:0/232: dread d1/d2/d13/f33 [0,4194304] 0 2024-06-15T01:06:58.272 INFO:tasks.workunit.client.0.smithi137.stdout:8/145: rmdir d9 39 2024-06-15T01:06:58.272 INFO:tasks.workunit.client.0.smithi137.stdout:0/233: chown d1/d2/dc 12543533 1 2024-06-15T01:06:58.273 INFO:tasks.workunit.client.1.smithi142.stdout:8/274: dread d0/f4c [0,4194304] 0 2024-06-15T01:06:58.273 INFO:tasks.workunit.client.1.smithi142.stdout:3/363: chown d0/d3/d15/c65 393276517 1 2024-06-15T01:06:58.275 INFO:tasks.workunit.client.0.smithi137.stdout:8/146: chown d9/f20 7370 1 2024-06-15T01:06:58.276 INFO:tasks.workunit.client.1.smithi142.stdout:5/295: dwrite d2/d18/f45 [0,4194304] 0 2024-06-15T01:06:58.276 INFO:tasks.workunit.client.1.smithi142.stdout:3/364: dread - d0/f66 zero size 2024-06-15T01:06:58.276 INFO:tasks.workunit.client.1.smithi142.stdout:9/258: dread d6/dc/dd/d22/f41 [0,4194304] 0 2024-06-15T01:06:58.277 INFO:tasks.workunit.client.0.smithi137.stdout:0/234: creat d1/d2/dc/d10/d14/d52/f58 x:0 0 0 2024-06-15T01:06:58.278 INFO:tasks.workunit.client.1.smithi142.stdout:5/296: stat d2/d3e/c3b 0 2024-06-15T01:06:58.279 INFO:tasks.workunit.client.1.smithi142.stdout:0/269: rename d3/d12/l47 to d3/d5/d1e/l69 0 2024-06-15T01:06:58.279 INFO:tasks.workunit.client.1.smithi142.stdout:3/365: fsync d0/d1f/d20/f32 0 2024-06-15T01:06:58.279 INFO:tasks.workunit.client.0.smithi137.stdout:2/181: unlink d0/l21 0 2024-06-15T01:06:58.280 INFO:tasks.workunit.client.1.smithi142.stdout:5/297: read - d2/d3e/f5c zero size 2024-06-15T01:06:58.281 INFO:tasks.workunit.client.0.smithi137.stdout:8/147: mkdir d9/d2a/d2f 0 2024-06-15T01:06:58.282 INFO:tasks.workunit.client.1.smithi142.stdout:8/275: mknod d0/d17/c5c 0 2024-06-15T01:06:58.283 INFO:tasks.workunit.client.1.smithi142.stdout:3/366: read d0/d3/d15/f2d [4191052,12440] 0 2024-06-15T01:06:58.284 INFO:tasks.workunit.client.1.smithi142.stdout:8/276: dread - d0/d25/d5a/f4d zero size 2024-06-15T01:06:58.285 INFO:tasks.workunit.client.1.smithi142.stdout:1/317: getdents d6/d12/d5b 0 2024-06-15T01:06:58.286 INFO:tasks.workunit.client.0.smithi137.stdout:1/246: unlink d0/d3/d14/c25 0 2024-06-15T01:06:58.288 INFO:tasks.workunit.client.1.smithi142.stdout:8/277: dread d0/d17/f27 [0,4194304] 0 2024-06-15T01:06:58.289 INFO:tasks.workunit.client.0.smithi137.stdout:1/247: write d0/d10/d41/f4b [4995855,18314] 0 2024-06-15T01:06:58.290 INFO:tasks.workunit.client.1.smithi142.stdout:5/298: symlink d2/d3e/d31/l5d 0 2024-06-15T01:06:58.291 INFO:tasks.workunit.client.0.smithi137.stdout:0/235: link d1/d2/dc/d10/d14/f39 d1/d2/dc/d10/d14/d15/f59 0 2024-06-15T01:06:58.294 INFO:tasks.workunit.client.0.smithi137.stdout:8/148: dwrite d9/de/f18 [0,4194304] 0 2024-06-15T01:06:58.294 INFO:tasks.workunit.client.0.smithi137.stdout:0/236: fdatasync d1/d2/dc/d10/f4f 0 2024-06-15T01:06:58.295 INFO:tasks.workunit.client.1.smithi142.stdout:0/270: dwrite d3/f2a [4194304,4194304] 0 2024-06-15T01:06:58.296 INFO:tasks.workunit.client.1.smithi142.stdout:1/318: rmdir d6/d12/d1c/d2c 39 2024-06-15T01:06:58.296 INFO:tasks.workunit.client.0.smithi137.stdout:8/149: chown d9/de/d1e/f27 31180669 1 2024-06-15T01:06:58.296 INFO:tasks.workunit.client.0.smithi137.stdout:1/248: rename d0/d2d/d3e/d58 to d0/d1f/d60 0 2024-06-15T01:06:58.297 INFO:tasks.workunit.client.0.smithi137.stdout:8/150: fsync d9/de/d1e/f28 0 2024-06-15T01:06:58.301 INFO:tasks.workunit.client.1.smithi142.stdout:6/270: rename d1a/l55 to d1a/d2d/l60 0 2024-06-15T01:06:58.301 INFO:tasks.workunit.client.0.smithi137.stdout:0/237: symlink d1/d2/dc/d10/d14/d15/d17/d54/l5a 0 2024-06-15T01:06:58.302 INFO:tasks.workunit.client.1.smithi142.stdout:5/299: mknod d2/de/d19/c5e 0 2024-06-15T01:06:58.303 INFO:tasks.workunit.client.0.smithi137.stdout:0/238: rename d1/d2/dc to d1/d2/dc/d10/d53/d5b 22 2024-06-15T01:06:58.304 INFO:tasks.workunit.client.1.smithi142.stdout:0/271: creat d3/d5/d4c/d61/f6a x:0 0 0 2024-06-15T01:06:58.305 INFO:tasks.workunit.client.1.smithi142.stdout:0/272: dread - d3/d5/d1e/f63 zero size 2024-06-15T01:06:58.305 INFO:tasks.workunit.client.0.smithi137.stdout:7/236: read df/d18/d1c/d1d/f28 [181951,63692] 0 2024-06-15T01:06:58.306 INFO:tasks.workunit.client.1.smithi142.stdout:8/278: mknod d0/d17/c5d 0 2024-06-15T01:06:58.306 INFO:tasks.workunit.client.0.smithi137.stdout:1/249: mknod d0/d3/d1b/c61 0 2024-06-15T01:06:58.306 INFO:tasks.workunit.client.1.smithi142.stdout:3/367: write d0/d1f/f27 [4372373,8569] 0 2024-06-15T01:06:58.307 INFO:tasks.workunit.client.0.smithi137.stdout:7/237: fsync df/d13/f20 0 2024-06-15T01:06:58.307 INFO:tasks.workunit.client.1.smithi142.stdout:0/273: read d3/f2a [2074576,99561] 0 2024-06-15T01:06:58.307 INFO:tasks.workunit.client.0.smithi137.stdout:8/151: mknod d9/d22/c30 0 2024-06-15T01:06:58.308 INFO:tasks.workunit.client.0.smithi137.stdout:1/250: write d0/d3/f5 [1122441,93036] 0 2024-06-15T01:06:58.308 INFO:tasks.workunit.client.0.smithi137.stdout:8/152: readlink d9/l26 0 2024-06-15T01:06:58.309 INFO:tasks.workunit.client.1.smithi142.stdout:2/260: sync 2024-06-15T01:06:58.309 INFO:tasks.workunit.client.1.smithi142.stdout:4/384: sync 2024-06-15T01:06:58.309 INFO:tasks.workunit.client.0.smithi137.stdout:6/186: write d0/d1c/d1d/d2e/f36 [1204546,32672] 0 2024-06-15T01:06:58.309 INFO:tasks.workunit.client.1.smithi142.stdout:7/247: sync 2024-06-15T01:06:58.311 INFO:tasks.workunit.client.1.smithi142.stdout:9/259: rmdir d6/dc/d1b/d25/d58/d62 0 2024-06-15T01:06:58.312 INFO:tasks.workunit.client.0.smithi137.stdout:1/251: chown d0/d2d/d3e/l5b 1 1 2024-06-15T01:06:58.314 INFO:tasks.workunit.client.1.smithi142.stdout:2/261: dread d8/fe [0,4194304] 0 2024-06-15T01:06:58.314 INFO:tasks.workunit.client.0.smithi137.stdout:5/442: write f7 [1036891,26461] 0 2024-06-15T01:06:58.315 INFO:tasks.workunit.client.1.smithi142.stdout:8/279: mkdir d0/d17/d40/d5e 0 2024-06-15T01:06:58.317 INFO:tasks.workunit.client.0.smithi137.stdout:8/153: rename c5 to d9/d2a/d2f/c31 0 2024-06-15T01:06:58.317 INFO:tasks.workunit.client.0.smithi137.stdout:1/252: read d0/d10/d41/f4b [2548664,75809] 0 2024-06-15T01:06:58.317 INFO:tasks.workunit.client.1.smithi142.stdout:7/248: creat d2/d8/dd/d14/f58 x:0 0 0 2024-06-15T01:06:58.318 INFO:tasks.workunit.client.1.smithi142.stdout:4/385: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/c89 0 2024-06-15T01:06:58.319 INFO:tasks.workunit.client.0.smithi137.stdout:6/187: creat d0/d1c/f3a x:0 0 0 2024-06-15T01:06:58.320 INFO:tasks.workunit.client.0.smithi137.stdout:6/188: dread - d0/d1a/f2f zero size 2024-06-15T01:06:58.320 INFO:tasks.workunit.client.1.smithi142.stdout:9/260: symlink d6/dc/d43/l69 0 2024-06-15T01:06:58.321 INFO:tasks.workunit.client.1.smithi142.stdout:9/261: fsync d6/dc/f1c 0 2024-06-15T01:06:58.322 INFO:tasks.workunit.client.0.smithi137.stdout:6/189: fdatasync d0/d1c/d1d/d2e/f38 0 2024-06-15T01:06:58.323 INFO:tasks.workunit.client.1.smithi142.stdout:4/386: dread d5/d9/d1a/f6a [0,4194304] 0 2024-06-15T01:06:58.324 INFO:tasks.workunit.client.0.smithi137.stdout:8/154: dread d9/f20 [0,4194304] 0 2024-06-15T01:06:58.326 INFO:tasks.workunit.client.1.smithi142.stdout:0/274: mknod d3/d39/d3e/c6b 0 2024-06-15T01:06:58.328 INFO:tasks.workunit.client.1.smithi142.stdout:1/319: rename d6/d12/d1c/c4e to d6/d12/d1c/d2c/c75 0 2024-06-15T01:06:58.328 INFO:tasks.workunit.client.1.smithi142.stdout:2/262: rename d8 to d8/d19/d23/d3a/d4a 22 2024-06-15T01:06:58.332 INFO:tasks.workunit.client.1.smithi142.stdout:3/368: creat d0/d1f/f69 x:0 0 0 2024-06-15T01:06:58.334 INFO:tasks.workunit.client.1.smithi142.stdout:6/271: write d1a/d2d/f2f [4800973,108492] 0 2024-06-15T01:06:58.335 INFO:tasks.workunit.client.1.smithi142.stdout:2/263: dwrite d8/df/f3f [0,4194304] 0 2024-06-15T01:06:58.336 INFO:tasks.workunit.client.0.smithi137.stdout:8/155: dread d9/d22/f25 [0,4194304] 0 2024-06-15T01:06:58.337 INFO:tasks.workunit.client.1.smithi142.stdout:2/264: chown d8/df/d10/d1c/c1e 7 1 2024-06-15T01:06:58.338 INFO:tasks.workunit.client.1.smithi142.stdout:4/387: write d5/d9/d1a/f6a [3736238,128868] 0 2024-06-15T01:06:58.339 INFO:tasks.workunit.client.1.smithi142.stdout:7/249: write d2/d6/f1d [1260558,38548] 0 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.1.smithi142.stdout:1/320: chown d6/d25/l31 1106 1 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.1.smithi142.stdout:4/388: write d5/d9/d1a/f6a [2406265,126719] 0 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.0.smithi137.stdout:7/238: truncate df/d18/d1c/d1d/f28 2851266 0 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.0.smithi137.stdout:2/182: dwrite d0/f1 [0,4194304] 0 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.0.smithi137.stdout:0/239: dread d1/f9 [0,4194304] 0 2024-06-15T01:06:58.342 INFO:tasks.workunit.client.0.smithi137.stdout:6/190: dwrite d0/f2a [0,4194304] 0 2024-06-15T01:06:58.343 INFO:tasks.workunit.client.1.smithi142.stdout:7/250: write d2/d8/da/d33/f35 [985971,69452] 0 2024-06-15T01:06:58.344 INFO:tasks.workunit.client.1.smithi142.stdout:5/300: getdents d2/de/d19 0 2024-06-15T01:06:58.346 INFO:tasks.workunit.client.1.smithi142.stdout:3/369: creat d0/f6a x:0 0 0 2024-06-15T01:06:58.348 INFO:tasks.workunit.client.1.smithi142.stdout:4/389: write d5/d9/da/f7c [858678,86977] 0 2024-06-15T01:06:58.348 INFO:tasks.workunit.client.0.smithi137.stdout:7/239: write df/d13/d2e/f19 [4188817,52550] 0 2024-06-15T01:06:58.349 INFO:tasks.workunit.client.0.smithi137.stdout:5/443: write df/d44/d54/d95/f94 [3924298,15660] 0 2024-06-15T01:06:58.352 INFO:tasks.workunit.client.1.smithi142.stdout:8/280: dwrite d0/d25/f35 [0,4194304] 0 2024-06-15T01:06:58.354 INFO:tasks.workunit.client.1.smithi142.stdout:8/281: write d0/d25/d53/f58 [371068,82551] 0 2024-06-15T01:06:58.354 INFO:tasks.workunit.client.0.smithi137.stdout:8/156: mknod d9/de/d1e/c32 0 2024-06-15T01:06:58.355 INFO:tasks.workunit.client.1.smithi142.stdout:8/282: write d0/f33 [4213399,45083] 0 2024-06-15T01:06:58.357 INFO:tasks.workunit.client.0.smithi137.stdout:1/253: dwrite d0/d10/d41/f4b [0,4194304] 0 2024-06-15T01:06:58.357 INFO:tasks.workunit.client.0.smithi137.stdout:6/191: mknod d0/d1c/d32/c3b 0 2024-06-15T01:06:58.357 INFO:tasks.workunit.client.1.smithi142.stdout:7/251: rmdir d2/d8/dd 39 2024-06-15T01:06:58.362 INFO:tasks.workunit.client.1.smithi142.stdout:7/252: truncate d2/d8/da/d33/d25/d27/f2f 1554334 0 2024-06-15T01:06:58.362 INFO:tasks.workunit.client.0.smithi137.stdout:2/183: readlink d0/da/l26 0 2024-06-15T01:06:58.362 INFO:tasks.workunit.client.1.smithi142.stdout:3/370: mkdir d0/d1b/d6b 0 2024-06-15T01:06:58.363 INFO:tasks.workunit.client.0.smithi137.stdout:7/240: chown df/d13/d2e/d1a/f2c 702 1 2024-06-15T01:06:58.363 INFO:tasks.workunit.client.0.smithi137.stdout:8/157: read d9/de/f18 [82210,115677] 0 2024-06-15T01:06:58.364 INFO:tasks.workunit.client.0.smithi137.stdout:5/444: write df/f41 [5072019,49069] 0 2024-06-15T01:06:58.364 INFO:tasks.workunit.client.1.smithi142.stdout:4/390: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d59/f8a x:0 0 0 2024-06-15T01:06:58.364 INFO:tasks.workunit.client.1.smithi142.stdout:8/283: dwrite d0/d25/d5a/f48 [0,4194304] 0 2024-06-15T01:06:58.366 INFO:tasks.workunit.client.0.smithi137.stdout:1/254: stat d0/d3/d1b/f4f 0 2024-06-15T01:06:58.367 INFO:tasks.workunit.client.1.smithi142.stdout:4/391: write d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f37 [1732636,12692] 0 2024-06-15T01:06:58.373 INFO:tasks.workunit.client.0.smithi137.stdout:8/158: unlink d9/l26 0 2024-06-15T01:06:58.373 INFO:tasks.workunit.client.0.smithi137.stdout:1/255: chown d0/d3/d14/d34/f45 735799 1 2024-06-15T01:06:58.374 INFO:tasks.workunit.client.0.smithi137.stdout:0/240: dwrite d1/d2/f47 [0,4194304] 0 2024-06-15T01:06:58.374 INFO:tasks.workunit.client.1.smithi142.stdout:4/392: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d59/f8a [0,4194304] 0 2024-06-15T01:06:58.376 INFO:tasks.workunit.client.0.smithi137.stdout:5/445: creat df/d3d/f97 x:0 0 0 2024-06-15T01:06:58.376 INFO:tasks.workunit.client.1.smithi142.stdout:9/262: rename d6/dc/f1c to d6/f6a 0 2024-06-15T01:06:58.378 INFO:tasks.workunit.client.1.smithi142.stdout:9/263: truncate d6/dc/dd/d22/f54 468585 0 2024-06-15T01:06:58.379 INFO:tasks.workunit.client.0.smithi137.stdout:8/159: creat d9/d22/f33 x:0 0 0 2024-06-15T01:06:58.380 INFO:tasks.workunit.client.1.smithi142.stdout:3/371: mknod d0/d3/c6c 0 2024-06-15T01:06:58.381 INFO:tasks.workunit.client.1.smithi142.stdout:3/372: chown d0/f6a 22817 1 2024-06-15T01:06:58.383 INFO:tasks.workunit.client.1.smithi142.stdout:0/275: link d3/d5/d4c/c53 d3/d12/d22/d59/c6c 0 2024-06-15T01:06:58.383 INFO:tasks.workunit.client.1.smithi142.stdout:0/276: dread - d3/d12/d22/f65 zero size 2024-06-15T01:06:58.385 INFO:tasks.workunit.client.0.smithi137.stdout:6/192: creat d0/f3c x:0 0 0 2024-06-15T01:06:58.385 INFO:tasks.workunit.client.1.smithi142.stdout:8/284: rename d0/f46 to d0/f5f 0 2024-06-15T01:06:58.385 INFO:tasks.workunit.client.0.smithi137.stdout:7/241: dwrite df/f32 [0,4194304] 0 2024-06-15T01:06:58.386 INFO:tasks.workunit.client.0.smithi137.stdout:5/446: readlink df/d3d/l8b 0 2024-06-15T01:06:58.388 INFO:tasks.workunit.client.1.smithi142.stdout:4/393: creat d5/d9/da/f8b x:0 0 0 2024-06-15T01:06:58.392 INFO:tasks.workunit.client.1.smithi142.stdout:3/373: rename d0/d3/da/c2f to d0/d1f/d20/d5d/c6d 0 2024-06-15T01:06:58.394 INFO:tasks.workunit.client.0.smithi137.stdout:0/241: rename d1/d2/dc/d10/d14/d15/d1c/l31 to d1/d2/dc/d10/d14/d15/l5c 0 2024-06-15T01:06:58.395 INFO:tasks.workunit.client.0.smithi137.stdout:5/447: read df/d11/d14/f21 [6037,77194] 0 2024-06-15T01:06:58.397 INFO:tasks.workunit.client.1.smithi142.stdout:9/264: symlink d6/d34/l6b 0 2024-06-15T01:06:58.397 INFO:tasks.workunit.client.0.smithi137.stdout:5/448: write df/d11/d30/f87 [126574,112204] 0 2024-06-15T01:06:58.398 INFO:tasks.workunit.client.0.smithi137.stdout:1/256: dwrite d0/d3/f5 [0,4194304] 0 2024-06-15T01:06:58.398 INFO:tasks.workunit.client.1.smithi142.stdout:4/394: rmdir d5/d1c/d2a/d30/d46 39 2024-06-15T01:06:58.398 INFO:tasks.workunit.client.1.smithi142.stdout:9/265: fsync d6/dc/d4e/f57 0 2024-06-15T01:06:58.406 INFO:tasks.workunit.client.0.smithi137.stdout:8/160: dread f6 [0,4194304] 0 2024-06-15T01:06:58.409 INFO:tasks.workunit.client.0.smithi137.stdout:1/257: getdents d0/d10 0 2024-06-15T01:06:58.409 INFO:tasks.workunit.client.0.smithi137.stdout:8/161: creat d9/d2a/f34 x:0 0 0 2024-06-15T01:06:58.412 INFO:tasks.workunit.client.0.smithi137.stdout:1/258: creat d0/d2d/f62 x:0 0 0 2024-06-15T01:06:58.413 INFO:tasks.workunit.client.1.smithi142.stdout:4/395: rmdir d5/d1c/d2a/d30 39 2024-06-15T01:06:58.413 INFO:tasks.workunit.client.0.smithi137.stdout:1/259: fsync d0/d3/f13 0 2024-06-15T01:06:58.414 INFO:tasks.workunit.client.1.smithi142.stdout:1/321: truncate d6/d12/d1c/d2c/d47/f5a 260565 0 2024-06-15T01:06:58.415 INFO:tasks.workunit.client.0.smithi137.stdout:1/260: fsync d0/d3/d1b/f50 0 2024-06-15T01:06:58.416 INFO:tasks.workunit.client.1.smithi142.stdout:9/266: symlink d6/dc/d1b/d25/l6c 0 2024-06-15T01:06:58.417 INFO:tasks.workunit.client.0.smithi137.stdout:8/162: dwrite f0 [0,4194304] 0 2024-06-15T01:06:58.418 INFO:tasks.workunit.client.1.smithi142.stdout:6/272: truncate d1a/f20 2853256 0 2024-06-15T01:06:58.419 INFO:tasks.workunit.client.1.smithi142.stdout:4/396: dwrite d5/d9/d18/f69 [0,4194304] 0 2024-06-15T01:06:58.419 INFO:tasks.workunit.client.0.smithi137.stdout:1/261: creat d0/d3/d1b/d4c/d59/f63 x:0 0 0 2024-06-15T01:06:58.420 INFO:tasks.workunit.client.0.smithi137.stdout:1/262: dread - d0/d2d/f62 zero size 2024-06-15T01:06:58.421 INFO:tasks.workunit.client.0.smithi137.stdout:8/163: mkdir d9/de/d1e/d35 0 2024-06-15T01:06:58.424 INFO:tasks.workunit.client.0.smithi137.stdout:1/263: mkdir d0/d3/d1b/d4c/d59/d64 0 2024-06-15T01:06:58.425 INFO:tasks.workunit.client.1.smithi142.stdout:0/277: truncate d3/f9 321795 0 2024-06-15T01:06:58.426 INFO:tasks.workunit.client.1.smithi142.stdout:0/278: write d3/d5/d1e/f5d [640908,87558] 0 2024-06-15T01:06:58.426 INFO:tasks.workunit.client.1.smithi142.stdout:1/322: creat d6/d46/d61/f76 x:0 0 0 2024-06-15T01:06:58.427 INFO:tasks.workunit.client.0.smithi137.stdout:1/264: mknod d0/d10/d1d/c65 0 2024-06-15T01:06:58.431 INFO:tasks.workunit.client.0.smithi137.stdout:8/164: rename d9/d22/f23 to d9/de/f36 0 2024-06-15T01:06:58.432 INFO:tasks.workunit.client.0.smithi137.stdout:1/265: mkdir d0/d3/d1b/d4c/d59/d66 0 2024-06-15T01:06:58.433 INFO:tasks.workunit.client.1.smithi142.stdout:9/267: mknod d6/dc/d1b/d25/c6d 0 2024-06-15T01:06:58.433 INFO:tasks.workunit.client.0.smithi137.stdout:8/165: read f0 [4602736,82632] 0 2024-06-15T01:06:58.434 INFO:tasks.workunit.client.0.smithi137.stdout:8/166: stat d9/de/c14 0 2024-06-15T01:06:58.434 INFO:tasks.workunit.client.1.smithi142.stdout:7/253: dwrite d2/d8/da/d33/f20 [0,4194304] 0 2024-06-15T01:06:58.434 INFO:tasks.workunit.client.1.smithi142.stdout:3/374: write d0/d3/f1a [4701443,71354] 0 2024-06-15T01:06:58.435 INFO:tasks.workunit.client.0.smithi137.stdout:8/167: fdatasync d9/f1b 0 2024-06-15T01:06:58.436 INFO:tasks.workunit.client.0.smithi137.stdout:1/266: mkdir d0/d1f/d67 0 2024-06-15T01:06:58.436 INFO:tasks.workunit.client.0.smithi137.stdout:1/267: truncate d0/d2d/f5e 2666581 0 2024-06-15T01:06:58.437 INFO:tasks.workunit.client.1.smithi142.stdout:3/375: chown d0/d3/d15/d4d/d55/l56 4257378 1 2024-06-15T01:06:58.437 INFO:tasks.workunit.client.0.smithi137.stdout:8/168: creat d9/d22/f37 x:0 0 0 2024-06-15T01:06:58.439 INFO:tasks.workunit.client.0.smithi137.stdout:1/268: rmdir d0/d1f 39 2024-06-15T01:06:58.439 INFO:tasks.workunit.client.1.smithi142.stdout:3/376: truncate d0/d3/d15/f64 4264136 0 2024-06-15T01:06:58.442 INFO:tasks.workunit.client.0.smithi137.stdout:1/269: dread - d0/d1f/d60/f55 zero size 2024-06-15T01:06:58.442 INFO:tasks.workunit.client.0.smithi137.stdout:8/169: creat d9/de/d1e/d35/f38 x:0 0 0 2024-06-15T01:06:58.443 INFO:tasks.workunit.client.1.smithi142.stdout:6/273: dwrite d1a/d39/d3a/d51/f5c [0,4194304] 0 2024-06-15T01:06:58.443 INFO:tasks.workunit.client.0.smithi137.stdout:1/270: write d0/f2 [4841571,35932] 0 2024-06-15T01:06:58.445 INFO:tasks.workunit.client.0.smithi137.stdout:1/271: write d0/d10/f23 [4773117,21259] 0 2024-06-15T01:06:58.445 INFO:tasks.workunit.client.1.smithi142.stdout:0/279: creat d3/d5/f6d x:0 0 0 2024-06-15T01:06:58.447 INFO:tasks.workunit.client.0.smithi137.stdout:8/170: write f1 [1625176,109544] 0 2024-06-15T01:06:58.447 INFO:tasks.workunit.client.0.smithi137.stdout:1/272: write d0/d10/f2e [9394573,23971] 0 2024-06-15T01:06:58.448 INFO:tasks.workunit.client.0.smithi137.stdout:1/273: readlink d0/d10/l16 0 2024-06-15T01:06:58.449 INFO:tasks.workunit.client.0.smithi137.stdout:1/274: dread - d0/d3/d1b/f50 zero size 2024-06-15T01:06:58.450 INFO:tasks.workunit.client.1.smithi142.stdout:6/274: dwrite d1a/d2d/f2f [0,4194304] 0 2024-06-15T01:06:58.453 INFO:tasks.workunit.client.1.smithi142.stdout:9/268: rmdir d6/dc 39 2024-06-15T01:06:58.454 INFO:tasks.workunit.client.0.smithi137.stdout:1/275: symlink d0/d3/d14/l68 0 2024-06-15T01:06:58.456 INFO:tasks.workunit.client.0.smithi137.stdout:1/276: fdatasync d0/d3/d14/f38 0 2024-06-15T01:06:58.458 INFO:tasks.workunit.client.0.smithi137.stdout:1/277: creat d0/d3/d1b/f69 x:0 0 0 2024-06-15T01:06:58.458 INFO:tasks.workunit.client.1.smithi142.stdout:3/377: mkdir d0/d1f/d3e/d6e 0 2024-06-15T01:06:58.460 INFO:tasks.workunit.client.1.smithi142.stdout:4/397: dread d5/d1c/d2a/d30/d46/f76 [0,4194304] 0 2024-06-15T01:06:58.461 INFO:tasks.workunit.client.0.smithi137.stdout:1/278: symlink d0/d1f/d60/l6a 0 2024-06-15T01:06:58.463 INFO:tasks.workunit.client.1.smithi142.stdout:9/269: stat d6/dc/d1b 0 2024-06-15T01:06:58.464 INFO:tasks.workunit.client.0.smithi137.stdout:1/279: rename d0/d10/l4e to d0/d1f/d67/l6b 0 2024-06-15T01:06:58.464 INFO:tasks.workunit.client.0.smithi137.stdout:1/280: read - d0/f3b zero size 2024-06-15T01:06:58.466 INFO:tasks.workunit.client.0.smithi137.stdout:1/281: mknod d0/d3/d1b/d4c/d59/d64/c6c 0 2024-06-15T01:06:58.467 INFO:tasks.workunit.client.1.smithi142.stdout:6/275: dwrite d1a/f28 [0,4194304] 0 2024-06-15T01:06:58.467 INFO:tasks.workunit.client.1.smithi142.stdout:4/398: unlink d5/d1c/l55 0 2024-06-15T01:06:58.469 INFO:tasks.workunit.client.1.smithi142.stdout:6/276: write d1a/d39/d3a/f5f [789280,39885] 0 2024-06-15T01:06:58.470 INFO:tasks.workunit.client.1.smithi142.stdout:6/277: fsync fc 0 2024-06-15T01:06:58.473 INFO:tasks.workunit.client.0.smithi137.stdout:1/282: dwrite d0/d3/d14/d34/f45 [0,4194304] 0 2024-06-15T01:06:58.474 INFO:tasks.workunit.client.1.smithi142.stdout:0/280: creat d3/f6e x:0 0 0 2024-06-15T01:06:58.475 INFO:tasks.workunit.client.1.smithi142.stdout:4/399: write f2 [3225146,43405] 0 2024-06-15T01:06:58.476 INFO:tasks.workunit.client.0.smithi137.stdout:1/283: symlink d0/d1f/d67/l6d 0 2024-06-15T01:06:58.476 INFO:tasks.workunit.client.1.smithi142.stdout:7/254: getdents d2/d8/da 0 2024-06-15T01:06:58.477 INFO:tasks.workunit.client.0.smithi137.stdout:1/284: fsync d0/d10/f40 0 2024-06-15T01:06:58.479 INFO:tasks.workunit.client.0.smithi137.stdout:1/285: rmdir d0/d10 39 2024-06-15T01:06:58.479 INFO:tasks.workunit.client.1.smithi142.stdout:4/400: read f3 [1763415,67283] 0 2024-06-15T01:06:58.482 INFO:tasks.workunit.client.1.smithi142.stdout:3/378: rename d0/d1f/d20/d5d/f5f to d0/d1f/f6f 0 2024-06-15T01:06:58.483 INFO:tasks.workunit.client.0.smithi137.stdout:1/286: link d0/d3/d1b/d4c/d59/f63 d0/d3/d1b/d4c/d59/d64/f6e 0 2024-06-15T01:06:58.484 INFO:tasks.workunit.client.1.smithi142.stdout:0/281: write d3/d5/d1e/f2b [798391,122064] 0 2024-06-15T01:06:58.491 INFO:tasks.workunit.client.1.smithi142.stdout:4/401: mknod d5/d1c/d2a/d30/d46/d50/d51/c8c 0 2024-06-15T01:06:58.492 INFO:tasks.workunit.client.1.smithi142.stdout:9/270: dread d6/dc/f15 [0,4194304] 0 2024-06-15T01:06:58.493 INFO:tasks.workunit.client.1.smithi142.stdout:9/271: readlink d6/dc/l13 0 2024-06-15T01:06:58.494 INFO:tasks.workunit.client.1.smithi142.stdout:6/278: creat d1a/f61 x:0 0 0 2024-06-15T01:06:58.502 INFO:tasks.workunit.client.1.smithi142.stdout:3/379: symlink d0/d1f/d3e/d6e/l70 0 2024-06-15T01:06:58.508 INFO:tasks.workunit.client.0.smithi137.stdout:2/184: truncate d0/f6 2423935 0 2024-06-15T01:06:58.509 INFO:tasks.workunit.client.0.smithi137.stdout:2/185: chown d0/de/f16 168 1 2024-06-15T01:06:58.510 INFO:tasks.workunit.client.1.smithi142.stdout:0/282: fdatasync d3/f2a 0 2024-06-15T01:06:58.512 INFO:tasks.workunit.client.1.smithi142.stdout:4/402: symlink d5/d1c/d6f/l8d 0 2024-06-15T01:06:58.512 INFO:tasks.workunit.client.0.smithi137.stdout:2/186: symlink d0/de/d2b/l36 0 2024-06-15T01:06:58.513 INFO:tasks.workunit.client.0.smithi137.stdout:6/193: truncate d0/f10 1001538 0 2024-06-15T01:06:58.514 INFO:tasks.workunit.client.0.smithi137.stdout:6/194: chown d0 29491830 1 2024-06-15T01:06:58.514 INFO:tasks.workunit.client.1.smithi142.stdout:7/255: creat d2/d8/da/d33/d25/f59 x:0 0 0 2024-06-15T01:06:58.515 INFO:tasks.workunit.client.0.smithi137.stdout:8/171: dread d9/de/ff [0,4194304] 0 2024-06-15T01:06:58.516 INFO:tasks.workunit.client.0.smithi137.stdout:0/242: truncate d1/d2/d13/f33 409921 0 2024-06-15T01:06:58.517 INFO:tasks.workunit.client.0.smithi137.stdout:6/195: creat d0/d1c/d1d/f3d x:0 0 0 2024-06-15T01:06:58.517 INFO:tasks.workunit.client.1.smithi142.stdout:8/285: dread d0/d17/f1f [0,4194304] 0 2024-06-15T01:06:58.523 INFO:tasks.workunit.client.0.smithi137.stdout:6/196: mkdir d0/d1a/d3e 0 2024-06-15T01:06:58.524 INFO:tasks.workunit.client.1.smithi142.stdout:3/380: dwrite d0/d1f/d20/f26 [4194304,4194304] 0 2024-06-15T01:06:58.525 INFO:tasks.workunit.client.0.smithi137.stdout:8/172: link d9/de/d1e/f28 d9/de/d1e/f39 0 2024-06-15T01:06:58.526 INFO:tasks.workunit.client.1.smithi142.stdout:7/256: unlink d2/d6/d22/f57 0 2024-06-15T01:06:58.526 INFO:tasks.workunit.client.1.smithi142.stdout:6/279: mkdir d1a/d39/d3a/d51/d62 0 2024-06-15T01:06:58.527 INFO:tasks.workunit.client.0.smithi137.stdout:8/173: write d9/d22/f37 [220287,73241] 0 2024-06-15T01:06:58.528 INFO:tasks.workunit.client.0.smithi137.stdout:0/243: symlink d1/d2/dc/d10/d14/d15/d17/l5d 0 2024-06-15T01:06:58.529 INFO:tasks.workunit.client.0.smithi137.stdout:2/187: link d0/de/d10/c13 d0/dc/d22/c37 0 2024-06-15T01:06:58.530 INFO:tasks.workunit.client.1.smithi142.stdout:0/283: creat d3/d39/f6f x:0 0 0 2024-06-15T01:06:58.532 INFO:tasks.workunit.client.0.smithi137.stdout:6/197: mkdir d0/d5/d9/d3f 0 2024-06-15T01:06:58.534 INFO:tasks.workunit.client.0.smithi137.stdout:8/174: symlink d9/de/d1e/d35/l3a 0 2024-06-15T01:06:58.535 INFO:tasks.workunit.client.1.smithi142.stdout:8/286: dread d0/f4c [0,4194304] 0 2024-06-15T01:06:58.536 INFO:tasks.workunit.client.1.smithi142.stdout:7/257: dwrite d2/d6/d22/d34/f36 [0,4194304] 0 2024-06-15T01:06:58.540 INFO:tasks.workunit.client.1.smithi142.stdout:6/280: rename d1a/d39/d42/f44 to d1a/d39/d3a/d51/f63 0 2024-06-15T01:06:58.542 INFO:tasks.workunit.client.0.smithi137.stdout:6/198: creat d0/d1c/d1d/f40 x:0 0 0 2024-06-15T01:06:58.546 INFO:tasks.workunit.client.0.smithi137.stdout:8/175: rename d9/ld to d9/d22/l3b 0 2024-06-15T01:06:58.546 INFO:tasks.workunit.client.1.smithi142.stdout:9/272: symlink d6/dc/l6e 0 2024-06-15T01:06:58.548 INFO:tasks.workunit.client.0.smithi137.stdout:2/188: chown d0/dc/d22/c37 57300 1 2024-06-15T01:06:58.551 INFO:tasks.workunit.client.0.smithi137.stdout:6/199: dwrite d0/f3c [0,4194304] 0 2024-06-15T01:06:58.552 INFO:tasks.workunit.client.0.smithi137.stdout:8/176: creat d9/de/f3c x:0 0 0 2024-06-15T01:06:58.558 INFO:tasks.workunit.client.0.smithi137.stdout:8/177: mknod d9/de/d1e/d35/c3d 0 2024-06-15T01:06:58.558 INFO:tasks.workunit.client.1.smithi142.stdout:7/258: creat d2/d6/d22/d47/f5a x:0 0 0 2024-06-15T01:06:58.560 INFO:tasks.workunit.client.1.smithi142.stdout:6/281: write f0 [2987217,54792] 0 2024-06-15T01:06:58.560 INFO:tasks.workunit.client.0.smithi137.stdout:2/189: truncate d0/da/f17 1040994 0 2024-06-15T01:06:58.561 INFO:tasks.workunit.client.1.smithi142.stdout:6/282: chown d1a/d39/d3a/d51/f63 0 1 2024-06-15T01:06:58.564 INFO:tasks.workunit.client.1.smithi142.stdout:7/259: read d2/d6/f1c [1337102,48354] 0 2024-06-15T01:06:58.564 INFO:tasks.workunit.client.1.smithi142.stdout:4/403: getdents d5/d9/d23 0 2024-06-15T01:06:58.565 INFO:tasks.workunit.client.1.smithi142.stdout:9/273: fsync d6/f6a 0 2024-06-15T01:06:58.566 INFO:tasks.workunit.client.1.smithi142.stdout:4/404: chown d5/d9/c7b 6 1 2024-06-15T01:06:58.566 INFO:tasks.workunit.client.1.smithi142.stdout:7/260: read - d2/d6/d22/d47/f4f zero size 2024-06-15T01:06:58.566 INFO:tasks.workunit.client.1.smithi142.stdout:4/405: dread - d5/d9/d41/f65 zero size 2024-06-15T01:06:58.566 INFO:tasks.workunit.client.1.smithi142.stdout:9/274: chown d6/dc/dd/le 971976 1 2024-06-15T01:06:58.571 INFO:tasks.workunit.client.1.smithi142.stdout:8/287: creat d0/d25/d53/f60 x:0 0 0 2024-06-15T01:06:58.572 INFO:tasks.workunit.client.0.smithi137.stdout:8/178: dread d9/de/d1e/f28 [0,4194304] 0 2024-06-15T01:06:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.576+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f47ac19a1f0 con 0x7f47ac0ffab0 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.577+0000 7f47a27fc700 1 -- 172.21.15.137:0/3037160451 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f47a8059750 con 0x7f47ac0ffab0 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-15T01:06:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 1 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 1 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:06:58.575 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 msgr2=0x7f4798072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 0x7f4798072540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c005dc0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 msgr2=0x7f47ac193ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 0x7f47ac193ec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47a8010aa0 tx=0x7f47a8010e60 comp rx=0 tx=0).stop 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 shutdown_connections 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4798070080 0x7f4798072540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47ac0ffab0 0x7f47ac193ec0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.581+0000 7f47b377c700 1 --2- 172.21.15.137:0/3037160451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac100cb0 0x7f47ac194400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.582+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 >> 172.21.15.137:0/3037160451 conn(0x7f47ac0fb030 msgr2=0x7f47ac0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.582+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 shutdown_connections 2024-06-15T01:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.582+0000 7f47b377c700 1 -- 172.21.15.137:0/3037160451 wait complete. 2024-06-15T01:06:58.579 INFO:tasks.workunit.client.0.smithi137.stdout:8/179: fsync d9/de/f18 0 2024-06-15T01:06:58.580 INFO:tasks.workunit.client.1.smithi142.stdout:6/283: write d1a/f1e [3306257,4140] 0 2024-06-15T01:06:58.580 INFO:tasks.workunit.client.1.smithi142.stdout:7/261: chown d2/d8/dd/d14/f46 120164946 1 2024-06-15T01:06:58.581 INFO:tasks.workunit.client.0.smithi137.stdout:8/180: dread d9/de/d1e/f28 [0,4194304] 0 2024-06-15T01:06:58.581 INFO:tasks.workunit.client.1.smithi142.stdout:7/262: chown d2/d8/dd/d14/f58 3101642 1 2024-06-15T01:06:58.581 INFO:tasks.workunit.client.1.smithi142.stdout:2/265: sync 2024-06-15T01:06:58.582 INFO:tasks.workunit.client.0.smithi137.stdout:8/181: fsync d9/de/d1e/d35/f38 0 2024-06-15T01:06:58.582 INFO:tasks.workunit.client.1.smithi142.stdout:2/266: rename d8/df to d8/df/d10/d1c/d4b 22 2024-06-15T01:06:58.582 INFO:tasks.workunit.client.0.smithi137.stdout:8/182: stat d9/d2a/d2f 0 2024-06-15T01:06:58.583 INFO:tasks.workunit.client.1.smithi142.stdout:0/284: link d3/d5/d1e/f5d d3/d12/d22/f70 0 2024-06-15T01:06:58.586 INFO:tasks.workunit.client.0.smithi137.stdout:0/244: write d1/d2/dc/d10/d14/f1b [2615343,124502] 0 2024-06-15T01:06:58.587 INFO:tasks.workunit.client.1.smithi142.stdout:6/284: mknod d1a/d39/d3a/c64 0 2024-06-15T01:06:58.587 INFO:tasks.workunit.client.0.smithi137.stdout:6/200: rmdir d0/d1c 39 2024-06-15T01:06:58.588 INFO:tasks.workunit.client.0.smithi137.stdout:6/201: write d0/f3c [568911,61576] 0 2024-06-15T01:06:58.590 INFO:tasks.workunit.client.1.smithi142.stdout:9/275: truncate d6/f6a 2567341 0 2024-06-15T01:06:58.593 INFO:tasks.workunit.client.1.smithi142.stdout:4/406: link d5/d1c/d2a/d30/d46/d50/d51/d5b/d59/f7a d5/d9/d23/d68/f8e 0 2024-06-15T01:06:58.594 INFO:tasks.workunit.client.1.smithi142.stdout:7/263: symlink d2/d8/dd/l5b 0 2024-06-15T01:06:58.595 INFO:tasks.workunit.client.1.smithi142.stdout:2/267: mknod d8/df/d1f/c4c 0 2024-06-15T01:06:58.596 INFO:tasks.workunit.client.1.smithi142.stdout:2/268: chown d8/c45 42 1 2024-06-15T01:06:58.597 INFO:tasks.workunit.client.1.smithi142.stdout:0/285: write d3/f28 [694862,42161] 0 2024-06-15T01:06:58.597 INFO:tasks.workunit.client.0.smithi137.stdout:6/202: dwrite d0/d5/f7 [4194304,4194304] 0 2024-06-15T01:06:58.597 INFO:tasks.workunit.client.1.smithi142.stdout:3/381: write d0/f39 [786422,38062] 0 2024-06-15T01:06:58.598 INFO:tasks.workunit.client.0.smithi137.stdout:0/245: rename d1/d2/d34/f41 to d1/d2/f5e 0 2024-06-15T01:06:58.600 INFO:tasks.workunit.client.1.smithi142.stdout:3/382: write d0/d3/f54 [143925,116921] 0 2024-06-15T01:06:58.601 INFO:tasks.workunit.client.1.smithi142.stdout:9/276: mkdir d6/dc/d1b/d6f 0 2024-06-15T01:06:58.602 INFO:tasks.workunit.client.0.smithi137.stdout:0/246: chown d1/d2/dc/d10/f4f 29 1 2024-06-15T01:06:58.602 INFO:tasks.workunit.client.1.smithi142.stdout:7/264: dread d2/d8/dd/d14/f4b [0,4194304] 0 2024-06-15T01:06:58.605 INFO:tasks.workunit.client.0.smithi137.stdout:8/183: write d9/de/d1e/f39 [1736490,114878] 0 2024-06-15T01:06:58.606 INFO:tasks.workunit.client.0.smithi137.stdout:0/247: symlink d1/d2/d34/l5f 0 2024-06-15T01:06:58.607 INFO:tasks.workunit.client.1.smithi142.stdout:8/288: link d0/d17/f1f d0/d17/f61 0 2024-06-15T01:06:58.607 INFO:tasks.workunit.client.1.smithi142.stdout:7/265: stat d2/d8/da/d33/f35 0 2024-06-15T01:06:58.609 INFO:tasks.workunit.client.1.smithi142.stdout:6/285: dwrite d1a/d39/d3a/d51/f63 [0,4194304] 0 2024-06-15T01:06:58.610 INFO:tasks.workunit.client.0.smithi137.stdout:6/203: link d0/d1c/d1d/f3d d0/d5/d9/f41 0 2024-06-15T01:06:58.610 INFO:tasks.workunit.client.1.smithi142.stdout:6/286: dread - d1a/d39/d3a/f5a zero size 2024-06-15T01:06:58.611 INFO:tasks.workunit.client.0.smithi137.stdout:2/190: dread d0/da/f17 [0,4194304] 0 2024-06-15T01:06:58.613 INFO:tasks.workunit.client.0.smithi137.stdout:8/184: creat d9/d22/f3e x:0 0 0 2024-06-15T01:06:58.615 INFO:tasks.workunit.client.1.smithi142.stdout:9/277: creat d6/dc/d1b/d25/d35/f70 x:0 0 0 2024-06-15T01:06:58.617 INFO:tasks.workunit.client.1.smithi142.stdout:4/407: dread d5/d1c/d2a/d30/d46/d50/f5e [0,4194304] 0 2024-06-15T01:06:58.617 INFO:tasks.workunit.client.1.smithi142.stdout:0/286: creat d3/d12/d22/d59/f71 x:0 0 0 2024-06-15T01:06:58.617 INFO:tasks.workunit.client.1.smithi142.stdout:7/266: dwrite d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:06:58.622 INFO:tasks.workunit.client.0.smithi137.stdout:0/248: dwrite d1/d2/dc/d10/d14/d15/f50 [0,4194304] 0 2024-06-15T01:06:58.625 INFO:tasks.workunit.client.0.smithi137.stdout:2/191: dwrite d0/f5 [4194304,4194304] 0 2024-06-15T01:06:58.628 INFO:tasks.workunit.client.1.smithi142.stdout:2/269: write d8/df/d10/f35 [559985,22246] 0 2024-06-15T01:06:58.629 INFO:tasks.workunit.client.1.smithi142.stdout:8/289: read d0/f47 [664134,9025] 0 2024-06-15T01:06:58.629 INFO:tasks.workunit.client.1.smithi142.stdout:4/408: dwrite d5/d9/d1a/f82 [0,4194304] 0 2024-06-15T01:06:58.629 INFO:tasks.workunit.client.1.smithi142.stdout:0/287: rmdir d3/d39/d3e 39 2024-06-15T01:06:58.635 INFO:tasks.workunit.client.0.smithi137.stdout:6/204: rename d0/c14 to d0/d5/d9/c42 0 2024-06-15T01:06:58.636 INFO:tasks.workunit.client.1.smithi142.stdout:4/409: dwrite d5/d9/d18/f80 [0,4194304] 0 2024-06-15T01:06:58.638 INFO:tasks.workunit.client.0.smithi137.stdout:2/192: dwrite d0/de/d10/f19 [4194304,4194304] 0 2024-06-15T01:06:58.641 INFO:tasks.workunit.client.1.smithi142.stdout:3/383: rename d0/d3/d15/d4d to d0/d3/da/d71 0 2024-06-15T01:06:58.645 INFO:tasks.workunit.client.1.smithi142.stdout:8/290: mknod d0/d17/d40/c62 0 2024-06-15T01:06:58.646 INFO:tasks.workunit.client.0.smithi137.stdout:6/205: mkdir d0/d5/d43 0 2024-06-15T01:06:58.647 INFO:tasks.workunit.client.1.smithi142.stdout:8/291: read d0/d17/f2f [3908663,48695] 0 2024-06-15T01:06:58.648 INFO:tasks.workunit.client.0.smithi137.stdout:2/193: dwrite d0/f1 [0,4194304] 0 2024-06-15T01:06:58.649 INFO:tasks.workunit.client.1.smithi142.stdout:4/410: creat d5/d9/d23/d32/f8f x:0 0 0 2024-06-15T01:06:58.653 INFO:tasks.workunit.client.0.smithi137.stdout:8/185: getdents d9/de 0 2024-06-15T01:06:58.656 INFO:tasks.workunit.client.0.smithi137.stdout:8/186: mkdir d9/de/d1e/d3f 0 2024-06-15T01:06:58.660 INFO:tasks.workunit.client.0.smithi137.stdout:8/187: fsync d9/d22/f25 0 2024-06-15T01:06:58.663 INFO:tasks.workunit.client.1.smithi142.stdout:8/292: mkdir d0/d17/d63 0 2024-06-15T01:06:58.665 INFO:tasks.workunit.client.0.smithi137.stdout:8/188: symlink d9/de/d1e/l40 0 2024-06-15T01:06:58.665 INFO:tasks.workunit.client.1.smithi142.stdout:9/278: rename d6/dc/dd/f1d to d6/f71 0 2024-06-15T01:06:58.668 INFO:tasks.workunit.client.1.smithi142.stdout:7/267: rename d2/d8/da/d33/d25/d27/l39 to d2/d6/l5c 0 2024-06-15T01:06:58.669 INFO:tasks.workunit.client.0.smithi137.stdout:8/189: creat d9/de/d1e/d3f/f41 x:0 0 0 2024-06-15T01:06:58.672 INFO:tasks.workunit.client.1.smithi142.stdout:9/279: dwrite d6/d31/f3e [0,4194304] 0 2024-06-15T01:06:58.673 INFO:tasks.workunit.client.1.smithi142.stdout:8/293: rename d0/d25/d53/c54 to d0/d25/c64 0 2024-06-15T01:06:58.675 INFO:tasks.workunit.client.1.smithi142.stdout:9/280: creat d6/dc/d43/f72 x:0 0 0 2024-06-15T01:06:58.679 INFO:tasks.workunit.client.1.smithi142.stdout:7/268: dwrite d2/f49 [0,4194304] 0 2024-06-15T01:06:58.680 INFO:tasks.workunit.client.1.smithi142.stdout:8/294: rmdir d0/d17/d63 0 2024-06-15T01:06:58.681 INFO:tasks.workunit.client.0.smithi137.stdout:2/194: fsync d0/de/f16 0 2024-06-15T01:06:58.681 INFO:tasks.workunit.client.1.smithi142.stdout:8/295: fdatasync d0/d25/d5a/f4d 0 2024-06-15T01:06:58.682 INFO:tasks.workunit.client.1.smithi142.stdout:7/269: write d2/d8/da/d33/d25/d27/f2f [2419559,68816] 0 2024-06-15T01:06:58.687 INFO:tasks.workunit.client.1.smithi142.stdout:9/281: read d6/dc/dd/d22/f49 [99009,46819] 0 2024-06-15T01:06:58.687 INFO:tasks.workunit.client.1.smithi142.stdout:8/296: write d0/d17/f2f [1185930,122903] 0 2024-06-15T01:06:58.688 INFO:tasks.workunit.client.1.smithi142.stdout:3/384: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:06:58.690 INFO:tasks.workunit.client.0.smithi137.stdout:2/195: dwrite d0/de/d10/f33 [0,4194304] 0 2024-06-15T01:06:58.690 INFO:tasks.workunit.client.1.smithi142.stdout:9/282: truncate d6/d31/f32 439314 0 2024-06-15T01:06:58.690 INFO:tasks.workunit.client.1.smithi142.stdout:3/385: write d0/d1f/f4b [13725,35928] 0 2024-06-15T01:06:58.691 INFO:tasks.workunit.client.0.smithi137.stdout:2/196: stat d0/da/l26 0 2024-06-15T01:06:58.691 INFO:tasks.workunit.client.1.smithi142.stdout:8/297: write d0/f15 [3790634,29734] 0 2024-06-15T01:06:58.693 INFO:tasks.workunit.client.1.smithi142.stdout:7/270: getdents d2/d6/d22/d3c 0 2024-06-15T01:06:58.694 INFO:tasks.workunit.client.0.smithi137.stdout:0/249: dwrite d1/d2/f6 [0,4194304] 0 2024-06-15T01:06:58.696 INFO:tasks.workunit.client.1.smithi142.stdout:9/283: creat d6/dc/d4e/d65/f73 x:0 0 0 2024-06-15T01:06:58.699 INFO:tasks.workunit.client.1.smithi142.stdout:3/386: mkdir d0/d1f/d20/d5d/d60/d72 0 2024-06-15T01:06:58.700 INFO:tasks.workunit.client.1.smithi142.stdout:8/298: write d0/f18 [2905411,991] 0 2024-06-15T01:06:58.703 INFO:tasks.workunit.client.1.smithi142.stdout:4/411: dwrite f3 [0,4194304] 0 2024-06-15T01:06:58.703 INFO:tasks.workunit.client.1.smithi142.stdout:0/288: dwrite d3/d5/fb [0,4194304] 0 2024-06-15T01:06:58.704 INFO:tasks.workunit.client.1.smithi142.stdout:9/284: mkdir d6/dc/d1b/d25/d58/d74 0 2024-06-15T01:06:58.704 INFO:tasks.workunit.client.0.smithi137.stdout:2/197: rename d0/dc/d22/d23/l31 to d0/de/d10/l38 0 2024-06-15T01:06:58.706 INFO:tasks.workunit.client.0.smithi137.stdout:8/190: write f6 [586202,84899] 0 2024-06-15T01:06:58.706 INFO:tasks.workunit.client.0.smithi137.stdout:0/250: dwrite d1/d2/f21 [0,4194304] 0 2024-06-15T01:06:58.711 INFO:tasks.workunit.client.1.smithi142.stdout:5/301: sync 2024-06-15T01:06:58.712 INFO:tasks.workunit.client.1.smithi142.stdout:3/387: dwrite d0/d3/f54 [0,4194304] 0 2024-06-15T01:06:58.712 INFO:tasks.workunit.client.1.smithi142.stdout:1/323: sync 2024-06-15T01:06:58.718 INFO:tasks.workunit.client.0.smithi137.stdout:2/198: getdents d0/dc/d2e 0 2024-06-15T01:06:58.721 INFO:tasks.workunit.client.0.smithi137.stdout:8/191: dwrite f1 [0,4194304] 0 2024-06-15T01:06:58.722 INFO:tasks.workunit.client.1.smithi142.stdout:8/299: rename d0/d25/d53/f60 to d0/d17/d40/f65 0 2024-06-15T01:06:58.724 INFO:tasks.workunit.client.0.smithi137.stdout:8/192: write f8 [858207,10306] 0 2024-06-15T01:06:58.730 INFO:tasks.workunit.client.1.smithi142.stdout:0/289: symlink d3/d5/d1b/l72 0 2024-06-15T01:06:58.731 INFO:tasks.workunit.client.1.smithi142.stdout:3/388: mknod d0/d1f/d20/d5d/c73 0 2024-06-15T01:06:58.732 INFO:tasks.workunit.client.1.smithi142.stdout:5/302: unlink d2/d3e/d31/c44 0 2024-06-15T01:06:58.732 INFO:tasks.workunit.client.0.smithi137.stdout:0/251: dwrite d1/d2/dc/d10/d14/d15/d1c/f1d [0,4194304] 0 2024-06-15T01:06:58.732 INFO:tasks.workunit.client.1.smithi142.stdout:1/324: rmdir d6/d12/d1c/d1e 39 2024-06-15T01:06:58.735 INFO:tasks.workunit.client.1.smithi142.stdout:4/412: mkdir d5/d9/d90 0 2024-06-15T01:06:58.736 INFO:tasks.workunit.client.1.smithi142.stdout:3/389: read d0/f3c [114247,91093] 0 2024-06-15T01:06:58.737 INFO:tasks.workunit.client.0.smithi137.stdout:8/193: dwrite f8 [0,4194304] 0 2024-06-15T01:06:58.737 INFO:tasks.workunit.client.0.smithi137.stdout:2/199: dread d0/de/f16 [0,4194304] 0 2024-06-15T01:06:58.737 INFO:tasks.workunit.client.0.smithi137.stdout:0/252: dread - d1/d2/dc/d10/d14/d15/d17/f4c zero size 2024-06-15T01:06:58.737 INFO:tasks.workunit.client.1.smithi142.stdout:8/300: mkdir d0/d25/d66 0 2024-06-15T01:06:58.738 INFO:tasks.workunit.client.1.smithi142.stdout:3/390: readlink d0/d3/l4c 0 2024-06-15T01:06:58.739 INFO:tasks.workunit.client.0.smithi137.stdout:0/253: dread - d1/d2/dc/d10/d14/d15/f30 zero size 2024-06-15T01:06:58.740 INFO:tasks.workunit.client.1.smithi142.stdout:0/290: rename d3/d5/d44/f4a to d3/d5/f73 0 2024-06-15T01:06:58.740 INFO:tasks.workunit.client.0.smithi137.stdout:2/200: write d0/de/d2b/f32 [568701,130960] 0 2024-06-15T01:06:58.741 INFO:tasks.workunit.client.0.smithi137.stdout:8/194: creat d9/de/d1e/f42 x:0 0 0 2024-06-15T01:06:58.743 INFO:tasks.workunit.client.0.smithi137.stdout:0/254: write d1/d2/dc/d10/f4f [163001,101686] 0 2024-06-15T01:06:58.744 INFO:tasks.workunit.client.0.smithi137.stdout:8/195: mknod d9/de/c43 0 2024-06-15T01:06:58.744 INFO:tasks.workunit.client.1.smithi142.stdout:4/413: dwrite d5/d9/d23/d32/f8f [0,4194304] 0 2024-06-15T01:06:58.746 INFO:tasks.workunit.client.1.smithi142.stdout:5/303: unlink d2/de/c4d 0 2024-06-15T01:06:58.748 INFO:tasks.workunit.client.0.smithi137.stdout:0/255: write d1/d2/f47 [1135326,47716] 0 2024-06-15T01:06:58.749 INFO:tasks.workunit.client.1.smithi142.stdout:7/271: truncate d2/d8/f2d 205507 0 2024-06-15T01:06:58.750 INFO:tasks.workunit.client.0.smithi137.stdout:8/196: creat d9/de/d1e/f44 x:0 0 0 2024-06-15T01:06:58.751 INFO:tasks.workunit.client.1.smithi142.stdout:0/291: readlink d3/l56 0 2024-06-15T01:06:58.753 INFO:tasks.workunit.client.0.smithi137.stdout:2/201: dwrite d0/de/f24 [0,4194304] 0 2024-06-15T01:06:58.753 INFO:tasks.workunit.client.0.smithi137.stdout:8/197: chown d9/d22/f33 7325 1 2024-06-15T01:06:58.756 INFO:tasks.workunit.client.1.smithi142.stdout:5/304: unlink d2/c11 0 2024-06-15T01:06:58.758 INFO:tasks.workunit.client.1.smithi142.stdout:4/414: creat d5/d1c/d2a/d30/d46/d50/f91 x:0 0 0 2024-06-15T01:06:58.760 INFO:tasks.workunit.client.1.smithi142.stdout:0/292: dwrite d3/d39/f6f [0,4194304] 0 2024-06-15T01:06:58.763 INFO:tasks.workunit.client.0.smithi137.stdout:0/256: rename d1/d2/dc/f3b to d1/d2/d34/f60 0 2024-06-15T01:06:58.763 INFO:tasks.workunit.client.0.smithi137.stdout:8/198: unlink f0 0 2024-06-15T01:06:58.764 INFO:tasks.workunit.client.1.smithi142.stdout:8/301: rename d0/c3d to d0/d25/d5a/c67 0 2024-06-15T01:06:58.767 INFO:tasks.workunit.client.0.smithi137.stdout:0/257: rmdir d1/d2/dc/d10/d14/d15/d17/d54 39 2024-06-15T01:06:58.770 INFO:tasks.workunit.client.1.smithi142.stdout:4/415: dwrite d5/d1c/d2a/d30/d46/d50/f91 [0,4194304] 0 2024-06-15T01:06:58.772 INFO:tasks.workunit.client.1.smithi142.stdout:4/416: truncate d5/d9/da/f7c 1626569 0 2024-06-15T01:06:58.772 INFO:tasks.workunit.client.0.smithi137.stdout:0/258: mknod d1/d2/dc/d10/d14/d56/c61 0 2024-06-15T01:06:58.775 INFO:tasks.workunit.client.0.smithi137.stdout:0/259: chown d1/d2/d13/f33 14294517 1 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 -- 172.21.15.137:0/532240131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84100bb0 msgr2=0x7f7d84100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 --2- 172.21.15.137:0/532240131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84100bb0 0x7f7d84100fd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009910 tx=0x7f7d74009c20 comp rx=0 tx=0).stop 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 -- 172.21.15.137:0/532240131 shutdown_connections 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 --2- 172.21.15.137:0/532240131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84101db0 0x7f7d84102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 --2- 172.21.15.137:0/532240131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84100bb0 0x7f7d84100fd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 -- 172.21.15.137:0/532240131 >> 172.21.15.137:0/532240131 conn(0x7f7d840fc110 msgr2=0x7f7d840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 -- 172.21.15.137:0/532240131 shutdown_connections 2024-06-15T01:06:58.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.780+0000 7f7d89220700 1 -- 172.21.15.137:0/532240131 wait complete. 2024-06-15T01:06:58.777 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:58 smithi137 ceph-mon[29354]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:58.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:58 smithi137 ceph-mon[29354]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:58.778 INFO:tasks.workunit.client.0.smithi137.stdout:0/260: fsync d1/f32 0 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 Processor -- start 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 -- start start 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 0x7f7d8419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8419d350 con 0x7f7d84101db0 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.781+0000 7f7d89220700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8419d490 con 0x7f7d84100bb0 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41976/0 (socket says 172.21.15.137:41976) 2024-06-15T01:06:58.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 -- 172.21.15.137:0/1866021125 learned_addr learned my addr 172.21.15.137:0/1866021125 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d82d9d700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 0x7f7d8419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 -- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 msgr2=0x7f7d8419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 0x7f7d8419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d8259c700 1 -- 172.21.15.137:0/1866021125 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740095f0 con 0x7f7d84101db0 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.782+0000 7f7d82d9d700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 0x7f7d8419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:06:58.780 INFO:tasks.workunit.client.0.smithi137.stdout:0/261: readlink d1/d2/le 0 2024-06-15T01:06:58.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.783+0000 7f7d8259c700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c930 tx=0x7f7d6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:58.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.783+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6c007ad0 con 0x7f7d84101db0 2024-06-15T01:06:58.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.783+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d841a1f50 con 0x7f7d84101db0 2024-06-15T01:06:58.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.783+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d841a24f0 con 0x7f7d84101db0 2024-06-15T01:06:58.781 INFO:tasks.workunit.client.1.smithi142.stdout:4/417: dread d5/d9/d18/f80 [0,4194304] 0 2024-06-15T01:06:58.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.784+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7d6c00ce80 con 0x7f7d84101db0 2024-06-15T01:06:58.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.784+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6c018630 con 0x7f7d84101db0 2024-06-15T01:06:58.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.785+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d64005320 con 0x7f7d84101db0 2024-06-15T01:06:58.782 INFO:tasks.workunit.client.0.smithi137.stdout:0/262: mknod d1/d2/dc/d10/d53/c62 0 2024-06-15T01:06:58.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.786+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f7d6c018790 con 0x7f7d84101db0 2024-06-15T01:06:58.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.786+0000 7f7d7bfff700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 0x7f7d7006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:06:58.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.786+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7d6c08ae70 con 0x7f7d84101db0 2024-06-15T01:06:58.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.786+0000 7f7d82d9d700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 0x7f7d7006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:06:58.783 INFO:tasks.workunit.client.1.smithi142.stdout:4/418: dread d5/d1c/d2a/d30/d46/d50/f91 [0,4194304] 0 2024-06-15T01:06:58.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.787+0000 7f7d82d9d700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 0x7f7d7006e020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006010 tx=0x7f7d7400b560 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:06:58.784 INFO:tasks.workunit.client.0.smithi137.stdout:0/263: mkdir d1/d2/dc/d10/d63 0 2024-06-15T01:06:58.785 INFO:tasks.workunit.client.0.smithi137.stdout:0/264: readlink d1/d2/dc/d10/d14/d15/l5c 0 2024-06-15T01:06:58.787 INFO:tasks.workunit.client.0.smithi137.stdout:0/265: symlink d1/d2/dc/l64 0 2024-06-15T01:06:58.788 INFO:tasks.workunit.client.0.smithi137.stdout:0/266: readlink d1/d2/dc/d10/d14/d15/l5c 0 2024-06-15T01:06:58.789 INFO:tasks.workunit.client.0.smithi137.stdout:0/267: chown d1/d2/dc/d10/d14/f39 0 1 2024-06-15T01:06:58.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:58.793+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f7d6c0598a0 con 0x7f7d84101db0 2024-06-15T01:06:58.792 INFO:tasks.workunit.client.0.smithi137.stdout:0/268: symlink d1/d2/dc/d10/d14/d15/d17/l65 0 2024-06-15T01:06:58.797 INFO:tasks.workunit.client.0.smithi137.stdout:8/199: rmdir d9/de/d1e 39 2024-06-15T01:06:58.799 INFO:tasks.workunit.client.0.smithi137.stdout:2/202: write d0/da/f17 [1813916,14445] 0 2024-06-15T01:06:58.802 INFO:tasks.workunit.client.0.smithi137.stdout:2/203: creat d0/dc/d22/f39 x:0 0 0 2024-06-15T01:06:58.804 INFO:tasks.workunit.client.0.smithi137.stdout:8/200: rename d9/d22/c30 to d9/c45 0 2024-06-15T01:06:58.804 INFO:tasks.workunit.client.0.smithi137.stdout:8/201: dread - d9/de/f3c zero size 2024-06-15T01:06:58.807 INFO:tasks.workunit.client.0.smithi137.stdout:8/202: truncate d9/d22/f25 1181382 0 2024-06-15T01:06:58.808 INFO:tasks.workunit.client.0.smithi137.stdout:8/203: chown d9/cb 5086827 1 2024-06-15T01:06:58.808 INFO:tasks.workunit.client.0.smithi137.stdout:0/269: write d1/d2/dc/d10/d14/d15/f57 [74189,26913] 0 2024-06-15T01:06:58.809 INFO:tasks.workunit.client.0.smithi137.stdout:2/204: dread d0/f1f [0,4194304] 0 2024-06-15T01:06:58.810 INFO:tasks.workunit.client.0.smithi137.stdout:0/270: readlink d1/d2/dc/d10/d14/d15/d17/l65 0 2024-06-15T01:06:58.810 INFO:tasks.workunit.client.0.smithi137.stdout:8/204: symlink d9/de/l46 0 2024-06-15T01:06:58.814 INFO:tasks.workunit.client.0.smithi137.stdout:2/205: rmdir d0/dc 39 2024-06-15T01:06:58.815 INFO:tasks.workunit.client.0.smithi137.stdout:2/206: fsync d0/da/f11 0 2024-06-15T01:06:58.816 INFO:tasks.workunit.client.0.smithi137.stdout:0/271: dwrite d1/d2/d34/f4a [0,4194304] 0 2024-06-15T01:06:58.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:58 smithi142 ceph-mon[37379]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:58.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:58 smithi142 ceph-mon[37379]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:06:58.818 INFO:tasks.workunit.client.0.smithi137.stdout:2/207: mkdir d0/de/d10/d3a 0 2024-06-15T01:06:58.828 INFO:tasks.workunit.client.1.smithi142.stdout:1/325: rmdir d6/d46/d6d 0 2024-06-15T01:06:58.829 INFO:tasks.workunit.client.0.smithi137.stdout:2/208: truncate d0/da/f11 185849 0 2024-06-15T01:06:58.829 INFO:tasks.workunit.client.1.smithi142.stdout:1/326: fdatasync d6/d12/d28/f39 0 2024-06-15T01:06:58.830 INFO:tasks.workunit.client.0.smithi137.stdout:0/272: write d1/d2/d34/f3f [2018495,82620] 0 2024-06-15T01:06:58.830 INFO:tasks.workunit.client.1.smithi142.stdout:7/272: creat d2/d8/da/d33/d25/d27/f5d x:0 0 0 2024-06-15T01:06:58.832 INFO:tasks.workunit.client.0.smithi137.stdout:2/209: unlink d0/da/f17 0 2024-06-15T01:06:58.832 INFO:tasks.workunit.client.0.smithi137.stdout:3/328: sync 2024-06-15T01:06:58.834 INFO:tasks.workunit.client.0.smithi137.stdout:3/329: read d0/d13/f26 [775990,90704] 0 2024-06-15T01:06:58.834 INFO:tasks.workunit.client.0.smithi137.stdout:0/273: unlink d1/d2/dc/d10/d14/d15/f19 0 2024-06-15T01:06:58.836 INFO:tasks.workunit.client.0.smithi137.stdout:0/274: dread - d1/d2/dc/d10/d14/d15/f30 zero size 2024-06-15T01:06:58.838 INFO:tasks.workunit.client.0.smithi137.stdout:0/275: truncate d1/d2/dc/d10/d14/d52/f58 173743 0 2024-06-15T01:06:58.839 INFO:tasks.workunit.client.0.smithi137.stdout:0/276: chown d1/d2/f5e 15520720 1 2024-06-15T01:06:58.840 INFO:tasks.workunit.client.1.smithi142.stdout:4/419: mkdir d5/d9/da/d92 0 2024-06-15T01:06:58.841 INFO:tasks.workunit.client.1.smithi142.stdout:1/327: creat d6/d25/f77 x:0 0 0 2024-06-15T01:06:58.842 INFO:tasks.workunit.client.1.smithi142.stdout:7/273: rmdir d2/d8/dd 39 2024-06-15T01:06:58.844 INFO:tasks.workunit.client.1.smithi142.stdout:7/274: truncate d2/d8/da/d33/d25/d27/f5d 748441 0 2024-06-15T01:06:58.845 INFO:tasks.workunit.client.1.smithi142.stdout:7/275: chown d2/d6/d22/d3c 7280 1 2024-06-15T01:06:58.845 INFO:tasks.workunit.client.1.smithi142.stdout:3/391: getdents d0/d1b 0 2024-06-15T01:06:58.846 INFO:tasks.workunit.client.1.smithi142.stdout:7/276: dread - d2/d6/d22/d47/f5a zero size 2024-06-15T01:06:58.849 INFO:tasks.workunit.client.1.smithi142.stdout:6/287: sync 2024-06-15T01:06:58.849 INFO:tasks.workunit.client.1.smithi142.stdout:2/270: sync 2024-06-15T01:06:58.849 INFO:tasks.workunit.client.1.smithi142.stdout:9/285: sync 2024-06-15T01:06:58.849 INFO:tasks.workunit.client.1.smithi142.stdout:9/286: fdatasync d6/f45 0 2024-06-15T01:06:58.850 INFO:tasks.workunit.client.1.smithi142.stdout:4/420: symlink d5/d9/da/l93 0 2024-06-15T01:06:58.852 INFO:tasks.workunit.client.1.smithi142.stdout:1/328: creat d6/d46/f78 x:0 0 0 2024-06-15T01:06:58.852 INFO:tasks.workunit.client.1.smithi142.stdout:1/329: fsync d6/fc 0 2024-06-15T01:06:58.854 INFO:tasks.workunit.client.1.smithi142.stdout:0/293: link d3/d5/d1e/c3b d3/d5/d1b/c74 0 2024-06-15T01:06:58.854 INFO:tasks.workunit.client.1.smithi142.stdout:0/294: chown d3/d12 5 1 2024-06-15T01:06:58.858 INFO:tasks.workunit.client.0.smithi137.stdout:2/210: dread d0/de/f1a [0,4194304] 0 2024-06-15T01:06:58.858 INFO:tasks.workunit.client.1.smithi142.stdout:7/277: rename d2/l4 to d2/d6/d22/d47/l5e 0 2024-06-15T01:06:58.860 INFO:tasks.workunit.client.1.smithi142.stdout:3/392: dwrite d0/d3/f3d [0,4194304] 0 2024-06-15T01:06:58.863 INFO:tasks.workunit.client.0.smithi137.stdout:2/211: dread d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:06:58.864 INFO:tasks.workunit.client.1.smithi142.stdout:2/271: mknod d8/df/d2a/c4d 0 2024-06-15T01:06:58.866 INFO:tasks.workunit.client.1.smithi142.stdout:9/287: unlink d6/f55 0 2024-06-15T01:06:58.866 INFO:tasks.workunit.client.1.smithi142.stdout:2/272: truncate d8/df/d2a/f2b 673369 0 2024-06-15T01:06:58.867 INFO:tasks.workunit.client.1.smithi142.stdout:9/288: write d6/d31/f3e [4059429,16660] 0 2024-06-15T01:06:58.868 INFO:tasks.workunit.client.1.smithi142.stdout:5/305: getdents d2/d3e/d31 0 2024-06-15T01:06:58.868 INFO:tasks.workunit.client.0.smithi137.stdout:2/212: truncate d0/f1f 347126 0 2024-06-15T01:06:58.869 INFO:tasks.workunit.client.0.smithi137.stdout:3/330: truncate d0/f1f 354708 0 2024-06-15T01:06:58.871 INFO:tasks.workunit.client.1.smithi142.stdout:0/295: mkdir d3/d5/d75 0 2024-06-15T01:06:58.872 INFO:tasks.workunit.client.0.smithi137.stdout:3/331: rename d0/d2/d23/c59 to d0/d56/c70 0 2024-06-15T01:06:58.874 INFO:tasks.workunit.client.0.smithi137.stdout:2/213: creat d0/dc/f3b x:0 0 0 2024-06-15T01:06:58.875 INFO:tasks.workunit.client.1.smithi142.stdout:9/289: dwrite d6/dc/d1b/d25/d35/f5c [0,4194304] 0 2024-06-15T01:06:58.876 INFO:tasks.workunit.client.0.smithi137.stdout:2/214: creat d0/de/d10/f3c x:0 0 0 2024-06-15T01:06:58.877 INFO:tasks.workunit.client.1.smithi142.stdout:8/302: symlink d0/d25/l68 0 2024-06-15T01:06:58.878 INFO:tasks.workunit.client.1.smithi142.stdout:8/303: chown d0 1649 1 2024-06-15T01:06:58.878 INFO:tasks.workunit.client.0.smithi137.stdout:2/215: getdents d0/de/d10/d3a 0 2024-06-15T01:06:58.882 INFO:tasks.workunit.client.1.smithi142.stdout:4/421: symlink d5/d9/da/d92/l94 0 2024-06-15T01:06:58.883 INFO:tasks.workunit.client.1.smithi142.stdout:5/306: mkdir d2/de/d19/d5f 0 2024-06-15T01:06:58.884 INFO:tasks.workunit.client.1.smithi142.stdout:4/422: read d5/d9/d23/f4e [858553,90848] 0 2024-06-15T01:06:58.887 INFO:tasks.workunit.client.1.smithi142.stdout:9/290: mknod d6/dc/d1b/d25/c75 0 2024-06-15T01:06:58.889 INFO:tasks.workunit.client.1.smithi142.stdout:3/393: rename d0/d1f/d20/c5a to d0/d3/da/d18/c74 0 2024-06-15T01:06:58.891 INFO:tasks.workunit.client.1.smithi142.stdout:0/296: dwrite d3/d39/d3e/f5f [0,4194304] 0 2024-06-15T01:06:58.892 INFO:tasks.workunit.client.1.smithi142.stdout:2/273: symlink d8/d19/d23/d3a/l4e 0 2024-06-15T01:06:58.893 INFO:tasks.workunit.client.0.smithi137.stdout:2/216: dread d0/f5 [0,4194304] 0 2024-06-15T01:06:58.894 INFO:tasks.workunit.client.1.smithi142.stdout:5/307: rmdir d2/d3e/d31 39 2024-06-15T01:06:58.896 INFO:tasks.workunit.client.1.smithi142.stdout:4/423: rmdir d5/d9/d1a 39 2024-06-15T01:06:58.897 INFO:tasks.workunit.client.1.smithi142.stdout:7/278: mknod d2/d6/c5f 0 2024-06-15T01:06:58.898 INFO:tasks.workunit.client.0.smithi137.stdout:3/332: dread d0/fd [0,4194304] 0 2024-06-15T01:06:58.899 INFO:tasks.workunit.client.1.smithi142.stdout:3/394: dwrite d0/d3/f1a [4194304,4194304] 0 2024-06-15T01:06:58.904 INFO:tasks.workunit.client.0.smithi137.stdout:3/333: rename d0/d2/d1e/l30 to d0/d2/d8/d67/l71 0 2024-06-15T01:06:58.905 INFO:tasks.workunit.client.1.smithi142.stdout:6/288: creat d1a/d2d/f65 x:0 0 0 2024-06-15T01:06:58.906 INFO:tasks.workunit.client.1.smithi142.stdout:8/304: dread d0/f14 [0,4194304] 0 2024-06-15T01:06:58.906 INFO:tasks.workunit.client.1.smithi142.stdout:9/291: rename d6/d34/f44 to d6/dc/d1b/f76 0 2024-06-15T01:06:58.907 INFO:tasks.workunit.client.0.smithi137.stdout:3/334: symlink d0/d13/l72 0 2024-06-15T01:06:58.908 INFO:tasks.workunit.client.1.smithi142.stdout:2/274: mknod d8/d19/d23/d25/d22/d43/c4f 0 2024-06-15T01:06:58.910 INFO:tasks.workunit.client.0.smithi137.stdout:3/335: unlink d0/d13/f26 0 2024-06-15T01:06:58.910 INFO:tasks.workunit.client.1.smithi142.stdout:5/308: unlink d2/d18/f45 0 2024-06-15T01:06:58.912 INFO:tasks.workunit.client.0.smithi137.stdout:3/336: write d0/d2/d1e/f5c [3504631,5312] 0 2024-06-15T01:06:58.913 INFO:tasks.workunit.client.1.smithi142.stdout:0/297: write d3/d39/d3e/f5f [4712130,57192] 0 2024-06-15T01:06:58.913 INFO:tasks.workunit.client.1.smithi142.stdout:0/298: chown d3/d5 342526 1 2024-06-15T01:06:58.914 INFO:tasks.workunit.client.0.smithi137.stdout:3/337: truncate d0/d2/d22/d24/f54 49454 0 2024-06-15T01:06:58.915 INFO:tasks.workunit.client.1.smithi142.stdout:3/395: mkdir d0/d3/da/d71/d75 0 2024-06-15T01:06:58.916 INFO:tasks.workunit.client.0.smithi137.stdout:3/338: creat d0/d2/d23/f73 x:0 0 0 2024-06-15T01:06:58.920 INFO:tasks.workunit.client.0.smithi137.stdout:3/339: mknod d0/d2/d4e/c74 0 2024-06-15T01:06:58.926 INFO:tasks.workunit.client.1.smithi142.stdout:7/279: stat d2/d8/dd/f10 0 2024-06-15T01:06:58.927 INFO:tasks.workunit.client.0.smithi137.stdout:3/340: dwrite d0/d2/d22/d24/f47 [4194304,4194304] 0 2024-06-15T01:06:58.927 INFO:tasks.workunit.client.1.smithi142.stdout:7/280: readlink d2/d8/da/d33/d25/d27/l53 0 2024-06-15T01:06:58.928 INFO:tasks.workunit.client.1.smithi142.stdout:4/424: truncate f2 19734 0 2024-06-15T01:06:58.929 INFO:tasks.workunit.client.1.smithi142.stdout:3/396: creat d0/d1b/f76 x:0 0 0 2024-06-15T01:06:58.931 INFO:tasks.workunit.client.0.smithi137.stdout:3/341: mknod d0/d2/d22/d24/c75 0 2024-06-15T01:06:58.931 INFO:tasks.workunit.client.1.smithi142.stdout:3/397: stat d0/d3/da/d71/d55/f57 0 2024-06-15T01:06:58.932 INFO:tasks.workunit.client.1.smithi142.stdout:8/305: chown d0/d25/d5a/f37 42 1 2024-06-15T01:06:58.932 INFO:tasks.workunit.client.1.smithi142.stdout:7/281: dread d2/d8/da/d33/f20 [0,4194304] 0 2024-06-15T01:06:58.933 INFO:tasks.workunit.client.1.smithi142.stdout:2/275: symlink d8/d19/d23/l50 0 2024-06-15T01:06:58.939 INFO:tasks.workunit.client.1.smithi142.stdout:6/289: truncate f7 2464458 0 2024-06-15T01:06:58.940 INFO:tasks.workunit.client.0.smithi137.stdout:3/342: dwrite d0/d13/d37/f43 [0,4194304] 0 2024-06-15T01:06:58.946 INFO:tasks.workunit.client.1.smithi142.stdout:3/398: chown d0/d3/da/c10 9 1 2024-06-15T01:06:58.947 INFO:tasks.workunit.client.1.smithi142.stdout:3/399: write d0/f6a [92558,50520] 0 2024-06-15T01:06:58.948 INFO:tasks.workunit.client.1.smithi142.stdout:7/282: read d2/f49 [3264861,353] 0 2024-06-15T01:06:58.949 INFO:tasks.workunit.client.1.smithi142.stdout:5/309: truncate d2/f21 828318 0 2024-06-15T01:06:58.952 INFO:tasks.workunit.client.1.smithi142.stdout:2/276: fdatasync d8/fe 0 2024-06-15T01:06:58.953 INFO:tasks.workunit.client.1.smithi142.stdout:2/277: chown d8/f40 4517 1 2024-06-15T01:06:58.961 INFO:tasks.workunit.client.1.smithi142.stdout:3/400: creat d0/d3/d15/f77 x:0 0 0 2024-06-15T01:06:58.963 INFO:tasks.workunit.client.1.smithi142.stdout:5/310: symlink d2/de/l60 0 2024-06-15T01:06:58.968 INFO:tasks.workunit.client.1.smithi142.stdout:2/278: mkdir d8/d19/d23/d25/d22/d43/d51 0 2024-06-15T01:06:58.968 INFO:tasks.workunit.client.1.smithi142.stdout:7/283: dwrite d2/f2e [0,4194304] 0 2024-06-15T01:06:58.969 INFO:tasks.workunit.client.1.smithi142.stdout:2/279: chown d8/f40 121 1 2024-06-15T01:06:58.969 INFO:tasks.workunit.client.1.smithi142.stdout:0/299: link d3/d5/d1b/l4e d3/d39/d3e/l76 0 2024-06-15T01:06:58.970 INFO:tasks.workunit.client.1.smithi142.stdout:2/280: fsync d8/df/f48 0 2024-06-15T01:06:58.972 INFO:tasks.workunit.client.1.smithi142.stdout:8/306: dread d0/d25/d5a/f2c [0,4194304] 0 2024-06-15T01:06:58.973 INFO:tasks.workunit.client.1.smithi142.stdout:4/425: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/f95 x:0 0 0 2024-06-15T01:06:58.976 INFO:tasks.workunit.client.1.smithi142.stdout:3/401: mknod d0/d3/d15/c78 0 2024-06-15T01:06:58.979 INFO:tasks.workunit.client.1.smithi142.stdout:5/311: rename d2/ff to d2/d18/f61 0 2024-06-15T01:06:58.979 INFO:tasks.workunit.client.1.smithi142.stdout:3/402: dread d0/d3/f54 [0,4194304] 0 2024-06-15T01:06:58.983 INFO:tasks.workunit.client.1.smithi142.stdout:2/281: mkdir d8/d19/d23/d25/d22/d43/d52 0 2024-06-15T01:06:58.985 INFO:tasks.workunit.client.1.smithi142.stdout:4/426: symlink d5/d9/d23/l96 0 2024-06-15T01:06:58.986 INFO:tasks.workunit.client.1.smithi142.stdout:7/284: dread d2/f49 [0,4194304] 0 2024-06-15T01:06:58.988 INFO:tasks.workunit.client.1.smithi142.stdout:6/290: getdents d1a/d39/d42/d4c 0 2024-06-15T01:06:58.990 INFO:tasks.workunit.client.1.smithi142.stdout:6/291: truncate d1a/d39/d3a/f59 1004874 0 2024-06-15T01:06:58.991 INFO:tasks.workunit.client.1.smithi142.stdout:7/285: dread d2/d8/da/d33/f35 [0,4194304] 0 2024-06-15T01:06:58.992 INFO:tasks.workunit.client.1.smithi142.stdout:8/307: write d0/d17/f56 [378235,12628] 0 2024-06-15T01:06:58.992 INFO:tasks.workunit.client.1.smithi142.stdout:7/286: chown d2/d6 62188739 1 2024-06-15T01:06:58.997 INFO:tasks.workunit.client.1.smithi142.stdout:6/292: dwrite d1a/f61 [0,4194304] 0 2024-06-15T01:06:58.999 INFO:tasks.workunit.client.1.smithi142.stdout:5/312: rename d2/d3e/l52 to d2/d18/l62 0 2024-06-15T01:06:59.001 INFO:tasks.workunit.client.1.smithi142.stdout:8/308: dwrite d0/d17/f2f [0,4194304] 0 2024-06-15T01:06:59.003 INFO:tasks.workunit.client.1.smithi142.stdout:7/287: symlink d2/d8/dd/l60 0 2024-06-15T01:06:59.005 INFO:tasks.workunit.client.1.smithi142.stdout:0/300: truncate d3/d5/d1b/f30 2770639 0 2024-06-15T01:06:59.006 INFO:tasks.workunit.client.1.smithi142.stdout:0/301: stat d3/d5/d1e/f63 0 2024-06-15T01:06:59.008 INFO:tasks.workunit.client.1.smithi142.stdout:6/293: symlink d1a/d39/d42/d4c/d58/l66 0 2024-06-15T01:06:59.009 INFO:tasks.workunit.client.1.smithi142.stdout:3/403: rename d0/d1f/d3e/d6e/l70 to d0/d1f/d20/d5d/l79 0 2024-06-15T01:06:59.011 INFO:tasks.workunit.client.1.smithi142.stdout:5/313: creat d2/d18/f63 x:0 0 0 2024-06-15T01:06:59.011 INFO:tasks.workunit.client.1.smithi142.stdout:3/404: chown d0/d1b/c23 145 1 2024-06-15T01:06:59.013 INFO:tasks.workunit.client.1.smithi142.stdout:2/282: truncate d8/fc 3986984 0 2024-06-15T01:06:59.014 INFO:tasks.workunit.client.1.smithi142.stdout:8/309: write d0/f33 [1733538,65170] 0 2024-06-15T01:06:59.017 INFO:tasks.workunit.client.1.smithi142.stdout:0/302: rmdir d3/d5/d75 0 2024-06-15T01:06:59.020 INFO:tasks.workunit.client.1.smithi142.stdout:4/427: dwrite d5/d1c/d2a/d30/d46/f76 [0,4194304] 0 2024-06-15T01:06:59.022 INFO:tasks.workunit.client.1.smithi142.stdout:1/330: sync 2024-06-15T01:06:59.023 INFO:tasks.workunit.client.1.smithi142.stdout:0/303: chown d3/d5/f34 362710 1 2024-06-15T01:06:59.023 INFO:tasks.workunit.client.1.smithi142.stdout:6/294: dwrite f0 [4194304,4194304] 0 2024-06-15T01:06:59.023 INFO:tasks.workunit.client.1.smithi142.stdout:8/310: truncate d0/d17/f2f 4530013 0 2024-06-15T01:06:59.025 INFO:tasks.workunit.client.1.smithi142.stdout:1/331: fsync d6/d3a/f70 0 2024-06-15T01:06:59.028 INFO:tasks.workunit.client.1.smithi142.stdout:4/428: fsync d5/d9/f22 0 2024-06-15T01:06:59.032 INFO:tasks.workunit.client.1.smithi142.stdout:0/304: unlink d3/d12/c62 0 2024-06-15T01:06:59.032 INFO:tasks.workunit.client.1.smithi142.stdout:4/429: chown d5/d1c/d2a/d30/d46/d50/d51 130932 1 2024-06-15T01:06:59.032 INFO:tasks.workunit.client.1.smithi142.stdout:2/283: creat d8/d19/d23/d25/f53 x:0 0 0 2024-06-15T01:06:59.032 INFO:tasks.workunit.client.1.smithi142.stdout:8/311: readlink d0/d25/d5a/l31 0 2024-06-15T01:06:59.033 INFO:tasks.workunit.client.1.smithi142.stdout:3/405: dread d0/d3/da/d71/d55/f57 [0,4194304] 0 2024-06-15T01:06:59.036 INFO:tasks.workunit.client.1.smithi142.stdout:4/430: chown d5/d9/d41/c86 203 1 2024-06-15T01:06:59.036 INFO:tasks.workunit.client.0.smithi137.stdout:9/133: sync 2024-06-15T01:06:59.037 INFO:tasks.workunit.client.0.smithi137.stdout:4/247: sync 2024-06-15T01:06:59.039 INFO:tasks.workunit.client.1.smithi142.stdout:8/312: unlink d0/d17/d40/l4b 0 2024-06-15T01:06:59.039 INFO:tasks.workunit.client.1.smithi142.stdout:6/295: dwrite d1a/f28 [4194304,4194304] 0 2024-06-15T01:06:59.040 INFO:tasks.workunit.client.0.smithi137.stdout:4/248: symlink dd/d15/d36/d45/d55/l5c 0 2024-06-15T01:06:59.041 INFO:tasks.workunit.client.1.smithi142.stdout:3/406: creat d0/d3/da/d71/d55/f7a x:0 0 0 2024-06-15T01:06:59.043 INFO:tasks.workunit.client.1.smithi142.stdout:5/314: dread d2/de/d27/f46 [0,4194304] 0 2024-06-15T01:06:59.044 INFO:tasks.workunit.client.1.smithi142.stdout:1/332: rename d6/d46/c58 to d6/d12/c79 0 2024-06-15T01:06:59.044 INFO:tasks.workunit.client.1.smithi142.stdout:2/284: mknod d8/df/c54 0 2024-06-15T01:06:59.045 INFO:tasks.workunit.client.1.smithi142.stdout:6/296: dread d1a/d39/d3a/d51/f5c [0,4194304] 0 2024-06-15T01:06:59.045 INFO:tasks.workunit.client.0.smithi137.stdout:4/249: symlink dd/d15/d21/d3a/d51/d5b/l5d 0 2024-06-15T01:06:59.045 INFO:tasks.workunit.client.0.smithi137.stdout:9/134: dwrite fd [8388608,4194304] 0 2024-06-15T01:06:59.046 INFO:tasks.workunit.client.1.smithi142.stdout:4/431: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/c97 0 2024-06-15T01:06:59.046 INFO:tasks.workunit.client.0.smithi137.stdout:9/135: chown df/d12 5195 1 2024-06-15T01:06:59.048 INFO:tasks.workunit.client.0.smithi137.stdout:9/136: truncate df/d12/d15/f1d 1537206 0 2024-06-15T01:06:59.050 INFO:tasks.workunit.client.0.smithi137.stdout:4/250: creat dd/d10/f5e x:0 0 0 2024-06-15T01:06:59.052 INFO:tasks.workunit.client.0.smithi137.stdout:9/137: creat df/f24 x:0 0 0 2024-06-15T01:06:59.053 INFO:tasks.workunit.client.1.smithi142.stdout:0/305: getdents d3/d12/d22/d59 0 2024-06-15T01:06:59.057 INFO:tasks.workunit.client.1.smithi142.stdout:6/297: symlink d1a/d2d/d3b/l67 0 2024-06-15T01:06:59.058 INFO:tasks.workunit.client.1.smithi142.stdout:6/298: write d1a/d2d/d3b/f41 [1624400,104109] 0 2024-06-15T01:06:59.059 INFO:tasks.workunit.client.1.smithi142.stdout:6/299: fdatasync d1a/d39/d3a/f5f 0 2024-06-15T01:06:59.060 INFO:tasks.workunit.client.1.smithi142.stdout:5/315: creat d2/d3e/d31/f64 x:0 0 0 2024-06-15T01:06:59.060 INFO:tasks.workunit.client.1.smithi142.stdout:5/316: fdatasync d2/de/f1f 0 2024-06-15T01:06:59.062 INFO:tasks.workunit.client.1.smithi142.stdout:0/306: rename d3/d5/d4c/d61/f6a to d3/d5/d4c/d61/f77 0 2024-06-15T01:06:59.063 INFO:tasks.workunit.client.1.smithi142.stdout:8/313: creat d0/d25/d5a/f69 x:0 0 0 2024-06-15T01:06:59.065 INFO:tasks.workunit.client.1.smithi142.stdout:3/407: creat d0/d3/da/f7b x:0 0 0 2024-06-15T01:06:59.070 INFO:tasks.workunit.client.0.smithi137.stdout:4/251: write dd/f17 [3357779,112276] 0 2024-06-15T01:06:59.073 INFO:tasks.workunit.client.1.smithi142.stdout:0/307: mknod d3/d5/d1e/c78 0 2024-06-15T01:06:59.073 INFO:tasks.workunit.client.1.smithi142.stdout:6/300: mkdir d1a/d2d/d68 0 2024-06-15T01:06:59.075 INFO:tasks.workunit.client.1.smithi142.stdout:1/333: getdents d6/d12/d1c 0 2024-06-15T01:06:59.075 INFO:tasks.workunit.client.1.smithi142.stdout:4/432: dread d5/d9/da/f7f [0,4194304] 0 2024-06-15T01:06:59.075 INFO:tasks.workunit.client.1.smithi142.stdout:5/317: mkdir d2/d3e/d65 0 2024-06-15T01:06:59.076 INFO:tasks.workunit.client.1.smithi142.stdout:8/314: getdents d0/d17/d40/d5e 0 2024-06-15T01:06:59.077 INFO:tasks.workunit.client.1.smithi142.stdout:0/308: mknod d3/d12/d22/c79 0 2024-06-15T01:06:59.078 INFO:tasks.workunit.client.1.smithi142.stdout:0/309: chown d3/d39/d3e 2 1 2024-06-15T01:06:59.078 INFO:tasks.workunit.client.0.smithi137.stdout:9/138: dwrite df/f18 [0,4194304] 0 2024-06-15T01:06:59.079 INFO:tasks.workunit.client.1.smithi142.stdout:6/301: dread d1a/d2d/f52 [0,4194304] 0 2024-06-15T01:06:59.079 INFO:tasks.workunit.client.1.smithi142.stdout:1/334: mknod d6/d12/d1c/c7a 0 2024-06-15T01:06:59.080 INFO:tasks.workunit.client.0.smithi137.stdout:4/252: dwrite f8 [4194304,4194304] 0 2024-06-15T01:06:59.082 INFO:tasks.workunit.client.1.smithi142.stdout:5/318: creat d2/de/f66 x:0 0 0 2024-06-15T01:06:59.082 INFO:tasks.workunit.client.1.smithi142.stdout:5/319: chown d2/d18/c39 15078 1 2024-06-15T01:06:59.083 INFO:tasks.workunit.client.1.smithi142.stdout:5/320: chown d2/d3e/d31 1162605 1 2024-06-15T01:06:59.083 INFO:tasks.workunit.client.1.smithi142.stdout:4/433: mkdir d5/d9/da/d92/d98 0 2024-06-15T01:06:59.083 INFO:tasks.workunit.client.1.smithi142.stdout:5/321: chown d2/d3e/f5c 12102300 1 2024-06-15T01:06:59.085 INFO:tasks.workunit.client.1.smithi142.stdout:5/322: write d2/de/d27/f38 [7077738,78547] 0 2024-06-15T01:06:59.085 INFO:tasks.workunit.client.0.smithi137.stdout:4/253: mknod dd/d15/d36/d45/c5f 0 2024-06-15T01:06:59.086 INFO:tasks.workunit.client.1.smithi142.stdout:5/323: write d2/d18/f55 [844092,101678] 0 2024-06-15T01:06:59.086 INFO:tasks.workunit.client.1.smithi142.stdout:5/324: stat d2/f12 0 2024-06-15T01:06:59.086 INFO:tasks.workunit.client.1.smithi142.stdout:0/310: readlink d3/l3a 0 2024-06-15T01:06:59.089 INFO:tasks.workunit.client.0.smithi137.stdout:9/139: rename c9 to df/d12/d15/c25 0 2024-06-15T01:06:59.090 INFO:tasks.workunit.client.0.smithi137.stdout:4/254: symlink dd/d15/d21/d3a/d51/d5b/l60 0 2024-06-15T01:06:59.091 INFO:tasks.workunit.client.0.smithi137.stdout:4/255: chown dd/d15/d36/d45/c57 12006569 1 2024-06-15T01:06:59.092 INFO:tasks.workunit.client.1.smithi142.stdout:3/408: getdents d0/d3 0 2024-06-15T01:06:59.093 INFO:tasks.workunit.client.0.smithi137.stdout:9/140: creat df/d12/d15/f26 x:0 0 0 2024-06-15T01:06:59.094 INFO:tasks.workunit.client.0.smithi137.stdout:9/141: truncate f5 2423020 0 2024-06-15T01:06:59.095 INFO:tasks.workunit.client.1.smithi142.stdout:8/315: link d0/d17/d40/f4a d0/d25/f6a 0 2024-06-15T01:06:59.095 INFO:tasks.workunit.client.1.smithi142.stdout:8/316: readlink d0/d17/l3e 0 2024-06-15T01:06:59.097 INFO:tasks.workunit.client.1.smithi142.stdout:4/434: mkdir d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/d99 0 2024-06-15T01:06:59.099 INFO:tasks.workunit.client.1.smithi142.stdout:6/302: symlink d1a/l69 0 2024-06-15T01:06:59.100 INFO:tasks.workunit.client.0.smithi137.stdout:4/256: dwrite dd/f3b [0,4194304] 0 2024-06-15T01:06:59.104 INFO:tasks.workunit.client.0.smithi137.stdout:9/142: dwrite fc [4194304,4194304] 0 2024-06-15T01:06:59.110 INFO:tasks.workunit.client.0.smithi137.stdout:4/257: creat dd/d15/d48/f61 x:0 0 0 2024-06-15T01:06:59.110 INFO:tasks.workunit.client.0.smithi137.stdout:4/258: readlink dd/lf 0 2024-06-15T01:06:59.111 INFO:tasks.workunit.client.0.smithi137.stdout:4/259: write dd/fe [853508,36766] 0 2024-06-15T01:06:59.111 INFO:tasks.workunit.client.1.smithi142.stdout:6/303: rename d1a/d2d/d3b/f41 to d1a/d39/d42/d4c/d58/f6a 0 2024-06-15T01:06:59.113 INFO:tasks.workunit.client.1.smithi142.stdout:3/409: unlink d0/d1f/d20/d5d/l79 0 2024-06-15T01:06:59.114 INFO:tasks.workunit.client.1.smithi142.stdout:4/435: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/d99/f9a x:0 0 0 2024-06-15T01:06:59.114 INFO:tasks.workunit.client.0.smithi137.stdout:4/260: creat dd/d15/d36/d45/d55/f62 x:0 0 0 2024-06-15T01:06:59.115 INFO:tasks.workunit.client.1.smithi142.stdout:1/335: fdatasync d6/d12/d1c/d2c/f62 0 2024-06-15T01:06:59.116 INFO:tasks.workunit.client.0.smithi137.stdout:4/261: creat dd/d10/f63 x:0 0 0 2024-06-15T01:06:59.117 INFO:tasks.workunit.client.1.smithi142.stdout:4/436: truncate d5/d9/d18/f6e 747738 0 2024-06-15T01:06:59.119 INFO:tasks.workunit.client.0.smithi137.stdout:9/143: dread f5 [0,4194304] 0 2024-06-15T01:06:59.120 INFO:tasks.workunit.client.1.smithi142.stdout:6/304: mknod d1a/d39/d3a/d51/c6b 0 2024-06-15T01:06:59.121 INFO:tasks.workunit.client.0.smithi137.stdout:9/144: chown df/d12/d15/f26 82 1 2024-06-15T01:06:59.121 INFO:tasks.workunit.client.1.smithi142.stdout:4/437: mknod d5/d9/d23/c9b 0 2024-06-15T01:06:59.121 INFO:tasks.workunit.client.0.smithi137.stdout:4/262: rename dd/d15/d30/l38 to dd/l64 0 2024-06-15T01:06:59.122 INFO:tasks.workunit.client.0.smithi137.stdout:4/263: chown dd/f47 344 1 2024-06-15T01:06:59.125 INFO:tasks.workunit.client.1.smithi142.stdout:6/305: mkdir d1a/d2d/d2e/d6c 0 2024-06-15T01:06:59.125 INFO:tasks.workunit.client.0.smithi137.stdout:4/264: read dd/f17 [552690,109768] 0 2024-06-15T01:06:59.126 INFO:tasks.workunit.client.1.smithi142.stdout:0/311: dread d3/f4 [0,4194304] 0 2024-06-15T01:06:59.127 INFO:tasks.workunit.client.1.smithi142.stdout:4/438: unlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f74 0 2024-06-15T01:06:59.129 INFO:tasks.workunit.client.1.smithi142.stdout:1/336: creat d6/d12/d3c/d63/d65/f7b x:0 0 0 2024-06-15T01:06:59.130 INFO:tasks.workunit.client.1.smithi142.stdout:6/306: fsync d1a/d39/d3a/d51/f5c 0 2024-06-15T01:06:59.132 INFO:tasks.workunit.client.1.smithi142.stdout:4/439: unlink d5/f8 0 2024-06-15T01:06:59.134 INFO:tasks.workunit.client.1.smithi142.stdout:1/337: write d6/d12/d1c/f53 [183879,125444] 0 2024-06-15T01:06:59.136 INFO:tasks.workunit.client.1.smithi142.stdout:6/307: dread d1a/d39/d42/d4c/d58/f6a [0,4194304] 0 2024-06-15T01:06:59.137 INFO:tasks.workunit.client.1.smithi142.stdout:0/312: getdents d3/d5/d1e 0 2024-06-15T01:06:59.138 INFO:tasks.workunit.client.0.smithi137.stdout:9/145: dread df/d12/d15/f1d [0,4194304] 0 2024-06-15T01:06:59.139 INFO:tasks.workunit.client.1.smithi142.stdout:6/308: write d1a/d2d/f32 [1065039,99522] 0 2024-06-15T01:06:59.139 INFO:tasks.workunit.client.0.smithi137.stdout:9/146: read f3 [540931,55115] 0 2024-06-15T01:06:59.140 INFO:tasks.workunit.client.1.smithi142.stdout:4/440: truncate d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/d99/f9a 208731 0 2024-06-15T01:06:59.140 INFO:tasks.workunit.client.0.smithi137.stdout:9/147: write fc [3196375,58348] 0 2024-06-15T01:06:59.141 INFO:tasks.workunit.client.1.smithi142.stdout:3/410: dread d0/d1b/f5b [0,4194304] 0 2024-06-15T01:06:59.141 INFO:tasks.workunit.client.1.smithi142.stdout:1/338: unlink d6/d12/d30/f5e 0 2024-06-15T01:06:59.142 INFO:tasks.workunit.client.1.smithi142.stdout:5/325: write d2/f14 [4028357,29279] 0 2024-06-15T01:06:59.144 INFO:tasks.workunit.client.1.smithi142.stdout:3/411: chown d0/d3/f12 0 1 2024-06-15T01:06:59.146 INFO:tasks.workunit.client.1.smithi142.stdout:3/412: write d0/d3/f43 [280060,54117] 0 2024-06-15T01:06:59.148 INFO:tasks.workunit.client.1.smithi142.stdout:6/309: dwrite f0 [0,4194304] 0 2024-06-15T01:06:59.148 INFO:tasks.workunit.client.1.smithi142.stdout:5/326: chown d2/d18/l43 619 1 2024-06-15T01:06:59.152 INFO:tasks.workunit.client.1.smithi142.stdout:3/413: truncate d0/d1b/f2a 1178642 0 2024-06-15T01:06:59.152 INFO:tasks.workunit.client.1.smithi142.stdout:3/414: stat d0/d3/da/f62 0 2024-06-15T01:06:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.161+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d64000bf0 con 0x7f7d7006bb60 2024-06-15T01:06:59.157 INFO:tasks.workunit.client.1.smithi142.stdout:0/313: link d3/d12/d22/l2c d3/d5/d1b/l7a 0 2024-06-15T01:06:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.162+0000 7f7d7bfff700 1 -- 172.21.15.137:0/1866021125 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7d64000bf0 con 0x7f7d7006bb60 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [], 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "1/2 daemons upgraded", 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "", 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:06:59.159 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 msgr2=0x7f7d7006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 0x7f7d7006e020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006010 tx=0x7f7d7400b560 comp rx=0 tx=0).stop 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 msgr2=0x7f7d8419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c930 tx=0x7f7d6c00cc40 comp rx=0 tx=0).stop 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 shutdown_connections 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f7d7006bb60 0x7f7d7006e020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d84100bb0 0x7f7d8419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 --2- 172.21.15.137:0/1866021125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d84101db0 0x7f7d8419cdc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 >> 172.21.15.137:0/1866021125 conn(0x7f7d840fc110 msgr2=0x7f7d84109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 shutdown_connections 2024-06-15T01:06:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:06:59.166+0000 7f7d89220700 1 -- 172.21.15.137:0/1866021125 wait complete. 2024-06-15T01:06:59.168 INFO:tasks.workunit.client.1.smithi142.stdout:4/441: truncate d5/d9/d18/f69 345439 0 2024-06-15T01:06:59.168 INFO:tasks.workunit.client.1.smithi142.stdout:1/339: truncate d6/d12/f2d 3449626 0 2024-06-15T01:06:59.169 INFO:tasks.workunit.client.1.smithi142.stdout:0/314: link d3/d5/d1b/l4e d3/l7b 0 2024-06-15T01:06:59.177 INFO:tasks.workunit.client.1.smithi142.stdout:0/315: rename d3/d12 to d3/d5/d44/d7c 0 2024-06-15T01:06:59.178 INFO:tasks.workunit.client.1.smithi142.stdout:1/340: dwrite d6/d3a/f70 [0,4194304] 0 2024-06-15T01:06:59.178 INFO:tasks.workunit.client.1.smithi142.stdout:4/442: mknod d5/c9c 0 2024-06-15T01:06:59.180 INFO:tasks.workunit.client.1.smithi142.stdout:1/341: mknod d6/d4d/c7c 0 2024-06-15T01:06:59.181 INFO:tasks.workunit.client.1.smithi142.stdout:1/342: chown d6/d25 46615 1 2024-06-15T01:06:59.189 INFO:tasks.workunit.client.1.smithi142.stdout:3/415: dread d0/f2c [0,4194304] 0 2024-06-15T01:06:59.211 INFO:tasks.workunit.client.1.smithi142.stdout:3/416: dread d0/d3/d15/f64 [0,4194304] 0 2024-06-15T01:06:59.224 INFO:tasks.workunit.client.1.smithi142.stdout:3/417: dread d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:06:59.302 INFO:tasks.workunit.client.1.smithi142.stdout:1/343: rename d6/l29 to d6/d46/d61/l7d 0 2024-06-15T01:06:59.324 INFO:tasks.workunit.client.1.smithi142.stdout:7/288: sync 2024-06-15T01:06:59.324 INFO:tasks.workunit.client.1.smithi142.stdout:9/292: sync 2024-06-15T01:06:59.334 INFO:tasks.workunit.client.0.smithi137.stdout:7/242: sync 2024-06-15T01:06:59.336 INFO:tasks.workunit.client.1.smithi142.stdout:3/418: creat d0/d1f/d20/d5d/d60/d72/f7c x:0 0 0 2024-06-15T01:06:59.337 INFO:tasks.workunit.client.1.smithi142.stdout:3/419: fdatasync d0/d3/f43 0 2024-06-15T01:06:59.338 INFO:tasks.workunit.client.1.smithi142.stdout:3/420: chown d0/d1f/d20/l3a 2140377 1 2024-06-15T01:06:59.340 INFO:tasks.workunit.client.1.smithi142.stdout:9/293: creat d6/dc/dd/f77 x:0 0 0 2024-06-15T01:06:59.342 INFO:tasks.workunit.client.1.smithi142.stdout:7/289: mknod d2/d8/dd/d14/d2c/d37/c61 0 2024-06-15T01:06:59.343 INFO:tasks.workunit.client.1.smithi142.stdout:3/421: dwrite d0/d3/da/d71/d55/f7a [0,4194304] 0 2024-06-15T01:06:59.346 INFO:tasks.workunit.client.1.smithi142.stdout:9/294: mkdir d6/dc/d4e/d78 0 2024-06-15T01:06:59.348 INFO:tasks.workunit.client.1.smithi142.stdout:3/422: truncate d0/d3/f54 2767452 0 2024-06-15T01:06:59.350 INFO:tasks.workunit.client.1.smithi142.stdout:7/290: dwrite d2/d6/f1d [0,4194304] 0 2024-06-15T01:06:59.351 INFO:tasks.workunit.client.1.smithi142.stdout:9/295: symlink d6/l79 0 2024-06-15T01:06:59.352 INFO:tasks.workunit.client.1.smithi142.stdout:9/296: chown d6/dc/dd/f77 1712 1 2024-06-15T01:06:59.354 INFO:tasks.workunit.client.1.smithi142.stdout:7/291: creat d2/d8/dd/d14/d2c/f62 x:0 0 0 2024-06-15T01:06:59.356 INFO:tasks.workunit.client.1.smithi142.stdout:9/297: unlink d6/dc/d1b/d25/d35/l5a 0 2024-06-15T01:06:59.357 INFO:tasks.workunit.client.1.smithi142.stdout:9/298: rename d6/dc/d1b to d6/dc/d1b/d25/d7a 22 2024-06-15T01:06:59.358 INFO:tasks.workunit.client.1.smithi142.stdout:7/292: creat d2/d6/d22/d47/f63 x:0 0 0 2024-06-15T01:06:59.358 INFO:tasks.workunit.client.1.smithi142.stdout:3/423: dwrite d0/d1f/f6f [0,4194304] 0 2024-06-15T01:06:59.362 INFO:tasks.workunit.client.1.smithi142.stdout:1/344: dread d6/d12/f2d [0,4194304] 0 2024-06-15T01:06:59.363 INFO:tasks.workunit.client.1.smithi142.stdout:9/299: read d6/dc/d1b/f76 [1610200,127728] 0 2024-06-15T01:06:59.364 INFO:tasks.workunit.client.1.smithi142.stdout:3/424: rename d0/d1f/d20/d59/c5e to d0/d1f/d20/d5d/c7d 0 2024-06-15T01:06:59.364 INFO:tasks.workunit.client.1.smithi142.stdout:9/300: dread - d6/dc/d1b/f40 zero size 2024-06-15T01:06:59.365 INFO:tasks.workunit.client.1.smithi142.stdout:1/345: symlink d6/d4d/l7e 0 2024-06-15T01:06:59.370 INFO:tasks.workunit.client.1.smithi142.stdout:7/293: dwrite d2/d8/da/d33/f20 [0,4194304] 0 2024-06-15T01:06:59.371 INFO:tasks.workunit.client.1.smithi142.stdout:7/294: readlink d2/d6/l3b 0 2024-06-15T01:06:59.374 INFO:tasks.workunit.client.1.smithi142.stdout:7/295: rename d2/d8/dd/l51 to d2/d6/l64 0 2024-06-15T01:06:59.377 INFO:tasks.workunit.client.1.smithi142.stdout:7/296: creat d2/d8/dd/d14/f65 x:0 0 0 2024-06-15T01:06:59.377 INFO:tasks.workunit.client.1.smithi142.stdout:9/301: dread d6/dc/dd/d22/f2b [0,4194304] 0 2024-06-15T01:06:59.378 INFO:tasks.workunit.client.1.smithi142.stdout:9/302: readlink d6/dc/dd/le 0 2024-06-15T01:06:59.383 INFO:tasks.workunit.client.1.smithi142.stdout:7/297: dread d2/f1a [0,4194304] 0 2024-06-15T01:06:59.383 INFO:tasks.workunit.client.1.smithi142.stdout:9/303: rename d6/d34/c47 to d6/dc/d1b/d25/c7b 0 2024-06-15T01:06:59.384 INFO:tasks.workunit.client.1.smithi142.stdout:7/298: dread - d2/d8/dd/d14/f58 zero size 2024-06-15T01:06:59.385 INFO:tasks.workunit.client.1.smithi142.stdout:9/304: truncate d6/d31/f32 690772 0 2024-06-15T01:06:59.386 INFO:tasks.workunit.client.1.smithi142.stdout:7/299: unlink d2/d8/dd/d14/f46 0 2024-06-15T01:06:59.387 INFO:tasks.workunit.client.1.smithi142.stdout:7/300: rename d2/d6/d22 to d2/d6/d22/d34/d66 22 2024-06-15T01:06:59.389 INFO:tasks.workunit.client.1.smithi142.stdout:1/346: truncate d6/f13 1520134 0 2024-06-15T01:06:59.389 INFO:tasks.workunit.client.1.smithi142.stdout:9/305: mknod d6/d34/c7c 0 2024-06-15T01:06:59.391 INFO:tasks.workunit.client.1.smithi142.stdout:7/301: rmdir d2/d8/dd/d14/d2c/d37 39 2024-06-15T01:06:59.392 INFO:tasks.workunit.client.1.smithi142.stdout:1/347: mknod d6/c7f 0 2024-06-15T01:06:59.392 INFO:tasks.workunit.client.1.smithi142.stdout:1/348: stat d6/d4d/c7c 0 2024-06-15T01:06:59.394 INFO:tasks.workunit.client.1.smithi142.stdout:5/327: sync 2024-06-15T01:06:59.394 INFO:tasks.workunit.client.1.smithi142.stdout:6/310: sync 2024-06-15T01:06:59.394 INFO:tasks.workunit.client.1.smithi142.stdout:2/285: sync 2024-06-15T01:06:59.394 INFO:tasks.workunit.client.1.smithi142.stdout:4/443: sync 2024-06-15T01:06:59.394 INFO:tasks.workunit.client.1.smithi142.stdout:0/316: sync 2024-06-15T01:06:59.395 INFO:tasks.workunit.client.1.smithi142.stdout:8/317: sync 2024-06-15T01:06:59.396 INFO:tasks.workunit.client.1.smithi142.stdout:1/349: mkdir d6/d12/d80 0 2024-06-15T01:06:59.397 INFO:tasks.workunit.client.1.smithi142.stdout:9/306: dwrite d6/dc/d1b/d25/f3f [0,4194304] 0 2024-06-15T01:06:59.399 INFO:tasks.workunit.client.1.smithi142.stdout:9/307: fsync d6/d31/f3e 0 2024-06-15T01:06:59.399 INFO:tasks.workunit.client.1.smithi142.stdout:1/350: write d6/d12/d28/f51 [1432686,42499] 0 2024-06-15T01:06:59.402 INFO:tasks.workunit.client.1.smithi142.stdout:3/425: truncate d0/d3/f54 59755 0 2024-06-15T01:06:59.403 INFO:tasks.workunit.client.1.smithi142.stdout:2/286: creat d8/d19/d23/d3a/f55 x:0 0 0 2024-06-15T01:06:59.403 INFO:tasks.workunit.client.1.smithi142.stdout:6/311: mknod d1a/d2d/c6d 0 2024-06-15T01:06:59.404 INFO:tasks.workunit.client.1.smithi142.stdout:7/302: write d2/d6/f9 [1897579,91491] 0 2024-06-15T01:06:59.407 INFO:tasks.workunit.client.1.smithi142.stdout:4/444: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/f9d x:0 0 0 2024-06-15T01:06:59.408 INFO:tasks.workunit.client.1.smithi142.stdout:9/308: mknod d6/dc/d1b/d25/d58/c7d 0 2024-06-15T01:06:59.409 INFO:tasks.workunit.client.1.smithi142.stdout:1/351: chown d6/d12/d1c/l74 3 1 2024-06-15T01:06:59.409 INFO:tasks.workunit.client.1.smithi142.stdout:3/426: creat d0/f7e x:0 0 0 2024-06-15T01:06:59.410 INFO:tasks.workunit.client.1.smithi142.stdout:5/328: symlink d2/de/d19/d5f/l67 0 2024-06-15T01:06:59.412 INFO:tasks.workunit.client.1.smithi142.stdout:2/287: unlink d8/d19/d23/l50 0 2024-06-15T01:06:59.414 INFO:tasks.workunit.client.1.smithi142.stdout:0/317: creat d3/d39/f7d x:0 0 0 2024-06-15T01:06:59.417 INFO:tasks.workunit.client.1.smithi142.stdout:3/427: fdatasync d0/d3/da/d18/f28 0 2024-06-15T01:06:59.418 INFO:tasks.workunit.client.1.smithi142.stdout:5/329: mkdir d2/de/d19/d5f/d68 0 2024-06-15T01:06:59.418 INFO:tasks.workunit.client.1.smithi142.stdout:9/309: dwrite d6/dc/d4e/f57 [0,4194304] 0 2024-06-15T01:06:59.422 INFO:tasks.workunit.client.1.smithi142.stdout:0/318: creat d3/d5/f7e x:0 0 0 2024-06-15T01:06:59.422 INFO:tasks.workunit.client.1.smithi142.stdout:3/428: write d0/d1f/f46 [1532778,34738] 0 2024-06-15T01:06:59.423 INFO:tasks.workunit.client.1.smithi142.stdout:8/318: dread d0/d25/d5a/f29 [0,4194304] 0 2024-06-15T01:06:59.423 INFO:tasks.workunit.client.1.smithi142.stdout:9/310: read d6/dc/dd/f46 [156548,105129] 0 2024-06-15T01:06:59.423 INFO:tasks.workunit.client.1.smithi142.stdout:5/330: write d2/de/d27/f54 [6654232,30474] 0 2024-06-15T01:06:59.424 INFO:tasks.workunit.client.1.smithi142.stdout:7/303: creat d2/d8/da/d33/d25/f67 x:0 0 0 2024-06-15T01:06:59.427 INFO:tasks.workunit.client.1.smithi142.stdout:5/331: chown d2/d3e/c3b 109019 1 2024-06-15T01:06:59.428 INFO:tasks.workunit.client.1.smithi142.stdout:7/304: chown d2/d6/d22/d47/f50 1163675 1 2024-06-15T01:06:59.428 INFO:tasks.workunit.client.1.smithi142.stdout:5/332: stat d2/d18/f37 0 2024-06-15T01:06:59.430 INFO:tasks.workunit.client.1.smithi142.stdout:4/445: dread d5/d9/da/f7f [0,4194304] 0 2024-06-15T01:06:59.430 INFO:tasks.workunit.client.1.smithi142.stdout:4/446: read f3 [4013012,20381] 0 2024-06-15T01:06:59.431 INFO:tasks.workunit.client.1.smithi142.stdout:4/447: chown d5/d1c/d2a/d30/d46/f76 34392362 1 2024-06-15T01:06:59.432 INFO:tasks.workunit.client.1.smithi142.stdout:2/288: rename d8/d19/d23/d3a/f55 to d8/d19/f56 0 2024-06-15T01:06:59.432 INFO:tasks.workunit.client.1.smithi142.stdout:6/312: link d1a/c1c d1a/d2d/d68/c6e 0 2024-06-15T01:06:59.433 INFO:tasks.workunit.client.1.smithi142.stdout:2/289: stat d8/df/d1f/c39 0 2024-06-15T01:06:59.433 INFO:tasks.workunit.client.1.smithi142.stdout:4/448: stat d5/d1c/d2a/d30/d46/d50/d51/c8c 0 2024-06-15T01:06:59.434 INFO:tasks.workunit.client.1.smithi142.stdout:9/311: read d6/dc/dd/d22/f2b [626434,92788] 0 2024-06-15T01:06:59.435 INFO:tasks.workunit.client.1.smithi142.stdout:3/429: mknod d0/d1f/d20/c7f 0 2024-06-15T01:06:59.437 INFO:tasks.workunit.client.1.smithi142.stdout:7/305: symlink d2/d8/dd/d14/l68 0 2024-06-15T01:06:59.439 INFO:tasks.workunit.client.1.smithi142.stdout:3/430: write d0/d1b/f41 [1316518,3454] 0 2024-06-15T01:06:59.441 INFO:tasks.workunit.client.1.smithi142.stdout:0/319: rename d3/d5/d1b/l72 to d3/d5/d1e/l7f 0 2024-06-15T01:06:59.443 INFO:tasks.workunit.client.1.smithi142.stdout:1/352: dwrite d6/d12/f3e [0,4194304] 0 2024-06-15T01:06:59.445 INFO:tasks.workunit.client.1.smithi142.stdout:0/320: fdatasync d3/d5/f1d 0 2024-06-15T01:06:59.446 INFO:tasks.workunit.client.1.smithi142.stdout:8/319: dwrite d0/f15 [4194304,4194304] 0 2024-06-15T01:06:59.447 INFO:tasks.workunit.client.0.smithi137.stdout:5/449: sync 2024-06-15T01:06:59.449 INFO:tasks.workunit.client.1.smithi142.stdout:5/333: link d2/de/f5a d2/d18/f69 0 2024-06-15T01:06:59.452 INFO:tasks.workunit.client.1.smithi142.stdout:6/313: truncate d1a/f25 3544292 0 2024-06-15T01:06:59.452 INFO:tasks.workunit.client.1.smithi142.stdout:3/431: creat d0/d1f/d20/d59/f80 x:0 0 0 2024-06-15T01:06:59.453 INFO:tasks.workunit.client.0.smithi137.stdout:5/450: symlink df/d44/d54/d95/d3b/d4b/d4f/l98 0 2024-06-15T01:06:59.455 INFO:tasks.workunit.client.0.smithi137.stdout:5/451: creat df/d11/d14/d6c/f99 x:0 0 0 2024-06-15T01:06:59.456 INFO:tasks.workunit.client.0.smithi137.stdout:5/452: readlink df/d3d/l8b 0 2024-06-15T01:06:59.456 INFO:tasks.workunit.client.1.smithi142.stdout:9/312: dread d6/dc/dd/d22/f41 [0,4194304] 0 2024-06-15T01:06:59.457 INFO:tasks.workunit.client.1.smithi142.stdout:9/313: fdatasync f2 0 2024-06-15T01:06:59.459 INFO:tasks.workunit.client.0.smithi137.stdout:5/453: symlink df/d44/d54/d95/d3b/d49/l9a 0 2024-06-15T01:06:59.460 INFO:tasks.workunit.client.1.smithi142.stdout:1/353: symlink d6/d12/d1c/l81 0 2024-06-15T01:06:59.461 INFO:tasks.workunit.client.1.smithi142.stdout:6/314: unlink d1a/d39/d42/c4b 0 2024-06-15T01:06:59.462 INFO:tasks.workunit.client.1.smithi142.stdout:9/314: unlink d6/dc/d1b/d25/c6d 0 2024-06-15T01:06:59.463 INFO:tasks.workunit.client.1.smithi142.stdout:2/290: dread d8/d19/d23/d25/f24 [0,4194304] 0 2024-06-15T01:06:59.463 INFO:tasks.workunit.client.1.smithi142.stdout:3/432: mknod d0/d3/da/c81 0 2024-06-15T01:06:59.464 INFO:tasks.workunit.client.0.smithi137.stdout:5/454: link df/d11/d14/f2f df/d44/d54/d95/d35/f9b 0 2024-06-15T01:06:59.464 INFO:tasks.workunit.client.1.smithi142.stdout:0/321: dwrite d3/f4 [4194304,4194304] 0 2024-06-15T01:06:59.465 INFO:tasks.workunit.client.0.smithi137.stdout:5/455: truncate df/d11/d14/d6c/f99 629743 0 2024-06-15T01:06:59.466 INFO:tasks.workunit.client.1.smithi142.stdout:8/320: link d0/d17/f44 d0/d17/f6b 0 2024-06-15T01:06:59.466 INFO:tasks.workunit.client.1.smithi142.stdout:1/354: truncate d6/d12/d1c/f34 8948517 0 2024-06-15T01:06:59.467 INFO:tasks.workunit.client.1.smithi142.stdout:9/315: truncate d6/dc/d1b/d25/d58/f63 278254 0 2024-06-15T01:06:59.469 INFO:tasks.workunit.client.1.smithi142.stdout:1/355: dread - d6/d12/d3c/d63/d65/f7b zero size 2024-06-15T01:06:59.470 INFO:tasks.workunit.client.1.smithi142.stdout:3/433: rename d0/d1f/d3e to d0/d1f/d20/d5d/d82 0 2024-06-15T01:06:59.473 INFO:tasks.workunit.client.1.smithi142.stdout:9/316: mknod d6/d31/c7e 0 2024-06-15T01:06:59.473 INFO:tasks.workunit.client.1.smithi142.stdout:3/434: write d0/d1b/f76 [610421,102411] 0 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:1/287: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:2/217: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:8/205: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:9/148: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:6/206: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:4/265: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:0/277: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:3/343: sync 2024-06-15T01:06:59.474 INFO:tasks.workunit.client.0.smithi137.stdout:7/243: sync 2024-06-15T01:06:59.475 INFO:tasks.workunit.client.1.smithi142.stdout:1/356: creat d6/d12/d28/f82 x:0 0 0 2024-06-15T01:06:59.475 INFO:tasks.workunit.client.1.smithi142.stdout:0/322: dwrite d3/d39/f45 [0,4194304] 0 2024-06-15T01:06:59.476 INFO:tasks.workunit.client.0.smithi137.stdout:0/278: fsync d1/d2/d34/f4a 0 2024-06-15T01:06:59.477 INFO:tasks.workunit.client.0.smithi137.stdout:2/218: creat d0/dc/f3d x:0 0 0 2024-06-15T01:06:59.478 INFO:tasks.workunit.client.1.smithi142.stdout:9/317: fsync d6/dc/f15 0 2024-06-15T01:06:59.478 INFO:tasks.workunit.client.0.smithi137.stdout:6/207: mknod d0/d1c/c44 0 2024-06-15T01:06:59.478 INFO:tasks.workunit.client.1.smithi142.stdout:3/435: dread d0/d3/da/d71/d55/f57 [0,4194304] 0 2024-06-15T01:06:59.479 INFO:tasks.workunit.client.0.smithi137.stdout:7/244: dread - df/d13/d2e/d1a/f26 zero size 2024-06-15T01:06:59.479 INFO:tasks.workunit.client.0.smithi137.stdout:4/266: creat dd/d10/f65 x:0 0 0 2024-06-15T01:06:59.479 INFO:tasks.workunit.client.0.smithi137.stdout:3/344: chown d0/d2/d22/d63/l6b 0 1 2024-06-15T01:06:59.481 INFO:tasks.workunit.client.1.smithi142.stdout:8/321: dread d0/f47 [0,4194304] 0 2024-06-15T01:06:59.481 INFO:tasks.workunit.client.1.smithi142.stdout:0/323: creat d3/d5/d44/d7c/d22/d59/f80 x:0 0 0 2024-06-15T01:06:59.485 INFO:tasks.workunit.client.0.smithi137.stdout:8/206: creat d9/f47 x:0 0 0 2024-06-15T01:06:59.486 INFO:tasks.workunit.client.0.smithi137.stdout:9/149: creat df/d12/d15/d16/f27 x:0 0 0 2024-06-15T01:06:59.488 INFO:tasks.workunit.client.1.smithi142.stdout:4/449: dread d5/d9/d23/f3b [0,4194304] 0 2024-06-15T01:06:59.489 INFO:tasks.workunit.client.1.smithi142.stdout:9/318: creat d6/dc/d4e/d78/f7f x:0 0 0 2024-06-15T01:06:59.489 INFO:tasks.workunit.client.0.smithi137.stdout:1/288: truncate d0/d3/f5 849691 0 2024-06-15T01:06:59.489 INFO:tasks.workunit.client.0.smithi137.stdout:7/245: rmdir df/d18/d1c/d3a 39 2024-06-15T01:06:59.490 INFO:tasks.workunit.client.0.smithi137.stdout:8/207: read f1 [1386041,36702] 0 2024-06-15T01:06:59.490 INFO:tasks.workunit.client.1.smithi142.stdout:9/319: read - d6/dc/d1b/d25/d35/f70 zero size 2024-06-15T01:06:59.490 INFO:tasks.workunit.client.1.smithi142.stdout:4/450: write d5/d1c/d2a/d30/d46/d50/f5e [4425334,95905] 0 2024-06-15T01:06:59.491 INFO:tasks.workunit.client.0.smithi137.stdout:3/345: mkdir d0/d2/d22/d63/d76 0 2024-06-15T01:06:59.492 INFO:tasks.workunit.client.1.smithi142.stdout:8/322: dread d0/d17/f61 [0,4194304] 0 2024-06-15T01:06:59.492 INFO:tasks.workunit.client.0.smithi137.stdout:7/246: chown df/c12 6756791 1 2024-06-15T01:06:59.493 INFO:tasks.workunit.client.0.smithi137.stdout:0/279: rename d1/d2/d34 to d1/d2/d13/d66 0 2024-06-15T01:06:59.495 INFO:tasks.workunit.client.1.smithi142.stdout:9/320: creat d6/dc/d51/f80 x:0 0 0 2024-06-15T01:06:59.496 INFO:tasks.workunit.client.0.smithi137.stdout:4/267: dwrite dd/d10/f39 [0,4194304] 0 2024-06-15T01:06:59.496 INFO:tasks.workunit.client.1.smithi142.stdout:9/321: dread - d6/dc/f61 zero size 2024-06-15T01:06:59.497 INFO:tasks.workunit.client.0.smithi137.stdout:9/150: creat df/f28 x:0 0 0 2024-06-15T01:06:59.497 INFO:tasks.workunit.client.1.smithi142.stdout:3/436: dwrite d0/d3/d15/f64 [0,4194304] 0 2024-06-15T01:06:59.499 INFO:tasks.workunit.client.0.smithi137.stdout:2/219: dwrite d0/da/f18 [0,4194304] 0 2024-06-15T01:06:59.502 INFO:tasks.workunit.client.0.smithi137.stdout:5/456: dread df/d11/d14/f15 [0,4194304] 0 2024-06-15T01:06:59.503 INFO:tasks.workunit.client.1.smithi142.stdout:9/322: fsync d6/dc/dd/d22/f2b 0 2024-06-15T01:06:59.505 INFO:tasks.workunit.client.1.smithi142.stdout:9/323: fdatasync d6/dc/dd/f4f 0 2024-06-15T01:06:59.506 INFO:tasks.workunit.client.1.smithi142.stdout:3/437: dwrite d0/d1f/f27 [0,4194304] 0 2024-06-15T01:06:59.509 INFO:tasks.workunit.client.1.smithi142.stdout:9/324: dread - d6/d34/f5f zero size 2024-06-15T01:06:59.509 INFO:tasks.workunit.client.1.smithi142.stdout:7/306: truncate d2/d8/da/d33/d25/f3f 3379993 0 2024-06-15T01:06:59.512 INFO:tasks.workunit.client.0.smithi137.stdout:9/151: unlink fa 0 2024-06-15T01:06:59.514 INFO:tasks.workunit.client.0.smithi137.stdout:1/289: creat d0/d10/f6f x:0 0 0 2024-06-15T01:06:59.516 INFO:tasks.workunit.client.0.smithi137.stdout:2/220: mkdir d0/da/d3e 0 2024-06-15T01:06:59.518 INFO:tasks.workunit.client.1.smithi142.stdout:7/307: dread - d2/d8/da/d33/d25/f67 zero size 2024-06-15T01:06:59.518 INFO:tasks.workunit.client.1.smithi142.stdout:8/323: fsync d0/d17/f44 0 2024-06-15T01:06:59.518 INFO:tasks.workunit.client.1.smithi142.stdout:7/308: truncate d2/d8/f42 461067 0 2024-06-15T01:06:59.519 INFO:tasks.workunit.client.1.smithi142.stdout:6/315: truncate d1a/f61 2971502 0 2024-06-15T01:06:59.519 INFO:tasks.workunit.client.1.smithi142.stdout:3/438: dwrite d0/d1f/f69 [0,4194304] 0 2024-06-15T01:06:59.519 INFO:tasks.workunit.client.0.smithi137.stdout:4/268: symlink dd/d15/d21/l66 0 2024-06-15T01:06:59.519 INFO:tasks.workunit.client.1.smithi142.stdout:9/325: creat d6/dc/d43/f81 x:0 0 0 2024-06-15T01:06:59.520 INFO:tasks.workunit.client.0.smithi137.stdout:4/269: chown dd/d15/d30/f40 12448007 1 2024-06-15T01:06:59.522 INFO:tasks.workunit.client.0.smithi137.stdout:3/346: mkdir d0/d77 0 2024-06-15T01:06:59.522 INFO:tasks.workunit.client.0.smithi137.stdout:8/208: rmdir d9/de/d1e/d35 39 2024-06-15T01:06:59.522 INFO:tasks.workunit.client.0.smithi137.stdout:2/221: dread d0/de/d2b/f32 [0,4194304] 0 2024-06-15T01:06:59.523 INFO:tasks.workunit.client.1.smithi142.stdout:6/316: stat d1a/f1f 0 2024-06-15T01:06:59.524 INFO:tasks.workunit.client.0.smithi137.stdout:6/208: getdents d0/d5/d9 0 2024-06-15T01:06:59.525 INFO:tasks.workunit.client.1.smithi142.stdout:8/324: mknod d0/d25/c6c 0 2024-06-15T01:06:59.529 INFO:tasks.workunit.client.1.smithi142.stdout:3/439: unlink d0/d1b/f76 0 2024-06-15T01:06:59.529 INFO:tasks.workunit.client.1.smithi142.stdout:0/324: rename d3/d5 to d3/d39/d3e/d81 0 2024-06-15T01:06:59.531 INFO:tasks.workunit.client.1.smithi142.stdout:1/357: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:06:59.531 INFO:tasks.workunit.client.0.smithi137.stdout:9/152: write f2 [800685,61508] 0 2024-06-15T01:06:59.532 INFO:tasks.workunit.client.0.smithi137.stdout:1/290: creat d0/d3/d14/d34/f70 x:0 0 0 2024-06-15T01:06:59.532 INFO:tasks.workunit.client.0.smithi137.stdout:3/347: dwrite d0/d2/d23/f48 [0,4194304] 0 2024-06-15T01:06:59.534 INFO:tasks.workunit.client.1.smithi142.stdout:6/317: truncate d1a/f38 1848328 0 2024-06-15T01:06:59.535 INFO:tasks.workunit.client.0.smithi137.stdout:4/270: mkdir dd/d15/d48/d67 0 2024-06-15T01:06:59.536 INFO:tasks.workunit.client.0.smithi137.stdout:0/280: dread d1/d2/dc/d10/d14/f39 [0,4194304] 0 2024-06-15T01:06:59.539 INFO:tasks.workunit.client.0.smithi137.stdout:6/209: dread d0/f2a [0,4194304] 0 2024-06-15T01:06:59.541 INFO:tasks.workunit.client.0.smithi137.stdout:9/153: truncate df/d12/d15/d16/f27 852598 0 2024-06-15T01:06:59.542 INFO:tasks.workunit.client.0.smithi137.stdout:8/209: dread - d9/de/d1e/d35/f38 zero size 2024-06-15T01:06:59.543 INFO:tasks.workunit.client.0.smithi137.stdout:3/348: read d0/d2/d23/f44 [102932,125451] 0 2024-06-15T01:06:59.544 INFO:tasks.workunit.client.0.smithi137.stdout:4/271: dread dd/d15/f2e [0,4194304] 0 2024-06-15T01:06:59.545 INFO:tasks.workunit.client.0.smithi137.stdout:4/272: fsync dd/f18 0 2024-06-15T01:06:59.546 INFO:tasks.workunit.client.0.smithi137.stdout:2/222: mknod d0/c3f 0 2024-06-15T01:06:59.547 INFO:tasks.workunit.client.1.smithi142.stdout:7/309: rename d2/d8/dd/d14/f30 to d2/d6/f69 0 2024-06-15T01:06:59.549 INFO:tasks.workunit.client.1.smithi142.stdout:0/325: readlink d3/d39/d3e/l76 0 2024-06-15T01:06:59.549 INFO:tasks.workunit.client.0.smithi137.stdout:0/281: dread d1/d2/d13/d66/f4a [0,4194304] 0 2024-06-15T01:06:59.550 INFO:tasks.workunit.client.1.smithi142.stdout:0/326: truncate d3/f6e 106460 0 2024-06-15T01:06:59.551 INFO:tasks.workunit.client.1.smithi142.stdout:1/358: mknod d6/d12/d5b/c83 0 2024-06-15T01:06:59.551 INFO:tasks.workunit.client.0.smithi137.stdout:1/291: dread d0/d2d/f5e [0,4194304] 0 2024-06-15T01:06:59.553 INFO:tasks.workunit.client.0.smithi137.stdout:0/282: write d1/d2/f21 [1972650,63217] 0 2024-06-15T01:06:59.553 INFO:tasks.workunit.client.1.smithi142.stdout:6/318: mknod d1a/d39/d42/c6f 0 2024-06-15T01:06:59.554 INFO:tasks.workunit.client.1.smithi142.stdout:0/327: dread - d3/d39/d3e/d81/d4c/d61/f77 zero size 2024-06-15T01:06:59.554 INFO:tasks.workunit.client.0.smithi137.stdout:9/154: unlink df/d12/d1b/l21 0 2024-06-15T01:06:59.554 INFO:tasks.workunit.client.1.smithi142.stdout:4/451: truncate d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f37 1416910 0 2024-06-15T01:06:59.555 INFO:tasks.workunit.client.0.smithi137.stdout:9/155: readlink l4 0 2024-06-15T01:06:59.555 INFO:tasks.workunit.client.0.smithi137.stdout:8/210: write d9/f1d [115420,37328] 0 2024-06-15T01:06:59.556 INFO:tasks.workunit.client.0.smithi137.stdout:9/156: write df/d12/f17 [10790103,13491] 0 2024-06-15T01:06:59.556 INFO:tasks.workunit.client.1.smithi142.stdout:3/440: truncate d0/d3/f54 400583 0 2024-06-15T01:06:59.557 INFO:tasks.workunit.client.1.smithi142.stdout:4/452: dread - d5/d9/da/f8b zero size 2024-06-15T01:06:59.557 INFO:tasks.workunit.client.0.smithi137.stdout:3/349: symlink d0/d2/d22/d24/l78 0 2024-06-15T01:06:59.557 INFO:tasks.workunit.client.0.smithi137.stdout:8/211: chown d9/c1a 1868381 1 2024-06-15T01:06:59.558 INFO:tasks.workunit.client.0.smithi137.stdout:0/283: stat d1/d2/dc/d10/d14/d15/f30 0 2024-06-15T01:06:59.558 INFO:tasks.workunit.client.1.smithi142.stdout:7/310: dwrite d2/d8/da/d33/d25/d27/f5d [0,4194304] 0 2024-06-15T01:06:59.559 INFO:tasks.workunit.client.1.smithi142.stdout:4/453: write d5/d1c/f34 [1667470,88135] 0 2024-06-15T01:06:59.560 INFO:tasks.workunit.client.0.smithi137.stdout:0/284: write d1/d2/dc/d10/d14/d15/f30 [478185,73769] 0 2024-06-15T01:06:59.560 INFO:tasks.workunit.client.1.smithi142.stdout:7/311: write d2/d6/f9 [1247694,49735] 0 2024-06-15T01:06:59.560 INFO:tasks.workunit.client.0.smithi137.stdout:4/273: symlink dd/d15/d48/l68 0 2024-06-15T01:06:59.561 INFO:tasks.workunit.client.0.smithi137.stdout:2/223: fdatasync d0/f5 0 2024-06-15T01:06:59.562 INFO:tasks.workunit.client.1.smithi142.stdout:7/312: read - d2/d8/dd/d14/f58 zero size 2024-06-15T01:06:59.563 INFO:tasks.workunit.client.0.smithi137.stdout:2/224: chown d0/f5 239983 1 2024-06-15T01:06:59.565 INFO:tasks.workunit.client.1.smithi142.stdout:6/319: creat d1a/d39/d3a/f70 x:0 0 0 2024-06-15T01:06:59.567 INFO:tasks.workunit.client.1.smithi142.stdout:0/328: creat d3/d39/d3e/d81/d44/d7c/d22/f82 x:0 0 0 2024-06-15T01:06:59.569 INFO:tasks.workunit.client.1.smithi142.stdout:3/441: creat d0/d3/d15/f83 x:0 0 0 2024-06-15T01:06:59.570 INFO:tasks.workunit.client.1.smithi142.stdout:2/291: sync 2024-06-15T01:06:59.570 INFO:tasks.workunit.client.1.smithi142.stdout:5/334: sync 2024-06-15T01:06:59.570 INFO:tasks.workunit.client.1.smithi142.stdout:6/320: dread d1a/d39/d42/f45 [0,4194304] 0 2024-06-15T01:06:59.571 INFO:tasks.workunit.client.1.smithi142.stdout:3/442: chown d0/f7e 154770636 1 2024-06-15T01:06:59.571 INFO:tasks.workunit.client.0.smithi137.stdout:1/292: rename d0/d10/f40 to d0/d3/d1b/d4c/d59/f71 0 2024-06-15T01:06:59.574 INFO:tasks.workunit.client.1.smithi142.stdout:1/359: symlink d6/d12/d52/l84 0 2024-06-15T01:06:59.575 INFO:tasks.workunit.client.1.smithi142.stdout:3/443: write d0/d1f/d20/d5d/d60/d72/f7c [811622,23098] 0 2024-06-15T01:06:59.575 INFO:tasks.workunit.client.1.smithi142.stdout:9/326: rename d6/dc/d43 to d6/dc/d82 0 2024-06-15T01:06:59.576 INFO:tasks.workunit.client.1.smithi142.stdout:4/454: rmdir d5/d9/d23/d32 39 2024-06-15T01:06:59.576 INFO:tasks.workunit.client.0.smithi137.stdout:7/247: dwrite df/d18/d1c/d1d/f28 [0,4194304] 0 2024-06-15T01:06:59.576 INFO:tasks.workunit.client.0.smithi137.stdout:9/157: mkdir df/d12/d29 0 2024-06-15T01:06:59.577 INFO:tasks.workunit.client.1.smithi142.stdout:9/327: chown d6/dc/dd/c2d 5836223 1 2024-06-15T01:06:59.577 INFO:tasks.workunit.client.0.smithi137.stdout:4/274: dwrite dd/d15/d21/d3a/d51/f52 [0,4194304] 0 2024-06-15T01:06:59.578 INFO:tasks.workunit.client.1.smithi142.stdout:3/444: chown d0/d3/l5 9517 1 2024-06-15T01:06:59.578 INFO:tasks.workunit.client.1.smithi142.stdout:5/335: dwrite d2/f14 [0,4194304] 0 2024-06-15T01:06:59.579 INFO:tasks.workunit.client.0.smithi137.stdout:7/248: fsync df/f32 0 2024-06-15T01:06:59.580 INFO:tasks.workunit.client.1.smithi142.stdout:5/336: chown d2/d18/f61 23879 1 2024-06-15T01:06:59.580 INFO:tasks.workunit.client.1.smithi142.stdout:3/445: stat d0/d3/da/d71/d55/f7a 0 2024-06-15T01:06:59.582 INFO:tasks.workunit.client.0.smithi137.stdout:3/350: mkdir d0/d42/d79 0 2024-06-15T01:06:59.583 INFO:tasks.workunit.client.1.smithi142.stdout:8/325: link d0/d25/d5a/c38 d0/d25/d53/c6d 0 2024-06-15T01:06:59.586 INFO:tasks.workunit.client.0.smithi137.stdout:2/225: stat d0/da/f11 0 2024-06-15T01:06:59.586 INFO:tasks.workunit.client.0.smithi137.stdout:0/285: chown d1/d2/dc/d10/d14/d15/d17/d54 348640 1 2024-06-15T01:06:59.593 INFO:tasks.workunit.client.0.smithi137.stdout:0/286: read d1/d2/f2f [52485,121363] 0 2024-06-15T01:06:59.598 INFO:tasks.workunit.client.1.smithi142.stdout:2/292: rename d8/df/d2a to d8/d19/d23/d25/d1a/d57 0 2024-06-15T01:06:59.599 INFO:tasks.workunit.client.1.smithi142.stdout:1/360: readlink d6/d12/d5b/l6e 0 2024-06-15T01:06:59.599 INFO:tasks.workunit.client.1.smithi142.stdout:2/293: dread - d8/df/f48 zero size 2024-06-15T01:06:59.601 INFO:tasks.workunit.client.1.smithi142.stdout:4/455: fdatasync d5/d9/d23/f3b 0 2024-06-15T01:06:59.601 INFO:tasks.workunit.client.1.smithi142.stdout:9/328: mknod d6/dc/d4e/d78/c83 0 2024-06-15T01:06:59.602 INFO:tasks.workunit.client.0.smithi137.stdout:4/275: mknod dd/d15/d21/d3a/d51/d5b/c69 0 2024-06-15T01:06:59.603 INFO:tasks.workunit.client.1.smithi142.stdout:2/294: write d8/d19/d23/d25/d1a/d57/f2b [437227,87259] 0 2024-06-15T01:06:59.603 INFO:tasks.workunit.client.0.smithi137.stdout:9/158: mkdir df/d2a 0 2024-06-15T01:06:59.603 INFO:tasks.workunit.client.1.smithi142.stdout:5/337: rmdir d2/d3e 39 2024-06-15T01:06:59.604 INFO:tasks.workunit.client.0.smithi137.stdout:7/249: creat df/d13/d2e/f4e x:0 0 0 2024-06-15T01:06:59.605 INFO:tasks.workunit.client.0.smithi137.stdout:3/351: stat d0/f1f 0 2024-06-15T01:06:59.605 INFO:tasks.workunit.client.1.smithi142.stdout:3/446: creat d0/d1f/d20/d5d/d60/f84 x:0 0 0 2024-06-15T01:06:59.608 INFO:tasks.workunit.client.1.smithi142.stdout:8/326: readlink d0/l39 0 2024-06-15T01:06:59.609 INFO:tasks.workunit.client.1.smithi142.stdout:7/313: symlink d2/d8/dd/d14/d2c/l6a 0 2024-06-15T01:06:59.611 INFO:tasks.workunit.client.0.smithi137.stdout:9/159: symlink df/d13/l2b 0 2024-06-15T01:06:59.612 INFO:tasks.workunit.client.1.smithi142.stdout:2/295: dwrite d8/df/f48 [0,4194304] 0 2024-06-15T01:06:59.613 INFO:tasks.workunit.client.1.smithi142.stdout:1/361: creat d6/d12/d3c/d63/f85 x:0 0 0 2024-06-15T01:06:59.614 INFO:tasks.workunit.client.0.smithi137.stdout:7/250: dwrite df/d13/d2e/f19 [0,4194304] 0 2024-06-15T01:06:59.615 INFO:tasks.workunit.client.0.smithi137.stdout:3/352: creat d0/d2/d8/f7a x:0 0 0 2024-06-15T01:06:59.618 INFO:tasks.workunit.client.1.smithi142.stdout:9/329: creat d6/d31/f84 x:0 0 0 2024-06-15T01:06:59.622 INFO:tasks.workunit.client.1.smithi142.stdout:4/456: symlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/l9e 0 2024-06-15T01:06:59.626 INFO:tasks.workunit.client.0.smithi137.stdout:7/251: dwrite df/f32 [0,4194304] 0 2024-06-15T01:06:59.626 INFO:tasks.workunit.client.0.smithi137.stdout:0/287: symlink d1/l67 0 2024-06-15T01:06:59.629 INFO:tasks.workunit.client.0.smithi137.stdout:0/288: write d1/d2/dc/d10/d14/d52/f58 [312488,97410] 0 2024-06-15T01:06:59.630 INFO:tasks.workunit.client.1.smithi142.stdout:1/362: rmdir d6/d46/d61 39 2024-06-15T01:06:59.631 INFO:tasks.workunit.client.1.smithi142.stdout:6/321: dread d1a/d39/f3f [0,4194304] 0 2024-06-15T01:06:59.631 INFO:tasks.workunit.client.0.smithi137.stdout:8/212: rename d9/c45 to d9/de/d1e/c48 0 2024-06-15T01:06:59.632 INFO:tasks.workunit.client.0.smithi137.stdout:2/226: write d0/de/f24 [4374482,115713] 0 2024-06-15T01:06:59.632 INFO:tasks.workunit.client.1.smithi142.stdout:9/330: mknod d6/dc/dd/c85 0 2024-06-15T01:06:59.633 INFO:tasks.workunit.client.0.smithi137.stdout:1/293: getdents d0/d3/d1b/d4c/d59 0 2024-06-15T01:06:59.634 INFO:tasks.workunit.client.0.smithi137.stdout:7/252: dread df/d13/d2e/f2b [0,4194304] 0 2024-06-15T01:06:59.635 INFO:tasks.workunit.client.1.smithi142.stdout:1/363: truncate d6/d12/d3c/d63/f85 450956 0 2024-06-15T01:06:59.636 INFO:tasks.workunit.client.1.smithi142.stdout:7/314: mkdir d2/d8/dd/d14/d2c/d37/d6b 0 2024-06-15T01:06:59.636 INFO:tasks.workunit.client.0.smithi137.stdout:1/294: readlink d0/d3/d14/l1a 0 2024-06-15T01:06:59.636 INFO:tasks.workunit.client.0.smithi137.stdout:9/160: truncate f3 2117341 0 2024-06-15T01:06:59.637 INFO:tasks.workunit.client.1.smithi142.stdout:7/315: chown d2/d6/d22/d47/f4f 1 1 2024-06-15T01:06:59.637 INFO:tasks.workunit.client.1.smithi142.stdout:6/322: fsync d1a/d2d/f52 0 2024-06-15T01:06:59.638 INFO:tasks.workunit.client.0.smithi137.stdout:9/161: dread - df/f28 zero size 2024-06-15T01:06:59.639 INFO:tasks.workunit.client.1.smithi142.stdout:4/457: fsync d5/d9/d18/f80 0 2024-06-15T01:06:59.640 INFO:tasks.workunit.client.0.smithi137.stdout:3/353: mkdir d0/d2/d50/d6c/d7b 0 2024-06-15T01:06:59.640 INFO:tasks.workunit.client.1.smithi142.stdout:9/331: rename d6/dc/dd/d22/f41 to d6/d31/f86 0 2024-06-15T01:06:59.641 INFO:tasks.workunit.client.0.smithi137.stdout:0/289: symlink d1/d2/dc/d10/d14/d15/d17/d54/l68 0 2024-06-15T01:06:59.641 INFO:tasks.workunit.client.0.smithi137.stdout:1/295: write d0/d10/f6f [923185,85430] 0 2024-06-15T01:06:59.642 INFO:tasks.workunit.client.1.smithi142.stdout:7/316: readlink d2/d6/d22/d47/l5e 0 2024-06-15T01:06:59.644 INFO:tasks.workunit.client.1.smithi142.stdout:1/364: rmdir d6/d46/d61 39 2024-06-15T01:06:59.644 INFO:tasks.workunit.client.0.smithi137.stdout:7/253: creat df/d13/d2e/f4f x:0 0 0 2024-06-15T01:06:59.645 INFO:tasks.workunit.client.1.smithi142.stdout:2/296: dread d8/d19/d23/d25/f18 [0,4194304] 0 2024-06-15T01:06:59.646 INFO:tasks.workunit.client.1.smithi142.stdout:6/323: symlink d1a/d39/d3a/d51/l71 0 2024-06-15T01:06:59.646 INFO:tasks.workunit.client.0.smithi137.stdout:3/354: truncate d0/d2/d1e/f6e 209758 0 2024-06-15T01:06:59.646 INFO:tasks.workunit.client.0.smithi137.stdout:8/213: symlink d9/de/d1e/l49 0 2024-06-15T01:06:59.647 INFO:tasks.workunit.client.0.smithi137.stdout:2/227: dwrite d0/de/f24 [0,4194304] 0 2024-06-15T01:06:59.647 INFO:tasks.workunit.client.1.smithi142.stdout:6/324: rename d1a/d39 to d1a/d39/d3a/d72 22 2024-06-15T01:06:59.648 INFO:tasks.workunit.client.1.smithi142.stdout:4/458: fdatasync d5/d9/d1a/f6a 0 2024-06-15T01:06:59.648 INFO:tasks.workunit.client.0.smithi137.stdout:1/296: chown d0/d3/d1b/d4c/d59/d64/f6e 94999584 1 2024-06-15T01:06:59.649 INFO:tasks.workunit.client.0.smithi137.stdout:3/355: chown d0/d4c 321847 1 2024-06-15T01:06:59.650 INFO:tasks.workunit.client.1.smithi142.stdout:9/332: creat d6/dc/d1b/d25/d58/f87 x:0 0 0 2024-06-15T01:06:59.650 INFO:tasks.workunit.client.0.smithi137.stdout:2/228: write d0/dc/d22/f39 [665266,49657] 0 2024-06-15T01:06:59.651 INFO:tasks.workunit.client.0.smithi137.stdout:3/356: stat d0/d2/d50/d6c/d7b 0 2024-06-15T01:06:59.652 INFO:tasks.workunit.client.0.smithi137.stdout:7/254: mknod df/c50 0 2024-06-15T01:06:59.653 INFO:tasks.workunit.client.0.smithi137.stdout:8/214: dread d9/f20 [0,4194304] 0 2024-06-15T01:06:59.654 INFO:tasks.workunit.client.1.smithi142.stdout:3/447: rmdir d0/d1f/d20/d5d/d60 39 2024-06-15T01:06:59.654 INFO:tasks.workunit.client.0.smithi137.stdout:7/255: write df/d13/d2e/f19 [4835969,44915] 0 2024-06-15T01:06:59.655 INFO:tasks.workunit.client.1.smithi142.stdout:2/297: mknod d8/d19/d23/d25/d22/c58 0 2024-06-15T01:06:59.656 INFO:tasks.workunit.client.0.smithi137.stdout:1/297: creat d0/d10/f72 x:0 0 0 2024-06-15T01:06:59.658 INFO:tasks.workunit.client.0.smithi137.stdout:2/229: rmdir d0/dc/d22 39 2024-06-15T01:06:59.659 INFO:tasks.workunit.client.1.smithi142.stdout:9/333: dwrite d6/dc/d1b/d25/d58/f87 [0,4194304] 0 2024-06-15T01:06:59.660 INFO:tasks.workunit.client.0.smithi137.stdout:1/298: chown d0/d10/f6f 55792 1 2024-06-15T01:06:59.660 INFO:tasks.workunit.client.0.smithi137.stdout:9/162: dread f3 [0,4194304] 0 2024-06-15T01:06:59.661 INFO:tasks.workunit.client.0.smithi137.stdout:1/299: stat d0/d3/d14/c3f 0 2024-06-15T01:06:59.662 INFO:tasks.workunit.client.1.smithi142.stdout:7/317: link d2/d8/da/d33/d25/f67 d2/d6/d22/f6c 0 2024-06-15T01:06:59.663 INFO:tasks.workunit.client.0.smithi137.stdout:7/256: creat df/d13/d2e/f51 x:0 0 0 2024-06-15T01:06:59.663 INFO:tasks.workunit.client.0.smithi137.stdout:2/230: read d0/f3 [62248,85744] 0 2024-06-15T01:06:59.664 INFO:tasks.workunit.client.0.smithi137.stdout:3/357: getdents d0/d42/d79 0 2024-06-15T01:06:59.664 INFO:tasks.workunit.client.1.smithi142.stdout:0/329: dwrite d3/f23 [0,4194304] 0 2024-06-15T01:06:59.665 INFO:tasks.workunit.client.0.smithi137.stdout:0/290: truncate d1/d2/d13/d66/f3f 2085972 0 2024-06-15T01:06:59.665 INFO:tasks.workunit.client.1.smithi142.stdout:0/330: stat d3/f23 0 2024-06-15T01:06:59.666 INFO:tasks.workunit.client.0.smithi137.stdout:3/358: fsync d0/d2/d1e/f5c 0 2024-06-15T01:06:59.669 INFO:tasks.workunit.client.0.smithi137.stdout:9/163: rmdir df 39 2024-06-15T01:06:59.669 INFO:tasks.workunit.client.1.smithi142.stdout:6/325: rename d1a/d39/d42/c54 to d1a/d2d/d3b/c73 0 2024-06-15T01:06:59.670 INFO:tasks.workunit.client.1.smithi142.stdout:8/327: write d0/d17/f6b [126187,117133] 0 2024-06-15T01:06:59.673 INFO:tasks.workunit.client.1.smithi142.stdout:4/459: symlink d5/d1c/d2a/d30/d46/d50/d51/d5b/l9f 0 2024-06-15T01:06:59.673 INFO:tasks.workunit.client.1.smithi142.stdout:5/338: write d2/f21 [501732,53509] 0 2024-06-15T01:06:59.676 INFO:tasks.workunit.client.0.smithi137.stdout:7/257: write df/d18/d1c/d3a/f37 [5051318,19291] 0 2024-06-15T01:06:59.676 INFO:tasks.workunit.client.1.smithi142.stdout:7/318: creat d2/d6/f6d x:0 0 0 2024-06-15T01:06:59.677 INFO:tasks.workunit.client.0.smithi137.stdout:2/231: dwrite d0/de/f16 [0,4194304] 0 2024-06-15T01:06:59.679 INFO:tasks.workunit.client.1.smithi142.stdout:8/328: dread d0/d17/f1f [0,4194304] 0 2024-06-15T01:06:59.680 INFO:tasks.workunit.client.1.smithi142.stdout:6/326: creat d1a/d39/d42/f74 x:0 0 0 2024-06-15T01:06:59.680 INFO:tasks.workunit.client.0.smithi137.stdout:9/164: chown df/d12/f23 232678987 1 2024-06-15T01:06:59.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/258: truncate df/d13/d2e/d1a/f2c 442383 0 2024-06-15T01:06:59.683 INFO:tasks.workunit.client.0.smithi137.stdout:9/165: creat df/d12/f2c x:0 0 0 2024-06-15T01:06:59.684 INFO:tasks.workunit.client.1.smithi142.stdout:5/339: creat d2/de/d19/f6a x:0 0 0 2024-06-15T01:06:59.685 INFO:tasks.workunit.client.0.smithi137.stdout:2/232: mkdir d0/dc/d40 0 2024-06-15T01:06:59.685 INFO:tasks.workunit.client.1.smithi142.stdout:1/365: getdents d6/d3a 0 2024-06-15T01:06:59.686 INFO:tasks.workunit.client.1.smithi142.stdout:1/366: dread - d6/d3a/f43 zero size 2024-06-15T01:06:59.687 INFO:tasks.workunit.client.1.smithi142.stdout:2/298: link d8/f40 d8/df/f59 0 2024-06-15T01:06:59.688 INFO:tasks.workunit.client.1.smithi142.stdout:7/319: truncate d2/d8/da/d33/f35 661617 0 2024-06-15T01:06:59.689 INFO:tasks.workunit.client.0.smithi137.stdout:0/291: getdents d1 0 2024-06-15T01:06:59.690 INFO:tasks.workunit.client.0.smithi137.stdout:2/233: mkdir d0/de/d41 0 2024-06-15T01:06:59.692 INFO:tasks.workunit.client.0.smithi137.stdout:8/215: dread d9/de/d1e/f27 [0,4194304] 0 2024-06-15T01:06:59.694 INFO:tasks.workunit.client.1.smithi142.stdout:9/334: link d6/dc/dd/d22/c2f d6/dc/d1b/c88 0 2024-06-15T01:06:59.694 INFO:tasks.workunit.client.1.smithi142.stdout:9/335: chown d6/dc/dd/c20 2320 1 2024-06-15T01:06:59.695 INFO:tasks.workunit.client.1.smithi142.stdout:9/336: chown d6/dc/d1b/d25/d58/c7d 105 1 2024-06-15T01:06:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:2/234: mkdir d0/d42 0 2024-06-15T01:06:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:9/166: truncate df/d12/d15/f1d 2741531 0 2024-06-15T01:06:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:1/300: dread d0/d3/d1b/d4c/f5c [0,4194304] 0 2024-06-15T01:06:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:7/259: dwrite df/f3d [0,4194304] 0 2024-06-15T01:06:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:2/235: readlink d0/d1c/l34 0 2024-06-15T01:06:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:3/359: truncate d0/d2/d23/f3d 3342334 0 2024-06-15T01:06:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:1/301: write d0/d2d/f62 [19241,75673] 0 2024-06-15T01:06:59.702 INFO:tasks.workunit.client.0.smithi137.stdout:7/260: truncate df/d13/d2e/f51 605333 0 2024-06-15T01:06:59.704 INFO:tasks.workunit.client.1.smithi142.stdout:2/299: mknod d8/d19/d23/d25/d22/c5a 0 2024-06-15T01:06:59.705 INFO:tasks.workunit.client.1.smithi142.stdout:2/300: fsync d8/d19/d23/d25/f18 0 2024-06-15T01:06:59.706 INFO:tasks.workunit.client.1.smithi142.stdout:0/331: link d3/d39/d3e/d81/d1e/c78 d3/d39/d3e/d81/d44/c83 0 2024-06-15T01:06:59.708 INFO:tasks.workunit.client.0.smithi137.stdout:2/236: rename d0/c7 to d0/dc/d40/c43 0 2024-06-15T01:06:59.709 INFO:tasks.workunit.client.0.smithi137.stdout:1/302: mkdir d0/d3/d47/d73 0 2024-06-15T01:06:59.710 INFO:tasks.workunit.client.0.smithi137.stdout:2/237: write d0/de/f24 [1637606,98616] 0 2024-06-15T01:06:59.711 INFO:tasks.workunit.client.1.smithi142.stdout:0/332: dwrite d3/d39/d3e/d81/f1d [0,4194304] 0 2024-06-15T01:06:59.715 INFO:tasks.workunit.client.1.smithi142.stdout:9/337: rename d6/d34/c7c to d6/dc/d4e/d65/c89 0 2024-06-15T01:06:59.715 INFO:tasks.workunit.client.0.smithi137.stdout:8/216: write d9/de/ff [1674244,1724] 0 2024-06-15T01:06:59.716 INFO:tasks.workunit.client.0.smithi137.stdout:0/292: read d1/d2/d13/d66/f3f [1510180,117903] 0 2024-06-15T01:06:59.716 INFO:tasks.workunit.client.0.smithi137.stdout:2/238: symlink d0/de/l44 0 2024-06-15T01:06:59.718 INFO:tasks.workunit.client.0.smithi137.stdout:1/303: creat d0/d1f/d60/f74 x:0 0 0 2024-06-15T01:06:59.722 INFO:tasks.workunit.client.0.smithi137.stdout:7/261: dwrite df/d13/d2e/f2b [0,4194304] 0 2024-06-15T01:06:59.723 INFO:tasks.workunit.client.0.smithi137.stdout:7/262: stat df/f14 0 2024-06-15T01:06:59.724 INFO:tasks.workunit.client.0.smithi137.stdout:8/217: unlink d9/f47 0 2024-06-15T01:06:59.725 INFO:tasks.workunit.client.1.smithi142.stdout:4/460: dwrite d5/d9/d41/f4c [4194304,4194304] 0 2024-06-15T01:06:59.725 INFO:tasks.workunit.client.0.smithi137.stdout:2/239: dread d0/de/f16 [0,4194304] 0 2024-06-15T01:06:59.726 INFO:tasks.workunit.client.1.smithi142.stdout:8/329: truncate d0/d17/f56 603471 0 2024-06-15T01:06:59.727 INFO:tasks.workunit.client.1.smithi142.stdout:9/338: write d6/dc/d1b/d25/d58/f64 [143908,109084] 0 2024-06-15T01:06:59.729 INFO:tasks.workunit.client.1.smithi142.stdout:6/327: truncate d1a/f1e 3987518 0 2024-06-15T01:06:59.731 INFO:tasks.workunit.client.1.smithi142.stdout:6/328: write d1a/d39/d3a/f5f [58624,66388] 0 2024-06-15T01:06:59.731 INFO:tasks.workunit.client.1.smithi142.stdout:7/320: rmdir d2/d6/d4a 0 2024-06-15T01:06:59.732 INFO:tasks.workunit.client.0.smithi137.stdout:1/304: creat d0/d3/d1b/d4c/f75 x:0 0 0 2024-06-15T01:06:59.733 INFO:tasks.workunit.client.1.smithi142.stdout:8/330: dread d0/d25/d53/f58 [0,4194304] 0 2024-06-15T01:06:59.733 INFO:tasks.workunit.client.1.smithi142.stdout:6/329: truncate d1a/d39/d3a/f5a 497857 0 2024-06-15T01:06:59.734 INFO:tasks.workunit.client.0.smithi137.stdout:8/218: symlink d9/d2a/d2f/l4a 0 2024-06-15T01:06:59.736 INFO:tasks.workunit.client.0.smithi137.stdout:8/219: write d9/de/d1e/f28 [482791,11195] 0 2024-06-15T01:06:59.736 INFO:tasks.workunit.client.0.smithi137.stdout:2/240: mknod d0/dc/d40/c45 0 2024-06-15T01:06:59.737 INFO:tasks.workunit.client.1.smithi142.stdout:4/461: creat d5/d9/da/d7e/fa0 x:0 0 0 2024-06-15T01:06:59.737 INFO:tasks.workunit.client.0.smithi137.stdout:1/305: mkdir d0/d1f/d76 0 2024-06-15T01:06:59.738 INFO:tasks.workunit.client.1.smithi142.stdout:4/462: truncate d5/d9/da/d7e/f56 1115281 0 2024-06-15T01:06:59.739 INFO:tasks.workunit.client.1.smithi142.stdout:2/301: mknod d8/d19/d23/d25/d22/d43/d51/c5b 0 2024-06-15T01:06:59.739 INFO:tasks.workunit.client.0.smithi137.stdout:7/263: rename f1 to df/d18/d1c/d1d/d27/d33/f52 0 2024-06-15T01:06:59.740 INFO:tasks.workunit.client.1.smithi142.stdout:5/340: truncate d2/de/d19/f2a 1041056 0 2024-06-15T01:06:59.740 INFO:tasks.workunit.client.0.smithi137.stdout:0/293: creat d1/d2/dc/d10/d14/d15/d17/f69 x:0 0 0 2024-06-15T01:06:59.741 INFO:tasks.workunit.client.1.smithi142.stdout:4/463: write d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/f6d [3086436,95788] 0 2024-06-15T01:06:59.741 INFO:tasks.workunit.client.0.smithi137.stdout:8/220: unlink d9/de/d1e/f39 0 2024-06-15T01:06:59.741 INFO:tasks.workunit.client.1.smithi142.stdout:0/333: unlink d3/cc 0 2024-06-15T01:06:59.742 INFO:tasks.workunit.client.1.smithi142.stdout:0/334: write d3/f4 [7996912,8305] 0 2024-06-15T01:06:59.744 INFO:tasks.workunit.client.0.smithi137.stdout:8/221: read d9/d22/f37 [201415,13872] 0 2024-06-15T01:06:59.744 INFO:tasks.workunit.client.1.smithi142.stdout:1/367: getdents d6/d46 0 2024-06-15T01:06:59.745 INFO:tasks.workunit.client.1.smithi142.stdout:9/339: write d6/f9 [1945090,2002] 0 2024-06-15T01:06:59.746 INFO:tasks.workunit.client.1.smithi142.stdout:8/331: mkdir d0/d17/d40/d6e 0 2024-06-15T01:06:59.747 INFO:tasks.workunit.client.0.smithi137.stdout:2/241: mknod d0/de/d10/d3a/c46 0 2024-06-15T01:06:59.747 INFO:tasks.workunit.client.1.smithi142.stdout:6/330: symlink d1a/d39/d3a/l75 0 2024-06-15T01:06:59.748 INFO:tasks.workunit.client.0.smithi137.stdout:7/264: symlink df/d13/d2e/d1a/d3f/l53 0 2024-06-15T01:06:59.752 INFO:tasks.workunit.client.1.smithi142.stdout:8/332: dread d0/d17/f1f [0,4194304] 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:1/368: symlink d6/d12/d52/l86 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:6/331: dread d1a/d39/d42/d4c/d58/f5d [4194304,4194304] 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:5/341: rmdir d2/de/d19 39 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:5/342: readlink d2/de/l60 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:1/369: symlink d6/d12/d3c/d63/d65/l87 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:5/343: dwrite d2/f14 [0,4194304] 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:4/464: mkdir d5/d9/d90/da1 0 2024-06-15T01:06:59.882 INFO:tasks.workunit.client.1.smithi142.stdout:5/344: dwrite d2/de/f50 [0,4194304] 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:1/370: creat d6/d3a/f88 x:0 0 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:8/333: creat d0/f6f x:0 0 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:9/340: dwrite d6/dc/f19 [0,4194304] 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:6/332: truncate d1a/d2d/f32 101775 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:9/341: read d6/dc/d1b/d25/d35/f5c [3210051,110895] 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:1/371: creat d6/d12/d52/f89 x:0 0 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:4/465: symlink d5/d9/d23/d32/la2 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:6/333: rmdir d1a/d39/d42/d4c/d58 39 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:4/466: mkdir d5/d9/da/da3 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:9/342: dwrite d6/dc/dd/f77 [0,4194304] 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:1/372: getdents d6/d12/d30 0 2024-06-15T01:06:59.883 INFO:tasks.workunit.client.1.smithi142.stdout:8/334: dwrite d0/d17/f6b [0,4194304] 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:4/467: unlink d5/d9/d23/c9b 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:1/373: truncate d6/d3a/f88 445509 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:6/334: creat d1a/f76 x:0 0 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:8/335: stat d0/f18 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:9/343: mkdir d6/d34/d8a 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:1/374: creat d6/d12/d5b/f8a x:0 0 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:6/335: mknod d1a/d2d/d2e/c77 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:4/468: creat d5/d1c/d2a/d30/d46/fa4 x:0 0 0 2024-06-15T01:06:59.884 INFO:tasks.workunit.client.1.smithi142.stdout:4/469: chown d5/d9/da/l67 134814829 1 2024-06-15T01:06:59.885 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:59 smithi142 ceph-mon[37379]: pgmap v15: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 91 MiB/s wr, 167 op/s 2024-06-15T01:06:59.885 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:06:59 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3037160451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:06:59.889 INFO:tasks.workunit.client.0.smithi137.stdout:8/222: dwrite d9/de/d1e/d3f/f41 [0,4194304] 0 2024-06-15T01:06:59.889 INFO:tasks.workunit.client.0.smithi137.stdout:2/242: write d0/dc/d22/f39 [1344594,41241] 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:8/223: creat d9/d2a/d2f/f4b x:0 0 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:7/265: rename df/d13/d2e/f2b to df/d13/d2e/d1a/d3f/f54 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:2/243: mkdir d0/de/d2b/d47 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:0/294: dread d1/d2/d13/d66/f3f [0,4194304] 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:1/306: write d0/f2 [573525,117962] 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:7/266: symlink df/d13/d3c/l55 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:8/224: dwrite f8 [0,4194304] 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:0/295: dread d1/d2/dc/d10/d14/f39 [0,4194304] 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:1/307: creat d0/d2d/d3e/f77 x:0 0 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:0/296: unlink d1/d2/c43 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:8/225: mknod d9/c4c 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:1/308: mknod d0/d10/c78 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:7/267: getdents df/d18 0 2024-06-15T01:06:59.890 INFO:tasks.workunit.client.0.smithi137.stdout:2/244: link d0/de/d10/l38 d0/da/l48 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:1/309: dread d0/d3/f5 [0,4194304] 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/226: symlink d9/de/d1e/l4d 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/297: dwrite d1/d2/f47 [4194304,4194304] 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:7/268: chown df/d13/d2e/d1a/f30 34091 1 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/298: chown d1/d2/l49 3 1 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/227: mkdir d9/d2a/d2f/d4e 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/299: readlink d1/d2/dc/d10/d14/d15/l37 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/228: dwrite d9/d2a/f2d [0,4194304] 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/229: dwrite f6 [0,4194304] 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/230: mknod d9/de/d1e/c4f 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/231: mknod d9/d22/c50 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/232: symlink d9/de/d1e/l51 0 2024-06-15T01:06:59.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/300: write d1/d2/dc/d10/d14/f39 [2892944,97283] 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/233: readlink d9/de/d1e/l51 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:0/301: fdatasync d1/d2/d13/d66/f4a 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/234: chown d9/f20 27191863 1 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:0/302: rename d1/d2/dc/d10 to d1/d2/dc/d10/d14/d56/d6a 22 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:1/310: dwrite d0/f3b [0,4194304] 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/269: dread df/d18/d1c/d1d/d27/f48 [0,4194304] 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:0/303: fdatasync d1/d2/dc/d10/d14/d15/d17/f69 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/270: unlink df/f38 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/271: fsync df/d13/d2e/f4e 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/235: dwrite d9/de/f13 [0,4194304] 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/272: write df/d13/d2e/f4f [332843,111416] 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/236: symlink d9/l52 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/273: mknod df/d13/d2e/d1a/c56 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/237: truncate d9/d22/f3e 481494 0 2024-06-15T01:06:59.892 INFO:tasks.workunit.client.0.smithi137.stdout:7/274: mknod df/d18/d1c/d1d/d27/d33/d43/c57 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/238: mknod d9/de/d1e/d35/c53 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/239: getdents d9/de 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/240: read - d9/de/d1e/d35/f38 zero size 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/241: truncate d9/d2a/f34 1006946 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/242: truncate d9/de/f13 5653684 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/243: dread d9/d2a/f2d [0,4194304] 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:8/244: unlink d9/f20 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:1/311: dwrite d0/d3/d1b/d4c/d59/d64/f6e [0,4194304] 0 2024-06-15T01:06:59.893 INFO:tasks.workunit.client.0.smithi137.stdout:0/304: dwrite d1/d2/dc/d10/d14/d15/d1c/f46 [0,4194304] 0 2024-06-15T01:06:59.894 INFO:tasks.workunit.client.1.smithi142.stdout:9/344: write d6/dc/d1b/d25/d35/f5c [4723390,41606] 0 2024-06-15T01:06:59.894 INFO:tasks.workunit.client.1.smithi142.stdout:1/375: symlink d6/d12/d5b/l8b 0 2024-06-15T01:06:59.894 INFO:tasks.workunit.client.1.smithi142.stdout:8/336: symlink d0/l70 0 2024-06-15T01:06:59.894 INFO:tasks.workunit.client.1.smithi142.stdout:9/345: write d6/dc/dd/d22/f54 [507709,88324] 0 2024-06-15T01:06:59.894 INFO:tasks.workunit.client.1.smithi142.stdout:4/470: rmdir d5/d1c/d2a/d30/d46/d50/d51/d5b/d20 39 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:4/471: dread - d5/d9/d41/f65 zero size 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/376: symlink d6/d3a/l8c 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:8/337: unlink d0/d17/l3e 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/336: mknod d1a/d39/d42/d4c/c78 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/377: dwrite d6/d12/d28/f82 [0,4194304] 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:4/472: symlink d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/la5 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/337: chown d1a/d3c/l3d 184890146 1 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:5/345: fsync d2/de/d19/f2a 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/378: mkdir d6/d12/d3c/d63/d8d 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:9/346: creat d6/d34/f8b x:0 0 0 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/379: dread - d6/d12/d5b/f8a zero size 2024-06-15T01:06:59.895 INFO:tasks.workunit.client.1.smithi142.stdout:8/338: dread d0/f23 [4194304,4194304] 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:5/346: mknod d2/de/d19/c6b 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:5/347: chown d2/d18/l29 13157099 1 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:5/348: stat d2/f25 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:1/380: mknod d6/d12/d3c/d63/c8e 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:6/338: creat d1a/d39/d3a/d51/d62/f79 x:0 0 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:9/347: write d6/dc/d1b/d25/d35/f5c [4482871,110884] 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:1/381: symlink d6/d46/l8f 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:1/382: creat d6/f90 x:0 0 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:1/383: write d6/d12/d1c/f53 [1231046,90718] 0 2024-06-15T01:06:59.896 INFO:tasks.workunit.client.1.smithi142.stdout:8/339: link d0/d25/d5a/c32 d0/d25/d5a/c71 0 2024-06-15T01:06:59.897 INFO:tasks.workunit.client.1.smithi142.stdout:5/349: truncate d2/f9 7050937 0 2024-06-15T01:06:59.897 INFO:tasks.workunit.client.1.smithi142.stdout:4/473: dwrite d5/d9/d41/f36 [0,4194304] 0 2024-06-15T01:06:59.897 INFO:tasks.workunit.client.0.smithi137.stdout:8/245: truncate f6 723064 0 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:8/246: dread - d9/de/d1e/d35/f38 zero size 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:1/312: creat d0/d2d/f79 x:0 0 0 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:1/313: write d0/d10/f72 [636584,127351] 0 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:8/247: creat d9/d2a/d2f/f54 x:0 0 0 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:0/305: creat d1/d2/dc/d10/d63/f6b x:0 0 0 2024-06-15T01:06:59.898 INFO:tasks.workunit.client.0.smithi137.stdout:0/306: stat d1/d2/dc/d10/d22 0 2024-06-15T01:06:59.899 INFO:tasks.workunit.client.1.smithi142.stdout:4/474: fsync d5/d9/da/d7e/f56 0 2024-06-15T01:06:59.899 INFO:tasks.workunit.client.1.smithi142.stdout:9/348: write d6/dc/dd/f46 [65897,25950] 0 2024-06-15T01:06:59.899 INFO:tasks.workunit.client.0.smithi137.stdout:0/307: fdatasync d1/d2/dc/d10/d14/d15/f4b 0 2024-06-15T01:06:59.902 INFO:tasks.workunit.client.0.smithi137.stdout:8/248: mkdir d9/d55 0 2024-06-15T01:06:59.903 INFO:tasks.workunit.client.0.smithi137.stdout:1/314: getdents d0/d3/d1b/d4c/d59/d66 0 2024-06-15T01:06:59.910 INFO:tasks.workunit.client.0.smithi137.stdout:0/308: dwrite d1/d2/dc/d10/d14/d15/d1c/f46 [0,4194304] 0 2024-06-15T01:06:59.911 INFO:tasks.workunit.client.1.smithi142.stdout:4/475: dwrite d5/d9/d23/f3b [4194304,4194304] 0 2024-06-15T01:06:59.912 INFO:tasks.workunit.client.1.smithi142.stdout:1/384: link d6/d46/f78 d6/d12/d1c/d2c/f91 0 2024-06-15T01:06:59.912 INFO:tasks.workunit.client.1.smithi142.stdout:5/350: dread d2/d3e/d31/f3c [0,4194304] 0 2024-06-15T01:06:59.914 INFO:tasks.workunit.client.1.smithi142.stdout:5/351: write d2/de/d19/f22 [9366701,97455] 0 2024-06-15T01:06:59.915 INFO:tasks.workunit.client.0.smithi137.stdout:0/309: mkdir d1/d2/dc/d6c 0 2024-06-15T01:06:59.915 INFO:tasks.workunit.client.0.smithi137.stdout:8/249: dwrite d9/d2a/f2d [0,4194304] 0 2024-06-15T01:06:59.921 INFO:tasks.workunit.client.1.smithi142.stdout:1/385: dwrite d6/fc [0,4194304] 0 2024-06-15T01:06:59.921 INFO:tasks.workunit.client.1.smithi142.stdout:5/352: read d2/f28 [688461,57698] 0 2024-06-15T01:06:59.922 INFO:tasks.workunit.client.0.smithi137.stdout:0/310: mknod d1/d2/d13/d66/c6d 0 2024-06-15T01:06:59.923 INFO:tasks.workunit.client.1.smithi142.stdout:5/353: stat d2/de/d27/f56 0 2024-06-15T01:06:59.924 INFO:tasks.workunit.client.0.smithi137.stdout:0/311: read d1/d2/dc/d10/d14/d15/d1c/f1d [3106543,47832] 0 2024-06-15T01:06:59.924 INFO:tasks.workunit.client.1.smithi142.stdout:1/386: write d6/d12/d28/f33 [9476128,5653] 0 2024-06-15T01:06:59.924 INFO:tasks.workunit.client.1.smithi142.stdout:1/387: chown d6/f13 19238 1 2024-06-15T01:06:59.926 INFO:tasks.workunit.client.0.smithi137.stdout:8/250: dread d9/d22/f37 [0,4194304] 0 2024-06-15T01:06:59.929 INFO:tasks.workunit.client.0.smithi137.stdout:1/315: dread d0/d10/d41/f4b [0,4194304] 0 2024-06-15T01:06:59.930 INFO:tasks.workunit.client.1.smithi142.stdout:1/388: symlink d6/d12/d3c/l92 0 2024-06-15T01:06:59.933 INFO:tasks.workunit.client.1.smithi142.stdout:1/389: mkdir d6/d12/d1c/d2c/d93 0 2024-06-15T01:06:59.934 INFO:tasks.workunit.client.1.smithi142.stdout:1/390: write d6/d12/d3c/d63/f85 [1076945,90001] 0 2024-06-15T01:06:59.935 INFO:tasks.workunit.client.0.smithi137.stdout:0/312: dwrite d1/d2/dc/d10/d14/d15/f35 [0,4194304] 0 2024-06-15T01:06:59.936 INFO:tasks.workunit.client.0.smithi137.stdout:1/316: rename d0/d3/d1b/l33 to d0/d2d/l7a 0 2024-06-15T01:06:59.936 INFO:tasks.workunit.client.0.smithi137.stdout:0/313: fsync d1/d2/dc/d10/d14/d15/d17/f69 0 2024-06-15T01:06:59.940 INFO:tasks.workunit.client.1.smithi142.stdout:1/391: dwrite d6/d25/f77 [0,4194304] 0 2024-06-15T01:06:59.941 INFO:tasks.workunit.client.0.smithi137.stdout:0/314: write d1/f9 [3844376,33196] 0 2024-06-15T01:06:59.941 INFO:tasks.workunit.client.0.smithi137.stdout:8/251: dwrite d9/d2a/d2f/f4b [0,4194304] 0 2024-06-15T01:06:59.945 INFO:tasks.workunit.client.1.smithi142.stdout:1/392: mknod d6/d12/d1c/d2c/c94 0 2024-06-15T01:06:59.945 INFO:tasks.workunit.client.0.smithi137.stdout:0/315: chown d1/d2/dc/d10/d14/d15/d17/f4c 0 1 2024-06-15T01:06:59.949 INFO:tasks.workunit.client.0.smithi137.stdout:8/252: creat d9/de/d1e/d35/f56 x:0 0 0 2024-06-15T01:06:59.951 INFO:tasks.workunit.client.0.smithi137.stdout:1/317: dwrite d0/d1f/f4d [0,4194304] 0 2024-06-15T01:06:59.956 INFO:tasks.workunit.client.0.smithi137.stdout:0/316: creat d1/d2/d55/f6e x:0 0 0 2024-06-15T01:06:59.956 INFO:tasks.workunit.client.0.smithi137.stdout:8/253: creat d9/d2a/d2f/d4e/f57 x:0 0 0 2024-06-15T01:06:59.956 INFO:tasks.workunit.client.0.smithi137.stdout:1/318: mkdir d0/d10/d41/d7b 0 2024-06-15T01:06:59.956 INFO:tasks.workunit.client.1.smithi142.stdout:8/340: write d0/f3f [3711299,71467] 0 2024-06-15T01:06:59.961 INFO:tasks.workunit.client.0.smithi137.stdout:1/319: mknod d0/d10/d41/d7b/c7c 0 2024-06-15T01:06:59.962 INFO:tasks.workunit.client.1.smithi142.stdout:8/341: dwrite d0/d17/f2f [0,4194304] 0 2024-06-15T01:06:59.964 INFO:tasks.workunit.client.0.smithi137.stdout:1/320: mkdir d0/d1f/d7d 0 2024-06-15T01:06:59.964 INFO:tasks.workunit.client.0.smithi137.stdout:0/317: dwrite d1/d2/dc/d10/d14/d15/f4b [0,4194304] 0 2024-06-15T01:06:59.966 INFO:tasks.workunit.client.1.smithi142.stdout:8/342: creat d0/d17/d40/d6e/f72 x:0 0 0 2024-06-15T01:06:59.968 INFO:tasks.workunit.client.0.smithi137.stdout:5/457: sync 2024-06-15T01:06:59.968 INFO:tasks.workunit.client.0.smithi137.stdout:0/318: symlink d1/d2/dc/d10/d53/l6f 0 2024-06-15T01:06:59.970 INFO:tasks.workunit.client.1.smithi142.stdout:8/343: creat d0/d17/d40/d6e/f73 x:0 0 0 2024-06-15T01:06:59.971 INFO:tasks.workunit.client.1.smithi142.stdout:9/349: truncate d6/d31/f32 238401 0 2024-06-15T01:06:59.972 INFO:tasks.workunit.client.1.smithi142.stdout:8/344: chown d0/d17/d40/d6e/f72 57 1 2024-06-15T01:06:59.972 INFO:tasks.workunit.client.1.smithi142.stdout:9/350: readlink d6/dc/l39 0 2024-06-15T01:06:59.974 INFO:tasks.workunit.client.0.smithi137.stdout:0/319: chown d1/d2/f21 60 1 2024-06-15T01:06:59.974 INFO:tasks.workunit.client.0.smithi137.stdout:5/458: rmdir df/d44/d54 39 2024-06-15T01:06:59.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:59 smithi137 ceph-mon[29354]: pgmap v15: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 91 MiB/s wr, 167 op/s 2024-06-15T01:06:59.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:06:59 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3037160451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:06:59.976 INFO:tasks.workunit.client.0.smithi137.stdout:1/321: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-15T01:06:59.979 INFO:tasks.workunit.client.0.smithi137.stdout:0/320: rename d1/l1a to d1/d2/dc/l70 0 2024-06-15T01:06:59.979 INFO:tasks.workunit.client.1.smithi142.stdout:9/351: dwrite d6/d34/f5f [0,4194304] 0 2024-06-15T01:06:59.981 INFO:tasks.workunit.client.0.smithi137.stdout:5/459: write df/d44/d54/d95/d3b/d49/d5c/f91 [311341,22491] 0 2024-06-15T01:06:59.983 INFO:tasks.workunit.client.0.smithi137.stdout:5/460: write df/d11/f31 [597847,27634] 0 2024-06-15T01:06:59.983 INFO:tasks.workunit.client.1.smithi142.stdout:9/352: chown d6/d31/l67 386457 1 2024-06-15T01:06:59.986 INFO:tasks.workunit.client.0.smithi137.stdout:1/322: link d0/d2d/l48 d0/d1f/d67/l7e 0 2024-06-15T01:06:59.987 INFO:tasks.workunit.client.1.smithi142.stdout:9/353: symlink d6/d34/d8a/l8c 0 2024-06-15T01:06:59.988 INFO:tasks.workunit.client.0.smithi137.stdout:5/461: mkdir df/d76/d7c/d9c 0 2024-06-15T01:06:59.989 INFO:tasks.workunit.client.0.smithi137.stdout:5/462: chown df/d44/d54/d95/c48 275984860 1 2024-06-15T01:06:59.990 INFO:tasks.workunit.client.0.smithi137.stdout:1/323: read d0/d3/d14/d34/f52 [605016,113111] 0 2024-06-15T01:06:59.991 INFO:tasks.workunit.client.0.smithi137.stdout:1/324: fsync d0/d2d/d3e/f77 0 2024-06-15T01:06:59.992 INFO:tasks.workunit.client.0.smithi137.stdout:5/463: symlink df/d76/d7c/d9c/l9d 0 2024-06-15T01:06:59.992 INFO:tasks.workunit.client.0.smithi137.stdout:5/464: fsync df/f2a 0 2024-06-15T01:06:59.994 INFO:tasks.workunit.client.0.smithi137.stdout:5/465: rmdir df/d76 39 2024-06-15T01:06:59.994 INFO:tasks.workunit.client.0.smithi137.stdout:1/325: symlink d0/d1f/d76/l7f 0 2024-06-15T01:06:59.995 INFO:tasks.workunit.client.0.smithi137.stdout:1/326: fdatasync d0/d2d/f79 0 2024-06-15T01:06:59.996 INFO:tasks.workunit.client.0.smithi137.stdout:5/466: unlink df/d11/d14/f21 0 2024-06-15T01:06:59.996 INFO:tasks.workunit.client.1.smithi142.stdout:5/354: dwrite d2/de/f5a [0,4194304] 0 2024-06-15T01:06:59.997 INFO:tasks.workunit.client.0.smithi137.stdout:5/467: write df/f2b [7922478,129873] 0 2024-06-15T01:06:59.998 INFO:tasks.workunit.client.0.smithi137.stdout:1/327: read d0/d3/f2f [1106196,96988] 0 2024-06-15T01:07:00.000 INFO:tasks.workunit.client.0.smithi137.stdout:5/468: unlink df/d11/d14/f5e 0 2024-06-15T01:07:00.001 INFO:tasks.workunit.client.0.smithi137.stdout:5/469: chown df/d11/d14/f15 9 1 2024-06-15T01:07:00.003 INFO:tasks.workunit.client.0.smithi137.stdout:1/328: creat d0/d1f/d76/f80 x:0 0 0 2024-06-15T01:07:00.003 INFO:tasks.workunit.client.1.smithi142.stdout:5/355: dwrite d2/de/d27/f38 [4194304,4194304] 0 2024-06-15T01:07:00.006 INFO:tasks.workunit.client.1.smithi142.stdout:5/356: symlink d2/d3e/d65/l6c 0 2024-06-15T01:07:00.008 INFO:tasks.workunit.client.1.smithi142.stdout:5/357: fsync d2/d3e/d31/f3c 0 2024-06-15T01:07:00.010 INFO:tasks.workunit.client.1.smithi142.stdout:8/345: dread d0/f4c [0,4194304] 0 2024-06-15T01:07:00.011 INFO:tasks.workunit.client.1.smithi142.stdout:8/346: chown d0/d17/d40/d5e 1634263 1 2024-06-15T01:07:00.012 INFO:tasks.workunit.client.1.smithi142.stdout:5/358: unlink d2/d18/l29 0 2024-06-15T01:07:00.013 INFO:tasks.workunit.client.0.smithi137.stdout:5/470: dwrite df/d11/d14/f15 [4194304,4194304] 0 2024-06-15T01:07:00.017 INFO:tasks.workunit.client.0.smithi137.stdout:5/471: write df/f12 [4973077,14198] 0 2024-06-15T01:07:00.023 INFO:tasks.workunit.client.1.smithi142.stdout:9/354: write d6/f6a [1189606,53842] 0 2024-06-15T01:07:00.023 INFO:tasks.workunit.client.1.smithi142.stdout:7/321: sync 2024-06-15T01:07:00.023 INFO:tasks.workunit.client.1.smithi142.stdout:0/335: sync 2024-06-15T01:07:00.023 INFO:tasks.workunit.client.1.smithi142.stdout:3/448: sync 2024-06-15T01:07:00.023 INFO:tasks.workunit.client.1.smithi142.stdout:2/302: sync 2024-06-15T01:07:00.024 INFO:tasks.workunit.client.0.smithi137.stdout:0/321: chown d1/d2/d13/d66/l38 83126616 1 2024-06-15T01:07:00.024 INFO:tasks.workunit.client.0.smithi137.stdout:8/254: dwrite f1 [4194304,4194304] 0 2024-06-15T01:07:00.024 INFO:tasks.workunit.client.1.smithi142.stdout:5/359: dwrite d2/f12 [0,4194304] 0 2024-06-15T01:07:00.026 INFO:tasks.workunit.client.1.smithi142.stdout:3/449: chown d0/d1f/d20/d59 115 1 2024-06-15T01:07:00.027 INFO:tasks.workunit.client.0.smithi137.stdout:6/210: sync 2024-06-15T01:07:00.028 INFO:tasks.workunit.client.1.smithi142.stdout:9/355: write d6/f9 [2418538,130470] 0 2024-06-15T01:07:00.028 INFO:tasks.workunit.client.0.smithi137.stdout:8/255: mknod d9/c58 0 2024-06-15T01:07:00.029 INFO:tasks.workunit.client.0.smithi137.stdout:0/322: mknod d1/d2/c71 0 2024-06-15T01:07:00.030 INFO:tasks.workunit.client.1.smithi142.stdout:5/360: creat d2/d3e/d31/f6d x:0 0 0 2024-06-15T01:07:00.030 INFO:tasks.workunit.client.1.smithi142.stdout:8/347: dwrite d0/f14 [0,4194304] 0 2024-06-15T01:07:00.030 INFO:tasks.workunit.client.1.smithi142.stdout:2/303: mknod d8/d19/c5c 0 2024-06-15T01:07:00.031 INFO:tasks.workunit.client.0.smithi137.stdout:0/323: stat d1/d2/dc/d10/d14/d15/d1c/f1d 0 2024-06-15T01:07:00.032 INFO:tasks.workunit.client.0.smithi137.stdout:6/211: creat d0/d1c/d1d/f45 x:0 0 0 2024-06-15T01:07:00.033 INFO:tasks.workunit.client.1.smithi142.stdout:7/322: read d2/d8/dd/f10 [38517,75915] 0 2024-06-15T01:07:00.034 INFO:tasks.workunit.client.0.smithi137.stdout:8/256: mkdir d9/d2a/d2f/d59 0 2024-06-15T01:07:00.034 INFO:tasks.workunit.client.0.smithi137.stdout:6/212: write d0/d1c/d1d/f45 [296392,101857] 0 2024-06-15T01:07:00.037 INFO:tasks.workunit.client.1.smithi142.stdout:7/323: fdatasync d2/d6/d22/d34/f36 0 2024-06-15T01:07:00.038 INFO:tasks.workunit.client.0.smithi137.stdout:8/257: write d9/de/d1e/f44 [240863,1755] 0 2024-06-15T01:07:00.038 INFO:tasks.workunit.client.0.smithi137.stdout:8/258: write d9/d2a/f34 [91998,94548] 0 2024-06-15T01:07:00.045 INFO:tasks.workunit.client.1.smithi142.stdout:7/324: dwrite d2/d6/f9 [0,4194304] 0 2024-06-15T01:07:00.046 INFO:tasks.workunit.client.1.smithi142.stdout:7/325: chown d2/d8/dd/d14/f4b 2382 1 2024-06-15T01:07:00.046 INFO:tasks.workunit.client.1.smithi142.stdout:7/326: chown d2/d8/da/d33/d25/f40 19912684 1 2024-06-15T01:07:00.047 INFO:tasks.workunit.client.1.smithi142.stdout:7/327: stat d2/d6/l21 0 2024-06-15T01:07:00.048 INFO:tasks.workunit.client.1.smithi142.stdout:7/328: write d2/d6/d22/d47/f50 [348674,105776] 0 2024-06-15T01:07:00.102 INFO:tasks.workunit.client.0.smithi137.stdout:8/259: dread - d9/de/d1e/f42 zero size 2024-06-15T01:07:00.103 INFO:tasks.workunit.client.0.smithi137.stdout:1/329: truncate d0/d3/f2f 3219162 0 2024-06-15T01:07:00.104 INFO:tasks.workunit.client.1.smithi142.stdout:5/361: symlink d2/d18/l6e 0 2024-06-15T01:07:00.106 INFO:tasks.workunit.client.0.smithi137.stdout:6/213: mkdir d0/d5/d46 0 2024-06-15T01:07:00.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/214: dread - d0/d1c/d1d/f40 zero size 2024-06-15T01:07:00.109 INFO:tasks.workunit.client.0.smithi137.stdout:1/330: mkdir d0/d1f/d67/d81 0 2024-06-15T01:07:00.112 INFO:tasks.workunit.client.1.smithi142.stdout:6/339: sync 2024-06-15T01:07:00.112 INFO:tasks.workunit.client.1.smithi142.stdout:4/476: sync 2024-06-15T01:07:00.114 INFO:tasks.workunit.client.0.smithi137.stdout:8/260: dwrite d9/de/d1e/d35/f56 [0,4194304] 0 2024-06-15T01:07:00.114 INFO:tasks.workunit.client.0.smithi137.stdout:1/331: write d0/d2d/d3e/f77 [380124,34185] 0 2024-06-15T01:07:00.114 INFO:tasks.workunit.client.1.smithi142.stdout:3/450: creat d0/d1f/d20/d5d/d60/f85 x:0 0 0 2024-06-15T01:07:00.114 INFO:tasks.workunit.client.1.smithi142.stdout:4/477: fsync d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f43 0 2024-06-15T01:07:00.120 INFO:tasks.workunit.client.0.smithi137.stdout:6/215: dwrite d0/d5/d9/f1b [4194304,4194304] 0 2024-06-15T01:07:00.122 INFO:tasks.workunit.client.0.smithi137.stdout:8/261: dread d9/d2a/f34 [0,4194304] 0 2024-06-15T01:07:00.122 INFO:tasks.workunit.client.1.smithi142.stdout:4/478: dwrite d5/d1c/f34 [0,4194304] 0 2024-06-15T01:07:00.123 INFO:tasks.workunit.client.0.smithi137.stdout:5/472: dread df/f4a [0,4194304] 0 2024-06-15T01:07:00.124 INFO:tasks.workunit.client.0.smithi137.stdout:8/262: read d9/d22/f37 [229427,80136] 0 2024-06-15T01:07:00.124 INFO:tasks.workunit.client.0.smithi137.stdout:1/332: rename d0/d2d/l56 to d0/d1f/l82 0 2024-06-15T01:07:00.125 INFO:tasks.workunit.client.0.smithi137.stdout:8/263: dread - d9/de/f3c zero size 2024-06-15T01:07:00.125 INFO:tasks.workunit.client.1.smithi142.stdout:4/479: chown d5/d1c/d2a/d30/d46/d50/d51/f75 493 1 2024-06-15T01:07:00.126 INFO:tasks.workunit.client.1.smithi142.stdout:2/304: creat d8/d19/d23/f5d x:0 0 0 2024-06-15T01:07:00.127 INFO:tasks.workunit.client.0.smithi137.stdout:8/264: stat d9/de/d1e/d35/f38 0 2024-06-15T01:07:00.131 INFO:tasks.workunit.client.0.smithi137.stdout:5/473: read df/d11/d30/f7b [3842234,48630] 0 2024-06-15T01:07:00.131 INFO:tasks.workunit.client.0.smithi137.stdout:6/216: unlink d0/d1c/f3a 0 2024-06-15T01:07:00.134 INFO:tasks.workunit.client.1.smithi142.stdout:8/348: rename d0/c21 to d0/d25/d5a/c74 0 2024-06-15T01:07:00.134 INFO:tasks.workunit.client.0.smithi137.stdout:1/333: dread d0/d1f/f4d [0,4194304] 0 2024-06-15T01:07:00.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/217: write d0/f39 [772396,85008] 0 2024-06-15T01:07:00.134 INFO:tasks.workunit.client.0.smithi137.stdout:5/474: chown df/d3d/c5a 44749 1 2024-06-15T01:07:00.136 INFO:tasks.workunit.client.0.smithi137.stdout:6/218: write d0/d1a/f26 [2434959,117039] 0 2024-06-15T01:07:00.136 INFO:tasks.workunit.client.1.smithi142.stdout:2/305: dwrite d8/d19/d33/f42 [0,4194304] 0 2024-06-15T01:07:00.140 INFO:tasks.workunit.client.0.smithi137.stdout:1/334: symlink d0/d2d/l83 0 2024-06-15T01:07:00.142 INFO:tasks.workunit.client.1.smithi142.stdout:2/306: dwrite d8/d19/d23/d25/d1a/f2e [0,4194304] 0 2024-06-15T01:07:00.143 INFO:tasks.workunit.client.1.smithi142.stdout:7/329: getdents d2/d8/da/d33/d25/d27 0 2024-06-15T01:07:00.145 INFO:tasks.workunit.client.0.smithi137.stdout:5/475: write df/d44/d54/d95/d3b/d49/f77 [2654488,110851] 0 2024-06-15T01:07:00.146 INFO:tasks.workunit.client.0.smithi137.stdout:5/476: chown df/d11/l1a 7 1 2024-06-15T01:07:00.149 INFO:tasks.workunit.client.1.smithi142.stdout:2/307: dread d8/df/d10/f35 [0,4194304] 0 2024-06-15T01:07:00.149 INFO:tasks.workunit.client.0.smithi137.stdout:5/477: dread df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:00.149 INFO:tasks.workunit.client.0.smithi137.stdout:8/265: link d9/l52 d9/de/d1e/l5a 0 2024-06-15T01:07:00.150 INFO:tasks.workunit.client.1.smithi142.stdout:7/330: dread d2/d6/f9 [0,4194304] 0 2024-06-15T01:07:00.151 INFO:tasks.workunit.client.1.smithi142.stdout:2/308: chown d8/d19/d33/f42 5 1 2024-06-15T01:07:00.152 INFO:tasks.workunit.client.1.smithi142.stdout:0/336: dwrite d3/d39/d3e/d81/d44/d7c/f19 [0,4194304] 0 2024-06-15T01:07:00.153 INFO:tasks.workunit.client.1.smithi142.stdout:8/349: symlink d0/d25/d53/l75 0 2024-06-15T01:07:00.154 INFO:tasks.workunit.client.1.smithi142.stdout:4/480: mkdir d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/da6 0 2024-06-15T01:07:00.154 INFO:tasks.workunit.client.1.smithi142.stdout:6/340: link d1a/d39/d42/d4c/d58/f5d d1a/d39/d42/d4c/d58/f7a 0 2024-06-15T01:07:00.155 INFO:tasks.workunit.client.1.smithi142.stdout:6/341: dread - d1a/d3c/f48 zero size 2024-06-15T01:07:00.156 INFO:tasks.workunit.client.0.smithi137.stdout:8/266: creat d9/de/d1e/d35/f5b x:0 0 0 2024-06-15T01:07:00.157 INFO:tasks.workunit.client.0.smithi137.stdout:8/267: write d9/d2a/f2d [3216266,54083] 0 2024-06-15T01:07:00.159 INFO:tasks.workunit.client.0.smithi137.stdout:8/268: stat d9/de/f18 0 2024-06-15T01:07:00.159 INFO:tasks.workunit.client.0.smithi137.stdout:5/478: mknod df/d76/d7c/c9e 0 2024-06-15T01:07:00.162 INFO:tasks.workunit.client.1.smithi142.stdout:5/362: rename d2/lb to d2/de/d19/d5f/l6f 0 2024-06-15T01:07:00.163 INFO:tasks.workunit.client.1.smithi142.stdout:3/451: link d0/d3/d15/l52 d0/d1f/l86 0 2024-06-15T01:07:00.164 INFO:tasks.workunit.client.1.smithi142.stdout:2/309: symlink d8/d19/d23/d3a/l5e 0 2024-06-15T01:07:00.166 INFO:tasks.workunit.client.0.smithi137.stdout:6/219: truncate d0/f6 3609817 0 2024-06-15T01:07:00.166 INFO:tasks.workunit.client.0.smithi137.stdout:6/220: chown d0/d1a/d3e 857500 1 2024-06-15T01:07:00.168 INFO:tasks.workunit.client.0.smithi137.stdout:8/269: dwrite d9/d22/f3e [0,4194304] 0 2024-06-15T01:07:00.169 INFO:tasks.workunit.client.0.smithi137.stdout:1/335: truncate d0/d3/d14/f19 2033428 0 2024-06-15T01:07:00.169 INFO:tasks.workunit.client.0.smithi137.stdout:8/270: chown d9/de/c14 1186006 1 2024-06-15T01:07:00.171 INFO:tasks.workunit.client.1.smithi142.stdout:8/350: rename d0/d17/f44 to d0/d17/d40/f76 0 2024-06-15T01:07:00.171 INFO:tasks.workunit.client.1.smithi142.stdout:1/393: sync 2024-06-15T01:07:00.174 INFO:tasks.workunit.client.1.smithi142.stdout:5/363: unlink d2/d3e/c35 0 2024-06-15T01:07:00.174 INFO:tasks.workunit.client.0.smithi137.stdout:1/336: symlink d0/d3/d14/d34/l84 0 2024-06-15T01:07:00.175 INFO:tasks.workunit.client.1.smithi142.stdout:7/331: mkdir d2/d8/dd/d14/d2c/d37/d6b/d6e 0 2024-06-15T01:07:00.175 INFO:tasks.workunit.client.0.smithi137.stdout:8/271: creat d9/d55/f5c x:0 0 0 2024-06-15T01:07:00.177 INFO:tasks.workunit.client.0.smithi137.stdout:6/221: rename d0/d1c to d0/d47 0 2024-06-15T01:07:00.180 INFO:tasks.workunit.client.1.smithi142.stdout:2/310: unlink d8/d19/f30 0 2024-06-15T01:07:00.180 INFO:tasks.workunit.client.1.smithi142.stdout:4/481: truncate d5/d1c/d2a/d30/d46/f52 747324 0 2024-06-15T01:07:00.181 INFO:tasks.workunit.client.0.smithi137.stdout:1/337: mknod d0/d1f/d7d/c85 0 2024-06-15T01:07:00.181 INFO:tasks.workunit.client.1.smithi142.stdout:7/332: chown d2/d6/d22/d47/f63 163 1 2024-06-15T01:07:00.182 INFO:tasks.workunit.client.1.smithi142.stdout:2/311: write d8/df/f48 [1398716,41469] 0 2024-06-15T01:07:00.183 INFO:tasks.workunit.client.1.smithi142.stdout:1/394: dwrite d6/d12/d5b/f8a [0,4194304] 0 2024-06-15T01:07:00.185 INFO:tasks.workunit.client.1.smithi142.stdout:2/312: read d8/d19/d33/f42 [1018843,17866] 0 2024-06-15T01:07:00.186 INFO:tasks.workunit.client.1.smithi142.stdout:6/342: creat d1a/d39/d42/d4c/f7b x:0 0 0 2024-06-15T01:07:00.186 INFO:tasks.workunit.client.1.smithi142.stdout:1/395: write d6/d12/d28/f33 [7211796,71231] 0 2024-06-15T01:07:00.186 INFO:tasks.workunit.client.0.smithi137.stdout:6/222: mkdir d0/d5/d46/d48 0 2024-06-15T01:07:00.187 INFO:tasks.workunit.client.1.smithi142.stdout:8/351: symlink d0/d17/l77 0 2024-06-15T01:07:00.188 INFO:tasks.workunit.client.0.smithi137.stdout:1/338: mknod d0/d10/d41/c86 0 2024-06-15T01:07:00.189 INFO:tasks.workunit.client.1.smithi142.stdout:7/333: rmdir d2/d8/da 39 2024-06-15T01:07:00.189 INFO:tasks.workunit.client.1.smithi142.stdout:8/352: truncate d0/d25/d5a/f37 616817 0 2024-06-15T01:07:00.190 INFO:tasks.workunit.client.0.smithi137.stdout:5/479: write df/d11/d14/f33 [332623,98111] 0 2024-06-15T01:07:00.190 INFO:tasks.workunit.client.1.smithi142.stdout:7/334: fsync d2/d8/dd/d14/d2c/f62 0 2024-06-15T01:07:00.194 INFO:tasks.workunit.client.1.smithi142.stdout:1/396: mknod d6/d12/d5b/c95 0 2024-06-15T01:07:00.194 INFO:tasks.workunit.client.1.smithi142.stdout:2/313: symlink d8/d19/d23/d25/d22/d43/l5f 0 2024-06-15T01:07:00.194 INFO:tasks.workunit.client.0.smithi137.stdout:8/272: truncate d9/d2a/f2d 2581823 0 2024-06-15T01:07:00.195 INFO:tasks.workunit.client.1.smithi142.stdout:2/314: fsync d8/d19/d23/f5d 0 2024-06-15T01:07:00.195 INFO:tasks.workunit.client.0.smithi137.stdout:1/339: mknod d0/d3/d1b/d4c/d59/d64/c87 0 2024-06-15T01:07:00.196 INFO:tasks.workunit.client.1.smithi142.stdout:2/315: stat d8/df/d1f/c4c 0 2024-06-15T01:07:00.196 INFO:tasks.workunit.client.0.smithi137.stdout:6/223: dwrite d0/d5/f7 [0,4194304] 0 2024-06-15T01:07:00.197 INFO:tasks.workunit.client.1.smithi142.stdout:2/316: chown d8/d19/d23/d3a/l5e 197 1 2024-06-15T01:07:00.198 INFO:tasks.workunit.client.0.smithi137.stdout:6/224: fdatasync d0/d5/f17 0 2024-06-15T01:07:00.198 INFO:tasks.workunit.client.0.smithi137.stdout:8/273: write d9/de/d1e/d3f/f41 [1036586,53478] 0 2024-06-15T01:07:00.198 INFO:tasks.workunit.client.1.smithi142.stdout:6/343: creat d1a/d2d/d2e/d6c/f7c x:0 0 0 2024-06-15T01:07:00.199 INFO:tasks.workunit.client.1.smithi142.stdout:1/397: mknod d6/d12/d52/c96 0 2024-06-15T01:07:00.205 INFO:tasks.workunit.client.0.smithi137.stdout:8/274: rmdir d9/de/d1e 39 2024-06-15T01:07:00.207 INFO:tasks.workunit.client.0.smithi137.stdout:1/340: dwrite d0/d2d/f62 [0,4194304] 0 2024-06-15T01:07:00.209 INFO:tasks.workunit.client.0.smithi137.stdout:1/341: chown d0/d2d/f79 6 1 2024-06-15T01:07:00.209 INFO:tasks.workunit.client.1.smithi142.stdout:5/364: chown d2/de/d19/d5f/l6f 13 1 2024-06-15T01:07:00.210 INFO:tasks.workunit.client.1.smithi142.stdout:5/365: chown d2/de/f5a 1894886 1 2024-06-15T01:07:00.210 INFO:tasks.workunit.client.0.smithi137.stdout:6/225: getdents d0/d5/d9/d29 0 2024-06-15T01:07:00.210 INFO:tasks.workunit.client.1.smithi142.stdout:5/366: dread - d2/d3e/d31/f64 zero size 2024-06-15T01:07:00.211 INFO:tasks.workunit.client.0.smithi137.stdout:1/342: fsync d0/d1f/d60/f74 0 2024-06-15T01:07:00.211 INFO:tasks.workunit.client.0.smithi137.stdout:5/480: getdents df/d44/d54/d95/d3b/d4b 0 2024-06-15T01:07:00.215 INFO:tasks.workunit.client.0.smithi137.stdout:8/275: dread d9/d2a/f2d [0,4194304] 0 2024-06-15T01:07:00.218 INFO:tasks.workunit.client.0.smithi137.stdout:1/343: dread d0/d3/f5 [0,4194304] 0 2024-06-15T01:07:00.220 INFO:tasks.workunit.client.1.smithi142.stdout:6/344: rename c1 to d1a/d39/d3a/d51/c7d 0 2024-06-15T01:07:00.221 INFO:tasks.workunit.client.0.smithi137.stdout:8/276: readlink d9/de/d1e/l1f 0 2024-06-15T01:07:00.221 INFO:tasks.workunit.client.0.smithi137.stdout:6/226: link d0/d47/d1d/d2e/l31 d0/l49 0 2024-06-15T01:07:00.222 INFO:tasks.workunit.client.1.smithi142.stdout:1/398: mkdir d6/d12/d3c/d63/d8d/d97 0 2024-06-15T01:07:00.223 INFO:tasks.workunit.client.1.smithi142.stdout:0/337: truncate d3/d39/d3e/d81/d44/d7c/f19 2027156 0 2024-06-15T01:07:00.224 INFO:tasks.workunit.client.0.smithi137.stdout:4/276: sync 2024-06-15T01:07:00.224 INFO:tasks.workunit.client.1.smithi142.stdout:0/338: write d3/d39/d3e/d81/d44/d7c/d22/f2d [178065,27726] 0 2024-06-15T01:07:00.225 INFO:tasks.workunit.client.0.smithi137.stdout:6/227: creat d0/d5/d9/d3f/f4a x:0 0 0 2024-06-15T01:07:00.226 INFO:tasks.workunit.client.0.smithi137.stdout:1/344: getdents d0/d10/d41/d7b 0 2024-06-15T01:07:00.226 INFO:tasks.workunit.client.1.smithi142.stdout:5/367: link d2/de/d27/f38 d2/d3e/d65/f70 0 2024-06-15T01:07:00.227 INFO:tasks.workunit.client.0.smithi137.stdout:4/277: mknod dd/d15/c6a 0 2024-06-15T01:07:00.227 INFO:tasks.workunit.client.1.smithi142.stdout:9/356: sync 2024-06-15T01:07:00.228 INFO:tasks.workunit.client.1.smithi142.stdout:6/345: truncate d1a/f2c 3097394 0 2024-06-15T01:07:00.228 INFO:tasks.workunit.client.0.smithi137.stdout:4/278: dread - dd/d10/f54 zero size 2024-06-15T01:07:00.228 INFO:tasks.workunit.client.1.smithi142.stdout:3/452: truncate d0/d1b/f5b 1115920 0 2024-06-15T01:07:00.229 INFO:tasks.workunit.client.0.smithi137.stdout:8/277: getdents d9/d22 0 2024-06-15T01:07:00.230 INFO:tasks.workunit.client.1.smithi142.stdout:1/399: mkdir d6/d12/d30/d98 0 2024-06-15T01:07:00.230 INFO:tasks.workunit.client.0.smithi137.stdout:6/228: truncate d0/d5/d9/fe 1606161 0 2024-06-15T01:07:00.231 INFO:tasks.workunit.client.0.smithi137.stdout:1/345: creat d0/d3/d47/f88 x:0 0 0 2024-06-15T01:07:00.231 INFO:tasks.workunit.client.1.smithi142.stdout:2/317: getdents d8/d19/d23/d25 0 2024-06-15T01:07:00.233 INFO:tasks.workunit.client.1.smithi142.stdout:0/339: stat d3/l2f 0 2024-06-15T01:07:00.234 INFO:tasks.workunit.client.1.smithi142.stdout:6/346: dwrite fc [0,4194304] 0 2024-06-15T01:07:00.236 INFO:tasks.workunit.client.0.smithi137.stdout:8/278: mknod d9/d2a/d2f/d59/c5d 0 2024-06-15T01:07:00.236 INFO:tasks.workunit.client.1.smithi142.stdout:5/368: creat d2/d18/d2e/f71 x:0 0 0 2024-06-15T01:07:00.237 INFO:tasks.workunit.client.1.smithi142.stdout:0/340: read d3/d39/f6f [2077100,117645] 0 2024-06-15T01:07:00.238 INFO:tasks.workunit.client.0.smithi137.stdout:6/229: rmdir d0/d5/d9/d3f 39 2024-06-15T01:07:00.239 INFO:tasks.workunit.client.1.smithi142.stdout:3/453: creat d0/d3/da/d18/f87 x:0 0 0 2024-06-15T01:07:00.239 INFO:tasks.workunit.client.1.smithi142.stdout:1/400: creat d6/d3a/f99 x:0 0 0 2024-06-15T01:07:00.241 INFO:tasks.workunit.client.0.smithi137.stdout:5/481: dread df/f12 [0,4194304] 0 2024-06-15T01:07:00.241 INFO:tasks.workunit.client.0.smithi137.stdout:8/279: symlink d9/d2a/d2f/d4e/l5e 0 2024-06-15T01:07:00.242 INFO:tasks.workunit.client.0.smithi137.stdout:6/230: fdatasync d0/d47/d1d/d2e/f38 0 2024-06-15T01:07:00.242 INFO:tasks.workunit.client.0.smithi137.stdout:4/279: dwrite dd/f17 [0,4194304] 0 2024-06-15T01:07:00.244 INFO:tasks.workunit.client.1.smithi142.stdout:9/357: dread d6/f21 [0,4194304] 0 2024-06-15T01:07:00.244 INFO:tasks.workunit.client.0.smithi137.stdout:1/346: dwrite d0/d10/f2e [8388608,4194304] 0 2024-06-15T01:07:00.246 INFO:tasks.workunit.client.1.smithi142.stdout:5/369: creat d2/de/d27/f72 x:0 0 0 2024-06-15T01:07:00.246 INFO:tasks.workunit.client.0.smithi137.stdout:6/231: chown d0/f6 1633732 1 2024-06-15T01:07:00.246 INFO:tasks.workunit.client.0.smithi137.stdout:5/482: symlink df/d44/d54/l9f 0 2024-06-15T01:07:00.248 INFO:tasks.workunit.client.1.smithi142.stdout:0/341: dwrite d3/d39/d3e/d81/d44/f4b [0,4194304] 0 2024-06-15T01:07:00.248 INFO:tasks.workunit.client.1.smithi142.stdout:1/401: creat d6/d12/d28/f9a x:0 0 0 2024-06-15T01:07:00.250 INFO:tasks.workunit.client.0.smithi137.stdout:1/347: write d0/d3/d47/f5d [280333,8071] 0 2024-06-15T01:07:00.250 INFO:tasks.workunit.client.0.smithi137.stdout:5/483: rmdir df/d44/d54/d80 39 2024-06-15T01:07:00.253 INFO:tasks.workunit.client.1.smithi142.stdout:4/482: dwrite d5/d9/f22 [0,4194304] 0 2024-06-15T01:07:00.253 INFO:tasks.workunit.client.1.smithi142.stdout:1/402: read d6/d12/d3c/d63/f6f [172385,110515] 0 2024-06-15T01:07:00.255 INFO:tasks.workunit.client.1.smithi142.stdout:1/403: chown d6/d12/d1c/d2c/d93 83302 1 2024-06-15T01:07:00.256 INFO:tasks.workunit.client.1.smithi142.stdout:7/335: truncate d2/d8/dd/f12 444004 0 2024-06-15T01:07:00.257 INFO:tasks.workunit.client.0.smithi137.stdout:5/484: creat df/d44/d54/d95/d3b/d4b/fa0 x:0 0 0 2024-06-15T01:07:00.257 INFO:tasks.workunit.client.1.smithi142.stdout:8/353: truncate d0/d17/f2f 1577391 0 2024-06-15T01:07:00.260 INFO:tasks.workunit.client.1.smithi142.stdout:9/358: mkdir d6/dc/d51/d53/d8d 0 2024-06-15T01:07:00.263 INFO:tasks.workunit.client.0.smithi137.stdout:9/167: sync 2024-06-15T01:07:00.263 INFO:tasks.workunit.client.0.smithi137.stdout:1/348: dwrite d0/d3/d1b/f69 [0,4194304] 0 2024-06-15T01:07:00.264 INFO:tasks.workunit.client.1.smithi142.stdout:0/342: mknod d3/d39/d3e/d81/d1b/d52/c84 0 2024-06-15T01:07:00.265 INFO:tasks.workunit.client.0.smithi137.stdout:9/168: stat df/d12/f2c 0 2024-06-15T01:07:00.266 INFO:tasks.workunit.client.0.smithi137.stdout:5/485: chown df/d11/d14/c1f 1197973 1 2024-06-15T01:07:00.267 INFO:tasks.workunit.client.1.smithi142.stdout:7/336: unlink d2/d6/d22/d47/f63 0 2024-06-15T01:07:00.268 INFO:tasks.workunit.client.0.smithi137.stdout:9/169: mknod df/d12/c2d 0 2024-06-15T01:07:00.270 INFO:tasks.workunit.client.0.smithi137.stdout:1/349: mkdir d0/d3/d89 0 2024-06-15T01:07:00.271 INFO:tasks.workunit.client.1.smithi142.stdout:3/454: truncate d0/d1f/d20/f32 2116657 0 2024-06-15T01:07:00.272 INFO:tasks.workunit.client.1.smithi142.stdout:5/370: creat d2/de/d19/d5f/d68/f73 x:0 0 0 2024-06-15T01:07:00.273 INFO:tasks.workunit.client.1.smithi142.stdout:3/455: truncate d0/d3/d15/f83 672051 0 2024-06-15T01:07:00.273 INFO:tasks.workunit.client.1.smithi142.stdout:5/371: write d2/d18/f69 [266600,87171] 0 2024-06-15T01:07:00.274 INFO:tasks.workunit.client.1.smithi142.stdout:4/483: rename d5/d1c/d2a/d30/d46/d50/d51/d5b/d2d/l9e to d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/la7 0 2024-06-15T01:07:00.274 INFO:tasks.workunit.client.1.smithi142.stdout:5/372: readlink d2/l10 0 2024-06-15T01:07:00.275 INFO:tasks.workunit.client.1.smithi142.stdout:2/318: getdents d8/d19/d23/d25/d22/d43 0 2024-06-15T01:07:00.275 INFO:tasks.workunit.client.0.smithi137.stdout:1/350: write d0/d3/d1b/d4c/f75 [816971,100543] 0 2024-06-15T01:07:00.276 INFO:tasks.workunit.client.1.smithi142.stdout:1/404: mkdir d6/d12/d80/d9b 0 2024-06-15T01:07:00.277 INFO:tasks.workunit.client.0.smithi137.stdout:5/486: dwrite df/d11/f24 [0,4194304] 0 2024-06-15T01:07:00.278 INFO:tasks.workunit.client.0.smithi137.stdout:9/170: dwrite df/f18 [0,4194304] 0 2024-06-15T01:07:00.278 INFO:tasks.workunit.client.1.smithi142.stdout:0/343: creat d3/d39/d3e/d81/d1b/d52/f85 x:0 0 0 2024-06-15T01:07:00.282 INFO:tasks.workunit.client.1.smithi142.stdout:6/347: link d1a/d39/d3a/c64 d1a/d39/d42/d4c/c7e 0 2024-06-15T01:07:00.282 INFO:tasks.workunit.client.0.smithi137.stdout:4/280: write dd/d15/f2e [471019,102881] 0 2024-06-15T01:07:00.288 INFO:tasks.workunit.client.0.smithi137.stdout:1/351: creat d0/d3/d1b/d4c/d59/f8a x:0 0 0 2024-06-15T01:07:00.289 INFO:tasks.workunit.client.0.smithi137.stdout:5/487: mkdir df/d76/d7c/d9c/da1 0 2024-06-15T01:07:00.291 INFO:tasks.workunit.client.0.smithi137.stdout:8/280: truncate d9/de/d1e/d35/f56 693399 0 2024-06-15T01:07:00.293 INFO:tasks.workunit.client.0.smithi137.stdout:6/232: truncate d0/d5/f12 3894356 0 2024-06-15T01:07:00.293 INFO:tasks.workunit.client.0.smithi137.stdout:1/352: read d0/d10/f72 [383367,93362] 0 2024-06-15T01:07:00.294 INFO:tasks.workunit.client.0.smithi137.stdout:6/233: fsync d0/d5/f7 0 2024-06-15T01:07:00.294 INFO:tasks.workunit.client.1.smithi142.stdout:2/319: mknod d8/d19/c60 0 2024-06-15T01:07:00.297 INFO:tasks.workunit.client.1.smithi142.stdout:2/320: write d8/d19/d23/d25/f53 [943643,94594] 0 2024-06-15T01:07:00.297 INFO:tasks.workunit.client.1.smithi142.stdout:7/337: dwrite d2/d8/da/d33/d25/f40 [0,4194304] 0 2024-06-15T01:07:00.297 INFO:tasks.workunit.client.1.smithi142.stdout:1/405: creat d6/d12/d3c/f9c x:0 0 0 2024-06-15T01:07:00.300 INFO:tasks.workunit.client.0.smithi137.stdout:8/281: mkdir d9/de/d1e/d35/d5f 0 2024-06-15T01:07:00.301 INFO:tasks.workunit.client.1.smithi142.stdout:6/348: stat d1a/d39/d3a/d51/c7d 0 2024-06-15T01:07:00.301 INFO:tasks.workunit.client.1.smithi142.stdout:0/344: truncate d3/d39/f6f 5000128 0 2024-06-15T01:07:00.301 INFO:tasks.workunit.client.0.smithi137.stdout:9/171: dwrite f5 [0,4194304] 0 2024-06-15T01:07:00.302 INFO:tasks.workunit.client.1.smithi142.stdout:8/354: creat d0/d25/d5a/f78 x:0 0 0 2024-06-15T01:07:00.304 INFO:tasks.workunit.client.0.smithi137.stdout:6/234: creat d0/d5/d9/d29/f4b x:0 0 0 2024-06-15T01:07:00.304 INFO:tasks.workunit.client.0.smithi137.stdout:1/353: mknod d0/d10/d41/c8b 0 2024-06-15T01:07:00.305 INFO:tasks.workunit.client.1.smithi142.stdout:9/359: creat d6/dc/f8e x:0 0 0 2024-06-15T01:07:00.306 INFO:tasks.workunit.client.1.smithi142.stdout:9/360: read - d6/dc/d82/f72 zero size 2024-06-15T01:07:00.308 INFO:tasks.workunit.client.0.smithi137.stdout:5/488: mknod df/d44/d54/d95/d3b/ca2 0 2024-06-15T01:07:00.310 INFO:tasks.workunit.client.0.smithi137.stdout:9/172: creat df/d12/d1b/f2e x:0 0 0 2024-06-15T01:07:00.312 INFO:tasks.workunit.client.1.smithi142.stdout:0/345: dwrite d3/d39/d3e/d81/d44/f57 [0,4194304] 0 2024-06-15T01:07:00.312 INFO:tasks.workunit.client.1.smithi142.stdout:7/338: symlink d2/d6/d22/d3c/l6f 0 2024-06-15T01:07:00.313 INFO:tasks.workunit.client.1.smithi142.stdout:1/406: symlink d6/d12/d52/l9d 0 2024-06-15T01:07:00.316 INFO:tasks.workunit.client.0.smithi137.stdout:5/489: mknod df/d11/d30/ca3 0 2024-06-15T01:07:00.316 INFO:tasks.workunit.client.0.smithi137.stdout:4/281: truncate f8 5233293 0 2024-06-15T01:07:00.317 INFO:tasks.workunit.client.1.smithi142.stdout:1/407: fsync d6/d12/d28/f5d 0 2024-06-15T01:07:00.318 INFO:tasks.workunit.client.0.smithi137.stdout:5/490: write df/d44/d54/d95/d3b/f59 [388706,83439] 0 2024-06-15T01:07:00.319 INFO:tasks.workunit.client.1.smithi142.stdout:6/349: creat d1a/d39/d42/d4c/d58/f7f x:0 0 0 2024-06-15T01:07:00.320 INFO:tasks.workunit.client.0.smithi137.stdout:9/173: rename df/d12/f23 to df/d12/d1b/f2f 0 2024-06-15T01:07:00.322 INFO:tasks.workunit.client.1.smithi142.stdout:3/456: dread d0/f3c [0,4194304] 0 2024-06-15T01:07:00.323 INFO:tasks.workunit.client.1.smithi142.stdout:8/355: creat d0/d25/d53/f79 x:0 0 0 2024-06-15T01:07:00.323 INFO:tasks.workunit.client.1.smithi142.stdout:7/339: write d2/d8/da/d33/d25/f40 [2759949,6910] 0 2024-06-15T01:07:00.323 INFO:tasks.workunit.client.0.smithi137.stdout:1/354: link d0/d10/f6f d0/d3/d1b/d4c/d59/f8c 0 2024-06-15T01:07:00.325 INFO:tasks.workunit.client.1.smithi142.stdout:0/346: mkdir d3/d39/d3e/d81/d1b/d86 0 2024-06-15T01:07:00.326 INFO:tasks.workunit.client.1.smithi142.stdout:1/408: truncate d6/d12/d3c/f54 90486 0 2024-06-15T01:07:00.326 INFO:tasks.workunit.client.1.smithi142.stdout:9/361: dwrite d6/dc/dd/f46 [0,4194304] 0 2024-06-15T01:07:00.327 INFO:tasks.workunit.client.0.smithi137.stdout:4/282: creat dd/d15/d30/f6b x:0 0 0 2024-06-15T01:07:00.327 INFO:tasks.workunit.client.0.smithi137.stdout:5/491: fsync f3 0 2024-06-15T01:07:00.328 INFO:tasks.workunit.client.0.smithi137.stdout:1/355: symlink d0/d3/d89/l8d 0 2024-06-15T01:07:00.328 INFO:tasks.workunit.client.1.smithi142.stdout:9/362: stat d6/dc/dd/d22 0 2024-06-15T01:07:00.330 INFO:tasks.workunit.client.0.smithi137.stdout:9/174: dwrite df/f24 [0,4194304] 0 2024-06-15T01:07:00.334 INFO:tasks.workunit.client.1.smithi142.stdout:8/356: write d0/f15 [6535118,128982] 0 2024-06-15T01:07:00.334 INFO:tasks.workunit.client.1.smithi142.stdout:5/373: getdents d2/de/d19 0 2024-06-15T01:07:00.337 INFO:tasks.workunit.client.1.smithi142.stdout:3/457: chown d0/d1f/l35 3656680 1 2024-06-15T01:07:00.338 INFO:tasks.workunit.client.1.smithi142.stdout:2/321: link d8/ld d8/d19/d23/d3a/l61 0 2024-06-15T01:07:00.339 INFO:tasks.workunit.client.0.smithi137.stdout:1/356: dwrite d0/d3/d14/d34/f45 [0,4194304] 0 2024-06-15T01:07:00.341 INFO:tasks.workunit.client.1.smithi142.stdout:7/340: dwrite d2/d6/d22/f4d [0,4194304] 0 2024-06-15T01:07:00.342 INFO:tasks.workunit.client.0.smithi137.stdout:9/175: creat df/d13/f30 x:0 0 0 2024-06-15T01:07:00.344 INFO:tasks.workunit.client.1.smithi142.stdout:0/347: chown d3/d39/d3e/d81/d44/c83 6 1 2024-06-15T01:07:00.344 INFO:tasks.workunit.client.1.smithi142.stdout:1/409: dwrite d6/d12/f3e [0,4194304] 0 2024-06-15T01:07:00.345 INFO:tasks.workunit.client.1.smithi142.stdout:6/350: mkdir d1a/d2d/d80 0 2024-06-15T01:07:00.346 INFO:tasks.workunit.client.0.smithi137.stdout:1/357: rename d0/d3/f5 to d0/d3/d1b/d4c/f8e 0 2024-06-15T01:07:00.349 INFO:tasks.workunit.client.0.smithi137.stdout:5/492: mkdir df/d44/d54/d95/d3b/d4b/d4f/da4 0 2024-06-15T01:07:00.350 INFO:tasks.workunit.client.0.smithi137.stdout:1/358: creat d0/d1f/d76/f8f x:0 0 0 2024-06-15T01:07:00.351 INFO:tasks.workunit.client.1.smithi142.stdout:1/410: write d6/d3a/f99 [524465,40957] 0 2024-06-15T01:07:00.351 INFO:tasks.workunit.client.1.smithi142.stdout:9/363: mknod d6/d34/d8a/c8f 0 2024-06-15T01:07:00.354 INFO:tasks.workunit.client.0.smithi137.stdout:9/176: fsync f3 0 2024-06-15T01:07:00.354 INFO:tasks.workunit.client.0.smithi137.stdout:6/235: write d0/d47/d1d/f3d [463716,76512] 0 2024-06-15T01:07:00.356 INFO:tasks.workunit.client.0.smithi137.stdout:9/177: truncate df/d12/d15/f26 171602 0 2024-06-15T01:07:00.356 INFO:tasks.workunit.client.1.smithi142.stdout:2/322: mknod d8/d19/d33/c62 0 2024-06-15T01:07:00.357 INFO:tasks.workunit.client.1.smithi142.stdout:8/357: dwrite d0/f18 [4194304,4194304] 0 2024-06-15T01:07:00.358 INFO:tasks.workunit.client.0.smithi137.stdout:9/178: write df/f28 [831497,1814] 0 2024-06-15T01:07:00.359 INFO:tasks.workunit.client.1.smithi142.stdout:5/374: mkdir d2/d3e/d74 0 2024-06-15T01:07:00.359 INFO:tasks.workunit.client.1.smithi142.stdout:0/348: creat d3/d39/d3e/d81/f87 x:0 0 0 2024-06-15T01:07:00.360 INFO:tasks.workunit.client.0.smithi137.stdout:1/359: creat d0/d10/d41/d7b/f90 x:0 0 0 2024-06-15T01:07:00.361 INFO:tasks.workunit.client.1.smithi142.stdout:1/411: fdatasync d6/d46/d61/f76 0 2024-06-15T01:07:00.361 INFO:tasks.workunit.client.1.smithi142.stdout:6/351: mkdir d1a/d39/d3a/d51/d81 0 2024-06-15T01:07:00.362 INFO:tasks.workunit.client.1.smithi142.stdout:9/364: symlink d6/dc/dd/d22/l90 0 2024-06-15T01:07:00.363 INFO:tasks.workunit.client.1.smithi142.stdout:7/341: mkdir d2/d8/dd/d14/d2c/d70 0 2024-06-15T01:07:00.365 INFO:tasks.workunit.client.1.smithi142.stdout:2/323: mkdir d8/d19/d23/d25/d1a/d63 0 2024-06-15T01:07:00.367 INFO:tasks.workunit.client.1.smithi142.stdout:3/458: dwrite d0/d3/da/d18/f87 [0,4194304] 0 2024-06-15T01:07:00.368 INFO:tasks.workunit.client.0.smithi137.stdout:9/179: mkdir df/d12/d1b/d31 0 2024-06-15T01:07:00.369 INFO:tasks.workunit.client.1.smithi142.stdout:8/358: dread d0/f47 [0,4194304] 0 2024-06-15T01:07:00.369 INFO:tasks.workunit.client.1.smithi142.stdout:6/352: creat d1a/d39/f82 x:0 0 0 2024-06-15T01:07:00.369 INFO:tasks.workunit.client.1.smithi142.stdout:7/342: fdatasync d2/f49 0 2024-06-15T01:07:00.370 INFO:tasks.workunit.client.0.smithi137.stdout:1/360: dwrite d0/d1f/d76/f80 [0,4194304] 0 2024-06-15T01:07:00.371 INFO:tasks.workunit.client.1.smithi142.stdout:7/343: fdatasync d2/d6/d22/d34/f36 0 2024-06-15T01:07:00.374 INFO:tasks.workunit.client.1.smithi142.stdout:9/365: rename d6/d34/f5f to d6/dc/dd/f91 0 2024-06-15T01:07:00.375 INFO:tasks.workunit.client.0.smithi137.stdout:9/180: creat df/d12/d15/f32 x:0 0 0 2024-06-15T01:07:00.376 INFO:tasks.workunit.client.0.smithi137.stdout:9/181: write f8 [2228778,62755] 0 2024-06-15T01:07:00.377 INFO:tasks.workunit.client.1.smithi142.stdout:2/324: truncate d8/fe 441641 0 2024-06-15T01:07:00.377 INFO:tasks.workunit.client.0.smithi137.stdout:5/493: rename df/d44/d54/d80/c83 to df/d44/d54/ca5 0 2024-06-15T01:07:00.378 INFO:tasks.workunit.client.1.smithi142.stdout:3/459: creat d0/d1b/f88 x:0 0 0 2024-06-15T01:07:00.378 INFO:tasks.workunit.client.1.smithi142.stdout:9/366: write d6/dc/d1b/f4b [219438,124304] 0 2024-06-15T01:07:00.381 INFO:tasks.workunit.client.1.smithi142.stdout:0/349: link d3/d39/d3e/d81/f73 d3/d39/d3e/d81/f88 0 2024-06-15T01:07:00.382 INFO:tasks.workunit.client.0.smithi137.stdout:9/182: rename f8 to df/d12/d1b/d31/f33 0 2024-06-15T01:07:00.385 INFO:tasks.workunit.client.1.smithi142.stdout:8/359: symlink d0/d17/d40/d5e/l7a 0 2024-06-15T01:07:00.385 INFO:tasks.workunit.client.1.smithi142.stdout:7/344: dwrite d2/d6/d22/f4d [0,4194304] 0 2024-06-15T01:07:00.385 INFO:tasks.workunit.client.1.smithi142.stdout:3/460: truncate d0/d3/da/f62 447784 0 2024-06-15T01:07:00.387 INFO:tasks.workunit.client.0.smithi137.stdout:5/494: dwrite df/d44/d54/d95/d3b/d49/f77 [0,4194304] 0 2024-06-15T01:07:00.387 INFO:tasks.workunit.client.1.smithi142.stdout:7/345: stat d2/d6/d22/d34/c3d 0 2024-06-15T01:07:00.387 INFO:tasks.workunit.client.1.smithi142.stdout:2/325: write d8/d19/d23/d25/f27 [569338,130503] 0 2024-06-15T01:07:00.388 INFO:tasks.workunit.client.0.smithi137.stdout:9/183: dread f3 [0,4194304] 0 2024-06-15T01:07:00.389 INFO:tasks.workunit.client.1.smithi142.stdout:0/350: truncate d3/d39/d3e/d81/d1e/f2b 1060115 0 2024-06-15T01:07:00.389 INFO:tasks.workunit.client.1.smithi142.stdout:0/351: read d3/d39/d3e/d81/f1d [2232985,36726] 0 2024-06-15T01:07:00.390 INFO:tasks.workunit.client.1.smithi142.stdout:6/353: rename d1a/d2d/d2e/d6c to d1a/d83 0 2024-06-15T01:07:00.390 INFO:tasks.workunit.client.0.smithi137.stdout:5/495: symlink df/d76/d8e/la6 0 2024-06-15T01:07:00.391 INFO:tasks.workunit.client.1.smithi142.stdout:3/461: readlink d0/d1f/l86 0 2024-06-15T01:07:00.392 INFO:tasks.workunit.client.1.smithi142.stdout:0/352: read - d3/d39/d3e/d81/d1b/d52/f85 zero size 2024-06-15T01:07:00.393 INFO:tasks.workunit.client.1.smithi142.stdout:9/367: creat d6/dc/d1b/d25/d58/d74/f92 x:0 0 0 2024-06-15T01:07:00.395 INFO:tasks.workunit.client.1.smithi142.stdout:8/360: write d0/d17/d40/f65 [488659,66791] 0 2024-06-15T01:07:00.398 INFO:tasks.workunit.client.1.smithi142.stdout:3/462: unlink d0/d3/c16 0 2024-06-15T01:07:00.399 INFO:tasks.workunit.client.0.smithi137.stdout:1/361: fsync d0/d10/d41/d7b/f90 0 2024-06-15T01:07:00.400 INFO:tasks.workunit.client.1.smithi142.stdout:3/463: fdatasync d0/d1b/f88 0 2024-06-15T01:07:00.400 INFO:tasks.workunit.client.1.smithi142.stdout:4/484: sync 2024-06-15T01:07:00.400 INFO:tasks.workunit.client.1.smithi142.stdout:7/346: dread d2/d8/da/d33/d25/d27/f5d [0,4194304] 0 2024-06-15T01:07:00.400 INFO:tasks.workunit.client.1.smithi142.stdout:6/354: truncate d1a/d39/d3a/d51/d62/f79 459417 0 2024-06-15T01:07:00.401 INFO:tasks.workunit.client.1.smithi142.stdout:0/353: symlink d3/d39/d3e/d81/d4c/l89 0 2024-06-15T01:07:00.401 INFO:tasks.workunit.client.0.smithi137.stdout:4/283: write f8 [3206254,72512] 0 2024-06-15T01:07:00.401 INFO:tasks.workunit.client.0.smithi137.stdout:4/284: stat dd/d15 0 2024-06-15T01:07:00.401 INFO:tasks.workunit.client.0.smithi137.stdout:4/285: readlink dd/d15/d30/l31 0 2024-06-15T01:07:00.402 INFO:tasks.workunit.client.1.smithi142.stdout:6/355: read - d1a/d39/d42/d4c/d58/f7f zero size 2024-06-15T01:07:00.402 INFO:tasks.workunit.client.1.smithi142.stdout:4/485: write f3 [555103,65130] 0 2024-06-15T01:07:00.404 INFO:tasks.workunit.client.0.smithi137.stdout:1/362: rename d0/d10/f72 to d0/d1f/d60/f91 0 2024-06-15T01:07:00.404 INFO:tasks.workunit.client.1.smithi142.stdout:2/326: mkdir d8/df/d10/d64 0 2024-06-15T01:07:00.405 INFO:tasks.workunit.client.1.smithi142.stdout:6/356: chown d1a/d39/d3a/d51/c7d 2 1 2024-06-15T01:07:00.405 INFO:tasks.workunit.client.1.smithi142.stdout:8/361: mknod d0/c7b 0 2024-06-15T01:07:00.406 INFO:tasks.workunit.client.1.smithi142.stdout:2/327: read d8/df/d10/f11 [349973,100799] 0 2024-06-15T01:07:00.408 INFO:tasks.workunit.client.0.smithi137.stdout:4/286: dwrite fa [0,4194304] 0 2024-06-15T01:07:00.410 INFO:tasks.workunit.client.1.smithi142.stdout:7/347: rmdir d2/d6 39 2024-06-15T01:07:00.413 INFO:tasks.workunit.client.1.smithi142.stdout:3/464: symlink d0/d1f/d20/d5d/d82/l89 0 2024-06-15T01:07:00.414 INFO:tasks.workunit.client.0.smithi137.stdout:9/184: fsync df/d12/d15/f32 0 2024-06-15T01:07:00.414 INFO:tasks.workunit.client.0.smithi137.stdout:9/185: readlink l4 0 2024-06-15T01:07:00.415 INFO:tasks.workunit.client.1.smithi142.stdout:3/465: write d0/d1f/f46 [649555,93918] 0 2024-06-15T01:07:00.415 INFO:tasks.workunit.client.0.smithi137.stdout:9/186: fsync df/f24 0 2024-06-15T01:07:00.416 INFO:tasks.workunit.client.1.smithi142.stdout:5/375: write d2/de/d19/f1a [3812087,36094] 0 2024-06-15T01:07:00.419 INFO:tasks.workunit.client.1.smithi142.stdout:8/362: unlink d0/d17/d40/c52 0 2024-06-15T01:07:00.420 INFO:tasks.workunit.client.1.smithi142.stdout:6/357: chown d1a/f61 14119776 1 2024-06-15T01:07:00.420 INFO:tasks.workunit.client.1.smithi142.stdout:3/466: dread d0/d3/f12 [4194304,4194304] 0 2024-06-15T01:07:00.421 INFO:tasks.workunit.client.1.smithi142.stdout:6/358: stat d1a/d39/d42/d4c/d58/f7f 0 2024-06-15T01:07:00.422 INFO:tasks.workunit.client.1.smithi142.stdout:3/467: fsync d0/d3/d15/f83 0 2024-06-15T01:07:00.422 INFO:tasks.workunit.client.0.smithi137.stdout:9/187: dread f5 [0,4194304] 0 2024-06-15T01:07:00.423 INFO:tasks.workunit.client.0.smithi137.stdout:9/188: readlink df/d12/l19 0 2024-06-15T01:07:00.423 INFO:tasks.workunit.client.0.smithi137.stdout:9/189: stat ce 0 2024-06-15T01:07:00.424 INFO:tasks.workunit.client.1.smithi142.stdout:6/359: truncate d1a/d83/f7c 679151 0 2024-06-15T01:07:00.425 INFO:tasks.workunit.client.1.smithi142.stdout:1/412: truncate d6/d12/d28/f40 2634693 0 2024-06-15T01:07:00.426 INFO:tasks.workunit.client.0.smithi137.stdout:9/190: creat df/d2a/f34 x:0 0 0 2024-06-15T01:07:00.429 INFO:tasks.workunit.client.1.smithi142.stdout:5/376: symlink d2/d3e/d31/l75 0 2024-06-15T01:07:00.430 INFO:tasks.workunit.client.0.smithi137.stdout:9/191: getdents df/d12/d1b 0 2024-06-15T01:07:00.431 INFO:tasks.workunit.client.0.smithi137.stdout:1/363: truncate d0/d3/d1b/f69 3909122 0 2024-06-15T01:07:00.433 INFO:tasks.workunit.client.0.smithi137.stdout:1/364: write d0/d3/d1b/f4f [1460215,96514] 0 2024-06-15T01:07:00.434 INFO:tasks.workunit.client.1.smithi142.stdout:5/377: dwrite d2/de/f5a [0,4194304] 0 2024-06-15T01:07:00.436 INFO:tasks.workunit.client.1.smithi142.stdout:3/468: rename d0/f7e to d0/d3/da/d18/f8a 0 2024-06-15T01:07:00.437 INFO:tasks.workunit.client.0.smithi137.stdout:1/365: creat d0/d3/d47/f92 x:0 0 0 2024-06-15T01:07:00.438 INFO:tasks.workunit.client.1.smithi142.stdout:9/368: link d6/dc/d1b/d25/c27 d6/d31/c93 0 2024-06-15T01:07:00.438 INFO:tasks.workunit.client.0.smithi137.stdout:1/366: chown d0/d3/d14/l1a 72775 1 2024-06-15T01:07:00.439 INFO:tasks.workunit.client.1.smithi142.stdout:9/369: stat d6/dc/d1b/d25/d58/f63 0 2024-06-15T01:07:00.439 INFO:tasks.workunit.client.0.smithi137.stdout:9/192: dwrite df/d12/f17 [8388608,4194304] 0 2024-06-15T01:07:00.441 INFO:tasks.workunit.client.1.smithi142.stdout:1/413: mkdir d6/d25/d9e 0 2024-06-15T01:07:00.445 INFO:tasks.workunit.client.0.smithi137.stdout:1/367: fsync d0/d1f/f4d 0 2024-06-15T01:07:00.446 INFO:tasks.workunit.client.1.smithi142.stdout:5/378: fsync d2/de/d19/f51 0 2024-06-15T01:07:00.447 INFO:tasks.workunit.client.1.smithi142.stdout:5/379: fdatasync d2/d3e/f5c 0 2024-06-15T01:07:00.448 INFO:tasks.workunit.client.0.smithi137.stdout:4/287: dread dd/fe [0,4194304] 0 2024-06-15T01:07:00.449 INFO:tasks.workunit.client.1.smithi142.stdout:5/380: stat d2/de/d19/f1a 0 2024-06-15T01:07:00.450 INFO:tasks.workunit.client.1.smithi142.stdout:9/370: dread d6/dc/f15 [0,4194304] 0 2024-06-15T01:07:00.450 INFO:tasks.workunit.client.1.smithi142.stdout:5/381: dread - d2/f58 zero size 2024-06-15T01:07:00.450 INFO:tasks.workunit.client.0.smithi137.stdout:1/368: mkdir d0/d3/d1b/d4c/d59/d93 0 2024-06-15T01:07:00.451 INFO:tasks.workunit.client.0.smithi137.stdout:9/193: dwrite df/d12/f17 [0,4194304] 0 2024-06-15T01:07:00.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/194: read - df/d13/f30 zero size 2024-06-15T01:07:00.453 INFO:tasks.workunit.client.0.smithi137.stdout:4/288: dread dd/fe [0,4194304] 0 2024-06-15T01:07:00.453 INFO:tasks.workunit.client.1.smithi142.stdout:3/469: mknod d0/d3/da/d18/c8b 0 2024-06-15T01:07:00.456 INFO:tasks.workunit.client.0.smithi137.stdout:9/195: creat df/d12/d1b/f35 x:0 0 0 2024-06-15T01:07:00.457 INFO:tasks.workunit.client.0.smithi137.stdout:4/289: truncate dd/d15/d21/d3a/d51/f52 4832668 0 2024-06-15T01:07:00.457 INFO:tasks.workunit.client.1.smithi142.stdout:5/382: dread - d2/d18/d2e/f71 zero size 2024-06-15T01:07:00.457 INFO:tasks.workunit.client.1.smithi142.stdout:1/414: unlink d6/d46/d61/c67 0 2024-06-15T01:07:00.458 INFO:tasks.workunit.client.0.smithi137.stdout:4/290: write dd/d10/f5e [264088,109305] 0 2024-06-15T01:07:00.459 INFO:tasks.workunit.client.1.smithi142.stdout:3/470: chown d0/d1b/l4e 29 1 2024-06-15T01:07:00.459 INFO:tasks.workunit.client.1.smithi142.stdout:2/328: write d8/d19/f56 [715536,61102] 0 2024-06-15T01:07:00.459 INFO:tasks.workunit.client.0.smithi137.stdout:9/196: rename df/d12/c2d to df/d1a/c36 0 2024-06-15T01:07:00.461 INFO:tasks.workunit.client.1.smithi142.stdout:7/348: dwrite d2/d6/f69 [0,4194304] 0 2024-06-15T01:07:00.462 INFO:tasks.workunit.client.0.smithi137.stdout:4/291: creat dd/d15/d48/d67/f6c x:0 0 0 2024-06-15T01:07:00.464 INFO:tasks.workunit.client.1.smithi142.stdout:9/371: write d6/dc/d1b/f40 [515787,120142] 0 2024-06-15T01:07:00.469 INFO:tasks.workunit.client.1.smithi142.stdout:1/415: read - d6/d46/d61/f76 zero size 2024-06-15T01:07:00.470 INFO:tasks.workunit.client.1.smithi142.stdout:5/383: mkdir d2/de/d19/d5f/d68/d76 0 2024-06-15T01:07:00.473 INFO:tasks.workunit.client.1.smithi142.stdout:0/354: dwrite d3/d39/d3e/d81/d44/d7c/d22/f70 [0,4194304] 0 2024-06-15T01:07:00.473 INFO:tasks.workunit.client.1.smithi142.stdout:1/416: fsync d6/d12/d3c/d63/d65/f7b 0 2024-06-15T01:07:00.474 INFO:tasks.workunit.client.1.smithi142.stdout:2/329: rename f6 to d8/f65 0 2024-06-15T01:07:00.474 INFO:tasks.workunit.client.1.smithi142.stdout:5/384: truncate d2/de/d19/f22 13090178 0 2024-06-15T01:07:00.475 INFO:tasks.workunit.client.0.smithi137.stdout:1/369: dread d0/d3/f2f [0,4194304] 0 2024-06-15T01:07:00.476 INFO:tasks.workunit.client.1.smithi142.stdout:5/385: write d2/de/f5a [2638335,48180] 0 2024-06-15T01:07:00.476 INFO:tasks.workunit.client.0.smithi137.stdout:1/370: fdatasync d0/f3b 0 2024-06-15T01:07:00.478 INFO:tasks.workunit.client.1.smithi142.stdout:1/417: truncate d6/d12/d28/f5d 815703 0 2024-06-15T01:07:00.479 INFO:tasks.workunit.client.0.smithi137.stdout:1/371: creat d0/d1f/d67/f94 x:0 0 0 2024-06-15T01:07:00.481 INFO:tasks.workunit.client.0.smithi137.stdout:1/372: write d0/d10/d41/d7b/f90 [444050,95579] 0 2024-06-15T01:07:00.481 INFO:tasks.workunit.client.1.smithi142.stdout:0/355: chown d3/d39/d3e/d81/d1b/l64 4157528 1 2024-06-15T01:07:00.482 INFO:tasks.workunit.client.1.smithi142.stdout:1/418: dread d6/f10 [0,4194304] 0 2024-06-15T01:07:00.482 INFO:tasks.workunit.client.0.smithi137.stdout:2/245: sync 2024-06-15T01:07:00.482 INFO:tasks.workunit.client.0.smithi137.stdout:3/360: sync 2024-06-15T01:07:00.482 INFO:tasks.workunit.client.0.smithi137.stdout:7/275: sync 2024-06-15T01:07:00.483 INFO:tasks.workunit.client.1.smithi142.stdout:0/356: fsync d3/d39/d3e/d81/d1b/f35 0 2024-06-15T01:07:00.483 INFO:tasks.workunit.client.0.smithi137.stdout:0/324: sync 2024-06-15T01:07:00.483 INFO:tasks.workunit.client.0.smithi137.stdout:4/292: dread dd/d15/f42 [0,4194304] 0 2024-06-15T01:07:00.484 INFO:tasks.workunit.client.0.smithi137.stdout:7/276: fsync df/d18/d1c/d1d/f28 0 2024-06-15T01:07:00.485 INFO:tasks.workunit.client.1.smithi142.stdout:0/357: dread - d3/d39/d3e/d81/d1b/f40 zero size 2024-06-15T01:07:00.485 INFO:tasks.workunit.client.1.smithi142.stdout:7/349: dread d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:07:00.485 INFO:tasks.workunit.client.0.smithi137.stdout:4/293: chown dd/f28 29286222 1 2024-06-15T01:07:00.486 INFO:tasks.workunit.client.1.smithi142.stdout:2/330: creat d8/d19/d33/f66 x:0 0 0 2024-06-15T01:07:00.486 INFO:tasks.workunit.client.0.smithi137.stdout:3/361: write d0/d2/d23/f3f [2615812,2463] 0 2024-06-15T01:07:00.486 INFO:tasks.workunit.client.0.smithi137.stdout:1/373: creat d0/d3/d1b/d4c/f95 x:0 0 0 2024-06-15T01:07:00.487 INFO:tasks.workunit.client.1.smithi142.stdout:7/350: dread - d2/d8/da/d33/f52 zero size 2024-06-15T01:07:00.488 INFO:tasks.workunit.client.0.smithi137.stdout:2/246: truncate d0/dc/f3b 671727 0 2024-06-15T01:07:00.488 INFO:tasks.workunit.client.0.smithi137.stdout:4/294: read dd/f47 [203657,4393] 0 2024-06-15T01:07:00.488 INFO:tasks.workunit.client.1.smithi142.stdout:5/386: creat d2/de/d27/f77 x:0 0 0 2024-06-15T01:07:00.489 INFO:tasks.workunit.client.1.smithi142.stdout:8/363: truncate d0/d25/d5a/f2c 4051577 0 2024-06-15T01:07:00.489 INFO:tasks.workunit.client.1.smithi142.stdout:5/387: stat d2/de/d19/d5f/d68/f73 0 2024-06-15T01:07:00.489 INFO:tasks.workunit.client.1.smithi142.stdout:7/351: chown d2/d8/dd/d14/d2c/d37/c45 27528 1 2024-06-15T01:07:00.490 INFO:tasks.workunit.client.0.smithi137.stdout:0/325: write d1/d2/dc/d10/d14/d15/f59 [5222835,73911] 0 2024-06-15T01:07:00.490 INFO:tasks.workunit.client.0.smithi137.stdout:4/295: chown l4 0 1 2024-06-15T01:07:00.491 INFO:tasks.workunit.client.1.smithi142.stdout:3/471: getdents d0/d3/da/d71/d55 0 2024-06-15T01:07:00.491 INFO:tasks.workunit.client.0.smithi137.stdout:1/374: truncate d0/d3/d14/d34/f52 1261371 0 2024-06-15T01:07:00.492 INFO:tasks.workunit.client.0.smithi137.stdout:0/326: chown d1/d2/d55 652 1 2024-06-15T01:07:00.494 INFO:tasks.workunit.client.1.smithi142.stdout:6/360: truncate d1a/d2d/f2f 18965 0 2024-06-15T01:07:00.494 INFO:tasks.workunit.client.0.smithi137.stdout:2/247: fsync d0/de/d10/f2a 0 2024-06-15T01:07:00.495 INFO:tasks.workunit.client.1.smithi142.stdout:5/388: dread d2/d18/f55 [0,4194304] 0 2024-06-15T01:07:00.496 INFO:tasks.workunit.client.1.smithi142.stdout:5/389: fdatasync d2/f21 0 2024-06-15T01:07:00.496 INFO:tasks.workunit.client.1.smithi142.stdout:1/419: mknod d6/d3a/c9f 0 2024-06-15T01:07:00.497 INFO:tasks.workunit.client.1.smithi142.stdout:5/390: write d2/f21 [135141,120616] 0 2024-06-15T01:07:00.498 INFO:tasks.workunit.client.1.smithi142.stdout:2/331: creat d8/d19/d23/d25/d1a/d57/f67 x:0 0 0 2024-06-15T01:07:00.499 INFO:tasks.workunit.client.0.smithi137.stdout:4/296: symlink dd/d10/d1f/l6d 0 2024-06-15T01:07:00.499 INFO:tasks.workunit.client.1.smithi142.stdout:5/391: dread - d2/d18/f63 zero size 2024-06-15T01:07:00.499 INFO:tasks.workunit.client.1.smithi142.stdout:1/420: truncate d6/d12/d1c/d2c/f91 67250 0 2024-06-15T01:07:00.500 INFO:tasks.workunit.client.1.smithi142.stdout:0/358: stat d3/d39/d3e/d81/d44/d7c/f14 0 2024-06-15T01:07:00.501 INFO:tasks.workunit.client.1.smithi142.stdout:5/392: write d2/de/d19/f51 [361298,74491] 0 2024-06-15T01:07:00.501 INFO:tasks.workunit.client.0.smithi137.stdout:1/375: creat d0/d2d/d3e/f96 x:0 0 0 2024-06-15T01:07:00.502 INFO:tasks.workunit.client.1.smithi142.stdout:0/359: write d3/d39/d3e/d81/d1e/f63 [424652,5459] 0 2024-06-15T01:07:00.503 INFO:tasks.workunit.client.0.smithi137.stdout:3/362: dwrite d0/d2/d22/d24/f54 [0,4194304] 0 2024-06-15T01:07:00.506 INFO:tasks.workunit.client.0.smithi137.stdout:1/376: write d0/d3/d1b/d4c/f75 [264832,83714] 0 2024-06-15T01:07:00.507 INFO:tasks.workunit.client.1.smithi142.stdout:3/472: creat d0/d3/d15/f8c x:0 0 0 2024-06-15T01:07:00.507 INFO:tasks.workunit.client.0.smithi137.stdout:0/327: dwrite d1/d2/d55/f6e [0,4194304] 0 2024-06-15T01:07:00.509 INFO:tasks.workunit.client.0.smithi137.stdout:0/328: write d1/d2/f21 [1053777,47707] 0 2024-06-15T01:07:00.509 INFO:tasks.workunit.client.1.smithi142.stdout:6/361: readlink d1a/l1b 0 2024-06-15T01:07:00.511 INFO:tasks.workunit.client.0.smithi137.stdout:4/297: mkdir dd/d15/d21/d3a/d6e 0 2024-06-15T01:07:00.513 INFO:tasks.workunit.client.1.smithi142.stdout:3/473: dread d0/d3/da/f62 [0,4194304] 0 2024-06-15T01:07:00.514 INFO:tasks.workunit.client.0.smithi137.stdout:2/248: mkdir d0/d42/d49 0 2024-06-15T01:07:00.515 INFO:tasks.workunit.client.1.smithi142.stdout:7/352: dread d2/d6/f1d [0,4194304] 0 2024-06-15T01:07:00.516 INFO:tasks.workunit.client.0.smithi137.stdout:3/363: mkdir d0/d2/d1e/d7c 0 2024-06-15T01:07:00.516 INFO:tasks.workunit.client.1.smithi142.stdout:1/421: unlink d6/d12/d3c/d63/d65/f7b 0 2024-06-15T01:07:00.517 INFO:tasks.workunit.client.1.smithi142.stdout:2/332: creat d8/d19/d23/d25/d1a/d57/f68 x:0 0 0 2024-06-15T01:07:00.519 INFO:tasks.workunit.client.0.smithi137.stdout:1/377: mknod d0/d3/d89/c97 0 2024-06-15T01:07:00.520 INFO:tasks.workunit.client.0.smithi137.stdout:7/277: getdents df/d13/d2e/d1a/d3f 0 2024-06-15T01:07:00.521 INFO:tasks.workunit.client.1.smithi142.stdout:3/474: dread d0/d3/da/d18/f87 [0,4194304] 0 2024-06-15T01:07:00.521 INFO:tasks.workunit.client.0.smithi137.stdout:7/278: fdatasync df/d13/d2e/f4e 0 2024-06-15T01:07:00.522 INFO:tasks.workunit.client.0.smithi137.stdout:0/329: truncate d1/d2/d13/f33 689348 0 2024-06-15T01:07:00.524 INFO:tasks.workunit.client.0.smithi137.stdout:4/298: mknod dd/d15/d30/c6f 0 2024-06-15T01:07:00.525 INFO:tasks.workunit.client.0.smithi137.stdout:2/249: chown d0/dc/d40/c43 368851 1 2024-06-15T01:07:00.525 INFO:tasks.workunit.client.0.smithi137.stdout:3/364: readlink d0/d2/d8/d67/l71 0 2024-06-15T01:07:00.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/378: symlink d0/d10/d41/l98 0 2024-06-15T01:07:00.527 INFO:tasks.workunit.client.1.smithi142.stdout:8/364: creat d0/d17/f7c x:0 0 0 2024-06-15T01:07:00.528 INFO:tasks.workunit.client.0.smithi137.stdout:7/279: rename df/d13/d2e/d1a/f30 to df/d13/d2e/f58 0 2024-06-15T01:07:00.530 INFO:tasks.workunit.client.0.smithi137.stdout:7/280: write df/d18/d1c/d3a/f37 [2488583,96890] 0 2024-06-15T01:07:00.530 INFO:tasks.workunit.client.0.smithi137.stdout:4/299: creat dd/d10/f70 x:0 0 0 2024-06-15T01:07:00.531 INFO:tasks.workunit.client.1.smithi142.stdout:6/362: dwrite d1a/f38 [0,4194304] 0 2024-06-15T01:07:00.532 INFO:tasks.workunit.client.1.smithi142.stdout:2/333: creat d8/d19/d23/d25/d1a/d57/f69 x:0 0 0 2024-06-15T01:07:00.533 INFO:tasks.workunit.client.1.smithi142.stdout:5/393: symlink d2/d3e/d74/l78 0 2024-06-15T01:07:00.534 INFO:tasks.workunit.client.0.smithi137.stdout:2/250: mknod d0/de/d10/d3a/c4a 0 2024-06-15T01:07:00.535 INFO:tasks.workunit.client.0.smithi137.stdout:1/379: write d0/d3/f22 [1086876,9784] 0 2024-06-15T01:07:00.536 INFO:tasks.workunit.client.0.smithi137.stdout:3/365: creat d0/d2/d22/d63/d76/f7d x:0 0 0 2024-06-15T01:07:00.536 INFO:tasks.workunit.client.1.smithi142.stdout:0/360: mknod d3/d39/c8a 0 2024-06-15T01:07:00.538 INFO:tasks.workunit.client.0.smithi137.stdout:4/300: mknod dd/d15/c71 0 2024-06-15T01:07:00.540 INFO:tasks.workunit.client.1.smithi142.stdout:8/365: chown d0/d25/c64 1908 1 2024-06-15T01:07:00.540 INFO:tasks.workunit.client.0.smithi137.stdout:4/301: chown dd/d15/f2e 10025305 1 2024-06-15T01:07:00.541 INFO:tasks.workunit.client.1.smithi142.stdout:2/334: dwrite d8/d19/d23/d25/f27 [0,4194304] 0 2024-06-15T01:07:00.541 INFO:tasks.workunit.client.1.smithi142.stdout:8/366: read d0/d17/f61 [4192306,53878] 0 2024-06-15T01:07:00.541 INFO:tasks.workunit.client.1.smithi142.stdout:0/361: dread d3/d39/d3e/d81/f73 [0,4194304] 0 2024-06-15T01:07:00.542 INFO:tasks.workunit.client.1.smithi142.stdout:2/335: dread - d8/df/d10/d1c/f2c zero size 2024-06-15T01:07:00.542 INFO:tasks.workunit.client.0.smithi137.stdout:4/302: chown dd/d15/d30/l3d 41963 1 2024-06-15T01:07:00.543 INFO:tasks.workunit.client.0.smithi137.stdout:7/281: mkdir df/d18/d1c/d1d/d27/d33/d59 0 2024-06-15T01:07:00.544 INFO:tasks.workunit.client.0.smithi137.stdout:2/251: creat d0/dc/d22/f4b x:0 0 0 2024-06-15T01:07:00.547 INFO:tasks.workunit.client.0.smithi137.stdout:1/380: dwrite d0/d10/f2e [4194304,4194304] 0 2024-06-15T01:07:00.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/282: fsync df/d13/d2e/f4f 0 2024-06-15T01:07:00.548 INFO:tasks.workunit.client.1.smithi142.stdout:2/336: dread d8/d19/d23/d25/d1a/f2e [0,4194304] 0 2024-06-15T01:07:00.549 INFO:tasks.workunit.client.1.smithi142.stdout:1/422: mkdir d6/d12/da0 0 2024-06-15T01:07:00.550 INFO:tasks.workunit.client.1.smithi142.stdout:3/475: dread d0/d3/f12 [0,4194304] 0 2024-06-15T01:07:00.550 INFO:tasks.workunit.client.0.smithi137.stdout:3/366: rename d0/d2/d50 to d0/d2/d8/d67/d7e 0 2024-06-15T01:07:00.551 INFO:tasks.workunit.client.0.smithi137.stdout:4/303: mkdir dd/d15/d48/d72 0 2024-06-15T01:07:00.552 INFO:tasks.workunit.client.1.smithi142.stdout:7/353: truncate d2/d8/da/d33/f35 1016914 0 2024-06-15T01:07:00.552 INFO:tasks.workunit.client.1.smithi142.stdout:6/363: creat d1a/d39/d3a/f84 x:0 0 0 2024-06-15T01:07:00.553 INFO:tasks.workunit.client.1.smithi142.stdout:7/354: dread - d2/d8/da/d33/f52 zero size 2024-06-15T01:07:00.554 INFO:tasks.workunit.client.1.smithi142.stdout:6/364: dread - d1a/d39/d3a/f84 zero size 2024-06-15T01:07:00.554 INFO:tasks.workunit.client.1.smithi142.stdout:7/355: chown d2/d8/dd 3 1 2024-06-15T01:07:00.555 INFO:tasks.workunit.client.1.smithi142.stdout:7/356: chown d2/d6/c5f 832976 1 2024-06-15T01:07:00.558 INFO:tasks.workunit.client.1.smithi142.stdout:0/362: creat d3/d39/d3e/f8b x:0 0 0 2024-06-15T01:07:00.560 INFO:tasks.workunit.client.0.smithi137.stdout:4/304: dwrite dd/d10/f54 [0,4194304] 0 2024-06-15T01:07:00.560 INFO:tasks.workunit.client.1.smithi142.stdout:7/357: dread d2/d8/da/d33/d25/d27/f5d [0,4194304] 0 2024-06-15T01:07:00.560 INFO:tasks.workunit.client.0.smithi137.stdout:7/283: creat df/d18/f5a x:0 0 0 2024-06-15T01:07:00.561 INFO:tasks.workunit.client.1.smithi142.stdout:7/358: write d2/d8/f42 [1361624,113777] 0 2024-06-15T01:07:00.561 INFO:tasks.workunit.client.1.smithi142.stdout:2/337: chown l5 512716067 1 2024-06-15T01:07:00.562 INFO:tasks.workunit.client.1.smithi142.stdout:7/359: read - d2/d6/d22/d47/f5a zero size 2024-06-15T01:07:00.565 INFO:tasks.workunit.client.0.smithi137.stdout:3/367: fdatasync d0/fd 0 2024-06-15T01:07:00.567 INFO:tasks.workunit.client.0.smithi137.stdout:0/330: dwrite d1/d2/dc/d10/d14/d15/d1c/f1d [4194304,4194304] 0 2024-06-15T01:07:00.568 INFO:tasks.workunit.client.0.smithi137.stdout:3/368: stat d0/d2/d22/d24/f47 0 2024-06-15T01:07:00.571 INFO:tasks.workunit.client.0.smithi137.stdout:0/331: write d1/d2/dc/d10/d14/d52/f58 [244005,105644] 0 2024-06-15T01:07:00.572 INFO:tasks.workunit.client.0.smithi137.stdout:0/332: chown d1/d2 97 1 2024-06-15T01:07:00.572 INFO:tasks.workunit.client.0.smithi137.stdout:2/252: rmdir d0/de/d41 0 2024-06-15T01:07:00.579 INFO:tasks.workunit.client.0.smithi137.stdout:2/253: dwrite d0/da/f18 [4194304,4194304] 0 2024-06-15T01:07:00.620 INFO:tasks.workunit.client.1.smithi142.stdout:2/338: dread d8/df/d10/f35 [0,4194304] 0 2024-06-15T01:07:00.621 INFO:tasks.workunit.client.0.smithi137.stdout:4/305: creat dd/d15/d21/d3a/f73 x:0 0 0 2024-06-15T01:07:00.622 INFO:tasks.workunit.client.0.smithi137.stdout:4/306: fdatasync dd/f12 0 2024-06-15T01:07:00.623 INFO:tasks.workunit.client.1.smithi142.stdout:3/476: symlink d0/d1f/d20/d5d/l8d 0 2024-06-15T01:07:00.625 INFO:tasks.workunit.client.1.smithi142.stdout:6/365: getdents d1a/d2d/d80 0 2024-06-15T01:07:00.628 INFO:tasks.workunit.client.1.smithi142.stdout:0/363: creat d3/d39/d3e/d81/d1b/d38/f8c x:0 0 0 2024-06-15T01:07:00.629 INFO:tasks.workunit.client.0.smithi137.stdout:0/333: mknod d1/d2/dc/d10/d14/d52/c72 0 2024-06-15T01:07:00.632 INFO:tasks.workunit.client.1.smithi142.stdout:2/339: rmdir d8/df 39 2024-06-15T01:07:00.633 INFO:tasks.workunit.client.1.smithi142.stdout:2/340: stat d8/d19/d23/f5d 0 2024-06-15T01:07:00.634 INFO:tasks.workunit.client.0.smithi137.stdout:4/307: creat dd/d15/d30/d49/f74 x:0 0 0 2024-06-15T01:07:00.635 INFO:tasks.workunit.client.0.smithi137.stdout:4/308: write dd/f28 [1160388,89859] 0 2024-06-15T01:07:00.636 INFO:tasks.workunit.client.1.smithi142.stdout:5/394: getdents d2/d3e 0 2024-06-15T01:07:00.636 INFO:tasks.workunit.client.1.smithi142.stdout:3/477: creat d0/d1f/d20/d59/f8e x:0 0 0 2024-06-15T01:07:00.637 INFO:tasks.workunit.client.0.smithi137.stdout:0/334: creat d1/d2/dc/f73 x:0 0 0 2024-06-15T01:07:00.639 INFO:tasks.workunit.client.1.smithi142.stdout:6/366: fdatasync d1a/f1f 0 2024-06-15T01:07:00.639 INFO:tasks.workunit.client.0.smithi137.stdout:2/254: symlink d0/da/d3e/l4c 0 2024-06-15T01:07:00.640 INFO:tasks.workunit.client.1.smithi142.stdout:8/367: getdents d0/d25/d53 0 2024-06-15T01:07:00.641 INFO:tasks.workunit.client.0.smithi137.stdout:2/255: write d0/dc/d22/f4b [855605,93172] 0 2024-06-15T01:07:00.641 INFO:tasks.workunit.client.1.smithi142.stdout:0/364: rename d3/d39/d3e/d81/f33 to d3/d39/d3e/d81/f8d 0 2024-06-15T01:07:00.642 INFO:tasks.workunit.client.1.smithi142.stdout:8/368: dread - d0/d17/d40/f4a zero size 2024-06-15T01:07:00.643 INFO:tasks.workunit.client.0.smithi137.stdout:4/309: dwrite dd/d10/f63 [0,4194304] 0 2024-06-15T01:07:00.644 INFO:tasks.workunit.client.1.smithi142.stdout:3/478: dwrite d0/d1b/f88 [0,4194304] 0 2024-06-15T01:07:00.644 INFO:tasks.workunit.client.1.smithi142.stdout:5/395: fsync d2/de/d19/f2a 0 2024-06-15T01:07:00.649 INFO:tasks.workunit.client.1.smithi142.stdout:6/367: symlink d1a/d39/d3a/d51/l85 0 2024-06-15T01:07:00.653 INFO:tasks.workunit.client.1.smithi142.stdout:0/365: creat d3/d39/d3e/d81/d1b/d38/f8e x:0 0 0 2024-06-15T01:07:00.654 INFO:tasks.workunit.client.1.smithi142.stdout:0/366: stat d3/d39/d3e/d81/d1b/d86 0 2024-06-15T01:07:00.654 INFO:tasks.workunit.client.1.smithi142.stdout:3/479: symlink d0/d1f/d20/d5d/d82/l8f 0 2024-06-15T01:07:00.655 INFO:tasks.workunit.client.1.smithi142.stdout:1/423: truncate d6/f13 2779610 0 2024-06-15T01:07:00.657 INFO:tasks.workunit.client.1.smithi142.stdout:6/368: dread d1a/d2d/d2e/f49 [0,4194304] 0 2024-06-15T01:07:00.657 INFO:tasks.workunit.client.1.smithi142.stdout:0/367: mkdir d3/d39/d3e/d8f 0 2024-06-15T01:07:00.658 INFO:tasks.workunit.client.1.smithi142.stdout:1/424: rmdir d6/d3a 39 2024-06-15T01:07:00.660 INFO:tasks.workunit.client.1.smithi142.stdout:7/360: write d2/d8/da/d33/f35 [1986855,29668] 0 2024-06-15T01:07:00.661 INFO:tasks.workunit.client.1.smithi142.stdout:7/361: fdatasync d2/d8/f42 0 2024-06-15T01:07:00.663 INFO:tasks.workunit.client.1.smithi142.stdout:1/425: creat d6/d12/fa1 x:0 0 0 2024-06-15T01:07:00.664 INFO:tasks.workunit.client.1.smithi142.stdout:0/368: dread d3/d39/d3e/d81/f1d [0,4194304] 0 2024-06-15T01:07:00.665 INFO:tasks.workunit.client.1.smithi142.stdout:7/362: mknod d2/d8/dd/d14/d2c/d37/c71 0 2024-06-15T01:07:00.665 INFO:tasks.workunit.client.1.smithi142.stdout:0/369: dread - d3/d39/d3e/d81/f7e zero size 2024-06-15T01:07:00.666 INFO:tasks.workunit.client.1.smithi142.stdout:7/363: fdatasync d2/d8/da/d33/f35 0 2024-06-15T01:07:00.669 INFO:tasks.workunit.client.1.smithi142.stdout:6/369: write d1a/d2d/d2e/f49 [901763,51503] 0 2024-06-15T01:07:00.670 INFO:tasks.workunit.client.1.smithi142.stdout:7/364: creat d2/d6/d22/d47/f72 x:0 0 0 2024-06-15T01:07:00.671 INFO:tasks.workunit.client.1.smithi142.stdout:8/369: dread d0/d25/f35 [0,4194304] 0 2024-06-15T01:07:00.673 INFO:tasks.workunit.client.1.smithi142.stdout:1/426: truncate d6/d25/f77 1570523 0 2024-06-15T01:07:00.674 INFO:tasks.workunit.client.1.smithi142.stdout:2/341: dwrite d8/fc [4194304,4194304] 0 2024-06-15T01:07:00.674 INFO:tasks.workunit.client.1.smithi142.stdout:0/370: getdents d3/d39/d3e/d81/d44 0 2024-06-15T01:07:00.675 INFO:tasks.workunit.client.1.smithi142.stdout:1/427: chown d6/d12/d28/f33 2315679 1 2024-06-15T01:07:00.676 INFO:tasks.workunit.client.1.smithi142.stdout:8/370: unlink d0/d17/l3c 0 2024-06-15T01:07:00.676 INFO:tasks.workunit.client.1.smithi142.stdout:6/370: mknod d1a/d2d/d80/c86 0 2024-06-15T01:07:00.678 INFO:tasks.workunit.client.1.smithi142.stdout:7/365: write d2/d8/da/d33/d25/d27/f5d [2757644,25625] 0 2024-06-15T01:07:00.680 INFO:tasks.workunit.client.1.smithi142.stdout:0/371: chown d3/d39/d3e/d81/d4c/c53 161132 1 2024-06-15T01:07:00.681 INFO:tasks.workunit.client.1.smithi142.stdout:2/342: rename d8/f65 to d8/d19/d23/d25/f6a 0 2024-06-15T01:07:00.681 INFO:tasks.workunit.client.1.smithi142.stdout:7/366: write d2/d6/d22/d3c/f56 [626280,77397] 0 2024-06-15T01:07:00.684 INFO:tasks.workunit.client.1.smithi142.stdout:7/367: readlink d2/d8/dd/l5b 0 2024-06-15T01:07:00.685 INFO:tasks.workunit.client.1.smithi142.stdout:1/428: dread d6/d12/d5b/f8a [0,4194304] 0 2024-06-15T01:07:00.686 INFO:tasks.workunit.client.1.smithi142.stdout:7/368: fdatasync d2/f2e 0 2024-06-15T01:07:00.689 INFO:tasks.workunit.client.1.smithi142.stdout:6/371: dread d1a/d39/d42/d4c/d58/f6a [0,4194304] 0 2024-06-15T01:07:00.690 INFO:tasks.workunit.client.1.smithi142.stdout:6/372: fdatasync d1a/f28 0 2024-06-15T01:07:00.691 INFO:tasks.workunit.client.1.smithi142.stdout:2/343: dwrite f4 [0,4194304] 0 2024-06-15T01:07:00.693 INFO:tasks.workunit.client.1.smithi142.stdout:7/369: symlink d2/d8/dd/d14/d2c/d70/l73 0 2024-06-15T01:07:00.694 INFO:tasks.workunit.client.1.smithi142.stdout:0/372: getdents d3/d39/d3e/d81/d1b/d38 0 2024-06-15T01:07:00.695 INFO:tasks.workunit.client.1.smithi142.stdout:0/373: readlink d3/l7b 0 2024-06-15T01:07:00.695 INFO:tasks.workunit.client.1.smithi142.stdout:6/373: mkdir d1a/d39/d42/d4c/d58/d87 0 2024-06-15T01:07:00.695 INFO:tasks.workunit.client.1.smithi142.stdout:2/344: chown d8/df/f48 702102738 1 2024-06-15T01:07:00.697 INFO:tasks.workunit.client.1.smithi142.stdout:1/429: getdents d6/d12/d3c/d63/d8d/d97 0 2024-06-15T01:07:00.697 INFO:tasks.workunit.client.1.smithi142.stdout:0/374: chown d3/d39/d3e/d81/c50 3417 1 2024-06-15T01:07:00.699 INFO:tasks.workunit.client.1.smithi142.stdout:7/370: write d2/f3 [2298567,88926] 0 2024-06-15T01:07:00.700 INFO:tasks.workunit.client.1.smithi142.stdout:8/371: dread d0/f33 [0,4194304] 0 2024-06-15T01:07:00.701 INFO:tasks.workunit.client.1.smithi142.stdout:2/345: rmdir d8 39 2024-06-15T01:07:00.701 INFO:tasks.workunit.client.1.smithi142.stdout:0/375: read - d3/d39/f7d zero size 2024-06-15T01:07:00.701 INFO:tasks.workunit.client.1.smithi142.stdout:7/371: fdatasync d2/d6/d22/d3c/f56 0 2024-06-15T01:07:00.704 INFO:tasks.workunit.client.1.smithi142.stdout:6/374: mknod d1a/d39/d42/d4c/c88 0 2024-06-15T01:07:00.705 INFO:tasks.workunit.client.1.smithi142.stdout:1/430: mkdir d6/d4d/da2 0 2024-06-15T01:07:00.707 INFO:tasks.workunit.client.1.smithi142.stdout:8/372: truncate d0/d17/d40/f4a 261629 0 2024-06-15T01:07:00.708 INFO:tasks.workunit.client.1.smithi142.stdout:8/373: write d0/f14 [2093753,50406] 0 2024-06-15T01:07:00.708 INFO:tasks.workunit.client.1.smithi142.stdout:7/372: mknod d2/d6/d22/c74 0 2024-06-15T01:07:00.708 INFO:tasks.workunit.client.1.smithi142.stdout:0/376: symlink d3/d39/d3e/d81/d4c/d61/l90 0 2024-06-15T01:07:00.710 INFO:tasks.workunit.client.1.smithi142.stdout:0/377: fsync d3/d39/d3e/d81/d44/d7c/d22/f82 0 2024-06-15T01:07:00.710 INFO:tasks.workunit.client.1.smithi142.stdout:6/375: rename d1a/d39/d42/d4c/f7b to d1a/d39/d42/d4c/d58/d87/f89 0 2024-06-15T01:07:00.711 INFO:tasks.workunit.client.1.smithi142.stdout:7/373: rmdir d2 39 2024-06-15T01:07:00.712 INFO:tasks.workunit.client.1.smithi142.stdout:8/374: readlink d0/d25/d5a/l3b 0 2024-06-15T01:07:00.714 INFO:tasks.workunit.client.1.smithi142.stdout:6/376: symlink d1a/d39/l8a 0 2024-06-15T01:07:00.715 INFO:tasks.workunit.client.1.smithi142.stdout:0/378: creat d3/d39/d3e/d81/d4c/d61/f91 x:0 0 0 2024-06-15T01:07:00.716 INFO:tasks.workunit.client.1.smithi142.stdout:2/346: dwrite d8/f9 [0,4194304] 0 2024-06-15T01:07:00.716 INFO:tasks.workunit.client.1.smithi142.stdout:8/375: symlink d0/d17/d40/d5e/l7d 0 2024-06-15T01:07:00.717 INFO:tasks.workunit.client.1.smithi142.stdout:7/374: dread - d2/d6/d22/d47/f5a zero size 2024-06-15T01:07:00.718 INFO:tasks.workunit.client.1.smithi142.stdout:6/377: creat d1a/d2d/d2e/f8b x:0 0 0 2024-06-15T01:07:00.719 INFO:tasks.workunit.client.1.smithi142.stdout:8/376: creat d0/d25/f7e x:0 0 0 2024-06-15T01:07:00.720 INFO:tasks.workunit.client.1.smithi142.stdout:0/379: mknod d3/d39/d3e/d81/d1b/d38/c92 0 2024-06-15T01:07:00.723 INFO:tasks.workunit.client.1.smithi142.stdout:6/378: mknod d1a/d39/d42/c8c 0 2024-06-15T01:07:00.723 INFO:tasks.workunit.client.1.smithi142.stdout:1/431: link d6/d3a/c57 d6/d46/d61/ca3 0 2024-06-15T01:07:00.725 INFO:tasks.workunit.client.1.smithi142.stdout:2/347: creat d8/f6b x:0 0 0 2024-06-15T01:07:00.725 INFO:tasks.workunit.client.1.smithi142.stdout:7/375: getdents d2/d6/d22/d34 0 2024-06-15T01:07:00.727 INFO:tasks.workunit.client.1.smithi142.stdout:0/380: rename d3/d39/d3e/l76 to d3/d39/d3e/d8f/l93 0 2024-06-15T01:07:00.727 INFO:tasks.workunit.client.1.smithi142.stdout:6/379: mknod d1a/d39/d42/c8d 0 2024-06-15T01:07:00.728 INFO:tasks.workunit.client.1.smithi142.stdout:1/432: symlink d6/d25/la4 0 2024-06-15T01:07:00.728 INFO:tasks.workunit.client.1.smithi142.stdout:7/376: rmdir d2 39 2024-06-15T01:07:00.728 INFO:tasks.workunit.client.1.smithi142.stdout:6/380: chown d1a/d39/d3a/d51/c7d 109 1 2024-06-15T01:07:00.730 INFO:tasks.workunit.client.1.smithi142.stdout:0/381: unlink d3/d39/d3e/f8b 0 2024-06-15T01:07:00.730 INFO:tasks.workunit.client.1.smithi142.stdout:0/382: fsync d3/d39/f7d 0 2024-06-15T01:07:00.734 INFO:tasks.workunit.client.1.smithi142.stdout:2/348: dread d8/d19/d23/d25/d1a/d57/f2b [0,4194304] 0 2024-06-15T01:07:00.735 INFO:tasks.workunit.client.1.smithi142.stdout:1/433: rename d6/c17 to d6/d12/ca5 0 2024-06-15T01:07:00.737 INFO:tasks.workunit.client.1.smithi142.stdout:4/486: sync 2024-06-15T01:07:00.738 INFO:tasks.workunit.client.1.smithi142.stdout:1/434: write d6/d12/d28/f64 [522396,55913] 0 2024-06-15T01:07:00.739 INFO:tasks.workunit.client.1.smithi142.stdout:0/383: dwrite d3/d39/d3e/d81/f87 [0,4194304] 0 2024-06-15T01:07:00.739 INFO:tasks.workunit.client.1.smithi142.stdout:2/349: write d8/d19/d23/d25/d22/d43/f46 [3586575,46736] 0 2024-06-15T01:07:00.743 INFO:tasks.workunit.client.1.smithi142.stdout:1/435: truncate d6/d12/d52/f89 403611 0 2024-06-15T01:07:00.745 INFO:tasks.workunit.client.1.smithi142.stdout:0/384: fsync d3/d39/d3e/d81/d44/d7c/d22/d59/f80 0 2024-06-15T01:07:00.748 INFO:tasks.workunit.client.1.smithi142.stdout:1/436: write d6/d12/f2d [812472,92001] 0 2024-06-15T01:07:00.750 INFO:tasks.workunit.client.1.smithi142.stdout:0/385: chown d3/d39/d3e/d81/d44/d7c/l25 38202 1 2024-06-15T01:07:00.752 INFO:tasks.workunit.client.1.smithi142.stdout:7/377: dread d2/f2e [0,4194304] 0 2024-06-15T01:07:00.753 INFO:tasks.workunit.client.1.smithi142.stdout:0/386: write d3/d39/d3e/d81/d1b/d38/f8e [923447,122926] 0 2024-06-15T01:07:00.754 INFO:tasks.workunit.client.1.smithi142.stdout:6/381: write d1a/f1e [4576302,100032] 0 2024-06-15T01:07:00.754 INFO:tasks.workunit.client.1.smithi142.stdout:0/387: write d3/f4 [2741861,102804] 0 2024-06-15T01:07:00.755 INFO:tasks.workunit.client.0.smithi137.stdout:0/335: symlink d1/d2/d13/d66/l74 0 2024-06-15T01:07:00.755 INFO:tasks.workunit.client.1.smithi142.stdout:2/350: getdents d8/df/d10 0 2024-06-15T01:07:00.758 INFO:tasks.workunit.client.1.smithi142.stdout:1/437: rename d6/d12/d1c/f53 to d6/d46/d61/fa6 0 2024-06-15T01:07:00.758 INFO:tasks.workunit.client.1.smithi142.stdout:1/438: chown d6/d12/d3c/d63/f85 838488795 1 2024-06-15T01:07:00.759 INFO:tasks.workunit.client.0.smithi137.stdout:7/284: fsync df/d18/d1c/d1d/d27/d33/f52 0 2024-06-15T01:07:00.760 INFO:tasks.workunit.client.0.smithi137.stdout:7/285: chown df/d13/l44 212284 1 2024-06-15T01:07:00.761 INFO:tasks.workunit.client.1.smithi142.stdout:7/378: creat d2/d8/dd/d14/d2c/d37/f75 x:0 0 0 2024-06-15T01:07:00.761 INFO:tasks.workunit.client.0.smithi137.stdout:7/286: chown df/f32 3806246 1 2024-06-15T01:07:00.762 INFO:tasks.workunit.client.1.smithi142.stdout:6/382: fdatasync d1a/f1e 0 2024-06-15T01:07:00.762 INFO:tasks.workunit.client.0.smithi137.stdout:0/336: creat d1/d2/dc/d10/f75 x:0 0 0 2024-06-15T01:07:00.762 INFO:tasks.workunit.client.1.smithi142.stdout:6/383: chown f6 7780 1 2024-06-15T01:07:00.763 INFO:tasks.workunit.client.1.smithi142.stdout:2/351: mknod d8/d19/d33/c6c 0 2024-06-15T01:07:00.763 INFO:tasks.workunit.client.0.smithi137.stdout:7/287: write df/d13/d2e/d1a/f26 [656198,42127] 0 2024-06-15T01:07:00.763 INFO:tasks.workunit.client.0.smithi137.stdout:0/337: fdatasync d1/d2/f47 0 2024-06-15T01:07:00.763 INFO:tasks.workunit.client.1.smithi142.stdout:6/384: readlink d1a/d39/d3a/l75 0 2024-06-15T01:07:00.764 INFO:tasks.workunit.client.0.smithi137.stdout:4/310: truncate dd/f18 2069853 0 2024-06-15T01:07:00.765 INFO:tasks.workunit.client.1.smithi142.stdout:6/385: chown d1a/f38 1805892 1 2024-06-15T01:07:00.765 INFO:tasks.workunit.client.1.smithi142.stdout:1/439: symlink d6/d12/d1c/d2c/la7 0 2024-06-15T01:07:00.767 INFO:tasks.workunit.client.1.smithi142.stdout:1/440: write d6/d12/f3e [4335479,5179] 0 2024-06-15T01:07:00.768 INFO:tasks.workunit.client.0.smithi137.stdout:3/369: truncate d0/d2/d22/d24/f47 8229551 0 2024-06-15T01:07:00.768 INFO:tasks.workunit.client.1.smithi142.stdout:7/379: chown d2/f1a 738088937 1 2024-06-15T01:07:00.769 INFO:tasks.workunit.client.0.smithi137.stdout:3/370: stat d0/d2/d22/d63/d76/f7d 0 2024-06-15T01:07:00.770 INFO:tasks.workunit.client.0.smithi137.stdout:2/256: rename d0/da/d3e to d0/d42/d4d 0 2024-06-15T01:07:00.770 INFO:tasks.workunit.client.1.smithi142.stdout:6/386: readlink d1a/l1d 0 2024-06-15T01:07:00.772 INFO:tasks.workunit.client.1.smithi142.stdout:1/441: dread d6/d12/d3c/f54 [0,4194304] 0 2024-06-15T01:07:00.772 INFO:tasks.workunit.client.1.smithi142.stdout:4/487: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f37 [0,4194304] 0 2024-06-15T01:07:00.775 INFO:tasks.workunit.client.1.smithi142.stdout:0/388: write d3/d39/f6f [3578098,81111] 0 2024-06-15T01:07:00.776 INFO:tasks.workunit.client.1.smithi142.stdout:0/389: fsync d3/d39/d3e/d81/d44/d7c/d22/d59/f80 0 2024-06-15T01:07:00.776 INFO:tasks.workunit.client.1.smithi142.stdout:2/352: link d8/d19/d33/f42 d8/d19/d23/d25/d22/f6d 0 2024-06-15T01:07:00.778 INFO:tasks.workunit.client.1.smithi142.stdout:0/390: read - d3/d39/d3e/d81/d44/d7c/d22/f36 zero size 2024-06-15T01:07:00.778 INFO:tasks.workunit.client.1.smithi142.stdout:0/391: chown d3/f28 47 1 2024-06-15T01:07:00.779 INFO:tasks.workunit.client.1.smithi142.stdout:6/387: mkdir d1a/d2d/d80/d8e 0 2024-06-15T01:07:00.782 INFO:tasks.workunit.client.1.smithi142.stdout:4/488: creat d5/d9/d18/fa8 x:0 0 0 2024-06-15T01:07:00.784 INFO:tasks.workunit.client.0.smithi137.stdout:4/311: dread dd/f22 [0,4194304] 0 2024-06-15T01:07:00.784 INFO:tasks.workunit.client.1.smithi142.stdout:2/353: creat d8/df/d1f/f6e x:0 0 0 2024-06-15T01:07:00.785 INFO:tasks.workunit.client.0.smithi137.stdout:3/371: creat d0/d2/d8/f7f x:0 0 0 2024-06-15T01:07:00.786 INFO:tasks.workunit.client.1.smithi142.stdout:7/380: link d2/d8/da/f55 d2/d8/dd/f76 0 2024-06-15T01:07:00.787 INFO:tasks.workunit.client.0.smithi137.stdout:3/372: write d0/d2/d22/d24/f54 [1824868,88615] 0 2024-06-15T01:07:00.787 INFO:tasks.workunit.client.0.smithi137.stdout:2/257: mkdir d0/da/d4e 0 2024-06-15T01:07:00.788 INFO:tasks.workunit.client.1.smithi142.stdout:4/489: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f43 [0,4194304] 0 2024-06-15T01:07:00.791 INFO:tasks.workunit.client.0.smithi137.stdout:4/312: mkdir dd/d15/d30/d49/d75 0 2024-06-15T01:07:00.792 INFO:tasks.workunit.client.0.smithi137.stdout:4/313: rename dd/d15/d21 to dd/d15/d21/d3a/d51/d76 22 2024-06-15T01:07:00.793 INFO:tasks.workunit.client.1.smithi142.stdout:6/388: link d1a/d2d/d80/c86 d1a/d3c/c8f 0 2024-06-15T01:07:00.794 INFO:tasks.workunit.client.0.smithi137.stdout:3/373: symlink d0/d2/d22/d63/l80 0 2024-06-15T01:07:00.795 INFO:tasks.workunit.client.1.smithi142.stdout:0/392: getdents d3/d39/d3e/d81/d1e 0 2024-06-15T01:07:00.796 INFO:tasks.workunit.client.0.smithi137.stdout:2/258: creat d0/dc/d40/f4f x:0 0 0 2024-06-15T01:07:00.797 INFO:tasks.workunit.client.0.smithi137.stdout:7/288: mkdir df/d18/d3b/d5b 0 2024-06-15T01:07:00.800 INFO:tasks.workunit.client.0.smithi137.stdout:4/314: symlink dd/d15/d48/d67/l77 0 2024-06-15T01:07:00.800 INFO:tasks.workunit.client.1.smithi142.stdout:4/490: dwrite d5/d1c/d2a/d30/d46/d50/f5e [4194304,4194304] 0 2024-06-15T01:07:00.801 INFO:tasks.workunit.client.1.smithi142.stdout:0/393: creat d3/d39/d3e/d81/d44/d7c/d22/d59/f94 x:0 0 0 2024-06-15T01:07:00.803 INFO:tasks.workunit.client.0.smithi137.stdout:3/374: unlink d0/d2/d22/c2b 0 2024-06-15T01:07:00.804 INFO:tasks.workunit.client.0.smithi137.stdout:3/375: readlink d0/d2/l38 0 2024-06-15T01:07:00.807 INFO:tasks.workunit.client.1.smithi142.stdout:1/442: write d6/d12/d28/f40 [2797693,59493] 0 2024-06-15T01:07:00.807 INFO:tasks.workunit.client.1.smithi142.stdout:4/491: dread d5/d1c/d2a/f3d [0,4194304] 0 2024-06-15T01:07:00.810 INFO:tasks.workunit.client.0.smithi137.stdout:4/315: unlink c2 0 2024-06-15T01:07:00.811 INFO:tasks.workunit.client.1.smithi142.stdout:0/394: creat d3/d39/d3e/d81/d44/d7c/f95 x:0 0 0 2024-06-15T01:07:00.811 INFO:tasks.workunit.client.0.smithi137.stdout:4/316: write f8 [5623428,76587] 0 2024-06-15T01:07:00.811 INFO:tasks.workunit.client.0.smithi137.stdout:2/259: symlink d0/dc/l50 0 2024-06-15T01:07:00.812 INFO:tasks.workunit.client.0.smithi137.stdout:8/282: sync 2024-06-15T01:07:00.812 INFO:tasks.workunit.client.0.smithi137.stdout:3/376: creat d0/d2/d1e/f81 x:0 0 0 2024-06-15T01:07:00.813 INFO:tasks.workunit.client.0.smithi137.stdout:7/289: mknod df/d18/c5c 0 2024-06-15T01:07:00.813 INFO:tasks.workunit.client.0.smithi137.stdout:8/283: dread - d9/d22/f33 zero size 2024-06-15T01:07:00.814 INFO:tasks.workunit.client.0.smithi137.stdout:4/317: creat dd/d15/d21/f78 x:0 0 0 2024-06-15T01:07:00.815 INFO:tasks.workunit.client.0.smithi137.stdout:8/284: write d9/f1b [101460,40152] 0 2024-06-15T01:07:00.816 INFO:tasks.workunit.client.1.smithi142.stdout:7/381: dwrite d2/d8/dd/f10 [0,4194304] 0 2024-06-15T01:07:00.816 INFO:tasks.workunit.client.0.smithi137.stdout:2/260: unlink d0/dc/l2d 0 2024-06-15T01:07:00.817 INFO:tasks.workunit.client.0.smithi137.stdout:2/261: chown d0/d42/d4d 9544 1 2024-06-15T01:07:00.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:00 smithi142 ceph-mon[37379]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:00.818 INFO:tasks.workunit.client.1.smithi142.stdout:4/492: rename d5/d9/l71 to d5/d9/d1a/la9 0 2024-06-15T01:07:00.818 INFO:tasks.workunit.client.0.smithi137.stdout:3/377: rename d0/d2f/f65 to d0/d4c/f82 0 2024-06-15T01:07:00.819 INFO:tasks.workunit.client.1.smithi142.stdout:0/395: creat d3/d39/d3e/d81/d1b/d38/f96 x:0 0 0 2024-06-15T01:07:00.820 INFO:tasks.workunit.client.0.smithi137.stdout:3/378: fsync d0/d2/d22/d24/d28/d2e/f3b 0 2024-06-15T01:07:00.821 INFO:tasks.workunit.client.0.smithi137.stdout:3/379: dread - d0/d56/f6a zero size 2024-06-15T01:07:00.822 INFO:tasks.workunit.client.1.smithi142.stdout:6/389: truncate d1a/d39/d3a/d51/f63 3873827 0 2024-06-15T01:07:00.822 INFO:tasks.workunit.client.1.smithi142.stdout:2/354: dread d8/d19/d23/d25/d1a/f1d [0,4194304] 0 2024-06-15T01:07:00.822 INFO:tasks.workunit.client.1.smithi142.stdout:0/396: dread - d3/d39/d3e/d81/d44/d7c/d22/d59/f94 zero size 2024-06-15T01:07:00.822 INFO:tasks.workunit.client.0.smithi137.stdout:8/285: mknod d9/de/d1e/c60 0 2024-06-15T01:07:00.823 INFO:tasks.workunit.client.1.smithi142.stdout:1/443: rename d6/d46/d61/fa6 to d6/d12/d3c/d63/fa8 0 2024-06-15T01:07:00.824 INFO:tasks.workunit.client.1.smithi142.stdout:4/493: creat d5/d9/da/d92/faa x:0 0 0 2024-06-15T01:07:00.825 INFO:tasks.workunit.client.1.smithi142.stdout:6/390: symlink d1a/d2d/d80/l90 0 2024-06-15T01:07:00.826 INFO:tasks.workunit.client.0.smithi137.stdout:7/290: creat df/d18/d3b/d5b/f5d x:0 0 0 2024-06-15T01:07:00.826 INFO:tasks.workunit.client.1.smithi142.stdout:0/397: write f1 [530011,56654] 0 2024-06-15T01:07:00.827 INFO:tasks.workunit.client.1.smithi142.stdout:2/355: creat d8/d19/d23/d3a/f6f x:0 0 0 2024-06-15T01:07:00.827 INFO:tasks.workunit.client.1.smithi142.stdout:0/398: stat d3/d39/d3e/d81/d1b/f26 0 2024-06-15T01:07:00.828 INFO:tasks.workunit.client.0.smithi137.stdout:8/286: truncate d9/d22/f25 1453731 0 2024-06-15T01:07:00.828 INFO:tasks.workunit.client.1.smithi142.stdout:7/382: creat d2/d8/f77 x:0 0 0 2024-06-15T01:07:00.829 INFO:tasks.workunit.client.0.smithi137.stdout:4/318: link dd/d15/d36/d45/c50 dd/d15/d36/d45/d55/c79 0 2024-06-15T01:07:00.829 INFO:tasks.workunit.client.1.smithi142.stdout:1/444: fdatasync d6/d46/f56 0 2024-06-15T01:07:00.830 INFO:tasks.workunit.client.1.smithi142.stdout:7/383: readlink d2/d8/dd/l5b 0 2024-06-15T01:07:00.830 INFO:tasks.workunit.client.1.smithi142.stdout:1/445: chown d6/d12/da0 2754 1 2024-06-15T01:07:00.830 INFO:tasks.workunit.client.0.smithi137.stdout:2/262: link d0/de/d2b/l36 d0/d1c/l51 0 2024-06-15T01:07:00.833 INFO:tasks.workunit.client.0.smithi137.stdout:8/287: write d9/de/f3c [227318,85299] 0 2024-06-15T01:07:00.833 INFO:tasks.workunit.client.0.smithi137.stdout:2/263: creat d0/de/f52 x:0 0 0 2024-06-15T01:07:00.834 INFO:tasks.workunit.client.1.smithi142.stdout:9/372: sync 2024-06-15T01:07:00.835 INFO:tasks.workunit.client.0.smithi137.stdout:4/319: creat dd/d15/d48/d72/f7a x:0 0 0 2024-06-15T01:07:00.835 INFO:tasks.workunit.client.1.smithi142.stdout:4/494: rename d5/d9/da/d7e/fa0 to d5/d1c/d2a/fab 0 2024-06-15T01:07:00.835 INFO:tasks.workunit.client.1.smithi142.stdout:6/391: link d1a/l69 d1a/d2d/d80/d8e/l91 0 2024-06-15T01:07:00.836 INFO:tasks.workunit.client.0.smithi137.stdout:2/264: chown d0/f5 7017 1 2024-06-15T01:07:00.836 INFO:tasks.workunit.client.0.smithi137.stdout:7/291: dwrite df/d13/d2e/d1a/f40 [0,4194304] 0 2024-06-15T01:07:00.836 INFO:tasks.workunit.client.1.smithi142.stdout:9/373: fsync d6/dc/d4e/d65/f73 0 2024-06-15T01:07:00.836 INFO:tasks.workunit.client.1.smithi142.stdout:4/495: chown d5/d9/da/f8b 3930491 1 2024-06-15T01:07:00.836 INFO:tasks.workunit.client.1.smithi142.stdout:7/384: symlink d2/d8/dd/d14/d2c/l78 0 2024-06-15T01:07:00.837 INFO:tasks.workunit.client.1.smithi142.stdout:4/496: chown d5/d1c/d2a/d30/c77 88734736 1 2024-06-15T01:07:00.837 INFO:tasks.workunit.client.0.smithi137.stdout:4/320: stat dd/d15/d30/f41 0 2024-06-15T01:07:00.838 INFO:tasks.workunit.client.1.smithi142.stdout:4/497: write d5/d1c/d2a/d30/d46/d50/f5e [478234,72571] 0 2024-06-15T01:07:00.839 INFO:tasks.workunit.client.0.smithi137.stdout:2/265: truncate d0/dc/d40/f4f 972162 0 2024-06-15T01:07:00.839 INFO:tasks.workunit.client.1.smithi142.stdout:6/392: mknod d1a/d83/c92 0 2024-06-15T01:07:00.839 INFO:tasks.workunit.client.0.smithi137.stdout:8/288: mknod d9/de/d1e/d35/d5f/c61 0 2024-06-15T01:07:00.839 INFO:tasks.workunit.client.0.smithi137.stdout:2/266: stat d0 0 2024-06-15T01:07:00.840 INFO:tasks.workunit.client.0.smithi137.stdout:2/267: stat d0/fb 0 2024-06-15T01:07:00.841 INFO:tasks.workunit.client.0.smithi137.stdout:8/289: write d9/de/d1e/f44 [540317,7977] 0 2024-06-15T01:07:00.841 INFO:tasks.workunit.client.1.smithi142.stdout:9/374: creat d6/dc/d4e/f94 x:0 0 0 2024-06-15T01:07:00.842 INFO:tasks.workunit.client.0.smithi137.stdout:4/321: mkdir dd/d15/d21/d3a/d6e/d7b 0 2024-06-15T01:07:00.846 INFO:tasks.workunit.client.1.smithi142.stdout:9/375: dread d6/dc/dd/d22/f49 [0,4194304] 0 2024-06-15T01:07:00.846 INFO:tasks.workunit.client.1.smithi142.stdout:4/498: creat d5/d9/da/d92/d98/fac x:0 0 0 2024-06-15T01:07:00.848 INFO:tasks.workunit.client.0.smithi137.stdout:2/268: creat d0/dc/d22/d23/f53 x:0 0 0 2024-06-15T01:07:00.849 INFO:tasks.workunit.client.1.smithi142.stdout:6/393: getdents d1a/d3c 0 2024-06-15T01:07:00.850 INFO:tasks.workunit.client.0.smithi137.stdout:2/269: stat d0/dc/d40/c45 0 2024-06-15T01:07:00.854 INFO:tasks.workunit.client.0.smithi137.stdout:4/322: dwrite dd/d15/d36/d45/f4b [0,4194304] 0 2024-06-15T01:07:00.854 INFO:tasks.workunit.client.0.smithi137.stdout:2/270: rmdir d0/de/d10/d3a 39 2024-06-15T01:07:00.854 INFO:tasks.workunit.client.1.smithi142.stdout:4/499: rename d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/l39 to d5/d9/d23/d32/lad 0 2024-06-15T01:07:00.854 INFO:tasks.workunit.client.1.smithi142.stdout:0/399: getdents d3/d39/d3e/d81/d1b/d38 0 2024-06-15T01:07:00.855 INFO:tasks.workunit.client.0.smithi137.stdout:7/292: dread df/d18/d1c/f1f [0,4194304] 0 2024-06-15T01:07:00.858 INFO:tasks.workunit.client.0.smithi137.stdout:7/293: rmdir df/d18/d1c/d1d/d27/d33 39 2024-06-15T01:07:00.859 INFO:tasks.workunit.client.1.smithi142.stdout:4/500: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/cae 0 2024-06-15T01:07:00.860 INFO:tasks.workunit.client.1.smithi142.stdout:0/400: mknod d3/d39/d3e/d81/d4c/c97 0 2024-06-15T01:07:00.860 INFO:tasks.workunit.client.0.smithi137.stdout:4/323: dwrite dd/f16 [0,4194304] 0 2024-06-15T01:07:00.861 INFO:tasks.workunit.client.1.smithi142.stdout:6/394: dwrite d1a/d39/d3a/f5f [0,4194304] 0 2024-06-15T01:07:00.863 INFO:tasks.workunit.client.0.smithi137.stdout:2/271: dread d0/dc/d22/f4b [0,4194304] 0 2024-06-15T01:07:00.865 INFO:tasks.workunit.client.1.smithi142.stdout:9/376: dread d6/f9 [0,4194304] 0 2024-06-15T01:07:00.867 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:00 smithi137 ceph-mon[29354]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:00.883 INFO:tasks.workunit.client.0.smithi137.stdout:7/294: rename df/f4c to df/d13/d3c/f5e 0 2024-06-15T01:07:00.884 INFO:tasks.workunit.client.1.smithi142.stdout:0/401: mkdir d3/d39/d3e/d81/d1e/d98 0 2024-06-15T01:07:00.885 INFO:tasks.workunit.client.0.smithi137.stdout:7/295: write df/d18/d1c/d1d/d27/f49 [637614,32698] 0 2024-06-15T01:07:00.889 INFO:tasks.workunit.client.1.smithi142.stdout:9/377: mknod d6/dc/d82/c95 0 2024-06-15T01:07:00.890 INFO:tasks.workunit.client.0.smithi137.stdout:4/324: creat dd/d15/d36/d45/d55/f7c x:0 0 0 2024-06-15T01:07:00.891 INFO:tasks.workunit.client.1.smithi142.stdout:0/402: fsync d3/d39/d3e/d81/f1d 0 2024-06-15T01:07:00.893 INFO:tasks.workunit.client.1.smithi142.stdout:7/385: dwrite d2/d8/da/d33/d25/f67 [0,4194304] 0 2024-06-15T01:07:00.895 INFO:tasks.workunit.client.1.smithi142.stdout:4/501: dwrite d5/d9/fd [0,4194304] 0 2024-06-15T01:07:00.895 INFO:tasks.workunit.client.0.smithi137.stdout:7/296: readlink df/d13/d2e/l3e 0 2024-06-15T01:07:00.898 INFO:tasks.workunit.client.0.smithi137.stdout:2/272: truncate d0/f1f 269181 0 2024-06-15T01:07:00.899 INFO:tasks.workunit.client.0.smithi137.stdout:2/273: chown d0/d42/d49 16128537 1 2024-06-15T01:07:00.901 INFO:tasks.workunit.client.0.smithi137.stdout:7/297: dread df/d13/d2e/f4f [0,4194304] 0 2024-06-15T01:07:00.901 INFO:tasks.workunit.client.0.smithi137.stdout:7/298: stat f3 0 2024-06-15T01:07:00.901 INFO:tasks.workunit.client.1.smithi142.stdout:4/502: mkdir d5/d9/d90/daf 0 2024-06-15T01:07:00.902 INFO:tasks.workunit.client.0.smithi137.stdout:2/274: chown d0/de/d10/l38 0 1 2024-06-15T01:07:00.903 INFO:tasks.workunit.client.0.smithi137.stdout:7/299: truncate df/d13/d2e/f4e 225360 0 2024-06-15T01:07:00.904 INFO:tasks.workunit.client.1.smithi142.stdout:9/378: link d6/d34/c36 d6/dc/d82/c96 0 2024-06-15T01:07:00.906 INFO:tasks.workunit.client.1.smithi142.stdout:7/386: dwrite d2/d6/f1d [0,4194304] 0 2024-06-15T01:07:00.907 INFO:tasks.workunit.client.1.smithi142.stdout:7/387: read d2/d6/d22/d47/f50 [405716,9091] 0 2024-06-15T01:07:00.908 INFO:tasks.workunit.client.0.smithi137.stdout:8/290: truncate d9/de/d1e/f44 383820 0 2024-06-15T01:07:00.908 INFO:tasks.workunit.client.0.smithi137.stdout:2/275: dread d0/da/f11 [0,4194304] 0 2024-06-15T01:07:00.908 INFO:tasks.workunit.client.1.smithi142.stdout:7/388: write d2/f1a [2452964,89354] 0 2024-06-15T01:07:00.910 INFO:tasks.workunit.client.1.smithi142.stdout:4/503: mkdir d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/da6/db0 0 2024-06-15T01:07:00.910 INFO:tasks.workunit.client.1.smithi142.stdout:7/389: stat d2/d6/d22/d47/f50 0 2024-06-15T01:07:00.911 INFO:tasks.workunit.client.1.smithi142.stdout:7/390: readlink l0 0 2024-06-15T01:07:00.912 INFO:tasks.workunit.client.0.smithi137.stdout:8/291: mknod d9/d2a/d2f/c62 0 2024-06-15T01:07:00.916 INFO:tasks.workunit.client.1.smithi142.stdout:4/504: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/fb1 x:0 0 0 2024-06-15T01:07:00.916 INFO:tasks.workunit.client.1.smithi142.stdout:0/403: dread d3/d39/d3e/d81/d1b/f35 [0,4194304] 0 2024-06-15T01:07:00.917 INFO:tasks.workunit.client.0.smithi137.stdout:2/276: mknod d0/de/d2b/d47/c54 0 2024-06-15T01:07:00.918 INFO:tasks.workunit.client.1.smithi142.stdout:4/505: write d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f6b [1695500,117672] 0 2024-06-15T01:07:00.918 INFO:tasks.workunit.client.1.smithi142.stdout:9/379: getdents d6/dc/d4e/d65 0 2024-06-15T01:07:00.919 INFO:tasks.workunit.client.0.smithi137.stdout:8/292: rename d9/de/d1e/d35/f38 to d9/de/d1e/d35/f63 0 2024-06-15T01:07:00.920 INFO:tasks.workunit.client.1.smithi142.stdout:9/380: truncate d6/f6a 3129181 0 2024-06-15T01:07:00.920 INFO:tasks.workunit.client.1.smithi142.stdout:4/506: truncate d5/d9/d41/f78 951701 0 2024-06-15T01:07:00.921 INFO:tasks.workunit.client.1.smithi142.stdout:0/404: write d3/d39/d3e/d81/f8d [761038,43830] 0 2024-06-15T01:07:00.923 INFO:tasks.workunit.client.1.smithi142.stdout:9/381: truncate d6/d31/f84 982479 0 2024-06-15T01:07:00.924 INFO:tasks.workunit.client.0.smithi137.stdout:7/300: dread df/d18/d1c/d1d/d27/d33/f52 [0,4194304] 0 2024-06-15T01:07:00.924 INFO:tasks.workunit.client.1.smithi142.stdout:6/395: truncate d1a/d39/d42/d4c/d58/f6a 1168366 0 2024-06-15T01:07:00.925 INFO:tasks.workunit.client.0.smithi137.stdout:8/293: mknod d9/de/d1e/d3f/c64 0 2024-06-15T01:07:00.925 INFO:tasks.workunit.client.1.smithi142.stdout:7/391: mkdir d2/d8/da/d33/d25/d79 0 2024-06-15T01:07:00.926 INFO:tasks.workunit.client.0.smithi137.stdout:7/301: fsync df/d13/d2e/d1a/f26 0 2024-06-15T01:07:00.926 INFO:tasks.workunit.client.0.smithi137.stdout:8/294: stat d9/de/d1e/d35/c3d 0 2024-06-15T01:07:00.928 INFO:tasks.workunit.client.1.smithi142.stdout:9/382: dread d6/d31/f84 [0,4194304] 0 2024-06-15T01:07:00.928 INFO:tasks.workunit.client.0.smithi137.stdout:2/277: link d0/de/d10/l38 d0/dc/d2e/l55 0 2024-06-15T01:07:00.930 INFO:tasks.workunit.client.1.smithi142.stdout:4/507: symlink d5/d9/d23/lb2 0 2024-06-15T01:07:00.930 INFO:tasks.workunit.client.0.smithi137.stdout:7/302: rmdir df/d18/d3b 39 2024-06-15T01:07:00.931 INFO:tasks.workunit.client.1.smithi142.stdout:0/405: dread d3/d39/d3e/d81/f34 [0,4194304] 0 2024-06-15T01:07:00.933 INFO:tasks.workunit.client.1.smithi142.stdout:7/392: creat d2/d6/d22/d47/f7a x:0 0 0 2024-06-15T01:07:00.933 INFO:tasks.workunit.client.0.smithi137.stdout:4/325: dread dd/f28 [0,4194304] 0 2024-06-15T01:07:00.936 INFO:tasks.workunit.client.1.smithi142.stdout:6/396: creat d1a/d39/d42/d4c/d58/d87/f93 x:0 0 0 2024-06-15T01:07:00.936 INFO:tasks.workunit.client.0.smithi137.stdout:8/295: dwrite d9/d22/f33 [0,4194304] 0 2024-06-15T01:07:00.937 INFO:tasks.workunit.client.0.smithi137.stdout:7/303: dread df/d18/d1c/f1f [0,4194304] 0 2024-06-15T01:07:00.938 INFO:tasks.workunit.client.1.smithi142.stdout:8/377: sync 2024-06-15T01:07:00.938 INFO:tasks.workunit.client.1.smithi142.stdout:5/396: sync 2024-06-15T01:07:00.938 INFO:tasks.workunit.client.1.smithi142.stdout:3/480: sync 2024-06-15T01:07:00.938 INFO:tasks.workunit.client.1.smithi142.stdout:9/383: creat d6/dc/d4e/d65/f97 x:0 0 0 2024-06-15T01:07:00.939 INFO:tasks.workunit.client.1.smithi142.stdout:8/378: write d0/d25/d53/f79 [111591,3991] 0 2024-06-15T01:07:00.942 INFO:tasks.workunit.client.0.smithi137.stdout:4/326: dread dd/d10/f63 [0,4194304] 0 2024-06-15T01:07:00.944 INFO:tasks.workunit.client.0.smithi137.stdout:7/304: write df/d13/d3c/f5e [134500,87721] 0 2024-06-15T01:07:00.947 INFO:tasks.workunit.client.0.smithi137.stdout:8/296: dwrite d9/d22/f33 [0,4194304] 0 2024-06-15T01:07:00.947 INFO:tasks.workunit.client.1.smithi142.stdout:0/406: symlink d3/d39/d3e/d81/d4c/l99 0 2024-06-15T01:07:00.948 INFO:tasks.workunit.client.1.smithi142.stdout:0/407: fdatasync d3/d39/d3e/d81/d44/f49 0 2024-06-15T01:07:00.948 INFO:tasks.workunit.client.1.smithi142.stdout:7/393: creat d2/d8/da/d33/d25/d27/f7b x:0 0 0 2024-06-15T01:07:00.949 INFO:tasks.workunit.client.1.smithi142.stdout:6/397: symlink d1a/d39/d3a/l94 0 2024-06-15T01:07:00.950 INFO:tasks.workunit.client.1.smithi142.stdout:0/408: chown d3/d39/d3e/d81/d1b/l1f 117 1 2024-06-15T01:07:00.951 INFO:tasks.workunit.client.1.smithi142.stdout:3/481: creat d0/d3/f90 x:0 0 0 2024-06-15T01:07:00.951 INFO:tasks.workunit.client.0.smithi137.stdout:7/305: creat df/d18/d3b/f5f x:0 0 0 2024-06-15T01:07:00.952 INFO:tasks.workunit.client.1.smithi142.stdout:4/508: symlink d5/d9/d23/lb3 0 2024-06-15T01:07:00.952 INFO:tasks.workunit.client.1.smithi142.stdout:6/398: dread d1a/d39/d3a/d51/d62/f79 [0,4194304] 0 2024-06-15T01:07:00.953 INFO:tasks.workunit.client.0.smithi137.stdout:4/327: truncate dd/d10/f1c 2082714 0 2024-06-15T01:07:00.954 INFO:tasks.workunit.client.0.smithi137.stdout:4/328: stat dd/d15/d30/f41 0 2024-06-15T01:07:00.955 INFO:tasks.workunit.client.0.smithi137.stdout:4/329: chown dd/d15/d21/d3a 980108 1 2024-06-15T01:07:00.957 INFO:tasks.workunit.client.1.smithi142.stdout:9/384: rename d6/dc/d51/d53/l56 to d6/dc/d1b/d25/d35/l98 0 2024-06-15T01:07:00.957 INFO:tasks.workunit.client.0.smithi137.stdout:7/306: rename df/f3d to df/d18/d1c/d1d/d27/d33/d59/f60 0 2024-06-15T01:07:00.958 INFO:tasks.workunit.client.1.smithi142.stdout:7/394: creat d2/d6/d22/d47/f7c x:0 0 0 2024-06-15T01:07:00.959 INFO:tasks.workunit.client.0.smithi137.stdout:8/297: link d9/de/d1e/c21 d9/d2a/d2f/c65 0 2024-06-15T01:07:00.959 INFO:tasks.workunit.client.1.smithi142.stdout:3/482: creat d0/d1b/f91 x:0 0 0 2024-06-15T01:07:00.961 INFO:tasks.workunit.client.1.smithi142.stdout:5/397: dread d2/de/f1f [0,4194304] 0 2024-06-15T01:07:00.961 INFO:tasks.workunit.client.0.smithi137.stdout:4/330: stat dd/d10/f1c 0 2024-06-15T01:07:00.961 INFO:tasks.workunit.client.0.smithi137.stdout:2/278: dread d0/de/f1a [0,4194304] 0 2024-06-15T01:07:00.963 INFO:tasks.workunit.client.1.smithi142.stdout:4/509: creat d5/d9/d23/d32/fb4 x:0 0 0 2024-06-15T01:07:00.964 INFO:tasks.workunit.client.0.smithi137.stdout:4/331: creat dd/d15/d48/d67/f7d x:0 0 0 2024-06-15T01:07:00.966 INFO:tasks.workunit.client.0.smithi137.stdout:8/298: fsync d9/de/d1e/d35/f56 0 2024-06-15T01:07:00.966 INFO:tasks.workunit.client.1.smithi142.stdout:9/385: creat d6/dc/d1b/d25/d58/f99 x:0 0 0 2024-06-15T01:07:00.968 INFO:tasks.workunit.client.0.smithi137.stdout:2/279: rename d0/dc/d40 to d0/d42/d4d/d56 0 2024-06-15T01:07:00.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/356: sync 2024-06-15T01:07:00.972 INFO:tasks.workunit.client.1.smithi142.stdout:5/398: mkdir d2/d18/d79 0 2024-06-15T01:07:00.972 INFO:tasks.workunit.client.0.smithi137.stdout:7/307: getdents df/d18/d1c/d1d/d27 0 2024-06-15T01:07:00.973 INFO:tasks.workunit.client.0.smithi137.stdout:4/332: dwrite dd/f14 [0,4194304] 0 2024-06-15T01:07:00.973 INFO:tasks.workunit.client.1.smithi142.stdout:6/399: mknod d1a/d39/d42/d4c/c95 0 2024-06-15T01:07:00.973 INFO:tasks.workunit.client.1.smithi142.stdout:2/357: chown d8/df/f2d 29248291 1 2024-06-15T01:07:00.973 INFO:tasks.workunit.client.1.smithi142.stdout:5/399: fsync d2/d18/f63 0 2024-06-15T01:07:00.974 INFO:tasks.workunit.client.1.smithi142.stdout:0/409: dread d3/d39/d3e/d81/f17 [0,4194304] 0 2024-06-15T01:07:00.977 INFO:tasks.workunit.client.1.smithi142.stdout:9/386: symlink d6/dc/dd/l9a 0 2024-06-15T01:07:00.977 INFO:tasks.workunit.client.1.smithi142.stdout:9/387: fdatasync d6/dc/d1b/f40 0 2024-06-15T01:07:00.978 INFO:tasks.workunit.client.1.smithi142.stdout:7/395: creat d2/d8/dd/d14/d2c/d37/d6b/d6e/f7d x:0 0 0 2024-06-15T01:07:00.980 INFO:tasks.workunit.client.0.smithi137.stdout:7/308: mknod df/d18/d3b/c61 0 2024-06-15T01:07:00.982 INFO:tasks.workunit.client.0.smithi137.stdout:4/333: creat dd/d15/d21/d3a/d6e/f7e x:0 0 0 2024-06-15T01:07:00.982 INFO:tasks.workunit.client.1.smithi142.stdout:2/358: creat d8/d19/f70 x:0 0 0 2024-06-15T01:07:00.983 INFO:tasks.workunit.client.1.smithi142.stdout:8/379: dwrite d0/f47 [0,4194304] 0 2024-06-15T01:07:00.984 INFO:tasks.workunit.client.1.smithi142.stdout:0/410: readlink d3/l6 0 2024-06-15T01:07:00.985 INFO:tasks.workunit.client.1.smithi142.stdout:0/411: dread - d3/d39/d3e/d81/d1b/d38/f96 zero size 2024-06-15T01:07:00.986 INFO:tasks.workunit.client.0.smithi137.stdout:7/309: mkdir df/d13/d2e/d62 0 2024-06-15T01:07:00.989 INFO:tasks.workunit.client.0.smithi137.stdout:4/334: creat dd/d15/d21/d3a/d51/f7f x:0 0 0 2024-06-15T01:07:00.989 INFO:tasks.workunit.client.1.smithi142.stdout:9/388: symlink d6/dc/d1b/d25/d58/d74/l9b 0 2024-06-15T01:07:00.989 INFO:tasks.workunit.client.1.smithi142.stdout:6/400: symlink d1a/d2d/l96 0 2024-06-15T01:07:00.990 INFO:tasks.workunit.client.1.smithi142.stdout:6/401: write f6 [3598509,125533] 0 2024-06-15T01:07:00.991 INFO:tasks.workunit.client.0.smithi137.stdout:2/280: rmdir d0/da/d4e 0 2024-06-15T01:07:00.991 INFO:tasks.workunit.client.1.smithi142.stdout:6/402: stat d1a/d39/d42/d4c/c88 0 2024-06-15T01:07:00.992 INFO:tasks.workunit.client.0.smithi137.stdout:8/299: dwrite d9/de/d1e/f28 [0,4194304] 0 2024-06-15T01:07:00.993 INFO:tasks.workunit.client.1.smithi142.stdout:3/483: write d0/d3/da/f62 [594102,3582] 0 2024-06-15T01:07:00.993 INFO:tasks.workunit.client.0.smithi137.stdout:4/335: write dd/d10/f54 [555876,4901] 0 2024-06-15T01:07:00.994 INFO:tasks.workunit.client.0.smithi137.stdout:7/310: mknod df/d18/d1c/d3a/c63 0 2024-06-15T01:07:00.995 INFO:tasks.workunit.client.1.smithi142.stdout:2/359: mknod d8/d19/d23/d25/d22/d43/c71 0 2024-06-15T01:07:00.995 INFO:tasks.workunit.client.0.smithi137.stdout:4/336: chown dd/d15/d21/d3a/d6e 4576 1 2024-06-15T01:07:00.996 INFO:tasks.workunit.client.1.smithi142.stdout:4/510: creat d5/d1c/d2a/fb5 x:0 0 0 2024-06-15T01:07:00.997 INFO:tasks.workunit.client.0.smithi137.stdout:4/337: dread - dd/d15/d36/d45/f4e zero size 2024-06-15T01:07:00.997 INFO:tasks.workunit.client.0.smithi137.stdout:2/281: mkdir d0/d42/d49/d57 0 2024-06-15T01:07:00.997 INFO:tasks.workunit.client.1.smithi142.stdout:2/360: truncate d8/df/d10/d1c/f2c 250017 0 2024-06-15T01:07:00.998 INFO:tasks.workunit.client.1.smithi142.stdout:0/412: dread d3/d39/d3e/d81/d44/d7c/d22/f2d [0,4194304] 0 2024-06-15T01:07:00.998 INFO:tasks.workunit.client.1.smithi142.stdout:9/389: write d6/f21 [3198783,110910] 0 2024-06-15T01:07:01.001 INFO:tasks.workunit.client.1.smithi142.stdout:0/413: truncate d3/d39/d3e/d81/d4c/d61/f91 214610 0 2024-06-15T01:07:01.003 INFO:tasks.workunit.client.0.smithi137.stdout:4/338: dread dd/d15/d21/f32 [0,4194304] 0 2024-06-15T01:07:01.003 INFO:tasks.workunit.client.0.smithi137.stdout:7/311: symlink df/d13/d2e/d1a/l64 0 2024-06-15T01:07:01.004 INFO:tasks.workunit.client.1.smithi142.stdout:5/400: creat d2/de/f7a x:0 0 0 2024-06-15T01:07:01.004 INFO:tasks.workunit.client.0.smithi137.stdout:4/339: fsync dd/d15/d48/d72/f7a 0 2024-06-15T01:07:01.005 INFO:tasks.workunit.client.1.smithi142.stdout:3/484: creat d0/d1f/d20/d59/f92 x:0 0 0 2024-06-15T01:07:01.007 INFO:tasks.workunit.client.1.smithi142.stdout:8/380: mknod d0/d17/c7f 0 2024-06-15T01:07:01.008 INFO:tasks.workunit.client.0.smithi137.stdout:4/340: unlink dd/d15/d48/l68 0 2024-06-15T01:07:01.010 INFO:tasks.workunit.client.1.smithi142.stdout:7/396: link d2/d6/l21 d2/d6/d22/d47/l7e 0 2024-06-15T01:07:01.010 INFO:tasks.workunit.client.0.smithi137.stdout:7/312: mknod df/c65 0 2024-06-15T01:07:01.011 INFO:tasks.workunit.client.1.smithi142.stdout:6/403: fdatasync d1a/f25 0 2024-06-15T01:07:01.011 INFO:tasks.workunit.client.0.smithi137.stdout:4/341: creat dd/d15/d48/f80 x:0 0 0 2024-06-15T01:07:01.012 INFO:tasks.workunit.client.0.smithi137.stdout:7/313: read df/d13/d2e/d1a/f2c [167768,71127] 0 2024-06-15T01:07:01.013 INFO:tasks.workunit.client.1.smithi142.stdout:1/446: sync 2024-06-15T01:07:01.013 INFO:tasks.workunit.client.1.smithi142.stdout:3/485: creat d0/d1f/d20/d5d/d82/d6e/f93 x:0 0 0 2024-06-15T01:07:01.014 INFO:tasks.workunit.client.1.smithi142.stdout:4/511: mknod d5/d9/da/da3/cb6 0 2024-06-15T01:07:01.014 INFO:tasks.workunit.client.1.smithi142.stdout:8/381: stat d0/c34 0 2024-06-15T01:07:01.015 INFO:tasks.workunit.client.1.smithi142.stdout:9/390: symlink d6/d34/l9c 0 2024-06-15T01:07:01.016 INFO:tasks.workunit.client.1.smithi142.stdout:2/361: creat d8/d19/d23/d25/d22/d43/d52/f72 x:0 0 0 2024-06-15T01:07:01.017 INFO:tasks.workunit.client.1.smithi142.stdout:0/414: mkdir d3/d39/d9a 0 2024-06-15T01:07:01.020 INFO:tasks.workunit.client.0.smithi137.stdout:2/282: truncate d0/dc/d22/f4b 801194 0 2024-06-15T01:07:01.021 INFO:tasks.workunit.client.1.smithi142.stdout:4/512: symlink d5/d9/d41/lb7 0 2024-06-15T01:07:01.023 INFO:tasks.workunit.client.0.smithi137.stdout:4/342: getdents dd/d15/d21/d3a/d51 0 2024-06-15T01:07:01.023 INFO:tasks.workunit.client.1.smithi142.stdout:3/486: creat d0/d3/da/d71/d55/f94 x:0 0 0 2024-06-15T01:07:01.024 INFO:tasks.workunit.client.0.smithi137.stdout:8/300: dwrite f8 [0,4194304] 0 2024-06-15T01:07:01.025 INFO:tasks.workunit.client.1.smithi142.stdout:9/391: creat d6/dc/d4e/d78/f9d x:0 0 0 2024-06-15T01:07:01.026 INFO:tasks.workunit.client.1.smithi142.stdout:9/392: chown d6/dc/d51/d53 673723 1 2024-06-15T01:07:01.027 INFO:tasks.workunit.client.1.smithi142.stdout:1/447: dread d6/d12/f22 [0,4194304] 0 2024-06-15T01:07:01.027 INFO:tasks.workunit.client.0.smithi137.stdout:4/343: truncate dd/d15/d36/d45/d55/f7c 356159 0 2024-06-15T01:07:01.027 INFO:tasks.workunit.client.0.smithi137.stdout:2/283: creat d0/d42/d49/d57/f58 x:0 0 0 2024-06-15T01:07:01.028 INFO:tasks.workunit.client.1.smithi142.stdout:7/397: unlink d2/d8/da/d33/d25/f3f 0 2024-06-15T01:07:01.028 INFO:tasks.workunit.client.0.smithi137.stdout:8/301: symlink d9/d2a/d2f/l66 0 2024-06-15T01:07:01.028 INFO:tasks.workunit.client.0.smithi137.stdout:2/284: fdatasync d0/de/f24 0 2024-06-15T01:07:01.032 INFO:tasks.workunit.client.0.smithi137.stdout:7/314: dread df/d18/d1c/d1d/d27/f48 [0,4194304] 0 2024-06-15T01:07:01.034 INFO:tasks.workunit.client.1.smithi142.stdout:2/362: mknod d8/d19/d23/d25/c73 0 2024-06-15T01:07:01.036 INFO:tasks.workunit.client.1.smithi142.stdout:5/401: getdents d2/d3e/d65 0 2024-06-15T01:07:01.037 INFO:tasks.workunit.client.1.smithi142.stdout:3/487: dwrite d0/d1f/f27 [0,4194304] 0 2024-06-15T01:07:01.042 INFO:tasks.workunit.client.1.smithi142.stdout:8/382: dread d0/d25/d5a/f29 [0,4194304] 0 2024-06-15T01:07:01.043 INFO:tasks.workunit.client.1.smithi142.stdout:6/404: dread f18 [0,4194304] 0 2024-06-15T01:07:01.045 INFO:tasks.workunit.client.1.smithi142.stdout:9/393: dwrite d6/dc/d1b/d25/d35/f5c [0,4194304] 0 2024-06-15T01:07:01.052 INFO:tasks.workunit.client.0.smithi137.stdout:6/236: sync 2024-06-15T01:07:01.055 INFO:tasks.workunit.client.1.smithi142.stdout:7/398: unlink d2/d8/dd/f10 0 2024-06-15T01:07:01.058 INFO:tasks.workunit.client.0.smithi137.stdout:7/315: mknod df/d13/d2e/d1a/d3f/c66 0 2024-06-15T01:07:01.058 INFO:tasks.workunit.client.0.smithi137.stdout:7/316: fdatasync df/d13/f20 0 2024-06-15T01:07:01.058 INFO:tasks.workunit.client.0.smithi137.stdout:8/302: mkdir d9/de/d1e/d67 0 2024-06-15T01:07:01.058 INFO:tasks.workunit.client.0.smithi137.stdout:7/317: chown df/d18/d1c/d1d/c29 110001 1 2024-06-15T01:07:01.061 INFO:tasks.workunit.client.0.smithi137.stdout:2/285: dread d0/dc/d22/f39 [0,4194304] 0 2024-06-15T01:07:01.062 INFO:tasks.workunit.client.1.smithi142.stdout:1/448: dread d6/d12/d28/f51 [0,4194304] 0 2024-06-15T01:07:01.062 INFO:tasks.workunit.client.0.smithi137.stdout:8/303: creat d9/de/d1e/d3f/f68 x:0 0 0 2024-06-15T01:07:01.062 INFO:tasks.workunit.client.1.smithi142.stdout:1/449: fdatasync d6/d12/d28/f39 0 2024-06-15T01:07:01.063 INFO:tasks.workunit.client.1.smithi142.stdout:1/450: stat d6/d12/d1c/f34 0 2024-06-15T01:07:01.063 INFO:tasks.workunit.client.0.smithi137.stdout:6/237: creat d0/d5/d9/d3f/f4c x:0 0 0 2024-06-15T01:07:01.064 INFO:tasks.workunit.client.0.smithi137.stdout:2/286: unlink d0/c3f 0 2024-06-15T01:07:01.066 INFO:tasks.workunit.client.0.smithi137.stdout:6/238: truncate d0/d47/f24 665860 0 2024-06-15T01:07:01.066 INFO:tasks.workunit.client.0.smithi137.stdout:8/304: creat d9/d2a/f69 x:0 0 0 2024-06-15T01:07:01.068 INFO:tasks.workunit.client.1.smithi142.stdout:6/405: rename d1a/d3c/l3d to d1a/d39/l97 0 2024-06-15T01:07:01.068 INFO:tasks.workunit.client.0.smithi137.stdout:8/305: fsync d9/de/d1e/d35/f56 0 2024-06-15T01:07:01.068 INFO:tasks.workunit.client.1.smithi142.stdout:6/406: stat d1a 0 2024-06-15T01:07:01.069 INFO:tasks.workunit.client.0.smithi137.stdout:7/318: dread df/f14 [0,4194304] 0 2024-06-15T01:07:01.069 INFO:tasks.workunit.client.1.smithi142.stdout:1/451: dwrite d6/d12/f2d [0,4194304] 0 2024-06-15T01:07:01.069 INFO:tasks.workunit.client.1.smithi142.stdout:9/394: mknod d6/dc/d1b/d25/d58/c9e 0 2024-06-15T01:07:01.071 INFO:tasks.workunit.client.1.smithi142.stdout:7/399: mknod d2/d8/da/d33/d25/d27/c7f 0 2024-06-15T01:07:01.073 INFO:tasks.workunit.client.1.smithi142.stdout:0/415: write d3/d39/f45 [4730546,38521] 0 2024-06-15T01:07:01.073 INFO:tasks.workunit.client.1.smithi142.stdout:7/400: readlink d2/d8/dd/l60 0 2024-06-15T01:07:01.073 INFO:tasks.workunit.client.0.smithi137.stdout:7/319: unlink c4 0 2024-06-15T01:07:01.074 INFO:tasks.workunit.client.1.smithi142.stdout:0/416: fdatasync d3/d39/d3e/d81/d1b/d38/f8c 0 2024-06-15T01:07:01.075 INFO:tasks.workunit.client.0.smithi137.stdout:2/287: dwrite d0/dc/f3b [0,4194304] 0 2024-06-15T01:07:01.076 INFO:tasks.workunit.client.1.smithi142.stdout:2/363: unlink d8/d19/d23/d25/f24 0 2024-06-15T01:07:01.077 INFO:tasks.workunit.client.0.smithi137.stdout:4/344: dread dd/d15/d21/d3a/d51/f52 [0,4194304] 0 2024-06-15T01:07:01.078 INFO:tasks.workunit.client.1.smithi142.stdout:5/402: mknod d2/de/d19/d5f/d68/d76/c7b 0 2024-06-15T01:07:01.079 INFO:tasks.workunit.client.0.smithi137.stdout:8/306: creat d9/de/d1e/d67/f6a x:0 0 0 2024-06-15T01:07:01.080 INFO:tasks.workunit.client.0.smithi137.stdout:2/288: fdatasync d0/da/f18 0 2024-06-15T01:07:01.081 INFO:tasks.workunit.client.0.smithi137.stdout:7/320: rename df/d18/d1c/d1d/f28 to df/d13/d3c/f67 0 2024-06-15T01:07:01.084 INFO:tasks.workunit.client.0.smithi137.stdout:4/345: truncate dd/d15/d48/f80 398535 0 2024-06-15T01:07:01.085 INFO:tasks.workunit.client.1.smithi142.stdout:1/452: mknod d6/d12/d52/ca9 0 2024-06-15T01:07:01.085 INFO:tasks.workunit.client.1.smithi142.stdout:4/513: write d5/d1c/d2a/d30/d46/f52 [537958,565] 0 2024-06-15T01:07:01.087 INFO:tasks.workunit.client.0.smithi137.stdout:2/289: rename d0/f9 to d0/d42/d4d/f59 0 2024-06-15T01:07:01.088 INFO:tasks.workunit.client.1.smithi142.stdout:1/453: fdatasync d6/d46/f56 0 2024-06-15T01:07:01.088 INFO:tasks.workunit.client.1.smithi142.stdout:6/407: rename d1a/d2d/l34 to d1a/d39/d3a/d51/d81/l98 0 2024-06-15T01:07:01.089 INFO:tasks.workunit.client.0.smithi137.stdout:2/290: truncate d0/de/f52 581365 0 2024-06-15T01:07:01.089 INFO:tasks.workunit.client.0.smithi137.stdout:7/321: creat df/d13/d2e/d1a/f68 x:0 0 0 2024-06-15T01:07:01.089 INFO:tasks.workunit.client.0.smithi137.stdout:8/307: getdents d9/d22 0 2024-06-15T01:07:01.090 INFO:tasks.workunit.client.1.smithi142.stdout:0/417: creat d3/d39/d3e/d81/d1b/d38/f9b x:0 0 0 2024-06-15T01:07:01.091 INFO:tasks.workunit.client.0.smithi137.stdout:4/346: mkdir dd/d15/d48/d72/d81 0 2024-06-15T01:07:01.092 INFO:tasks.workunit.client.0.smithi137.stdout:8/308: write d9/de/f3c [709973,29185] 0 2024-06-15T01:07:01.092 INFO:tasks.workunit.client.0.smithi137.stdout:7/322: write df/d18/f5a [603725,117347] 0 2024-06-15T01:07:01.092 INFO:tasks.workunit.client.1.smithi142.stdout:5/403: unlink d2/de/d19/f1a 0 2024-06-15T01:07:01.095 INFO:tasks.workunit.client.1.smithi142.stdout:3/488: dwrite d0/d3/da/d71/d55/f57 [0,4194304] 0 2024-06-15T01:07:01.096 INFO:tasks.workunit.client.0.smithi137.stdout:4/347: creat dd/d15/d36/d45/d55/f82 x:0 0 0 2024-06-15T01:07:01.097 INFO:tasks.workunit.client.1.smithi142.stdout:8/383: dwrite d0/d25/d5a/f29 [4194304,4194304] 0 2024-06-15T01:07:01.098 INFO:tasks.workunit.client.1.smithi142.stdout:5/404: read d2/d3e/d65/f70 [3234059,19969] 0 2024-06-15T01:07:01.099 INFO:tasks.workunit.client.0.smithi137.stdout:8/309: rename f1 to d9/de/d1e/d67/f6b 0 2024-06-15T01:07:01.100 INFO:tasks.workunit.client.1.smithi142.stdout:1/454: chown d6/d12/c79 997787 1 2024-06-15T01:07:01.101 INFO:tasks.workunit.client.0.smithi137.stdout:7/323: dwrite df/d13/d2e/f4f [0,4194304] 0 2024-06-15T01:07:01.102 INFO:tasks.workunit.client.1.smithi142.stdout:6/408: rmdir d1a/d83 39 2024-06-15T01:07:01.102 INFO:tasks.workunit.client.0.smithi137.stdout:8/310: truncate d9/d2a/d2f/d4e/f57 145782 0 2024-06-15T01:07:01.102 INFO:tasks.workunit.client.0.smithi137.stdout:7/324: chown df/c42 165958 1 2024-06-15T01:07:01.103 INFO:tasks.workunit.client.0.smithi137.stdout:8/311: chown d9/de/d1e 369 1 2024-06-15T01:07:01.104 INFO:tasks.workunit.client.1.smithi142.stdout:4/514: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/d47/d99/f9a [0,4194304] 0 2024-06-15T01:07:01.105 INFO:tasks.workunit.client.0.smithi137.stdout:4/348: mkdir dd/d15/d21/d3a/d51/d83 0 2024-06-15T01:07:01.106 INFO:tasks.workunit.client.1.smithi142.stdout:4/515: write d5/d9/d1a/f82 [4280084,46270] 0 2024-06-15T01:07:01.107 INFO:tasks.workunit.client.1.smithi142.stdout:4/516: chown d5/d1c/d2a/d30/d46/fa4 10 1 2024-06-15T01:07:01.108 INFO:tasks.workunit.client.1.smithi142.stdout:9/395: rename d6/d31/f3e to d6/dc/d51/d53/f9f 0 2024-06-15T01:07:01.110 INFO:tasks.workunit.client.0.smithi137.stdout:8/312: chown d9/d2a/d2f/c65 0 1 2024-06-15T01:07:01.110 INFO:tasks.workunit.client.1.smithi142.stdout:9/396: write d6/dc/dd/f46 [3145642,93916] 0 2024-06-15T01:07:01.113 INFO:tasks.workunit.client.0.smithi137.stdout:7/325: mknod df/d13/d2e/c69 0 2024-06-15T01:07:01.113 INFO:tasks.workunit.client.0.smithi137.stdout:2/291: write d0/f3 [1863584,26397] 0 2024-06-15T01:07:01.114 INFO:tasks.workunit.client.0.smithi137.stdout:4/349: readlink dd/l64 0 2024-06-15T01:07:01.117 INFO:tasks.workunit.client.0.smithi137.stdout:4/350: fdatasync dd/d15/d36/d45/f4b 0 2024-06-15T01:07:01.117 INFO:tasks.workunit.client.0.smithi137.stdout:2/292: creat d0/d42/f5a x:0 0 0 2024-06-15T01:07:01.120 INFO:tasks.workunit.client.0.smithi137.stdout:2/293: write d0/dc/f2c [2492187,20892] 0 2024-06-15T01:07:01.121 INFO:tasks.workunit.client.0.smithi137.stdout:8/313: dwrite d9/d22/f33 [0,4194304] 0 2024-06-15T01:07:01.123 INFO:tasks.workunit.client.1.smithi142.stdout:3/489: mkdir d0/d95 0 2024-06-15T01:07:01.126 INFO:tasks.workunit.client.0.smithi137.stdout:8/314: mknod d9/de/d1e/d67/c6c 0 2024-06-15T01:07:01.127 INFO:tasks.workunit.client.1.smithi142.stdout:8/384: mknod d0/d17/d40/c80 0 2024-06-15T01:07:01.128 INFO:tasks.workunit.client.0.smithi137.stdout:2/294: creat d0/f5b x:0 0 0 2024-06-15T01:07:01.130 INFO:tasks.workunit.client.0.smithi137.stdout:8/315: link d9/de/d1e/d67/f6a d9/d2a/f6d 0 2024-06-15T01:07:01.130 INFO:tasks.workunit.client.1.smithi142.stdout:5/405: creat d2/de/d19/d5f/d68/d76/f7c x:0 0 0 2024-06-15T01:07:01.131 INFO:tasks.workunit.client.1.smithi142.stdout:4/517: mknod d5/d9/d23/d32/cb8 0 2024-06-15T01:07:01.131 INFO:tasks.workunit.client.0.smithi137.stdout:2/295: mkdir d0/dc/d22/d23/d5c 0 2024-06-15T01:07:01.132 INFO:tasks.workunit.client.1.smithi142.stdout:5/406: readlink d2/de/d27/l3f 0 2024-06-15T01:07:01.132 INFO:tasks.workunit.client.0.smithi137.stdout:2/296: fdatasync d0/f5b 0 2024-06-15T01:07:01.133 INFO:tasks.workunit.client.1.smithi142.stdout:7/401: rename d2/d6/d22/d34 to d2/d8/dd/d14/d2c/d37/d6b/d80 0 2024-06-15T01:07:01.134 INFO:tasks.workunit.client.0.smithi137.stdout:8/316: mknod d9/d2a/d2f/c6e 0 2024-06-15T01:07:01.136 INFO:tasks.workunit.client.1.smithi142.stdout:2/364: creat d8/d19/d23/d25/f74 x:0 0 0 2024-06-15T01:07:01.136 INFO:tasks.workunit.client.1.smithi142.stdout:8/385: dwrite d0/d17/d40/f65 [0,4194304] 0 2024-06-15T01:07:01.137 INFO:tasks.workunit.client.0.smithi137.stdout:2/297: symlink d0/dc/d2e/l5d 0 2024-06-15T01:07:01.138 INFO:tasks.workunit.client.1.smithi142.stdout:2/365: read d8/df/f48 [246691,2875] 0 2024-06-15T01:07:01.138 INFO:tasks.workunit.client.1.smithi142.stdout:3/490: unlink d0/d1f/d20/d5d/d60/d72/f7c 0 2024-06-15T01:07:01.139 INFO:tasks.workunit.client.0.smithi137.stdout:2/298: chown d0/de/f1a 12160 1 2024-06-15T01:07:01.142 INFO:tasks.workunit.client.0.smithi137.stdout:4/351: write dd/f18 [1044759,121782] 0 2024-06-15T01:07:01.143 INFO:tasks.workunit.client.0.smithi137.stdout:8/317: dwrite d9/d2a/d2f/d4e/f57 [0,4194304] 0 2024-06-15T01:07:01.144 INFO:tasks.workunit.client.1.smithi142.stdout:4/518: mkdir d5/d1c/d6f/db9 0 2024-06-15T01:07:01.145 INFO:tasks.workunit.client.0.smithi137.stdout:8/318: fdatasync d9/de/d1e/d3f/f41 0 2024-06-15T01:07:01.148 INFO:tasks.workunit.client.0.smithi137.stdout:7/326: dread f3 [0,4194304] 0 2024-06-15T01:07:01.150 INFO:tasks.workunit.client.0.smithi137.stdout:4/352: symlink dd/d15/d36/l84 0 2024-06-15T01:07:01.151 INFO:tasks.workunit.client.0.smithi137.stdout:8/319: fsync d9/de/d1e/f27 0 2024-06-15T01:07:01.152 INFO:tasks.workunit.client.0.smithi137.stdout:7/327: mkdir df/d13/d2e/d1a/d6a 0 2024-06-15T01:07:01.155 INFO:tasks.workunit.client.1.smithi142.stdout:8/386: creat d0/d17/d40/d5e/f81 x:0 0 0 2024-06-15T01:07:01.156 INFO:tasks.workunit.client.1.smithi142.stdout:8/387: rename d0/d17/d40 to d0/d17/d40/d82 22 2024-06-15T01:07:01.157 INFO:tasks.workunit.client.1.smithi142.stdout:3/491: rmdir d0/d3/d15 39 2024-06-15T01:07:01.158 INFO:tasks.workunit.client.0.smithi137.stdout:7/328: symlink df/d18/d1c/d1d/d27/d33/d43/l6b 0 2024-06-15T01:07:01.158 INFO:tasks.workunit.client.0.smithi137.stdout:4/353: dwrite dd/d15/f2e [0,4194304] 0 2024-06-15T01:07:01.159 INFO:tasks.workunit.client.1.smithi142.stdout:8/388: write d0/d17/d40/d5e/f81 [516037,96206] 0 2024-06-15T01:07:01.160 INFO:tasks.workunit.client.1.smithi142.stdout:2/366: creat d8/d19/f75 x:0 0 0 2024-06-15T01:07:01.161 INFO:tasks.workunit.client.0.smithi137.stdout:8/320: getdents d9 0 2024-06-15T01:07:01.162 INFO:tasks.workunit.client.1.smithi142.stdout:5/407: dread d2/de/f50 [0,4194304] 0 2024-06-15T01:07:01.162 INFO:tasks.workunit.client.0.smithi137.stdout:7/329: fdatasync df/d13/d3c/f67 0 2024-06-15T01:07:01.163 INFO:tasks.workunit.client.1.smithi142.stdout:9/397: creat d6/d34/fa0 x:0 0 0 2024-06-15T01:07:01.163 INFO:tasks.workunit.client.0.smithi137.stdout:7/330: write df/d13/d2e/d1a/f26 [1146680,37867] 0 2024-06-15T01:07:01.164 INFO:tasks.workunit.client.1.smithi142.stdout:5/408: read d2/de/d19/f22 [6228273,62] 0 2024-06-15T01:07:01.165 INFO:tasks.workunit.client.0.smithi137.stdout:2/299: write d0/f6 [1687418,2264] 0 2024-06-15T01:07:01.165 INFO:tasks.workunit.client.1.smithi142.stdout:9/398: write d6/dc/d1b/d25/f3f [4255900,17079] 0 2024-06-15T01:07:01.166 INFO:tasks.workunit.client.0.smithi137.stdout:7/331: readlink df/d18/d1c/d3a/l4d 0 2024-06-15T01:07:01.166 INFO:tasks.workunit.client.0.smithi137.stdout:7/332: chown df/d13/d2e/f51 39 1 2024-06-15T01:07:01.167 INFO:tasks.workunit.client.1.smithi142.stdout:5/409: readlink d2/d18/l6e 0 2024-06-15T01:07:01.167 INFO:tasks.workunit.client.1.smithi142.stdout:7/402: mkdir d2/d8/da/d33/d25/d81 0 2024-06-15T01:07:01.168 INFO:tasks.workunit.client.1.smithi142.stdout:0/418: getdents d3/d39 0 2024-06-15T01:07:01.168 INFO:tasks.workunit.client.0.smithi137.stdout:4/354: unlink dd/d15/d21/d3a/d51/d5b/c69 0 2024-06-15T01:07:01.170 INFO:tasks.workunit.client.1.smithi142.stdout:9/399: write d6/dc/d4e/f57 [4659367,86328] 0 2024-06-15T01:07:01.172 INFO:tasks.workunit.client.1.smithi142.stdout:1/455: write d6/d25/f77 [1283101,67280] 0 2024-06-15T01:07:01.173 INFO:tasks.workunit.client.0.smithi137.stdout:7/333: dread df/d13/d2e/f4f [0,4194304] 0 2024-06-15T01:07:01.173 INFO:tasks.workunit.client.0.smithi137.stdout:4/355: creat dd/d15/d21/d3a/d51/f85 x:0 0 0 2024-06-15T01:07:01.174 INFO:tasks.workunit.client.0.smithi137.stdout:7/334: rmdir df/d18 39 2024-06-15T01:07:01.174 INFO:tasks.workunit.client.1.smithi142.stdout:6/409: write d1a/f20 [3231203,130461] 0 2024-06-15T01:07:01.175 INFO:tasks.workunit.client.0.smithi137.stdout:4/356: chown dd/d10/c5a 4908 1 2024-06-15T01:07:01.177 INFO:tasks.workunit.client.1.smithi142.stdout:4/519: symlink d5/d1c/d6f/db9/lba 0 2024-06-15T01:07:01.181 INFO:tasks.workunit.client.0.smithi137.stdout:7/335: rename df/d18/d1c/d3a to df/d18/d1c/d1d/d27/d33/d43/d6c 0 2024-06-15T01:07:01.183 INFO:tasks.workunit.client.0.smithi137.stdout:8/321: write d9/d22/f25 [1983382,75938] 0 2024-06-15T01:07:01.183 INFO:tasks.workunit.client.0.smithi137.stdout:7/336: rmdir df/d18 39 2024-06-15T01:07:01.184 INFO:tasks.workunit.client.0.smithi137.stdout:4/357: dwrite dd/f22 [0,4194304] 0 2024-06-15T01:07:01.185 INFO:tasks.workunit.client.1.smithi142.stdout:0/419: dread d3/d39/d3e/d81/f17 [0,4194304] 0 2024-06-15T01:07:01.187 INFO:tasks.workunit.client.1.smithi142.stdout:4/520: chown d5/d9/da/f5a 192819995 1 2024-06-15T01:07:01.187 INFO:tasks.workunit.client.0.smithi137.stdout:8/322: dread - d9/de/d1e/d35/f5b zero size 2024-06-15T01:07:01.188 INFO:tasks.workunit.client.1.smithi142.stdout:7/403: rmdir d2/d8/dd 39 2024-06-15T01:07:01.188 INFO:tasks.workunit.client.0.smithi137.stdout:8/323: chown d9/d2a/f6d 134227 1 2024-06-15T01:07:01.189 INFO:tasks.workunit.client.0.smithi137.stdout:4/358: read dd/d10/f63 [110816,122423] 0 2024-06-15T01:07:01.189 INFO:tasks.workunit.client.0.smithi137.stdout:7/337: creat df/d18/d3b/d5b/f6d x:0 0 0 2024-06-15T01:07:01.190 INFO:tasks.workunit.client.0.smithi137.stdout:7/338: fdatasync df/d13/f20 0 2024-06-15T01:07:01.191 INFO:tasks.workunit.client.0.smithi137.stdout:4/359: write dd/d15/d21/f29 [547354,24709] 0 2024-06-15T01:07:01.192 INFO:tasks.workunit.client.0.smithi137.stdout:7/339: chown df/d13/l44 58508 1 2024-06-15T01:07:01.193 INFO:tasks.workunit.client.0.smithi137.stdout:8/324: creat d9/d2a/f6f x:0 0 0 2024-06-15T01:07:01.193 INFO:tasks.workunit.client.1.smithi142.stdout:3/492: dwrite d0/d3/f12 [4194304,4194304] 0 2024-06-15T01:07:01.194 INFO:tasks.workunit.client.0.smithi137.stdout:2/300: dread d0/f1 [0,4194304] 0 2024-06-15T01:07:01.195 INFO:tasks.workunit.client.1.smithi142.stdout:2/367: dread d8/df/f2d [0,4194304] 0 2024-06-15T01:07:01.196 INFO:tasks.workunit.client.0.smithi137.stdout:4/360: mkdir dd/d10/d1f/d86 0 2024-06-15T01:07:01.196 INFO:tasks.workunit.client.0.smithi137.stdout:4/361: chown dd/fe 9881 1 2024-06-15T01:07:01.197 INFO:tasks.workunit.client.0.smithi137.stdout:7/340: dread df/d18/d1c/f1f [0,4194304] 0 2024-06-15T01:07:01.197 INFO:tasks.workunit.client.1.smithi142.stdout:9/400: mkdir d6/dc/d1b/d6f/da1 0 2024-06-15T01:07:01.197 INFO:tasks.workunit.client.0.smithi137.stdout:2/301: fdatasync d0/f1 0 2024-06-15T01:07:01.198 INFO:tasks.workunit.client.1.smithi142.stdout:0/420: creat d3/d39/d3e/d81/d44/d7c/d22/d59/f9c x:0 0 0 2024-06-15T01:07:01.199 INFO:tasks.workunit.client.0.smithi137.stdout:7/341: creat df/d13/d3c/f6e x:0 0 0 2024-06-15T01:07:01.199 INFO:tasks.workunit.client.1.smithi142.stdout:3/493: chown d0/d1b 1944799 1 2024-06-15T01:07:01.200 INFO:tasks.workunit.client.1.smithi142.stdout:5/410: write d2/d18/f55 [1911265,122062] 0 2024-06-15T01:07:01.201 INFO:tasks.workunit.client.0.smithi137.stdout:7/342: write df/d13/d2e/f51 [1443,11081] 0 2024-06-15T01:07:01.202 INFO:tasks.workunit.client.1.smithi142.stdout:4/521: write d5/d9/da/d92/faa [131795,84927] 0 2024-06-15T01:07:01.203 INFO:tasks.workunit.client.1.smithi142.stdout:1/456: dwrite d6/d12/d28/f64 [0,4194304] 0 2024-06-15T01:07:01.203 INFO:tasks.workunit.client.1.smithi142.stdout:7/404: symlink d2/d8/da/d33/d25/l82 0 2024-06-15T01:07:01.204 INFO:tasks.workunit.client.1.smithi142.stdout:9/401: dread - d6/dc/d4e/d78/f7f zero size 2024-06-15T01:07:01.204 INFO:tasks.workunit.client.0.smithi137.stdout:4/362: dwrite dd/f13 [4194304,4194304] 0 2024-06-15T01:07:01.205 INFO:tasks.workunit.client.1.smithi142.stdout:2/368: stat d8/d19/d23/d25/c3c 0 2024-06-15T01:07:01.207 INFO:tasks.workunit.client.0.smithi137.stdout:8/325: rmdir d9/d2a 39 2024-06-15T01:07:01.209 INFO:tasks.workunit.client.1.smithi142.stdout:5/411: mknod d2/d18/c7d 0 2024-06-15T01:07:01.210 INFO:tasks.workunit.client.0.smithi137.stdout:4/363: mknod dd/c87 0 2024-06-15T01:07:01.211 INFO:tasks.workunit.client.1.smithi142.stdout:4/522: chown d5/d9/da/f3c 120210 1 2024-06-15T01:07:01.212 INFO:tasks.workunit.client.1.smithi142.stdout:8/389: dwrite d0/d25/d53/f58 [0,4194304] 0 2024-06-15T01:07:01.213 INFO:tasks.workunit.client.0.smithi137.stdout:8/326: write d9/d2a/f6f [199150,92588] 0 2024-06-15T01:07:01.215 INFO:tasks.workunit.client.0.smithi137.stdout:7/343: rename df/d18/d1c/d1d/d27/d33/d43/d6c/c63 to df/d18/c6f 0 2024-06-15T01:07:01.215 INFO:tasks.workunit.client.1.smithi142.stdout:9/402: write d6/dc/dd/f4f [448232,129504] 0 2024-06-15T01:07:01.216 INFO:tasks.workunit.client.0.smithi137.stdout:4/364: mknod dd/d10/c88 0 2024-06-15T01:07:01.217 INFO:tasks.workunit.client.0.smithi137.stdout:5/496: sync 2024-06-15T01:07:01.217 INFO:tasks.workunit.client.1.smithi142.stdout:3/494: mkdir d0/d1f/d20/d59/d61/d96 0 2024-06-15T01:07:01.219 INFO:tasks.workunit.client.1.smithi142.stdout:6/410: dread d1a/f61 [0,4194304] 0 2024-06-15T01:07:01.219 INFO:tasks.workunit.client.0.smithi137.stdout:5/497: fsync df/d44/f84 0 2024-06-15T01:07:01.221 INFO:tasks.workunit.client.0.smithi137.stdout:5/498: write df/d44/f84 [389298,25553] 0 2024-06-15T01:07:01.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/365: mknod dd/d15/d30/d49/c89 0 2024-06-15T01:07:01.223 INFO:tasks.workunit.client.1.smithi142.stdout:4/523: creat d5/d1c/fbb x:0 0 0 2024-06-15T01:07:01.223 INFO:tasks.workunit.client.0.smithi137.stdout:4/366: chown dd/lf 23 1 2024-06-15T01:07:01.223 INFO:tasks.workunit.client.0.smithi137.stdout:8/327: read d9/f1d [627231,91449] 0 2024-06-15T01:07:01.224 INFO:tasks.workunit.client.1.smithi142.stdout:9/403: creat d6/dc/d4e/d65/fa2 x:0 0 0 2024-06-15T01:07:01.224 INFO:tasks.workunit.client.1.smithi142.stdout:2/369: truncate d8/df/f48 2551974 0 2024-06-15T01:07:01.225 INFO:tasks.workunit.client.1.smithi142.stdout:1/457: truncate d6/d12/d28/f64 4578617 0 2024-06-15T01:07:01.226 INFO:tasks.workunit.client.0.smithi137.stdout:7/344: dread df/d18/d1c/d1d/d27/f49 [0,4194304] 0 2024-06-15T01:07:01.227 INFO:tasks.workunit.client.0.smithi137.stdout:5/499: rename l8 to df/la7 0 2024-06-15T01:07:01.227 INFO:tasks.workunit.client.1.smithi142.stdout:6/411: readlink d1a/l69 0 2024-06-15T01:07:01.231 INFO:tasks.workunit.client.1.smithi142.stdout:3/495: creat d0/d3/da/d71/f97 x:0 0 0 2024-06-15T01:07:01.231 INFO:tasks.workunit.client.0.smithi137.stdout:4/367: creat dd/d10/d1f/d86/f8a x:0 0 0 2024-06-15T01:07:01.231 INFO:tasks.workunit.client.0.smithi137.stdout:5/500: fdatasync df/f41 0 2024-06-15T01:07:01.231 INFO:tasks.workunit.client.0.smithi137.stdout:7/345: symlink df/d13/d2e/d62/l70 0 2024-06-15T01:07:01.232 INFO:tasks.workunit.client.0.smithi137.stdout:5/501: stat df/d44/d54 0 2024-06-15T01:07:01.234 INFO:tasks.workunit.client.1.smithi142.stdout:9/404: mknod d6/dc/d82/ca3 0 2024-06-15T01:07:01.235 INFO:tasks.workunit.client.0.smithi137.stdout:5/502: write df/d11/f31 [1655715,49979] 0 2024-06-15T01:07:01.236 INFO:tasks.workunit.client.0.smithi137.stdout:7/346: mknod df/d13/d3c/c71 0 2024-06-15T01:07:01.237 INFO:tasks.workunit.client.1.smithi142.stdout:8/390: dwrite d0/f18 [4194304,4194304] 0 2024-06-15T01:07:01.238 INFO:tasks.workunit.client.0.smithi137.stdout:7/347: dread - df/d13/d3c/f6e zero size 2024-06-15T01:07:01.238 INFO:tasks.workunit.client.1.smithi142.stdout:6/412: mknod d1a/d2d/d3b/c99 0 2024-06-15T01:07:01.240 INFO:tasks.workunit.client.0.smithi137.stdout:2/302: dread d0/da/f18 [0,4194304] 0 2024-06-15T01:07:01.240 INFO:tasks.workunit.client.1.smithi142.stdout:0/421: dwrite d3/d39/d3e/d81/d1b/d38/f8c [0,4194304] 0 2024-06-15T01:07:01.240 INFO:tasks.workunit.client.1.smithi142.stdout:2/370: mkdir d8/d19/d23/d25/d76 0 2024-06-15T01:07:01.241 INFO:tasks.workunit.client.1.smithi142.stdout:5/412: dwrite d2/de/f5a [0,4194304] 0 2024-06-15T01:07:01.243 INFO:tasks.workunit.client.0.smithi137.stdout:2/303: write d0/de/f16 [2118089,111366] 0 2024-06-15T01:07:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:9/197: sync 2024-06-15T01:07:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:0/338: sync 2024-06-15T01:07:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:3/380: sync 2024-06-15T01:07:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:6/239: sync 2024-06-15T01:07:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:1/381: sync 2024-06-15T01:07:01.246 INFO:tasks.workunit.client.1.smithi142.stdout:5/413: stat d2/de/d19/d5f/d68 0 2024-06-15T01:07:01.247 INFO:tasks.workunit.client.0.smithi137.stdout:5/503: dread df/d11/d30/f61 [0,4194304] 0 2024-06-15T01:07:01.248 INFO:tasks.workunit.client.1.smithi142.stdout:5/414: chown d2/de/d19/f22 30901901 1 2024-06-15T01:07:01.250 INFO:tasks.workunit.client.0.smithi137.stdout:3/381: stat d0/d2/d23/f3f 0 2024-06-15T01:07:01.250 INFO:tasks.workunit.client.0.smithi137.stdout:0/339: write d1/d2/dc/d10/d14/d15/f4b [3940824,69187] 0 2024-06-15T01:07:01.251 INFO:tasks.workunit.client.1.smithi142.stdout:5/415: write d2/d18/f55 [1070881,100845] 0 2024-06-15T01:07:01.252 INFO:tasks.workunit.client.0.smithi137.stdout:7/348: dread df/d13/d2e/f4e [0,4194304] 0 2024-06-15T01:07:01.254 INFO:tasks.workunit.client.1.smithi142.stdout:5/416: fsync d2/d3e/d31/f64 0 2024-06-15T01:07:01.254 INFO:tasks.workunit.client.1.smithi142.stdout:7/405: dread d2/d6/d22/f4d [0,4194304] 0 2024-06-15T01:07:01.254 INFO:tasks.workunit.client.0.smithi137.stdout:3/382: write d0/d13/f20 [1740914,98562] 0 2024-06-15T01:07:01.254 INFO:tasks.workunit.client.0.smithi137.stdout:1/382: write d0/d1f/d76/f80 [4758123,73665] 0 2024-06-15T01:07:01.255 INFO:tasks.workunit.client.1.smithi142.stdout:6/413: symlink d1a/d39/d42/d4c/d58/d87/l9a 0 2024-06-15T01:07:01.256 INFO:tasks.workunit.client.1.smithi142.stdout:6/414: readlink d1a/d2d/l96 0 2024-06-15T01:07:01.256 INFO:tasks.workunit.client.1.smithi142.stdout:3/496: dwrite d0/f6a [0,4194304] 0 2024-06-15T01:07:01.257 INFO:tasks.workunit.client.1.smithi142.stdout:7/406: fsync d2/f3 0 2024-06-15T01:07:01.259 INFO:tasks.workunit.client.1.smithi142.stdout:2/371: creat d8/d19/d23/d25/d1a/d63/f77 x:0 0 0 2024-06-15T01:07:01.260 INFO:tasks.workunit.client.1.smithi142.stdout:4/524: mkdir d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/da6/db0/dbc 0 2024-06-15T01:07:01.261 INFO:tasks.workunit.client.1.smithi142.stdout:0/422: mkdir d3/d39/d9a/d9d 0 2024-06-15T01:07:01.262 INFO:tasks.workunit.client.1.smithi142.stdout:3/497: write d0/d1f/f46 [1160523,128993] 0 2024-06-15T01:07:01.263 INFO:tasks.workunit.client.0.smithi137.stdout:8/328: dread d9/de/f36 [0,4194304] 0 2024-06-15T01:07:01.265 INFO:tasks.workunit.client.1.smithi142.stdout:1/458: dread d6/d12/d3c/d63/fa8 [0,4194304] 0 2024-06-15T01:07:01.265 INFO:tasks.workunit.client.0.smithi137.stdout:9/198: dread df/f28 [0,4194304] 0 2024-06-15T01:07:01.268 INFO:tasks.workunit.client.1.smithi142.stdout:3/498: read - d0/d1f/d20/d5d/d60/f84 zero size 2024-06-15T01:07:01.271 INFO:tasks.workunit.client.1.smithi142.stdout:9/405: getdents d6/dc/d51/d53 0 2024-06-15T01:07:01.272 INFO:tasks.workunit.client.0.smithi137.stdout:7/349: rmdir df 39 2024-06-15T01:07:01.273 INFO:tasks.workunit.client.0.smithi137.stdout:4/368: truncate dd/d10/f54 2419179 0 2024-06-15T01:07:01.274 INFO:tasks.workunit.client.0.smithi137.stdout:2/304: link d0/de/l44 d0/de/d2b/d47/l5e 0 2024-06-15T01:07:01.275 INFO:tasks.workunit.client.1.smithi142.stdout:8/391: dread d0/d17/d40/d5e/f81 [0,4194304] 0 2024-06-15T01:07:01.275 INFO:tasks.workunit.client.0.smithi137.stdout:2/305: write d0/dc/d22/d23/f53 [966201,69255] 0 2024-06-15T01:07:01.276 INFO:tasks.workunit.client.1.smithi142.stdout:5/417: dwrite d2/d3e/d31/f3c [0,4194304] 0 2024-06-15T01:07:01.278 INFO:tasks.workunit.client.1.smithi142.stdout:3/499: creat d0/d3/da/d18/f98 x:0 0 0 2024-06-15T01:07:01.279 INFO:tasks.workunit.client.1.smithi142.stdout:4/525: fsync d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/f6b 0 2024-06-15T01:07:01.279 INFO:tasks.workunit.client.0.smithi137.stdout:5/504: rename df/d44/d54/d95/d3b/d4b/d4f to df/d76/d7c/d9c/da8 0 2024-06-15T01:07:01.279 INFO:tasks.workunit.client.1.smithi142.stdout:1/459: dread d6/d12/d1c/d1e/f4b [0,4194304] 0 2024-06-15T01:07:01.281 INFO:tasks.workunit.client.0.smithi137.stdout:5/505: write df/d11/d30/f87 [178403,10779] 0 2024-06-15T01:07:01.282 INFO:tasks.workunit.client.1.smithi142.stdout:1/460: write d6/d3a/f99 [1416051,13515] 0 2024-06-15T01:07:01.284 INFO:tasks.workunit.client.0.smithi137.stdout:0/340: write d1/d2/f2f [1493108,115657] 0 2024-06-15T01:07:01.286 INFO:tasks.workunit.client.0.smithi137.stdout:5/506: chown df/d76/d7c/d9c/da8/l98 723 1 2024-06-15T01:07:01.286 INFO:tasks.workunit.client.0.smithi137.stdout:9/199: mknod df/d12/d15/c37 0 2024-06-15T01:07:01.289 INFO:tasks.workunit.client.0.smithi137.stdout:9/200: dread - df/d12/d1b/f2f zero size 2024-06-15T01:07:01.290 INFO:tasks.workunit.client.0.smithi137.stdout:7/350: write df/d18/f5a [1261458,108850] 0 2024-06-15T01:07:01.290 INFO:tasks.workunit.client.0.smithi137.stdout:3/383: mkdir d0/d13/d37/d83 0 2024-06-15T01:07:01.292 INFO:tasks.workunit.client.1.smithi142.stdout:9/406: dwrite d6/dc/dd/d22/f49 [0,4194304] 0 2024-06-15T01:07:01.292 INFO:tasks.workunit.client.1.smithi142.stdout:5/418: mkdir d2/d3e/d31/d7e 0 2024-06-15T01:07:01.292 INFO:tasks.workunit.client.1.smithi142.stdout:6/415: dread d1a/f28 [0,4194304] 0 2024-06-15T01:07:01.293 INFO:tasks.workunit.client.0.smithi137.stdout:4/369: rename dd/d15/d36/d45/c5f to dd/d10/c8b 0 2024-06-15T01:07:01.293 INFO:tasks.workunit.client.0.smithi137.stdout:9/201: chown df/d13/f22 1759479115 1 2024-06-15T01:07:01.293 INFO:tasks.workunit.client.0.smithi137.stdout:9/202: readlink l6 0 2024-06-15T01:07:01.294 INFO:tasks.workunit.client.1.smithi142.stdout:3/500: mkdir d0/d1f/d20/d5d/d60/d99 0 2024-06-15T01:07:01.297 INFO:tasks.workunit.client.0.smithi137.stdout:7/351: symlink df/d18/d1c/d1d/l72 0 2024-06-15T01:07:01.298 INFO:tasks.workunit.client.0.smithi137.stdout:5/507: rename df/d11/d14/f2f to df/d76/d7c/d9c/da1/fa9 0 2024-06-15T01:07:01.298 INFO:tasks.workunit.client.1.smithi142.stdout:1/461: creat d6/d46/d61/faa x:0 0 0 2024-06-15T01:07:01.300 INFO:tasks.workunit.client.1.smithi142.stdout:3/501: dread - d0/d3/f90 zero size 2024-06-15T01:07:01.302 INFO:tasks.workunit.client.0.smithi137.stdout:1/383: getdents d0/d3/d14/d34 0 2024-06-15T01:07:01.303 INFO:tasks.workunit.client.1.smithi142.stdout:4/526: rmdir d5/d9/d90/da1 0 2024-06-15T01:07:01.306 INFO:tasks.workunit.client.1.smithi142.stdout:2/372: truncate d8/df/f2d 489749 0 2024-06-15T01:07:01.306 INFO:tasks.workunit.client.1.smithi142.stdout:0/423: dwrite d3/f9 [0,4194304] 0 2024-06-15T01:07:01.308 INFO:tasks.workunit.client.0.smithi137.stdout:9/203: dwrite df/d2a/f34 [0,4194304] 0 2024-06-15T01:07:01.310 INFO:tasks.workunit.client.1.smithi142.stdout:3/502: fdatasync d0/d3/da/d71/d55/f7a 0 2024-06-15T01:07:01.312 INFO:tasks.workunit.client.0.smithi137.stdout:1/384: mkdir d0/d1f/d60/d99 0 2024-06-15T01:07:01.314 INFO:tasks.workunit.client.1.smithi142.stdout:2/373: stat d8/d19/d23/d25/d76 0 2024-06-15T01:07:01.314 INFO:tasks.workunit.client.0.smithi137.stdout:0/341: rename d1/ld to d1/d2/dc/d10/d14/d15/d17/d54/l76 0 2024-06-15T01:07:01.314 INFO:tasks.workunit.client.0.smithi137.stdout:9/204: write df/d12/d15/f26 [1129620,80954] 0 2024-06-15T01:07:01.315 INFO:tasks.workunit.client.1.smithi142.stdout:7/407: dread d2/d8/da/d33/d25/f16 [0,4194304] 0 2024-06-15T01:07:01.315 INFO:tasks.workunit.client.0.smithi137.stdout:5/508: dwrite df/f2b [8388608,4194304] 0 2024-06-15T01:07:01.317 INFO:tasks.workunit.client.0.smithi137.stdout:1/385: stat d0/d10/cf 0 2024-06-15T01:07:01.317 INFO:tasks.workunit.client.1.smithi142.stdout:5/419: dwrite d2/d18/f63 [0,4194304] 0 2024-06-15T01:07:01.319 INFO:tasks.workunit.client.0.smithi137.stdout:1/386: fdatasync d0/d2d/f62 0 2024-06-15T01:07:01.322 INFO:tasks.workunit.client.1.smithi142.stdout:0/424: dread d3/d39/d3e/d81/f34 [0,4194304] 0 2024-06-15T01:07:01.326 INFO:tasks.workunit.client.1.smithi142.stdout:5/420: chown d2/de/d27/f46 3283891 1 2024-06-15T01:07:01.326 INFO:tasks.workunit.client.1.smithi142.stdout:6/416: dwrite d1a/d83/f7c [0,4194304] 0 2024-06-15T01:07:01.329 INFO:tasks.workunit.client.1.smithi142.stdout:5/421: chown d2/d18/c39 0 1 2024-06-15T01:07:01.330 INFO:tasks.workunit.client.1.smithi142.stdout:3/503: creat d0/d1f/d20/d5d/d82/f9a x:0 0 0 2024-06-15T01:07:01.334 INFO:tasks.workunit.client.0.smithi137.stdout:1/387: unlink d0/d1f/d60/l29 0 2024-06-15T01:07:01.335 INFO:tasks.workunit.client.0.smithi137.stdout:8/329: dwrite d9/d22/f37 [0,4194304] 0 2024-06-15T01:07:01.336 INFO:tasks.workunit.client.1.smithi142.stdout:0/425: dread - d3/d39/d3e/d81/d44/d7c/d22/d59/f80 zero size 2024-06-15T01:07:01.339 INFO:tasks.workunit.client.1.smithi142.stdout:8/392: dwrite d0/d25/d5a/f2c [0,4194304] 0 2024-06-15T01:07:01.340 INFO:tasks.workunit.client.0.smithi137.stdout:7/352: getdents df/d18 0 2024-06-15T01:07:01.340 INFO:tasks.workunit.client.0.smithi137.stdout:2/306: dread d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:07:01.340 INFO:tasks.workunit.client.0.smithi137.stdout:7/353: chown df/d13/d2e/f4f 177 1 2024-06-15T01:07:01.341 INFO:tasks.workunit.client.1.smithi142.stdout:6/417: rmdir d1a/d39/d3a/d51 39 2024-06-15T01:07:01.344 INFO:tasks.workunit.client.0.smithi137.stdout:9/205: dwrite df/d12/f1f [0,4194304] 0 2024-06-15T01:07:01.344 INFO:tasks.workunit.client.0.smithi137.stdout:3/384: rename d0/d2/d22/f58 to d0/d56/f84 0 2024-06-15T01:07:01.345 INFO:tasks.workunit.client.1.smithi142.stdout:5/422: symlink d2/de/d19/d5f/d68/l7f 0 2024-06-15T01:07:01.346 INFO:tasks.workunit.client.1.smithi142.stdout:8/393: dread - d0/d17/d40/d6e/f73 zero size 2024-06-15T01:07:01.349 INFO:tasks.workunit.client.1.smithi142.stdout:3/504: dwrite d0/d1f/f69 [4194304,4194304] 0 2024-06-15T01:07:01.351 INFO:tasks.workunit.client.1.smithi142.stdout:9/407: dread f2 [0,4194304] 0 2024-06-15T01:07:01.352 INFO:tasks.workunit.client.0.smithi137.stdout:9/206: unlink fc 0 2024-06-15T01:07:01.352 INFO:tasks.workunit.client.1.smithi142.stdout:4/527: rename d5/d1c/d6f/db9/lba to d5/d9/lbd 0 2024-06-15T01:07:01.354 INFO:tasks.workunit.client.1.smithi142.stdout:6/418: rmdir d1a/d2d/d68 39 2024-06-15T01:07:01.354 INFO:tasks.workunit.client.1.smithi142.stdout:4/528: dread - d5/d1c/fbb zero size 2024-06-15T01:07:01.354 INFO:tasks.workunit.client.1.smithi142.stdout:9/408: chown d6/dc/dd/d22/f49 605 1 2024-06-15T01:07:01.354 INFO:tasks.workunit.client.0.smithi137.stdout:5/509: rename df/d44 to df/d44/daa 22 2024-06-15T01:07:01.357 INFO:tasks.workunit.client.1.smithi142.stdout:1/462: rename d6/d46/c50 to d6/d46/cab 0 2024-06-15T01:07:01.357 INFO:tasks.workunit.client.0.smithi137.stdout:9/207: truncate df/d12/d1b/f2e 718381 0 2024-06-15T01:07:01.358 INFO:tasks.workunit.client.0.smithi137.stdout:8/330: creat d9/d2a/d2f/d4e/f70 x:0 0 0 2024-06-15T01:07:01.360 INFO:tasks.workunit.client.1.smithi142.stdout:3/505: rmdir d0/d3/da/d18 39 2024-06-15T01:07:01.361 INFO:tasks.workunit.client.0.smithi137.stdout:0/342: mkdir d1/d2/dc/d10/d63/d77 0 2024-06-15T01:07:01.363 INFO:tasks.workunit.client.0.smithi137.stdout:7/354: read df/d18/d1c/d1d/d27/d33/d59/f60 [2936614,61619] 0 2024-06-15T01:07:01.363 INFO:tasks.workunit.client.0.smithi137.stdout:2/307: symlink d0/l5f 0 2024-06-15T01:07:01.364 INFO:tasks.workunit.client.1.smithi142.stdout:0/426: rename d3/d39/f6f to d3/d39/d9a/d9d/f9e 0 2024-06-15T01:07:01.366 INFO:tasks.workunit.client.0.smithi137.stdout:3/385: dread d0/f1f [0,4194304] 0 2024-06-15T01:07:01.366 INFO:tasks.workunit.client.0.smithi137.stdout:9/208: unlink l4 0 2024-06-15T01:07:01.368 INFO:tasks.workunit.client.1.smithi142.stdout:8/394: mknod d0/d25/d5a/c83 0 2024-06-15T01:07:01.368 INFO:tasks.workunit.client.1.smithi142.stdout:1/463: unlink d6/d12/d52/c96 0 2024-06-15T01:07:01.368 INFO:tasks.workunit.client.0.smithi137.stdout:7/355: dread df/d18/d1c/f1f [0,4194304] 0 2024-06-15T01:07:01.369 INFO:tasks.workunit.client.1.smithi142.stdout:1/464: readlink d6/d12/d1c/d2c/la7 0 2024-06-15T01:07:01.370 INFO:tasks.workunit.client.0.smithi137.stdout:1/388: link d0/d10/d1d/c53 d0/d2d/d3e/c9a 0 2024-06-15T01:07:01.370 INFO:tasks.workunit.client.0.smithi137.stdout:0/343: rmdir d1/d2/dc/d10/d14/d52 39 2024-06-15T01:07:01.371 INFO:tasks.workunit.client.0.smithi137.stdout:7/356: stat df/d18/d3b/d5b/f5d 0 2024-06-15T01:07:01.372 INFO:tasks.workunit.client.0.smithi137.stdout:7/357: chown c8 573 1 2024-06-15T01:07:01.376 INFO:tasks.workunit.client.0.smithi137.stdout:8/331: truncate d9/de/f18 2443581 0 2024-06-15T01:07:01.376 INFO:tasks.workunit.client.0.smithi137.stdout:7/358: dread - df/d18/d3b/d5b/f6d zero size 2024-06-15T01:07:01.376 INFO:tasks.workunit.client.1.smithi142.stdout:4/529: mkdir d5/d9/d23/dbe 0 2024-06-15T01:07:01.376 INFO:tasks.workunit.client.1.smithi142.stdout:2/374: write d8/f40 [936789,23676] 0 2024-06-15T01:07:01.378 INFO:tasks.workunit.client.1.smithi142.stdout:4/530: readlink d5/d1c/d2a/d30/d46/d50/d51/d5b/l9f 0 2024-06-15T01:07:01.379 INFO:tasks.workunit.client.0.smithi137.stdout:2/308: symlink d0/de/d10/d3a/l60 0 2024-06-15T01:07:01.379 INFO:tasks.workunit.client.0.smithi137.stdout:7/359: stat df/d13/d3c/f6e 0 2024-06-15T01:07:01.379 INFO:tasks.workunit.client.0.smithi137.stdout:3/386: write d0/d13/d37/f5f [714229,108850] 0 2024-06-15T01:07:01.381 INFO:tasks.workunit.client.0.smithi137.stdout:1/389: mkdir d0/d1f/d60/d99/d9b 0 2024-06-15T01:07:01.382 INFO:tasks.workunit.client.0.smithi137.stdout:8/332: write d9/de/d1e/d35/f63 [274704,93767] 0 2024-06-15T01:07:01.382 INFO:tasks.workunit.client.1.smithi142.stdout:3/506: rename d0/d1b/f5b to d0/d3/da/f9b 0 2024-06-15T01:07:01.383 INFO:tasks.workunit.client.0.smithi137.stdout:7/360: mknod df/d18/c73 0 2024-06-15T01:07:01.385 INFO:tasks.workunit.client.0.smithi137.stdout:9/209: dwrite df/d12/f2c [0,4194304] 0 2024-06-15T01:07:01.385 INFO:tasks.workunit.client.0.smithi137.stdout:2/309: creat d0/dc/d22/f61 x:0 0 0 2024-06-15T01:07:01.385 INFO:tasks.workunit.client.1.smithi142.stdout:1/465: getdents d6/d12/d5b 0 2024-06-15T01:07:01.386 INFO:tasks.workunit.client.1.smithi142.stdout:7/408: dread d2/d6/f1c [0,4194304] 0 2024-06-15T01:07:01.387 INFO:tasks.workunit.client.1.smithi142.stdout:2/375: link d8/df/d10/d1c/l2f d8/d19/d23/d25/l78 0 2024-06-15T01:07:01.387 INFO:tasks.workunit.client.0.smithi137.stdout:2/310: chown d0/de/d10/f2a 601 1 2024-06-15T01:07:01.388 INFO:tasks.workunit.client.1.smithi142.stdout:4/531: dwrite d5/d1c/d2a/f3d [0,4194304] 0 2024-06-15T01:07:01.389 INFO:tasks.workunit.client.1.smithi142.stdout:4/532: chown d5/d9/d23 598124 1 2024-06-15T01:07:01.391 INFO:tasks.workunit.client.0.smithi137.stdout:3/387: write d0/d2/d23/d68/f3c [62278,99876] 0 2024-06-15T01:07:01.391 INFO:tasks.workunit.client.0.smithi137.stdout:2/311: write d0/dc/f3b [1122244,40739] 0 2024-06-15T01:07:01.391 INFO:tasks.workunit.client.0.smithi137.stdout:1/390: unlink d0/d2d/d3e/c9a 0 2024-06-15T01:07:01.395 INFO:tasks.workunit.client.0.smithi137.stdout:3/388: truncate d0/d2/d23/f48 4487298 0 2024-06-15T01:07:01.396 INFO:tasks.workunit.client.0.smithi137.stdout:3/389: stat d0/d2/d1e/l3e 0 2024-06-15T01:07:01.396 INFO:tasks.workunit.client.0.smithi137.stdout:2/312: creat d0/d42/d49/f62 x:0 0 0 2024-06-15T01:07:01.397 INFO:tasks.workunit.client.0.smithi137.stdout:3/390: readlink d0/d2/d22/d24/l25 0 2024-06-15T01:07:01.398 INFO:tasks.workunit.client.0.smithi137.stdout:9/210: mknod df/d12/d15/d16/c38 0 2024-06-15T01:07:01.399 INFO:tasks.workunit.client.1.smithi142.stdout:5/423: dread d2/d3e/d65/f70 [0,4194304] 0 2024-06-15T01:07:01.402 INFO:tasks.workunit.client.1.smithi142.stdout:7/409: dwrite d2/f49 [0,4194304] 0 2024-06-15T01:07:01.402 INFO:tasks.workunit.client.0.smithi137.stdout:9/211: write df/d12/d15/f26 [2194963,76711] 0 2024-06-15T01:07:01.403 INFO:tasks.workunit.client.1.smithi142.stdout:6/419: write f7 [2333459,126641] 0 2024-06-15T01:07:01.404 INFO:tasks.workunit.client.0.smithi137.stdout:9/212: fsync df/d12/f17 0 2024-06-15T01:07:01.406 INFO:tasks.workunit.client.1.smithi142.stdout:6/420: chown d1a/d2d/d80/d8e 1451 1 2024-06-15T01:07:01.409 INFO:tasks.workunit.client.1.smithi142.stdout:6/421: read - d1a/d3c/f48 zero size 2024-06-15T01:07:01.410 INFO:tasks.workunit.client.1.smithi142.stdout:2/376: mknod d8/d19/d23/d25/c79 0 2024-06-15T01:07:01.411 INFO:tasks.workunit.client.0.smithi137.stdout:8/333: getdents d9/d2a/d2f/d59 0 2024-06-15T01:07:01.411 INFO:tasks.workunit.client.0.smithi137.stdout:3/391: dread d0/d2/d1e/f6e [0,4194304] 0 2024-06-15T01:07:01.411 INFO:tasks.workunit.client.0.smithi137.stdout:5/510: write df/d44/d54/d95/d35/f81 [2626789,7237] 0 2024-06-15T01:07:01.413 INFO:tasks.workunit.client.0.smithi137.stdout:1/391: symlink d0/d1f/d60/d99/d9b/l9c 0 2024-06-15T01:07:01.413 INFO:tasks.workunit.client.1.smithi142.stdout:2/377: dread - d8/d19/f75 zero size 2024-06-15T01:07:01.413 INFO:tasks.workunit.client.1.smithi142.stdout:8/395: dwrite d0/d17/f61 [0,4194304] 0 2024-06-15T01:07:01.417 INFO:tasks.workunit.client.0.smithi137.stdout:5/511: write df/d11/f31 [2559228,124147] 0 2024-06-15T01:07:01.417 INFO:tasks.workunit.client.0.smithi137.stdout:2/313: dwrite d0/d42/d4d/d56/f4f [0,4194304] 0 2024-06-15T01:07:01.419 INFO:tasks.workunit.client.0.smithi137.stdout:2/314: chown d0/de/f1a 1413 1 2024-06-15T01:07:01.420 INFO:tasks.workunit.client.1.smithi142.stdout:2/378: chown d8/d19/d23/d25/d1a/f2e 7 1 2024-06-15T01:07:01.420 INFO:tasks.workunit.client.0.smithi137.stdout:7/361: chown df/d18/c73 37 1 2024-06-15T01:07:01.420 INFO:tasks.workunit.client.1.smithi142.stdout:1/466: creat d6/d46/d61/fac x:0 0 0 2024-06-15T01:07:01.421 INFO:tasks.workunit.client.0.smithi137.stdout:9/213: mknod df/d12/d15/c39 0 2024-06-15T01:07:01.421 INFO:tasks.workunit.client.1.smithi142.stdout:3/507: write d0/d1b/f2a [1410805,95840] 0 2024-06-15T01:07:01.424 INFO:tasks.workunit.client.1.smithi142.stdout:7/410: creat d2/d6/f83 x:0 0 0 2024-06-15T01:07:01.426 INFO:tasks.workunit.client.1.smithi142.stdout:6/422: dwrite d1a/d39/d42/d4c/d58/d87/f93 [0,4194304] 0 2024-06-15T01:07:01.427 INFO:tasks.workunit.client.1.smithi142.stdout:1/467: dread - d6/d46/d61/f76 zero size 2024-06-15T01:07:01.427 INFO:tasks.workunit.client.1.smithi142.stdout:0/427: creat d3/d39/d9a/d9d/f9f x:0 0 0 2024-06-15T01:07:01.427 INFO:tasks.workunit.client.1.smithi142.stdout:8/396: mkdir d0/d25/d5a/d84 0 2024-06-15T01:07:01.428 INFO:tasks.workunit.client.0.smithi137.stdout:3/392: stat d0/d56/c70 0 2024-06-15T01:07:01.428 INFO:tasks.workunit.client.0.smithi137.stdout:1/392: creat d0/d3/d1b/f9d x:0 0 0 2024-06-15T01:07:01.430 INFO:tasks.workunit.client.1.smithi142.stdout:2/379: mkdir d8/d19/d23/d25/d22/d43/d52/d7a 0 2024-06-15T01:07:01.431 INFO:tasks.workunit.client.0.smithi137.stdout:5/512: rename df/d11/d14/f33 to df/d11/d70/fab 0 2024-06-15T01:07:01.431 INFO:tasks.workunit.client.0.smithi137.stdout:7/362: dwrite df/f32 [0,4194304] 0 2024-06-15T01:07:01.432 INFO:tasks.workunit.client.0.smithi137.stdout:3/393: chown d0/d2/d23/f3f 7863 1 2024-06-15T01:07:01.432 INFO:tasks.workunit.client.0.smithi137.stdout:5/513: fsync df/d11/d14/d6c/f90 0 2024-06-15T01:07:01.432 INFO:tasks.workunit.client.1.smithi142.stdout:1/468: fdatasync d6/d12/d3c/d63/f85 0 2024-06-15T01:07:01.435 INFO:tasks.workunit.client.0.smithi137.stdout:3/394: write d0/d2/d23/f48 [1592524,18560] 0 2024-06-15T01:07:01.441 INFO:tasks.workunit.client.0.smithi137.stdout:1/393: write d0/d2d/f5e [1688850,126554] 0 2024-06-15T01:07:01.442 INFO:tasks.workunit.client.1.smithi142.stdout:6/423: creat d1a/d39/d3a/d51/f9b x:0 0 0 2024-06-15T01:07:01.443 INFO:tasks.workunit.client.1.smithi142.stdout:8/397: truncate d0/d25/d5a/f4d 415567 0 2024-06-15T01:07:01.443 INFO:tasks.workunit.client.1.smithi142.stdout:5/424: dwrite d2/de/d19/f22 [8388608,4194304] 0 2024-06-15T01:07:01.444 INFO:tasks.workunit.client.0.smithi137.stdout:1/394: dread - d0/d3/d1b/d4c/f95 zero size 2024-06-15T01:07:01.444 INFO:tasks.workunit.client.0.smithi137.stdout:9/214: rename df/c1c to df/d2a/c3a 0 2024-06-15T01:07:01.444 INFO:tasks.workunit.client.0.smithi137.stdout:1/395: fsync d0/d2d/f79 0 2024-06-15T01:07:01.446 INFO:tasks.workunit.client.1.smithi142.stdout:1/469: dwrite d6/d3a/f70 [0,4194304] 0 2024-06-15T01:07:01.446 INFO:tasks.workunit.client.0.smithi137.stdout:7/363: unlink df/c50 0 2024-06-15T01:07:01.447 INFO:tasks.workunit.client.0.smithi137.stdout:7/364: chown df/d13/d2e/d1a 17675176 1 2024-06-15T01:07:01.447 INFO:tasks.workunit.client.1.smithi142.stdout:3/508: rename d0/d1f/l35 to d0/d3/l9c 0 2024-06-15T01:07:01.448 INFO:tasks.workunit.client.0.smithi137.stdout:7/365: chown df/d13/d2e/d1a/c46 12488 1 2024-06-15T01:07:01.449 INFO:tasks.workunit.client.1.smithi142.stdout:4/533: dread d5/d1c/d2a/d30/d46/f76 [0,4194304] 0 2024-06-15T01:07:01.449 INFO:tasks.workunit.client.1.smithi142.stdout:2/380: mknod d8/d19/d23/d25/d76/c7b 0 2024-06-15T01:07:01.450 INFO:tasks.workunit.client.1.smithi142.stdout:7/411: creat d2/d8/f84 x:0 0 0 2024-06-15T01:07:01.451 INFO:tasks.workunit.client.0.smithi137.stdout:2/315: truncate d0/da/f18 2783374 0 2024-06-15T01:07:01.452 INFO:tasks.workunit.client.0.smithi137.stdout:3/395: creat d0/d2/d22/d24/d5e/f85 x:0 0 0 2024-06-15T01:07:01.454 INFO:tasks.workunit.client.0.smithi137.stdout:9/215: symlink df/d12/l3b 0 2024-06-15T01:07:01.454 INFO:tasks.workunit.client.0.smithi137.stdout:9/216: readlink df/l14 0 2024-06-15T01:07:01.457 INFO:tasks.workunit.client.1.smithi142.stdout:5/425: symlink d2/de/d19/l80 0 2024-06-15T01:07:01.457 INFO:tasks.workunit.client.1.smithi142.stdout:0/428: dread - d3/d39/d3e/d81/d44/d7c/f3c zero size 2024-06-15T01:07:01.457 INFO:tasks.workunit.client.1.smithi142.stdout:1/470: creat d6/d12/d5b/fad x:0 0 0 2024-06-15T01:07:01.458 INFO:tasks.workunit.client.1.smithi142.stdout:5/426: write d2/de/d27/f54 [1567571,23473] 0 2024-06-15T01:07:01.460 INFO:tasks.workunit.client.1.smithi142.stdout:4/534: fsync d5/d1c/d2a/d30/d46/d50/d51/d5b/d12/f43 0 2024-06-15T01:07:01.460 INFO:tasks.workunit.client.1.smithi142.stdout:2/381: write d8/d19/d33/f66 [48420,86616] 0 2024-06-15T01:07:01.461 INFO:tasks.workunit.client.0.smithi137.stdout:1/396: getdents d0/d3/d1b/d4c/d59/d93 0 2024-06-15T01:07:01.462 INFO:tasks.workunit.client.1.smithi142.stdout:0/429: stat d3/d39/d3e/d81/d1b/d86 0 2024-06-15T01:07:01.463 INFO:tasks.workunit.client.0.smithi137.stdout:1/397: fdatasync d0/d1f/d76/f80 0 2024-06-15T01:07:01.464 INFO:tasks.workunit.client.1.smithi142.stdout:3/509: write d0/d3/d15/f2d [1856348,12627] 0 2024-06-15T01:07:01.464 INFO:tasks.workunit.client.1.smithi142.stdout:1/471: write d6/d3a/f88 [1060008,90308] 0 2024-06-15T01:07:01.465 INFO:tasks.workunit.client.1.smithi142.stdout:8/398: dwrite d0/d17/d40/d6e/f73 [0,4194304] 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.1.smithi142.stdout:7/412: mkdir d2/d8/dd/d14/d2c/d37/d6b/d80/d85 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.1.smithi142.stdout:5/427: write d2/de/d19/d5f/d68/f73 [585043,13669] 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.1.smithi142.stdout:3/510: write d0/d1f/f4b [2858520,87008] 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.0.smithi137.stdout:7/366: dwrite df/d13/d2e/f4f [0,4194304] 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.0.smithi137.stdout:2/316: rename d0/dc/d2e/l55 to d0/d42/d49/l63 0 2024-06-15T01:07:01.470 INFO:tasks.workunit.client.0.smithi137.stdout:1/398: read d0/d10/d41/d7b/f90 [12531,65368] 0 2024-06-15T01:07:01.471 INFO:tasks.workunit.client.1.smithi142.stdout:4/535: symlink d5/d9/da/d92/lbf 0 2024-06-15T01:07:01.473 INFO:tasks.workunit.client.1.smithi142.stdout:2/382: mkdir d8/df/d10/d1c/d7c 0 2024-06-15T01:07:01.473 INFO:tasks.workunit.client.0.smithi137.stdout:2/317: write d0/f1 [1287238,63418] 0 2024-06-15T01:07:01.474 INFO:tasks.workunit.client.0.smithi137.stdout:3/396: link d0/d56/f6a d0/d2/d8/d67/d7e/f86 0 2024-06-15T01:07:01.476 INFO:tasks.workunit.client.1.smithi142.stdout:1/472: mkdir d6/d12/d80/dae 0 2024-06-15T01:07:01.477 INFO:tasks.workunit.client.0.smithi137.stdout:3/397: read - d0/d2/d22/d24/d5e/f85 zero size 2024-06-15T01:07:01.478 INFO:tasks.workunit.client.1.smithi142.stdout:3/511: mknod d0/d1b/c9d 0 2024-06-15T01:07:01.479 INFO:tasks.workunit.client.1.smithi142.stdout:5/428: creat d2/de/d19/d5f/d68/d76/f81 x:0 0 0 2024-06-15T01:07:01.480 INFO:tasks.workunit.client.1.smithi142.stdout:0/430: truncate d3/d39/d3e/d81/f88 1242620 0 2024-06-15T01:07:01.480 INFO:tasks.workunit.client.1.smithi142.stdout:7/413: read d2/d6/f9 [4075829,9454] 0 2024-06-15T01:07:01.481 INFO:tasks.workunit.client.1.smithi142.stdout:4/536: mknod d5/d1c/d2a/d30/d46/d50/d51/d5b/d20/da6/cc0 0 2024-06-15T01:07:01.486 INFO:tasks.workunit.client.1.smithi142.stdout:2/383: dwrite d8/df/d10/d1c/f2c [0,4194304] 0 2024-06-15T01:07:01.486 INFO:tasks.workunit.client.1.smithi142.stdout:7/414: chown d2/d6/d22/d47/f72 0 1 2024-06-15T01:07:01.487 INFO:tasks.workunit.client.0.smithi137.stdout:1/399: symlink d0/d3/d1b/d4c/d59/d64/l9e 0 2024-06-15T01:07:01.488 INFO:tasks.workunit.client.1.smithi142.stdout:0/431: write d3/d39/f7d [132854,108489] 0 2024-06-15T01:07:01.489 INFO:tasks.workunit.client.0.smithi137.stdout:1/400: chown d0/d1f/d7d/c85 293466941 1 2024-06-15T01:07:01.490 INFO:tasks.workunit.client.1.smithi142.stdout:5/429: creat d2/d18/d79/f82 x:0 0 0 2024-06-15T01:07:01.491 INFO:tasks.workunit.client.1.smithi142.stdout:4/537: mkdir d5/d9/da/d92/d98/dc1 0 2024-06-15T01:07:01.491 INFO:tasks.workunit.client.1.smithi142.stdout:7/415: chown d2/d8/dd/d14/d2c/d37/d6b/d6e/f7d 156425 1 2024-06-15T01:07:01.493 INFO:tasks.workunit.client.0.smithi137.stdout:3/398: creat d0/d2/d23/f87 x:0 0 0 2024-06-15T01:07:01.493 INFO:tasks.workunit.client.0.smithi137.stdout:9/217: dread df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:01.493 INFO:tasks.workunit.client.0.smithi137.stdout:9/218: stat f5 0 2024-06-15T01:07:01.495 INFO:tasks.workunit.client.1.smithi142.stdout:2/384: chown d8/d19/d23/d25/d22/d43/l5f 1679 1 2024-06-15T01:07:01.495 INFO:tasks.workunit.client.1.smithi142.stdout:7/416: dread - d2/d6/d22/d47/f7a zero size 2024-06-15T01:07:01.497 INFO:tasks.workunit.client.0.smithi137.stdout:3/399: unlink d0/d2/d1e/f5c 0 2024-06-15T01:07:01.497 INFO:tasks.workunit.client.1.smithi142.stdout:8/399: link d0/f43 d0/d25/d5a/f85 0 2024-06-15T01:07:01.499 INFO:tasks.workunit.client.0.smithi137.stdout:9/219: fdatasync df/f28 0 2024-06-15T01:07:01.501 INFO:tasks.workunit.client.1.smithi142.stdout:0/432: mkdir d3/d39/d3e/d81/d4c/da0 0 2024-06-15T01:07:01.503 INFO:tasks.workunit.client.0.smithi137.stdout:1/401: creat d0/d3/f9f x:0 0 0 2024-06-15T01:07:01.503 INFO:tasks.workunit.client.1.smithi142.stdout:2/385: mknod d8/d19/d23/d25/d76/c7d 0 2024-06-15T01:07:01.504 INFO:tasks.workunit.client.1.smithi142.stdout:4/538: dwrite d5/d9/d18/f2f [0,4194304] 0 2024-06-15T01:07:01.505 INFO:tasks.workunit.client.0.smithi137.stdout:1/402: write d0/d3/f13 [2551674,54253] 0 2024-06-15T01:07:01.507 INFO:tasks.workunit.client.0.smithi137.stdout:9/220: mkdir df/d12/d15/d16/d3c 0 2024-06-15T01:07:01.508 INFO:tasks.workunit.client.0.smithi137.stdout:3/400: dwrite d0/d2/d22/d63/d76/f7d [0,4194304] 0 2024-06-15T01:07:01.509 INFO:tasks.workunit.client.1.smithi142.stdout:0/433: mknod d3/d39/d3e/d81/d1b/ca1 0 2024-06-15T01:07:01.509 INFO:tasks.workunit.client.1.smithi142.stdout:6/424: dwrite d1a/d2d/f32 [0,4194304] 0 2024-06-15T01:07:01.511 INFO:tasks.workunit.client.0.smithi137.stdout:2/318: link d0/d1c/l51 d0/l64 0 2024-06-15T01:07:01.516 INFO:tasks.workunit.client.0.smithi137.stdout:1/403: mkdir d0/d3/d14/da0 0 2024-06-15T01:07:01.518 INFO:tasks.workunit.client.0.smithi137.stdout:9/221: dread df/d12/d15/d16/f27 [0,4194304] 0 2024-06-15T01:07:01.521 INFO:tasks.workunit.client.0.smithi137.stdout:1/404: chown d0/d3/d1b/d4c/d59/f71 5 1 2024-06-15T01:07:01.521 INFO:tasks.workunit.client.0.smithi137.stdout:7/367: write df/d13/d2e/d1a/f2c [473837,3384] 0 2024-06-15T01:07:01.522 INFO:tasks.workunit.client.1.smithi142.stdout:1/473: truncate d6/d12/d5b/f8a 1648077 0 2024-06-15T01:07:01.523 INFO:tasks.workunit.client.1.smithi142.stdout:3/512: write d0/d3/f54 [368174,7095] 0 2024-06-15T01:07:01.523 INFO:tasks.workunit.client.1.smithi142.stdout:4/539: creat d5/d1c/d2a/d30/d46/d50/d51/d5b/d15/fc2 x:0 0 0 2024-06-15T01:07:01.524 INFO:tasks.workunit.client.1.smithi142.stdout:1/474: stat d6/d12/d30/c3f 0 2024-06-15T01:07:01.525 INFO:tasks.workunit.client.1.smithi142.stdout:0/434: mkdir d3/d39/d9a/da2 0 2024-06-15T01:07:01.527 INFO:tasks.workunit.client.1.smithi142.stdout:5/430: link d2/de/c3a d2/de/c83 0 2024-06-15T01:07:01.528 INFO:tasks.workunit.client.0.smithi137.stdout:3/401: creat d0/d13/d37/d66/f88 x:0 0 0 2024-06-15T01:07:01.528 INFO:tasks.workunit.client.1.smithi142.stdout:7/417: getdents d2/d8/dd/d14/d2c/d37/d6b 0 2024-06-15T01:07:01.528 INFO:tasks.workunit.client.1.smithi142.stdout:4/540: mknod d5/d9/d90/cc3 0 2024-06-15T01:07:01.529 INFO:tasks.workunit.client.1.smithi142.stdout:8/400: dread d0/d17/f56 [0,4194304] 0 2024-06-15T01:07:01.531 INFO:tasks.workunit.client.0.smithi137.stdout:1/405: mkdir d0/d1f/d76/da1 0 2024-06-15T01:07:01.532 INFO:tasks.workunit.client.0.smithi137.stdout:3/402: truncate d0/fd 1836786 0 2024-06-15T01:07:01.533 INFO:tasks.workunit.client.1.smithi142.stdout:8/401: write d0/f47 [2651229,59344] 0 2024-06-15T01:07:01.533 INFO:tasks.workunit.client.0.smithi137.stdout:1/406: dread - d0/d3/d14/d34/f70 zero size 2024-06-15T01:07:01.533 INFO:tasks.workunit.client.0.smithi137.stdout:2/319: creat d0/dc/f65 x:0 0 0 2024-06-15T01:07:01.535 INFO:tasks.workunit.client.1.smithi142.stdout:1/475: dwrite d6/d3a/f88 [0,4194304] 0 2024-06-15T01:07:01.536 INFO:tasks.workunit.client.1.smithi142.stdout:1/476: write d6/d12/d52/f89 [806593,93750] 0 2024-06-15T01:07:01.537 INFO:tasks.workunit.client.0.smithi137.stdout:3/403: creat d0/d2/d22/d24/f89 x:0 0 0 2024-06-15T01:07:01.537 INFO:tasks.workunit.client.0.smithi137.stdout:2/320: creat d0/d1c/f66 x:0 0 0 2024-06-15T01:07:01.539 INFO:tasks.workunit.client.0.smithi137.stdout:9/222: dwrite df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:01.543 INFO:tasks.workunit.client.0.smithi137.stdout:9/223: readlink df/d13/l2b 0 2024-06-15T01:07:01.546 INFO:tasks.workunit.client.0.smithi137.stdout:1/407: getdents d0/d3/d1b 0 2024-06-15T01:07:01.547 INFO:tasks.workunit.client.0.smithi137.stdout:2/321: dread d0/f6 [0,4194304] 0 2024-06-15T01:07:01.549 INFO:tasks.workunit.client.0.smithi137.stdout:9/224: getdents df/d12/d15/d16/d3c 0 2024-06-15T01:07:01.550 INFO:tasks.workunit.client.1.smithi142.stdout:2/386: dread d8/df/f3f [0,4194304] 0 2024-06-15T01:07:01.552 INFO:tasks.workunit.client.0.smithi137.stdout:9/225: fdatasync df/d12/d1b/f2f 0 2024-06-15T01:07:01.553 INFO:tasks.workunit.client.0.smithi137.stdout:3/404: dwrite d0/d2/d8/f41 [0,4194304] 0 2024-06-15T01:07:01.554 INFO:tasks.workunit.client.0.smithi137.stdout:9/226: fdatasync fd 0 2024-06-15T01:07:01.555 INFO:tasks.workunit.client.0.smithi137.stdout:2/322: truncate d0/de/f24 1408947 0 2024-06-15T01:07:01.559 INFO:tasks.workunit.client.0.smithi137.stdout:9/227: mknod df/d13/c3d 0 2024-06-15T01:07:01.561 INFO:tasks.workunit.client.0.smithi137.stdout:9/228: write df/d2a/f34 [688863,110252] 0 2024-06-15T01:07:01.565 INFO:tasks.workunit.client.0.smithi137.stdout:3/405: unlink d0/d2/c6 0 2024-06-15T01:07:01.567 INFO:tasks.workunit.client.0.smithi137.stdout:2/323: dwrite d0/d42/d4d/d56/f4f [0,4194304] 0 2024-06-15T01:07:01.570 INFO:tasks.workunit.client.0.smithi137.stdout:1/408: write d0/d10/f6f [180390,45401] 0 2024-06-15T01:07:01.572 INFO:tasks.workunit.client.0.smithi137.stdout:1/409: fsync d0/d2d/f62 0 2024-06-15T01:07:01.574 INFO:tasks.workunit.client.0.smithi137.stdout:3/406: symlink d0/d13/d37/d83/l8a 0 2024-06-15T01:07:01.574 INFO:tasks.workunit.client.0.smithi137.stdout:1/410: write d0/d3/f22 [675535,75028] 0 2024-06-15T01:07:01.574 INFO:tasks.workunit.client.0.smithi137.stdout:2/324: dread d0/d42/d4d/d56/f4f [0,4194304] 0 2024-06-15T01:07:01.580 INFO:tasks.workunit.client.0.smithi137.stdout:2/325: dread d0/dc/d22/f39 [0,4194304] 0 2024-06-15T01:07:01.584 INFO:tasks.workunit.client.0.smithi137.stdout:1/411: dwrite d0/d2d/f62 [0,4194304] 0 2024-06-15T01:07:01.584 INFO:tasks.workunit.client.0.smithi137.stdout:9/229: getdents df/d12/d15/d16 0 2024-06-15T01:07:01.584 INFO:tasks.workunit.client.0.smithi137.stdout:2/326: read d0/d42/d4d/d56/f4f [1730791,46920] 0 2024-06-15T01:07:01.585 INFO:tasks.workunit.client.0.smithi137.stdout:9/230: readlink df/l11 0 2024-06-15T01:07:01.586 INFO:tasks.workunit.client.0.smithi137.stdout:9/231: write f2 [3742157,97505] 0 2024-06-15T01:07:01.591 INFO:tasks.workunit.client.0.smithi137.stdout:9/232: unlink df/l11 0 2024-06-15T01:07:01.592 INFO:tasks.workunit.client.0.smithi137.stdout:9/233: read - df/d13/f30 zero size 2024-06-15T01:07:01.593 INFO:tasks.workunit.client.0.smithi137.stdout:1/412: creat d0/d1f/d76/da1/fa2 x:0 0 0 2024-06-15T01:07:01.593 INFO:tasks.workunit.client.0.smithi137.stdout:9/234: truncate df/d12/f17 13258695 0 2024-06-15T01:07:01.595 INFO:tasks.workunit.client.0.smithi137.stdout:9/235: write df/d12/f17 [3402092,72964] 0 2024-06-15T01:07:01.595 INFO:tasks.workunit.client.0.smithi137.stdout:1/413: rmdir d0/d3/d1b 39 2024-06-15T01:07:01.598 INFO:tasks.workunit.client.0.smithi137.stdout:3/407: write d0/d2/d22/d24/f47 [2871630,106316] 0 2024-06-15T01:07:01.600 INFO:tasks.workunit.client.0.smithi137.stdout:3/408: fsync d0/f1a 0 2024-06-15T01:07:01.602 INFO:tasks.workunit.client.0.smithi137.stdout:3/409: rmdir d0/d2/d22/d63 39 2024-06-15T01:07:01.609 INFO:tasks.workunit.client.0.smithi137.stdout:3/410: dwrite d0/d2/d23/f87 [0,4194304] 0 2024-06-15T01:07:01.647 INFO:tasks.workunit.client.1.smithi142.stdout:9/409: sync 2024-06-15T01:07:01.648 INFO:tasks.workunit.client.1.smithi142.stdout:9/410: write d6/dc/dd/f4f [1607660,51502] 0 2024-06-15T01:07:01.655 INFO:tasks.workunit.client.1.smithi142.stdout:9/411: dread d6/f9 [0,4194304] 0 2024-06-15T01:07:01.696 INFO:tasks.workunit.client.0.smithi137.stdout:6/240: sync 2024-06-15T01:07:01.700 INFO:tasks.workunit.client.0.smithi137.stdout:1/414: truncate d0/d10/f23 2287240 0 2024-06-15T01:07:01.703 INFO:tasks.workunit.client.0.smithi137.stdout:2/327: dread d0/f5 [4194304,4194304] 0 2024-06-15T01:07:01.704 INFO:tasks.workunit.client.1.smithi142.stdout:0/435: mkdir d3/d39/d3e/d81/d1b/d38/da3 0 2024-06-15T01:07:01.705 INFO:tasks.workunit.client.1.smithi142.stdout:0/436: chown d3/d39/d3e/d81/d44/d7c/d22/f36 1204 1 2024-06-15T01:07:01.706 INFO:tasks.workunit.client.1.smithi142.stdout:5/431: mkdir d2/d18/d2e/d84 0 2024-06-15T01:07:01.708 INFO:tasks.workunit.client.1.smithi142.stdout:4/541: creat d5/d9/d90/fc4 x:0 0 0 2024-06-15T01:07:01.708 INFO:tasks.workunit.client.1.smithi142.stdout:4/542: chown d5/d9/d18 185522 1 2024-06-15T01:07:01.711 INFO:tasks.workunit.client.1.smithi142.stdout:7/418: mknod d2/d8/dd/d14/d2c/d37/d6b/c86 0 2024-06-15T01:07:01.713 INFO:tasks.workunit.client.1.smithi142.stdout:8/402: mkdir d0/d17/d40/d86 0 2024-06-15T01:07:01.719 INFO:tasks.workunit.client.0.smithi137.stdout:6/241: dwrite d0/f39 [0,4194304] 0 2024-06-15T01:07:01.720 INFO:tasks.workunit.client.1.smithi142.stdout:1/477: mknod d6/d46/d61/caf 0 2024-06-15T01:07:01.721 INFO:tasks.workunit.client.1.smithi142.stdout:7/419: dwrite d2/d8/da/d33/d25/d27/f7b [0,4194304] 0 2024-06-15T01:07:01.722 INFO:tasks.workunit.client.1.smithi142.stdout:1/478: truncate d6/d46/d61/f76 422344 0 2024-06-15T01:07:01.723 INFO:tasks.workunit.client.0.smithi137.stdout:3/411: dwrite d0/d2/d22/d24/d28/d2e/f3b [0,4194304] 0 2024-06-15T01:07:01.723 INFO:tasks.workunit.client.0.smithi137.stdout:1/415: dwrite d0/d1f/d60/f74 [0,4194304] 0 2024-06-15T01:07:01.725 INFO:tasks.workunit.client.0.smithi137.stdout:3/412: write d0/d2/d22/d24/f89 [407814,109411] 0 2024-06-15T01:07:01.725 INFO:tasks.workunit.client.1.smithi142.stdout:2/387: symlink d8/d19/d23/d25/d1a/d57/l7e 0 2024-06-15T01:07:01.726 INFO:tasks.workunit.client.1.smithi142.stdout:6/425: write d1a/f46 [1680961,102270] 0 2024-06-15T01:07:01.733 INFO:tasks.workunit.client.0.smithi137.stdout:1/416: rename d0/d1f/d60/d99/d9b to d0/d1f/d60/d99/da3 0 2024-06-15T01:07:01.733 INFO:tasks.workunit.client.1.smithi142.stdout:0/437: mknod d3/d39/d3e/d81/ca4 0 2024-06-15T01:07:01.734 INFO:tasks.workunit.client.0.smithi137.stdout:6/242: creat d0/d5/d9/d29/f4d x:0 0 0 2024-06-15T01:07:01.734 INFO:tasks.workunit.client.0.smithi137.stdout:3/413: mknod d0/d2f/c8b 0 2024-06-15T01:07:01.735 INFO:tasks.workunit.client.1.smithi142.stdout:2/388: dread d8/d19/d23/d25/d1a/d57/f2b [0,4194304] 0 2024-06-15T01:07:01.735 INFO:tasks.workunit.client.0.smithi137.stdout:2/328: dwrite d0/de/f1a [0,4194304] 0 2024-06-15T01:07:01.735 INFO:tasks.workunit.client.1.smithi142.stdout:0/438: dread - d3/d39/d3e/d81/f7e zero size 2024-06-15T01:07:01.736 INFO:tasks.workunit.client.1.smithi142.stdout:3/513: sync 2024-06-15T01:07:01.736 INFO:tasks.workunit.client.1.smithi142.stdout:2/389: write d8/df/d10/f3b [4461496,40291] 0 2024-06-15T01:07:01.742 INFO:tasks.workunit.client.1.smithi142.stdout:2/390: dread d8/df/f59 [0,4194304] 0 2024-06-15T01:07:01.743 INFO:tasks.workunit.client.0.smithi137.stdout:3/414: readlink d0/d2/d22/d24/l78 0 2024-06-15T01:07:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:1/417: dwrite d0/d3/d14/d34/f70 [0,4194304] 0 2024-06-15T01:07:01.750 INFO:tasks.workunit.client.1.smithi142.stdout:5/432: mkdir d2/de/d85 0 2024-06-15T01:07:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:3/415: readlink d0/d2/d1e/l3e 0 2024-06-15T01:07:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:6/243: write d0/d1a/f35 [66842,94162] 0 2024-06-15T01:07:01.751 INFO:tasks.workunit.client.1.smithi142.stdout:5/433: dread - d2/de/d27/f77 zero size 2024-06-15T01:07:01.751 INFO:tasks.workunit.client.1.smithi142.stdout:3/514: mkdir d0/d1b/d9e 0 2024-06-15T01:07:01.752 INFO:tasks.workunit.client.1.smithi142.stdout:5/434: fsync d2/d18/f55 0 2024-06-15T01:07:01.754 INFO:tasks.workunit.client.1.smithi142.stdout:5/435: fsync d2/de/d19/d5f/d68/d76/f7c 0 2024-06-15T01:07:01.756 INFO:tasks.workunit.client.0.smithi137.stdout:1/418: fdatasync d0/d3/d1b/d4c/d59/f63 0 2024-06-15T01:07:01.757 INFO:tasks.workunit.client.1.smithi142.stdout:5/436: write d2/de/d19/f51 [1273182,104972] 0 2024-06-15T01:07:01.757 INFO:tasks.workunit.client.0.smithi137.stdout:6/244: readlink d0/d47/d1d/d2e/l31 0 2024-06-15T01:07:01.758 INFO:tasks.workunit.client.1.smithi142.stdout:8/403: creat d0/f87 x:0 0 0 2024-06-15T01:07:01.759 INFO:tasks.workunit.client.0.smithi137.stdout:3/416: symlink d0/d2/d8/d67/d7e/d6c/d7b/l8c 0 2024-06-15T01:07:01.760 INFO:tasks.workunit.client.1.smithi142.stdout:8/404: stat d0/d25/d5a/f4d 0 2024-06-15T01:07:01.761 INFO:tasks.workunit.client.0.smithi137.stdout:2/329: dwrite d0/d42/d4d/f59 [4194304,4194304] 0 2024-06-15T01:07:01.762 INFO:tasks.workunit.client.1.smithi142.stdout:7/420: creat d2/d8/da/d33/f87 x:0 0 0 2024-06-15T01:07:01.762 INFO:tasks.workunit.client.0.smithi137.stdout:6/245: write d0/d47/d1d/d2e/f36 [2396392,101900] 0 2024-06-15T01:07:01.763 INFO:tasks.workunit.client.0.smithi137.stdout:6/246: rename d0/d5 to d0/d5/d46/d4e 22 2024-06-15T01:07:01.764 INFO:tasks.workunit.client.0.smithi137.stdout:1/419: rmdir d0/d10/d41/d7b 39 2024-06-15T01:07:01.764 INFO:tasks.workunit.client.1.smithi142.stdout:3/515: dwrite d0/d3/f90 [0,4194304] 0 2024-06-15T01:07:01.765 INFO:tasks.workunit.client.1.smithi142.stdout:9/412: rename d6/d31/c93 to d6/d34/ca4 0 2024-06-15T01:07:01.769 INFO:tasks.workunit.client.0.smithi137.stdout:3/417: mknod d0/d2/d8/d67/d7e/c8d 0 2024-06-15T01:07:01.769 INFO:tasks.workunit.client.1.smithi142.stdout:5/437: mkdir d2/de/d27/d86 0 2024-06-15T01:07:01.772 INFO:tasks.workunit.client.1.smithi142.stdout:4/543: rename d5/d9/d41/f36 to d5/d9/da/d92/fc5 0 2024-06-15T01:07:01.774 INFO:tasks.workunit.client.0.smithi137.stdout:1/420: creat d0/d2d/d3e/fa4 x:0 0 0 2024-06-15T01:07:01.775 INFO:tasks.workunit.client.1.smithi142.stdout:7/421: rename d2/d8/dd/d14/d2c/d37/d6b/d6e to d2/d8/dd/d14/d2c/d37/d6b/d88 0 2024-06-15T01:07:01.777 INFO:tasks.workunit.client.1.smithi142.stdout:9/413: creat d6/dc/d4e/d65/fa5 x:0 0 0 2024-06-15T01:07:01.780 INFO:tasks.workunit.client.1.smithi142.stdout:9/414: truncate d6/dc/d1b/d25/d58/f63 682337 0 2024-06-15T01:07:01.783 INFO:tasks.workunit.client.1.smithi142.stdout:3/516: creat d0/d3/d15/f9f x:0 0 0 2024-06-15T01:07:01.786 INFO:tasks.workunit.client.1.smithi142.stdout:5/438: symlink d2/d3e/d65/l87 0 2024-06-15T01:07:01.787 INFO:tasks.workunit.client.0.smithi137.stdout:2/330: truncate d0/f5 2706384 0 2024-06-15T01:07:01.788 INFO:tasks.workunit.client.0.smithi137.stdout:3/418: rename d0/d56/c70 to d0/d42/c8e 0 2024-06-15T01:07:01.790 INFO:tasks.workunit.client.0.smithi137.stdout:6/247: creat d0/d5/d9/f4f x:0 0 0 2024-06-15T01:07:01.792 INFO:tasks.workunit.client.1.smithi142.stdout:1/479: rename d6/d12/d30 to d6/d12/da0/db0 0 2024-06-15T01:07:01.793 INFO:tasks.workunit.client.0.smithi137.stdout:1/421: rename d0/d10/f2e to d0/d3/d14/fa5 0 2024-06-15T01:07:01.794 INFO:tasks.workunit.client.1.smithi142.stdout:4/544: creat d5/d1c/d2a/d30/d46/fc6 x:0 0 0 2024-06-15T01:07:01.796 INFO:tasks.workunit.client.0.smithi137.stdout:3/419: dwrite d0/d2/f15 [4194304,4194304] 0 2024-06-15T01:07:01.797 INFO:tasks.workunit.client.1.smithi142.stdout:8/405: creat d0/d25/d5a/f88 x:0 0 0 2024-06-15T01:07:01.798 INFO:tasks.workunit.client.1.smithi142.stdout:7/422: dread d2/d8/f2d [0,4194304] 0 2024-06-15T01:07:01.799 INFO:tasks.workunit.client.0.smithi137.stdout:6/248: read d0/f10 [849676,21646] 0 2024-06-15T01:07:01.800 INFO:tasks.workunit.client.1.smithi142.stdout:7/423: write d2/d8/dd/d14/d2c/d37/d6b/d80/f36 [3408528,96380] 0 2024-06-15T01:07:01.803 INFO:tasks.workunit.client.1.smithi142.stdout:7/424: read d2/d6/d22/f6c [3134243,9579] 0 2024-06-15T01:07:01.805 INFO:tasks.workunit.client.1.smithi142.stdout:2/391: rename d8/df/d10/c12 to d8/d19/d23/d25/d22/d43/c7f 0 2024-06-15T01:07:01.806 INFO:tasks.workunit.client.1.smithi142.stdout:1/480: symlink d6/d12/d80/lb1 0 2024-06-15T01:07:01.807 INFO:tasks.workunit.client.0.smithi137.stdout:6/249: symlink d0/d5/l50 0 2024-06-15T01:07:01.808 INFO:tasks.workunit.client.0.smithi137.stdout:6/250: readlink d0/d5/d9/d29/l2b 0 2024-06-15T01:07:01.809 INFO:tasks.workunit.client.0.smithi137.stdout:2/331: write d0/dc/d22/f4b [234176,66753] 0 2024-06-15T01:07:01.809 INFO:tasks.workunit.client.0.smithi137.stdout:6/251: readlink d0/ld 0 2024-06-15T01:07:01.811 INFO:tasks.workunit.client.0.smithi137.stdout:3/420: rename d0/d2/d8/d67/d7e/d6c to d0/d56/d8f 0 2024-06-15T01:07:01.813 INFO:tasks.workunit.client.0.smithi137.stdout:1/422: truncate d0/d1f/d60/f91 224693 0 2024-06-15T01:07:01.815 INFO:tasks.workunit.client.1.smithi142.stdout:9/415: rename c1 to d6/dc/dd/ca6 0 2024-06-15T01:07:01.815 INFO:tasks.workunit.client.0.smithi137.stdout:6/252: rename d0/d5/f8 to d0/d47/d1d/d2e/f51 0 2024-06-15T01:07:01.816 INFO:tasks.workunit.client.1.smithi142.stdout:6/426: dwrite d1a/d39/d3a/d51/f63 [0,4194304] 0 2024-06-15T01:07:01.816 INFO:tasks.workunit.client.1.smithi142.stdout:2/392: symlink d8/d19/d23/d25/d22/d43/l80 0 2024-06-15T01:07:01.817 INFO:tasks.workunit.client.0.smithi137.stdout:3/421: mkdir d0/d56/d8f/d90 0 2024-06-15T01:07:01.817 INFO:tasks.workunit.client.1.smithi142.stdout:0/439: truncate d3/d39/d3e/d81/f87 1012291 0 2024-06-15T01:07:01.819 INFO:tasks.workunit.client.1.smithi142.stdout:0/440: write d3/d39/d3e/d81/d44/f4b [3021582,44321] 0 2024-06-15T01:07:01.824 INFO:tasks.workunit.client.0.smithi137.stdout:1/423: dwrite d0/d1f/d76/f8f [0,4194304] 0 2024-06-15T01:07:01.827 INFO:tasks.workunit.client.1.smithi142.stdout:0/441: dwrite d3/d39/d3e/d81/d1b/d38/f8e [0,4194304] 0 2024-06-15T01:07:01.828 INFO:tasks.workunit.client.0.smithi137.stdout:1/424: creat d0/d3/d1b/d4c/d59/fa6 x:0 0 0 2024-06-15T01:07:01.829 INFO:tasks.workunit.client.1.smithi142.stdout:7/425: symlink d2/d8/dd/l89 0 2024-06-15T01:07:01.832 INFO:tasks.workunit.client.0.smithi137.stdout:6/253: dwrite d0/d5/d9/d29/f4d [0,4194304] 0 2024-06-15T01:07:01.841 INFO:tasks.workunit.client.1.smithi142.stdout:1/481: mkdir d6/d12/d80/d9b/db2 0 2024-06-15T01:07:01.842 INFO:tasks.workunit.client.0.smithi137.stdout:1/425: creat d0/d10/d41/fa7 x:0 0 0 2024-06-15T01:07:01.843 INFO:tasks.workunit.client.1.smithi142.stdout:1/482: read d6/d3a/f88 [1285296,41899] 0 2024-06-15T01:07:01.845 INFO:tasks.workunit.client.1.smithi142.stdout:9/416: dwrite d6/f9 [4194304,4194304] 0 2024-06-15T01:07:01.849 INFO:tasks.workunit.client.1.smithi142.stdout:7/426: chown d2/d8/da/d33/c2a 30069453 1 2024-06-15T01:07:01.851 INFO:tasks.workunit.client.1.smithi142.stdout:8/406: write d0/d25/d5a/f85 [239027,89901] 0 2024-06-15T01:07:01.852 INFO:tasks.workunit.client.0.smithi137.stdout:3/422: dread d0/d2/d23/d68/f3c [0,4194304] 0 2024-06-15T01:07:01.852 INFO:tasks.workunit.client.0.smithi137.stdout:3/423: dread - d0/d56/f6a zero size 2024-06-15T01:07:01.858 INFO:tasks.workunit.client.1.smithi142.stdout:5/439: rename d2/de/l60 to d2/l88 0 2024-06-15T01:07:01.860 INFO:tasks.workunit.client.1.smithi142.stdout:4/545: dwrite d5/d1c/d2a/d30/d46/d50/d51/d5b/d59/f7a [0,4194304] 0 2024-06-15T01:07:01.862 INFO:tasks.workunit.client.1.smithi142.stdout:7/427: write d2/d6/d22/f6c [1756460,40541] 0 2024-06-15T01:07:01.863 INFO:tasks.workunit.client.0.smithi137.stdout:3/424: dwrite d0/d13/d37/f43 [4194304,4194304] 0 2024-06-15T01:07:01.863 INFO:tasks.workunit.client.1.smithi142.stdout:1/483: creat d6/d12/d80/dae/fb3 x:0 0 0 2024-06-15T01:07:01.865 INFO:tasks.workunit.client.1.smithi142.stdout:5/440: truncate d2/d18/f37 4569571 0 2024-06-15T01:07:01.866 INFO:tasks.workunit.client.1.smithi142.stdout:1/484: chown d6/d12/d80/d9b/db2 0 1 2024-06-15T01:07:01.869 INFO:tasks.workunit.client.1.smithi142.stdout:2/393: dwrite d8/d19/d23/d25/f6a [0,4194304] 0 2024-06-15T01:07:01.875 INFO:tasks.workunit.client.1.smithi142.stdout:6/427: rename d1a/d39/d3a/l75 to d1a/d39/d42/d4c/l9c 0 2024-06-15T01:07:01.875 INFO:tasks.workunit.client.1.smithi142.stdout:7/428: write d2/d8/dd/d14/d2c/f62 [736157,42450] 0 2024-06-15T01:07:01.877 INFO:tasks.workunit.client.1.smithi142.stdout:6/428: fdatasync d1a/d83/f7c 0 2024-06-15T01:07:01.879 INFO:tasks.workunit.client.0.smithi137.stdout:1/426: dread d0/f2 [0,4194304] 0 2024-06-15T01:07:01.882 INFO:tasks.workunit.client.1.smithi142.stdout:8/407: dread d0/d25/f35 [0,4194304] 0 2024-06-15T01:07:01.883 INFO:tasks.workunit.client.1.smithi142.stdout:4/546: rename d5/d1c/d2a to d5/d1c/d6f/dc7 0 2024-06-15T01:07:01.883 INFO:tasks.workunit.client.1.smithi142.stdout:8/408: stat d0/c7b 0 2024-06-15T01:07:01.886 INFO:tasks.workunit.client.1.smithi142.stdout:8/409: read d0/d17/f6b [670435,14472] 0 2024-06-15T01:07:01.886 INFO:tasks.workunit.client.1.smithi142.stdout:4/547: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d [282395,10234] 0 2024-06-15T01:07:01.890 INFO:tasks.workunit.client.1.smithi142.stdout:2/394: creat d8/d19/d23/d25/d22/d43/d52/d7a/f81 x:0 0 0 2024-06-15T01:07:01.890 INFO:tasks.workunit.client.1.smithi142.stdout:7/429: mkdir d2/d8/dd/d8a 0 2024-06-15T01:07:01.892 INFO:tasks.workunit.client.1.smithi142.stdout:0/442: dread d3/d39/d3e/d81/f87 [0,4194304] 0 2024-06-15T01:07:01.894 INFO:tasks.workunit.client.1.smithi142.stdout:2/395: write d8/d19/d23/d25/f74 [438878,22702] 0 2024-06-15T01:07:01.898 INFO:tasks.workunit.client.1.smithi142.stdout:0/443: chown d3/d39/d3e/d81/d4c/l51 5029090 1 2024-06-15T01:07:01.898 INFO:tasks.workunit.client.1.smithi142.stdout:8/410: dread d0/d17/d40/f65 [0,4194304] 0 2024-06-15T01:07:01.900 INFO:tasks.workunit.client.1.smithi142.stdout:7/430: symlink d2/d8/dd/d14/d2c/d37/d6b/d80/d85/l8b 0 2024-06-15T01:07:01.900 INFO:tasks.workunit.client.1.smithi142.stdout:4/548: dwrite d5/d1c/d6f/dc7/d30/d46/d50/f5e [4194304,4194304] 0 2024-06-15T01:07:01.901 INFO:tasks.workunit.client.1.smithi142.stdout:2/396: symlink d8/d19/d23/d25/d76/l82 0 2024-06-15T01:07:01.903 INFO:tasks.workunit.client.1.smithi142.stdout:0/444: creat d3/d39/d3e/fa5 x:0 0 0 2024-06-15T01:07:01.905 INFO:tasks.workunit.client.1.smithi142.stdout:7/431: creat d2/d8/dd/d14/d2c/d37/d6b/f8c x:0 0 0 2024-06-15T01:07:01.909 INFO:tasks.workunit.client.1.smithi142.stdout:5/441: dread d2/f9 [0,4194304] 0 2024-06-15T01:07:01.910 INFO:tasks.workunit.client.1.smithi142.stdout:0/445: fdatasync d3/d39/d3e/d81/d44/d7c/d22/d59/f71 0 2024-06-15T01:07:01.911 INFO:tasks.workunit.client.1.smithi142.stdout:4/549: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/fc8 x:0 0 0 2024-06-15T01:07:01.914 INFO:tasks.workunit.client.1.smithi142.stdout:7/432: symlink d2/d8/da/d33/d25/d27/l8d 0 2024-06-15T01:07:01.916 INFO:tasks.workunit.client.1.smithi142.stdout:0/446: symlink d3/d39/d9a/la6 0 2024-06-15T01:07:01.920 INFO:tasks.workunit.client.1.smithi142.stdout:1/485: dwrite d6/d12/d5b/f8a [0,4194304] 0 2024-06-15T01:07:01.921 INFO:tasks.workunit.client.1.smithi142.stdout:5/442: mknod d2/de/d19/d5f/d68/c89 0 2024-06-15T01:07:01.921 INFO:tasks.workunit.client.1.smithi142.stdout:8/411: dwrite d0/d25/f35 [0,4194304] 0 2024-06-15T01:07:01.923 INFO:tasks.workunit.client.1.smithi142.stdout:6/429: dwrite d1a/d39/d42/d4c/d58/f6a [0,4194304] 0 2024-06-15T01:07:01.924 INFO:tasks.workunit.client.1.smithi142.stdout:5/443: readlink d2/d3e/d65/l6c 0 2024-06-15T01:07:01.925 INFO:tasks.workunit.client.1.smithi142.stdout:4/550: rename d5/d9/d23/lb2 to d5/d1c/d6f/lc9 0 2024-06-15T01:07:01.932 INFO:tasks.workunit.client.1.smithi142.stdout:0/447: chown d3/d39/d3e/d81/f73 1242 1 2024-06-15T01:07:01.934 INFO:tasks.workunit.client.1.smithi142.stdout:2/397: dwrite d8/d19/d23/d25/d1a/f1d [0,4194304] 0 2024-06-15T01:07:01.935 INFO:tasks.workunit.client.1.smithi142.stdout:1/486: mknod d6/d12/da0/cb4 0 2024-06-15T01:07:01.935 INFO:tasks.workunit.client.1.smithi142.stdout:4/551: fdatasync d5/d9/d90/fc4 0 2024-06-15T01:07:01.938 INFO:tasks.workunit.client.1.smithi142.stdout:5/444: stat d2/d18/l62 0 2024-06-15T01:07:01.939 INFO:tasks.workunit.client.1.smithi142.stdout:2/398: fdatasync d8/d19/d23/d25/d1a/d57/f69 0 2024-06-15T01:07:01.940 INFO:tasks.workunit.client.1.smithi142.stdout:5/445: dread - d2/de/d27/f77 zero size 2024-06-15T01:07:01.940 INFO:tasks.workunit.client.1.smithi142.stdout:0/448: unlink d3/d39/d3e/d81/d4c/d61/l90 0 2024-06-15T01:07:01.944 INFO:tasks.workunit.client.1.smithi142.stdout:7/433: link d2/d8/f17 d2/d8/dd/d14/d2c/d37/d6b/d80/d85/f8e 0 2024-06-15T01:07:01.944 INFO:tasks.workunit.client.1.smithi142.stdout:6/430: creat d1a/d2d/d80/d8e/f9d x:0 0 0 2024-06-15T01:07:01.947 INFO:tasks.workunit.client.1.smithi142.stdout:5/446: mknod d2/d18/d2e/c8a 0 2024-06-15T01:07:01.948 INFO:tasks.workunit.client.1.smithi142.stdout:1/487: dread d6/d3a/f99 [0,4194304] 0 2024-06-15T01:07:01.949 INFO:tasks.workunit.client.1.smithi142.stdout:0/449: creat d3/d39/d3e/d81/d44/fa7 x:0 0 0 2024-06-15T01:07:01.949 INFO:tasks.workunit.client.0.smithi137.stdout:4/370: sync 2024-06-15T01:07:01.950 INFO:tasks.workunit.client.1.smithi142.stdout:4/552: dread d5/d1c/d6f/dc7/f3d [0,4194304] 0 2024-06-15T01:07:01.950 INFO:tasks.workunit.client.0.smithi137.stdout:6/254: write d0/d47/f1e [1416507,2909] 0 2024-06-15T01:07:01.951 INFO:tasks.workunit.client.1.smithi142.stdout:7/434: creat d2/d6/f8f x:0 0 0 2024-06-15T01:07:01.952 INFO:tasks.workunit.client.0.smithi137.stdout:3/425: chown d0/fd 20628995 1 2024-06-15T01:07:01.952 INFO:tasks.workunit.client.0.smithi137.stdout:1/427: mknod d0/d3/d1b/d4c/ca8 0 2024-06-15T01:07:01.954 INFO:tasks.workunit.client.0.smithi137.stdout:0/344: sync 2024-06-15T01:07:01.954 INFO:tasks.workunit.client.1.smithi142.stdout:8/412: dwrite d0/d17/f27 [0,4194304] 0 2024-06-15T01:07:01.956 INFO:tasks.workunit.client.0.smithi137.stdout:4/371: dwrite dd/f18 [0,4194304] 0 2024-06-15T01:07:01.958 INFO:tasks.workunit.client.1.smithi142.stdout:1/488: write d6/d12/d3c/d63/f85 [2113997,69214] 0 2024-06-15T01:07:01.958 INFO:tasks.workunit.client.1.smithi142.stdout:8/413: fsync d0/f43 0 2024-06-15T01:07:01.958 INFO:tasks.workunit.client.1.smithi142.stdout:4/553: read d5/d9/d41/f4c [541218,39184] 0 2024-06-15T01:07:01.961 INFO:tasks.workunit.client.1.smithi142.stdout:2/399: dwrite d8/d19/d23/d25/d1a/d63/f77 [0,4194304] 0 2024-06-15T01:07:01.961 INFO:tasks.workunit.client.1.smithi142.stdout:8/414: write d0/d25/d5a/f4d [788953,38350] 0 2024-06-15T01:07:01.963 INFO:tasks.workunit.client.1.smithi142.stdout:8/415: chown d0/d25/d53/f79 56978394 1 2024-06-15T01:07:01.965 INFO:tasks.workunit.client.1.smithi142.stdout:8/416: write d0/f15 [5249884,126065] 0 2024-06-15T01:07:01.968 INFO:tasks.workunit.client.1.smithi142.stdout:1/489: dwrite d6/d12/da0/db0/f73 [0,4194304] 0 2024-06-15T01:07:01.969 INFO:tasks.workunit.client.1.smithi142.stdout:1/490: fdatasync d6/f9 0 2024-06-15T01:07:01.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: pgmap v16: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 91 MiB/s wr, 167 op/s 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:01.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:01 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:07:01.977 INFO:tasks.workunit.client.1.smithi142.stdout:5/447: creat d2/f8b x:0 0 0 2024-06-15T01:07:01.978 INFO:tasks.workunit.client.1.smithi142.stdout:5/448: write d2/d18/f69 [2800705,25495] 0 2024-06-15T01:07:01.978 INFO:tasks.workunit.client.1.smithi142.stdout:0/450: unlink d3/d39/d3e/d81/d44/c83 0 2024-06-15T01:07:01.980 INFO:tasks.workunit.client.1.smithi142.stdout:0/451: chown d3/l56 667595827 1 2024-06-15T01:07:01.982 INFO:tasks.workunit.client.1.smithi142.stdout:6/431: link d1a/d2d/d2e/l36 d1a/d2d/d80/l9e 0 2024-06-15T01:07:01.983 INFO:tasks.workunit.client.0.smithi137.stdout:3/426: dwrite d0/d2/d8/f41 [0,4194304] 0 2024-06-15T01:07:01.986 INFO:tasks.workunit.client.1.smithi142.stdout:4/554: chown d5/d9/c84 6715 1 2024-06-15T01:07:01.988 INFO:tasks.workunit.client.1.smithi142.stdout:2/400: creat d8/d19/d33/f83 x:0 0 0 2024-06-15T01:07:01.988 INFO:tasks.workunit.client.0.smithi137.stdout:0/345: symlink d1/d2/dc/d10/d53/l78 0 2024-06-15T01:07:01.990 INFO:tasks.workunit.client.0.smithi137.stdout:4/372: rename dd/d15/d48/d72/f7a to dd/d15/d30/f8c 0 2024-06-15T01:07:01.995 INFO:tasks.workunit.client.0.smithi137.stdout:4/373: unlink l4 0 2024-06-15T01:07:01.998 INFO:tasks.workunit.client.1.smithi142.stdout:5/449: mkdir d2/de/d19/d5f/d68/d76/d8c 0 2024-06-15T01:07:02.002 INFO:tasks.workunit.client.0.smithi137.stdout:3/427: dwrite d0/d2/d23/f44 [0,4194304] 0 2024-06-15T01:07:02.004 INFO:tasks.workunit.client.0.smithi137.stdout:4/374: rename dd/d10 to dd/d15/d36/d45/d55/d8d 0 2024-06-15T01:07:02.007 INFO:tasks.workunit.client.1.smithi142.stdout:7/435: symlink d2/d8/da/d33/d25/d79/l90 0 2024-06-15T01:07:02.008 INFO:tasks.workunit.client.0.smithi137.stdout:0/346: dread d1/d2/dc/d10/d14/d15/f30 [0,4194304] 0 2024-06-15T01:07:02.008 INFO:tasks.workunit.client.1.smithi142.stdout:4/555: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/c11 739 1 2024-06-15T01:07:02.008 INFO:tasks.workunit.client.1.smithi142.stdout:4/556: fdatasync f2 0 2024-06-15T01:07:02.009 INFO:tasks.workunit.client.1.smithi142.stdout:2/401: mknod d8/d19/d23/d25/d22/d43/d51/c84 0 2024-06-15T01:07:02.009 INFO:tasks.workunit.client.1.smithi142.stdout:4/557: chown d5/d9/d23/d32/cb8 1108 1 2024-06-15T01:07:02.010 INFO:tasks.workunit.client.1.smithi142.stdout:8/417: symlink d0/d17/l89 0 2024-06-15T01:07:02.010 INFO:tasks.workunit.client.1.smithi142.stdout:1/491: mkdir d6/d12/db5 0 2024-06-15T01:07:02.011 INFO:tasks.workunit.client.1.smithi142.stdout:2/402: chown d8/d19/d23/d25/c73 5 1 2024-06-15T01:07:02.011 INFO:tasks.workunit.client.1.smithi142.stdout:4/558: chown d5/d9/da/f8b 30199214 1 2024-06-15T01:07:02.013 INFO:tasks.workunit.client.1.smithi142.stdout:1/492: chown d6/d12/d1c/d2c/d47/f5f 467351 1 2024-06-15T01:07:02.014 INFO:tasks.workunit.client.0.smithi137.stdout:4/375: link dd/d15/d21/f29 dd/f8e 0 2024-06-15T01:07:02.015 INFO:tasks.workunit.client.1.smithi142.stdout:7/436: getdents d2/d8/dd/d8a 0 2024-06-15T01:07:02.015 INFO:tasks.workunit.client.0.smithi137.stdout:3/428: read d0/d13/d37/f43 [1010501,130731] 0 2024-06-15T01:07:02.015 INFO:tasks.workunit.client.1.smithi142.stdout:8/418: read d0/f14 [593898,94641] 0 2024-06-15T01:07:02.017 INFO:tasks.workunit.client.1.smithi142.stdout:4/559: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/dca 0 2024-06-15T01:07:02.017 INFO:tasks.workunit.client.0.smithi137.stdout:3/429: write d0/d2/d22/d24/d5e/f85 [641728,53034] 0 2024-06-15T01:07:02.017 INFO:tasks.workunit.client.0.smithi137.stdout:4/376: mkdir dd/d15/d36/d45/d8f 0 2024-06-15T01:07:02.019 INFO:tasks.workunit.client.0.smithi137.stdout:6/255: write d0/d5/d9/fe [212447,14780] 0 2024-06-15T01:07:02.019 INFO:tasks.workunit.client.1.smithi142.stdout:1/493: symlink d6/d12/d80/d9b/db2/lb6 0 2024-06-15T01:07:02.019 INFO:tasks.workunit.client.0.smithi137.stdout:0/347: creat d1/f79 x:0 0 0 2024-06-15T01:07:02.020 INFO:tasks.workunit.client.1.smithi142.stdout:2/403: link d8/d19/f70 d8/df/f85 0 2024-06-15T01:07:02.022 INFO:tasks.workunit.client.1.smithi142.stdout:7/437: creat d2/f91 x:0 0 0 2024-06-15T01:07:02.023 INFO:tasks.workunit.client.1.smithi142.stdout:8/419: dread d0/d17/d40/f65 [0,4194304] 0 2024-06-15T01:07:02.024 INFO:tasks.workunit.client.1.smithi142.stdout:4/560: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d59 to d5/d9/d90/daf/dcb 0 2024-06-15T01:07:02.027 INFO:tasks.workunit.client.1.smithi142.stdout:4/561: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f37 [1682175,53650] 0 2024-06-15T01:07:02.027 INFO:tasks.workunit.client.1.smithi142.stdout:8/420: dread d0/d17/d40/d6e/f73 [0,4194304] 0 2024-06-15T01:07:02.029 INFO:tasks.workunit.client.1.smithi142.stdout:5/450: truncate d2/de/f5a 1955612 0 2024-06-15T01:07:02.029 INFO:tasks.workunit.client.1.smithi142.stdout:4/562: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d [973342,28869] 0 2024-06-15T01:07:02.030 INFO:tasks.workunit.client.0.smithi137.stdout:1/428: dwrite d0/d3/d14/f19 [0,4194304] 0 2024-06-15T01:07:02.030 INFO:tasks.workunit.client.1.smithi142.stdout:5/451: readlink d2/de/d19/d5f/d68/l7f 0 2024-06-15T01:07:02.031 INFO:tasks.workunit.client.1.smithi142.stdout:5/452: fsync d2/de/d27/f54 0 2024-06-15T01:07:02.032 INFO:tasks.workunit.client.1.smithi142.stdout:2/404: fsync d8/d19/d23/d25/d1a/f2e 0 2024-06-15T01:07:02.032 INFO:tasks.workunit.client.0.smithi137.stdout:6/256: dread d0/d5/d9/d29/f4d [0,4194304] 0 2024-06-15T01:07:02.033 INFO:tasks.workunit.client.0.smithi137.stdout:3/430: getdents d0/d4c 0 2024-06-15T01:07:02.034 INFO:tasks.workunit.client.0.smithi137.stdout:6/257: stat d0/f3c 0 2024-06-15T01:07:02.035 INFO:tasks.workunit.client.0.smithi137.stdout:3/431: stat d0/d2/d22/d24/d28/d2e/f3b 0 2024-06-15T01:07:02.037 INFO:tasks.workunit.client.0.smithi137.stdout:0/348: mkdir d1/d2/dc/d10/d14/d15/d17/d54/d7a 0 2024-06-15T01:07:02.041 INFO:tasks.workunit.client.1.smithi142.stdout:8/421: mknod d0/d17/d40/d86/c8a 0 2024-06-15T01:07:02.041 INFO:tasks.workunit.client.0.smithi137.stdout:6/258: fdatasync d0/fc 0 2024-06-15T01:07:02.043 INFO:tasks.workunit.client.1.smithi142.stdout:2/405: symlink d8/d19/d23/d25/d22/d43/l86 0 2024-06-15T01:07:02.043 INFO:tasks.workunit.client.1.smithi142.stdout:4/563: dwrite d5/d1c/d6f/dc7/fb5 [0,4194304] 0 2024-06-15T01:07:02.044 INFO:tasks.workunit.client.0.smithi137.stdout:3/432: write d0/d2/d8/f14 [865335,25892] 0 2024-06-15T01:07:02.049 INFO:tasks.workunit.client.1.smithi142.stdout:1/494: getdents d6/d12/d80 0 2024-06-15T01:07:02.049 INFO:tasks.workunit.client.1.smithi142.stdout:5/453: dwrite d2/de/d27/f72 [0,4194304] 0 2024-06-15T01:07:02.052 INFO:tasks.workunit.client.1.smithi142.stdout:8/422: rmdir d0/d17/d40/d86 39 2024-06-15T01:07:02.053 INFO:tasks.workunit.client.0.smithi137.stdout:3/433: rmdir d0/d4c 39 2024-06-15T01:07:02.054 INFO:tasks.workunit.client.0.smithi137.stdout:4/377: dwrite f8 [0,4194304] 0 2024-06-15T01:07:02.058 INFO:tasks.workunit.client.0.smithi137.stdout:1/429: dwrite d0/d3/d14/fa5 [8388608,4194304] 0 2024-06-15T01:07:02.061 INFO:tasks.workunit.client.0.smithi137.stdout:0/349: dwrite d1/d2/dc/d10/d14/d15/f4b [0,4194304] 0 2024-06-15T01:07:02.064 INFO:tasks.workunit.client.0.smithi137.stdout:4/378: chown dd/d15/d36/d45/d55/d8d/d1f 761082740 1 2024-06-15T01:07:02.065 INFO:tasks.workunit.client.1.smithi142.stdout:7/438: dwrite d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:07:02.065 INFO:tasks.workunit.client.1.smithi142.stdout:5/454: mknod d2/d3e/d65/c8d 0 2024-06-15T01:07:02.066 INFO:tasks.workunit.client.0.smithi137.stdout:1/430: mkdir d0/d1f/d60/da9 0 2024-06-15T01:07:02.068 INFO:tasks.workunit.client.0.smithi137.stdout:0/350: symlink d1/d2/dc/d10/l7b 0 2024-06-15T01:07:02.068 INFO:tasks.workunit.client.0.smithi137.stdout:4/379: creat dd/d15/d36/d45/d55/d8d/f90 x:0 0 0 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: pgmap v16: 65 pgs: 65 active+clean; 887 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 91 MiB/s wr, 167 op/s 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:02.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:01 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:07:02.070 INFO:tasks.workunit.client.1.smithi142.stdout:7/439: chown d2/d6/d22/d3c/c4c 560222081 1 2024-06-15T01:07:02.070 INFO:tasks.workunit.client.1.smithi142.stdout:4/564: dread d5/d9/f1f [0,4194304] 0 2024-06-15T01:07:02.071 INFO:tasks.workunit.client.1.smithi142.stdout:4/565: truncate d5/d9/d18/f6e 1785008 0 2024-06-15T01:07:02.072 INFO:tasks.workunit.client.1.smithi142.stdout:8/423: mknod d0/c8b 0 2024-06-15T01:07:02.073 INFO:tasks.workunit.client.1.smithi142.stdout:4/566: chown d5/d1c/d6f/dc7 22086 1 2024-06-15T01:07:02.074 INFO:tasks.workunit.client.1.smithi142.stdout:8/424: chown d0/f4c 459455 1 2024-06-15T01:07:02.074 INFO:tasks.workunit.client.0.smithi137.stdout:4/380: mkdir dd/d15/d30/d49/d91 0 2024-06-15T01:07:02.077 INFO:tasks.workunit.client.0.smithi137.stdout:1/431: write d0/d3/d14/f38 [1152946,66997] 0 2024-06-15T01:07:02.079 INFO:tasks.workunit.client.1.smithi142.stdout:5/455: mknod d2/de/d19/d5f/c8e 0 2024-06-15T01:07:02.079 INFO:tasks.workunit.client.1.smithi142.stdout:5/456: read d2/de/d27/f38 [2124408,32861] 0 2024-06-15T01:07:02.079 INFO:tasks.workunit.client.1.smithi142.stdout:7/440: creat d2/d8/da/d33/d25/d79/f92 x:0 0 0 2024-06-15T01:07:02.082 INFO:tasks.workunit.client.1.smithi142.stdout:1/495: dwrite d6/f10 [4194304,4194304] 0 2024-06-15T01:07:02.083 INFO:tasks.workunit.client.1.smithi142.stdout:7/441: readlink d2/d6/d22/d47/l5e 0 2024-06-15T01:07:02.083 INFO:tasks.workunit.client.1.smithi142.stdout:8/425: creat d0/d25/d5a/f8c x:0 0 0 2024-06-15T01:07:02.085 INFO:tasks.workunit.client.1.smithi142.stdout:1/496: chown d6/d12/d1c/d2c/d47/c49 2879 1 2024-06-15T01:07:02.085 INFO:tasks.workunit.client.0.smithi137.stdout:0/351: dwrite d1/d2/dc/d10/d14/d15/f59 [4194304,4194304] 0 2024-06-15T01:07:02.085 INFO:tasks.workunit.client.0.smithi137.stdout:1/432: write d0/d3/d14/d34/f70 [4497073,99151] 0 2024-06-15T01:07:02.086 INFO:tasks.workunit.client.0.smithi137.stdout:3/434: dwrite d0/d56/f6a [0,4194304] 0 2024-06-15T01:07:02.089 INFO:tasks.workunit.client.0.smithi137.stdout:6/259: dread d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:02.091 INFO:tasks.workunit.client.1.smithi142.stdout:5/457: rename d2/d3e/c47 to d2/d3e/d31/d7e/c8f 0 2024-06-15T01:07:02.091 INFO:tasks.workunit.client.1.smithi142.stdout:1/497: truncate d6/d3a/f43 56457 0 2024-06-15T01:07:02.093 INFO:tasks.workunit.client.0.smithi137.stdout:0/352: symlink d1/d2/dc/d10/d22/l7c 0 2024-06-15T01:07:02.095 INFO:tasks.workunit.client.0.smithi137.stdout:6/260: unlink d0/d5/d9/f1b 0 2024-06-15T01:07:02.099 INFO:tasks.workunit.client.0.smithi137.stdout:4/381: dwrite dd/d15/d36/d45/d55/d8d/d1f/f56 [0,4194304] 0 2024-06-15T01:07:02.099 INFO:tasks.workunit.client.1.smithi142.stdout:7/442: dwrite d2/d6/f8f [0,4194304] 0 2024-06-15T01:07:02.100 INFO:tasks.workunit.client.0.smithi137.stdout:4/382: fsync dd/f14 0 2024-06-15T01:07:02.102 INFO:tasks.workunit.client.0.smithi137.stdout:6/261: chown d0/d47/d1d/d2e/f51 1294918612 1 2024-06-15T01:07:02.102 INFO:tasks.workunit.client.1.smithi142.stdout:1/498: rename d6/d25/f77 to d6/d12/d80/dae/fb7 0 2024-06-15T01:07:02.108 INFO:tasks.workunit.client.0.smithi137.stdout:1/433: dwrite d0/d3/d1b/d4c/d59/d64/f6e [0,4194304] 0 2024-06-15T01:07:02.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/262: read - d0/d47/d1d/f40 zero size 2024-06-15T01:07:02.111 INFO:tasks.workunit.client.0.smithi137.stdout:4/383: creat dd/f92 x:0 0 0 2024-06-15T01:07:02.112 INFO:tasks.workunit.client.0.smithi137.stdout:0/353: creat d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d x:0 0 0 2024-06-15T01:07:02.114 INFO:tasks.workunit.client.0.smithi137.stdout:6/263: mknod d0/d47/d32/c52 0 2024-06-15T01:07:02.115 INFO:tasks.workunit.client.0.smithi137.stdout:1/434: symlink d0/d1f/d60/da9/laa 0 2024-06-15T01:07:02.116 INFO:tasks.workunit.client.1.smithi142.stdout:7/443: dwrite d2/d6/d22/f6c [4194304,4194304] 0 2024-06-15T01:07:02.117 INFO:tasks.workunit.client.0.smithi137.stdout:0/354: write d1/d2/dc/d10/d14/d15/f30 [81510,90183] 0 2024-06-15T01:07:02.118 INFO:tasks.workunit.client.0.smithi137.stdout:1/435: write d0/d2d/f79 [795408,9244] 0 2024-06-15T01:07:02.119 INFO:tasks.workunit.client.0.smithi137.stdout:0/355: readlink d1/d2/dc/d10/d14/d15/l37 0 2024-06-15T01:07:02.119 INFO:tasks.workunit.client.0.smithi137.stdout:4/384: creat dd/d15/d21/d3a/d51/f93 x:0 0 0 2024-06-15T01:07:02.122 INFO:tasks.workunit.client.0.smithi137.stdout:1/436: mknod d0/d10/cab 0 2024-06-15T01:07:02.123 INFO:tasks.workunit.client.1.smithi142.stdout:3/517: sync 2024-06-15T01:07:02.123 INFO:tasks.workunit.client.1.smithi142.stdout:1/499: dwrite d6/d12/d1c/d2c/f59 [0,4194304] 0 2024-06-15T01:07:02.123 INFO:tasks.workunit.client.0.smithi137.stdout:1/437: readlink d0/d3/d1b/l20 0 2024-06-15T01:07:02.123 INFO:tasks.workunit.client.0.smithi137.stdout:6/264: link d0/d47/d32/c52 d0/d5/c53 0 2024-06-15T01:07:02.126 INFO:tasks.workunit.client.0.smithi137.stdout:1/438: write d0/d3/d14/f38 [2100526,3315] 0 2024-06-15T01:07:02.127 INFO:tasks.workunit.client.1.smithi142.stdout:4/567: dread d5/d9/da/f3c [0,4194304] 0 2024-06-15T01:07:02.129 INFO:tasks.workunit.client.0.smithi137.stdout:0/356: getdents d1/d2/d13 0 2024-06-15T01:07:02.130 INFO:tasks.workunit.client.0.smithi137.stdout:1/439: read d0/d10/d41/f4b [2930505,36602] 0 2024-06-15T01:07:02.130 INFO:tasks.workunit.client.0.smithi137.stdout:4/385: getdents dd/d15/d48/d67 0 2024-06-15T01:07:02.131 INFO:tasks.workunit.client.1.smithi142.stdout:2/406: dread d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:07:02.131 INFO:tasks.workunit.client.1.smithi142.stdout:1/500: stat d6/d46/cab 0 2024-06-15T01:07:02.132 INFO:tasks.workunit.client.0.smithi137.stdout:0/357: mkdir d1/d7e 0 2024-06-15T01:07:02.132 INFO:tasks.workunit.client.0.smithi137.stdout:4/386: write dd/f12 [3935061,29206] 0 2024-06-15T01:07:02.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/265: dwrite d0/d47/d1d/f3d [0,4194304] 0 2024-06-15T01:07:02.134 INFO:tasks.workunit.client.1.smithi142.stdout:5/458: dwrite d2/f12 [4194304,4194304] 0 2024-06-15T01:07:02.135 INFO:tasks.workunit.client.1.smithi142.stdout:3/518: truncate d0/d3/da/f62 1197801 0 2024-06-15T01:07:02.136 INFO:tasks.workunit.client.1.smithi142.stdout:5/459: write d2/d18/f55 [638836,99218] 0 2024-06-15T01:07:02.138 INFO:tasks.workunit.client.1.smithi142.stdout:3/519: mknod d0/d3/da/d71/d55/ca0 0 2024-06-15T01:07:02.140 INFO:tasks.workunit.client.1.smithi142.stdout:2/407: unlink d8/d19/d23/d25/d1a/f2e 0 2024-06-15T01:07:02.143 INFO:tasks.workunit.client.1.smithi142.stdout:5/460: symlink d2/d3e/d31/d7e/l90 0 2024-06-15T01:07:02.149 INFO:tasks.workunit.client.1.smithi142.stdout:1/501: dwrite d6/d12/d1c/f32 [0,4194304] 0 2024-06-15T01:07:02.153 INFO:tasks.workunit.client.1.smithi142.stdout:1/502: creat d6/d12/d3c/d63/d8d/fb8 x:0 0 0 2024-06-15T01:07:02.155 INFO:tasks.workunit.client.1.smithi142.stdout:5/461: dwrite d2/de/d19/f6a [0,4194304] 0 2024-06-15T01:07:02.157 INFO:tasks.workunit.client.1.smithi142.stdout:1/503: write f5 [15245668,83068] 0 2024-06-15T01:07:02.158 INFO:tasks.workunit.client.1.smithi142.stdout:1/504: chown d6/d12/da0/cb4 0 1 2024-06-15T01:07:02.165 INFO:tasks.workunit.client.1.smithi142.stdout:1/505: dwrite d6/d3a/f44 [0,4194304] 0 2024-06-15T01:07:02.171 INFO:tasks.workunit.client.1.smithi142.stdout:1/506: write d6/d46/f56 [922018,2142] 0 2024-06-15T01:07:02.174 INFO:tasks.workunit.client.0.smithi137.stdout:1/440: truncate d0/d1f/d76/f8f 1390015 0 2024-06-15T01:07:02.176 INFO:tasks.workunit.client.0.smithi137.stdout:6/266: dread d0/d5/f13 [0,4194304] 0 2024-06-15T01:07:02.176 INFO:tasks.workunit.client.0.smithi137.stdout:0/358: truncate d1/f32 3070689 0 2024-06-15T01:07:02.177 INFO:tasks.workunit.client.0.smithi137.stdout:1/441: write d0/d3/d14/d34/f52 [1312655,89044] 0 2024-06-15T01:07:02.178 INFO:tasks.workunit.client.0.smithi137.stdout:1/442: fsync d0/f3b 0 2024-06-15T01:07:02.181 INFO:tasks.workunit.client.1.smithi142.stdout:3/520: dread d0/d3/da/d71/d55/f7a [0,4194304] 0 2024-06-15T01:07:02.181 INFO:tasks.workunit.client.0.smithi137.stdout:0/359: symlink d1/d2/dc/d10/d14/d56/l7f 0 2024-06-15T01:07:02.182 INFO:tasks.workunit.client.0.smithi137.stdout:1/443: rmdir d0 39 2024-06-15T01:07:02.182 INFO:tasks.workunit.client.0.smithi137.stdout:4/387: dwrite dd/d15/d30/f41 [0,4194304] 0 2024-06-15T01:07:02.186 INFO:tasks.workunit.client.0.smithi137.stdout:1/444: write d0/d3/d47/f5d [1248186,102034] 0 2024-06-15T01:07:02.187 INFO:tasks.workunit.client.0.smithi137.stdout:4/388: mknod dd/d15/d48/c94 0 2024-06-15T01:07:02.188 INFO:tasks.workunit.client.0.smithi137.stdout:4/389: readlink dd/l64 0 2024-06-15T01:07:02.191 INFO:tasks.workunit.client.0.smithi137.stdout:1/445: readlink d0/d2d/l7a 0 2024-06-15T01:07:02.191 INFO:tasks.workunit.client.1.smithi142.stdout:2/408: truncate f4 3628705 0 2024-06-15T01:07:02.192 INFO:tasks.workunit.client.0.smithi137.stdout:4/390: creat dd/d15/d36/d45/d55/f95 x:0 0 0 2024-06-15T01:07:02.195 INFO:tasks.workunit.client.0.smithi137.stdout:0/360: dwrite d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:02.197 INFO:tasks.workunit.client.1.smithi142.stdout:2/409: chown d8/d19/d23/d3a/l61 228476904 1 2024-06-15T01:07:02.197 INFO:tasks.workunit.client.1.smithi142.stdout:4/568: dread d5/d9/d41/f4c [0,4194304] 0 2024-06-15T01:07:02.198 INFO:tasks.workunit.client.0.smithi137.stdout:1/446: write d0/d3/d1b/d4c/d59/f71 [765893,107609] 0 2024-06-15T01:07:02.200 INFO:tasks.workunit.client.0.smithi137.stdout:0/361: readlink d1/d2/l7 0 2024-06-15T01:07:02.202 INFO:tasks.workunit.client.0.smithi137.stdout:4/391: mkdir dd/d15/d21/d3a/d6e/d7b/d96 0 2024-06-15T01:07:02.203 INFO:tasks.workunit.client.0.smithi137.stdout:1/447: rename d0/d10/d41/f4b to d0/d1f/d60/da9/fac 0 2024-06-15T01:07:02.203 INFO:tasks.workunit.client.0.smithi137.stdout:4/392: readlink dd/d15/d36/d45/d55/l5c 0 2024-06-15T01:07:02.204 INFO:tasks.workunit.client.0.smithi137.stdout:0/362: symlink d1/d2/dc/d10/d22/l80 0 2024-06-15T01:07:02.205 INFO:tasks.workunit.client.1.smithi142.stdout:3/521: dwrite d0/d1f/f46 [0,4194304] 0 2024-06-15T01:07:02.206 INFO:tasks.workunit.client.1.smithi142.stdout:3/522: stat d0/d1b/l50 0 2024-06-15T01:07:02.206 INFO:tasks.workunit.client.0.smithi137.stdout:1/448: unlink d0/d10/d41/l98 0 2024-06-15T01:07:02.210 INFO:tasks.workunit.client.1.smithi142.stdout:7/444: dwrite d2/d8/dd/f76 [0,4194304] 0 2024-06-15T01:07:02.211 INFO:tasks.workunit.client.0.smithi137.stdout:0/363: link d1/d2/dc/d10/l44 d1/d2/d55/l81 0 2024-06-15T01:07:02.211 INFO:tasks.workunit.client.1.smithi142.stdout:1/507: dread d6/f2a [0,4194304] 0 2024-06-15T01:07:02.213 INFO:tasks.workunit.client.1.smithi142.stdout:2/410: creat d8/df/d10/f87 x:0 0 0 2024-06-15T01:07:02.214 INFO:tasks.workunit.client.1.smithi142.stdout:1/508: chown d6/d12/l48 186097280 1 2024-06-15T01:07:02.214 INFO:tasks.workunit.client.0.smithi137.stdout:1/449: creat d0/d3/d1b/d4c/d59/d66/fad x:0 0 0 2024-06-15T01:07:02.216 INFO:tasks.workunit.client.1.smithi142.stdout:3/523: dwrite d0/f39 [0,4194304] 0 2024-06-15T01:07:02.217 INFO:tasks.workunit.client.0.smithi137.stdout:0/364: creat d1/d2/dc/d10/d14/d56/f82 x:0 0 0 2024-06-15T01:07:02.218 INFO:tasks.workunit.client.1.smithi142.stdout:3/524: dread - d0/d3/d15/f9f zero size 2024-06-15T01:07:02.222 INFO:tasks.workunit.client.0.smithi137.stdout:1/450: rename d0/d3/d1b/d4c/d59/f8a to d0/d1f/d7d/fae 0 2024-06-15T01:07:02.223 INFO:tasks.workunit.client.1.smithi142.stdout:7/445: unlink d2/d8/da/d33/d25/d27/c54 0 2024-06-15T01:07:02.223 INFO:tasks.workunit.client.1.smithi142.stdout:3/525: read d0/d1b/f2a [697513,13346] 0 2024-06-15T01:07:02.226 INFO:tasks.workunit.client.1.smithi142.stdout:3/526: read - d0/d1b/f91 zero size 2024-06-15T01:07:02.226 INFO:tasks.workunit.client.1.smithi142.stdout:5/462: dread d2/de/d19/f22 [4194304,4194304] 0 2024-06-15T01:07:02.227 INFO:tasks.workunit.client.0.smithi137.stdout:1/451: read d0/d3/f22 [1017008,37458] 0 2024-06-15T01:07:02.228 INFO:tasks.workunit.client.1.smithi142.stdout:3/527: write d0/d3/f3d [4415623,38459] 0 2024-06-15T01:07:02.230 INFO:tasks.workunit.client.0.smithi137.stdout:0/365: dwrite d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [0,4194304] 0 2024-06-15T01:07:02.231 INFO:tasks.workunit.client.1.smithi142.stdout:4/569: rename d5/d9/da/d7e to d5/d1c/d6f/dc7/d30/dcc 0 2024-06-15T01:07:02.235 INFO:tasks.workunit.client.1.smithi142.stdout:1/509: mkdir d6/d12/d1c/d2c/d93/db9 0 2024-06-15T01:07:02.236 INFO:tasks.workunit.client.0.smithi137.stdout:0/366: write d1/d2/f6 [238150,33409] 0 2024-06-15T01:07:02.239 INFO:tasks.workunit.client.0.smithi137.stdout:1/452: dwrite d0/d1f/d7d/fae [0,4194304] 0 2024-06-15T01:07:02.240 INFO:tasks.workunit.client.0.smithi137.stdout:0/367: chown d1/d2/dc/d10/d14/d15/l37 208362850 1 2024-06-15T01:07:02.242 INFO:tasks.workunit.client.1.smithi142.stdout:3/528: creat d0/d3/da/d71/fa1 x:0 0 0 2024-06-15T01:07:02.244 INFO:tasks.workunit.client.1.smithi142.stdout:9/417: sync 2024-06-15T01:07:02.244 INFO:tasks.workunit.client.1.smithi142.stdout:5/463: rename d2/d3e/c3b to d2/d3e/d31/c91 0 2024-06-15T01:07:02.245 INFO:tasks.workunit.client.1.smithi142.stdout:9/418: dread - d6/dc/f61 zero size 2024-06-15T01:07:02.246 INFO:tasks.workunit.client.1.smithi142.stdout:4/570: rmdir d5/d9/d18 39 2024-06-15T01:07:02.248 INFO:tasks.workunit.client.1.smithi142.stdout:1/510: chown d6/d46/f78 1752145 1 2024-06-15T01:07:02.251 INFO:tasks.workunit.client.1.smithi142.stdout:7/446: rename d2/d6/d22/c74 to d2/d8/dd/d14/d2c/d37/d6b/d80/d85/c93 0 2024-06-15T01:07:02.253 INFO:tasks.workunit.client.1.smithi142.stdout:1/511: mkdir d6/d12/d1c/d2c/d93/dba 0 2024-06-15T01:07:02.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/368: dread d1/d2/dc/d10/d14/d52/f58 [0,4194304] 0 2024-06-15T01:07:02.254 INFO:tasks.workunit.client.1.smithi142.stdout:3/529: symlink d0/d1b/d6b/la2 0 2024-06-15T01:07:02.254 INFO:tasks.workunit.client.1.smithi142.stdout:5/464: mkdir d2/de/d27/d86/d92 0 2024-06-15T01:07:02.255 INFO:tasks.workunit.client.1.smithi142.stdout:5/465: dread - d2/d3e/d31/f64 zero size 2024-06-15T01:07:02.256 INFO:tasks.workunit.client.0.smithi137.stdout:0/369: write d1/d2/dc/d10/d14/d56/f82 [154407,93107] 0 2024-06-15T01:07:02.256 INFO:tasks.workunit.client.1.smithi142.stdout:3/530: truncate d0/d3/da/f7b 861304 0 2024-06-15T01:07:02.257 INFO:tasks.workunit.client.1.smithi142.stdout:2/411: write d8/d19/d23/f26 [877454,101961] 0 2024-06-15T01:07:02.258 INFO:tasks.workunit.client.0.smithi137.stdout:0/370: getdents d1/d7e 0 2024-06-15T01:07:02.259 INFO:tasks.workunit.client.1.smithi142.stdout:2/412: readlink d8/d19/d23/d3a/l61 0 2024-06-15T01:07:02.260 INFO:tasks.workunit.client.1.smithi142.stdout:3/531: creat d0/d1f/d20/fa3 x:0 0 0 2024-06-15T01:07:02.262 INFO:tasks.workunit.client.1.smithi142.stdout:2/413: symlink d8/d19/l88 0 2024-06-15T01:07:02.264 INFO:tasks.workunit.client.0.smithi137.stdout:0/371: fdatasync d1/d2/dc/d10/d14/d15/d1c/f46 0 2024-06-15T01:07:02.264 INFO:tasks.workunit.client.1.smithi142.stdout:5/466: dwrite d2/f25 [0,4194304] 0 2024-06-15T01:07:02.266 INFO:tasks.workunit.client.0.smithi137.stdout:0/372: write d1/d2/dc/d10/d14/d15/d17/f69 [929652,43066] 0 2024-06-15T01:07:02.267 INFO:tasks.workunit.client.0.smithi137.stdout:0/373: truncate d1/d2/dc/d10/f25 548552 0 2024-06-15T01:07:02.268 INFO:tasks.workunit.client.1.smithi142.stdout:3/532: symlink d0/d1f/d20/d5d/la4 0 2024-06-15T01:07:02.273 INFO:tasks.workunit.client.1.smithi142.stdout:2/414: mknod d8/d19/d23/d25/c89 0 2024-06-15T01:07:02.273 INFO:tasks.workunit.client.1.smithi142.stdout:3/533: unlink d0/d1b/c36 0 2024-06-15T01:07:02.274 INFO:tasks.workunit.client.1.smithi142.stdout:5/467: dread d2/f12 [4194304,4194304] 0 2024-06-15T01:07:02.276 INFO:tasks.workunit.client.1.smithi142.stdout:9/419: dread d6/dc/dd/f77 [0,4194304] 0 2024-06-15T01:07:02.276 INFO:tasks.workunit.client.1.smithi142.stdout:2/415: write d8/d19/d23/d25/f74 [1490654,65470] 0 2024-06-15T01:07:02.277 INFO:tasks.workunit.client.1.smithi142.stdout:1/512: dread d6/d12/d28/f40 [0,4194304] 0 2024-06-15T01:07:02.277 INFO:tasks.workunit.client.0.smithi137.stdout:0/374: dread d1/d2/dc/d10/d14/d15/f57 [0,4194304] 0 2024-06-15T01:07:02.279 INFO:tasks.workunit.client.0.smithi137.stdout:0/375: dread - d1/d2/dc/d10/d14/d15/d17/f4c zero size 2024-06-15T01:07:02.279 INFO:tasks.workunit.client.1.smithi142.stdout:2/416: dread - d8/df/d1f/f3e zero size 2024-06-15T01:07:02.279 INFO:tasks.workunit.client.1.smithi142.stdout:3/534: truncate d0/d1f/d20/d59/f80 3764 0 2024-06-15T01:07:02.279 INFO:tasks.workunit.client.1.smithi142.stdout:1/513: readlink d6/d25/l31 0 2024-06-15T01:07:02.280 INFO:tasks.workunit.client.0.smithi137.stdout:0/376: write d1/d2/dc/d10/f25 [1488592,122904] 0 2024-06-15T01:07:02.280 INFO:tasks.workunit.client.1.smithi142.stdout:7/447: dread d2/d6/f1c [0,4194304] 0 2024-06-15T01:07:02.281 INFO:tasks.workunit.client.1.smithi142.stdout:2/417: chown d8/d19/f75 334095820 1 2024-06-15T01:07:02.282 INFO:tasks.workunit.client.1.smithi142.stdout:2/418: fdatasync d8/fe 0 2024-06-15T01:07:02.283 INFO:tasks.workunit.client.1.smithi142.stdout:5/468: dread d2/f12 [4194304,4194304] 0 2024-06-15T01:07:02.285 INFO:tasks.workunit.client.0.smithi137.stdout:0/377: link d1/d2/dc/d10/d14/l27 d1/d2/d13/d66/l83 0 2024-06-15T01:07:02.286 INFO:tasks.workunit.client.1.smithi142.stdout:9/420: link d6/dc/d1b/d25/d58/f99 d6/dc/d1b/d6f/da1/fa7 0 2024-06-15T01:07:02.286 INFO:tasks.workunit.client.1.smithi142.stdout:8/426: sync 2024-06-15T01:07:02.286 INFO:tasks.workunit.client.1.smithi142.stdout:0/452: sync 2024-06-15T01:07:02.286 INFO:tasks.workunit.client.1.smithi142.stdout:6/432: sync 2024-06-15T01:07:02.288 INFO:tasks.workunit.client.0.smithi137.stdout:0/378: mkdir d1/d2/dc/d6c/d84 0 2024-06-15T01:07:02.288 INFO:tasks.workunit.client.1.smithi142.stdout:6/433: truncate d1a/d83/f7c 4418454 0 2024-06-15T01:07:02.288 INFO:tasks.workunit.client.1.smithi142.stdout:3/535: getdents d0/d3 0 2024-06-15T01:07:02.289 INFO:tasks.workunit.client.0.smithi137.stdout:0/379: fsync d1/d2/d13/f33 0 2024-06-15T01:07:02.291 INFO:tasks.workunit.client.1.smithi142.stdout:5/469: mkdir d2/de/d19/d93 0 2024-06-15T01:07:02.292 INFO:tasks.workunit.client.1.smithi142.stdout:5/470: readlink d2/de/d19/d5f/l6f 0 2024-06-15T01:07:02.292 INFO:tasks.workunit.client.1.smithi142.stdout:7/448: mkdir d2/d8/dd/d8a/d94 0 2024-06-15T01:07:02.293 INFO:tasks.workunit.client.1.smithi142.stdout:5/471: chown d2/de/f1f 2541893 1 2024-06-15T01:07:02.294 INFO:tasks.workunit.client.1.smithi142.stdout:5/472: chown d2/d3e/d65/c8d 12310219 1 2024-06-15T01:07:02.294 INFO:tasks.workunit.client.1.smithi142.stdout:0/453: creat d3/d39/d9a/fa8 x:0 0 0 2024-06-15T01:07:02.296 INFO:tasks.workunit.client.1.smithi142.stdout:1/514: creat d6/d12/d1c/d2c/d93/db9/fbb x:0 0 0 2024-06-15T01:07:02.298 INFO:tasks.workunit.client.1.smithi142.stdout:2/419: creat d8/df/d10/d1c/f8a x:0 0 0 2024-06-15T01:07:02.299 INFO:tasks.workunit.client.1.smithi142.stdout:6/434: fsync d1a/f22 0 2024-06-15T01:07:02.301 INFO:tasks.workunit.client.1.smithi142.stdout:5/473: fdatasync d2/de/d19/f2a 0 2024-06-15T01:07:02.301 INFO:tasks.workunit.client.1.smithi142.stdout:3/536: fdatasync d0/d3/da/d18/f8a 0 2024-06-15T01:07:02.303 INFO:tasks.workunit.client.1.smithi142.stdout:6/435: write f7 [3118699,110687] 0 2024-06-15T01:07:02.303 INFO:tasks.workunit.client.1.smithi142.stdout:5/474: readlink d2/l3d 0 2024-06-15T01:07:02.303 INFO:tasks.workunit.client.1.smithi142.stdout:0/454: link d3/d39/d3e/d81/d4c/l58 d3/d39/d3e/d81/d4c/la9 0 2024-06-15T01:07:02.306 INFO:tasks.workunit.client.1.smithi142.stdout:5/475: readlink d2/d3e/d31/d7e/l90 0 2024-06-15T01:07:02.307 INFO:tasks.workunit.client.1.smithi142.stdout:6/436: write d1a/f20 [4251469,26093] 0 2024-06-15T01:07:02.309 INFO:tasks.workunit.client.1.smithi142.stdout:1/515: dwrite d6/d46/d61/f76 [0,4194304] 0 2024-06-15T01:07:02.310 INFO:tasks.workunit.client.1.smithi142.stdout:0/455: truncate d3/d39/d9a/fa8 247275 0 2024-06-15T01:07:02.313 INFO:tasks.workunit.client.1.smithi142.stdout:7/449: dread d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:07:02.313 INFO:tasks.workunit.client.0.smithi137.stdout:8/334: sync 2024-06-15T01:07:02.314 INFO:tasks.workunit.client.1.smithi142.stdout:3/537: dwrite d0/d1f/f4b [4194304,4194304] 0 2024-06-15T01:07:02.315 INFO:tasks.workunit.client.1.smithi142.stdout:7/450: write d2/d8/da/d33/d25/d27/f7b [4222732,43337] 0 2024-06-15T01:07:02.315 INFO:tasks.workunit.client.1.smithi142.stdout:3/538: chown d0/d3/d15/c19 1850 1 2024-06-15T01:07:02.315 INFO:tasks.workunit.client.0.smithi137.stdout:8/335: chown d9/c1a 1054 1 2024-06-15T01:07:02.316 INFO:tasks.workunit.client.0.smithi137.stdout:0/380: read d1/d2/f47 [7234712,82861] 0 2024-06-15T01:07:02.326 INFO:tasks.workunit.client.0.smithi137.stdout:0/381: dwrite d1/d2/dc/d10/d14/d15/f50 [0,4194304] 0 2024-06-15T01:07:02.329 INFO:tasks.workunit.client.0.smithi137.stdout:0/382: creat d1/d2/dc/d10/d14/d56/f85 x:0 0 0 2024-06-15T01:07:02.336 INFO:tasks.workunit.client.0.smithi137.stdout:8/336: truncate d9/d2a/f2d 1205075 0 2024-06-15T01:07:02.336 INFO:tasks.workunit.client.1.smithi142.stdout:8/427: dread d0/f5f [0,4194304] 0 2024-06-15T01:07:02.337 INFO:tasks.workunit.client.1.smithi142.stdout:8/428: dread d0/f5f [0,4194304] 0 2024-06-15T01:07:02.338 INFO:tasks.workunit.client.0.smithi137.stdout:8/337: write d9/de/f3c [351234,21334] 0 2024-06-15T01:07:02.340 INFO:tasks.workunit.client.0.smithi137.stdout:0/383: dwrite d1/d2/dc/d10/d14/d52/f58 [0,4194304] 0 2024-06-15T01:07:02.340 INFO:tasks.workunit.client.0.smithi137.stdout:8/338: write d9/d2a/d2f/f4b [3285187,75535] 0 2024-06-15T01:07:02.344 INFO:tasks.workunit.client.0.smithi137.stdout:0/384: chown d1/d2/dc/d10/d14/l27 1133 1 2024-06-15T01:07:02.345 INFO:tasks.workunit.client.0.smithi137.stdout:8/339: creat d9/d2a/d2f/d59/f71 x:0 0 0 2024-06-15T01:07:02.348 INFO:tasks.workunit.client.0.smithi137.stdout:0/385: getdents d1/d2/dc/d6c/d84 0 2024-06-15T01:07:02.349 INFO:tasks.workunit.client.0.smithi137.stdout:0/386: write d1/d2/dc/d10/d14/f1b [367649,78139] 0 2024-06-15T01:07:02.350 INFO:tasks.workunit.client.1.smithi142.stdout:2/420: rename d8/d19/d23/d25/d1a/d57/f2b to d8/d19/d23/d3a/f8b 0 2024-06-15T01:07:02.350 INFO:tasks.workunit.client.0.smithi137.stdout:8/340: creat d9/d2a/d2f/f72 x:0 0 0 2024-06-15T01:07:02.353 INFO:tasks.workunit.client.0.smithi137.stdout:0/387: write d1/d2/dc/d10/d14/d15/f50 [3962400,393] 0 2024-06-15T01:07:02.355 INFO:tasks.workunit.client.0.smithi137.stdout:8/341: write d9/de/ff [110427,16486] 0 2024-06-15T01:07:02.355 INFO:tasks.workunit.client.0.smithi137.stdout:0/388: creat d1/d2/d13/d66/f86 x:0 0 0 2024-06-15T01:07:02.357 INFO:tasks.workunit.client.1.smithi142.stdout:1/516: unlink d6/d12/d28/f64 0 2024-06-15T01:07:02.357 INFO:tasks.workunit.client.0.smithi137.stdout:0/389: write d1/d2/f21 [209179,38131] 0 2024-06-15T01:07:02.357 INFO:tasks.workunit.client.1.smithi142.stdout:1/517: fdatasync d6/d12/fa1 0 2024-06-15T01:07:02.358 INFO:tasks.workunit.client.0.smithi137.stdout:8/342: symlink d9/d2a/d2f/d4e/l73 0 2024-06-15T01:07:02.360 INFO:tasks.workunit.client.0.smithi137.stdout:0/390: chown d1/c20 917013 1 2024-06-15T01:07:02.362 INFO:tasks.workunit.client.0.smithi137.stdout:8/343: creat d9/de/d1e/d35/d5f/f74 x:0 0 0 2024-06-15T01:07:02.362 INFO:tasks.workunit.client.1.smithi142.stdout:3/539: mknod d0/ca5 0 2024-06-15T01:07:02.363 INFO:tasks.workunit.client.0.smithi137.stdout:0/391: write d1/d2/dc/d10/d14/d15/d17/f4c [49722,10564] 0 2024-06-15T01:07:02.363 INFO:tasks.workunit.client.0.smithi137.stdout:8/344: chown d9/de/d1e/d35/f63 46 1 2024-06-15T01:07:02.364 INFO:tasks.workunit.client.1.smithi142.stdout:7/451: creat d2/d8/da/d33/d25/d79/f95 x:0 0 0 2024-06-15T01:07:02.367 INFO:tasks.workunit.client.1.smithi142.stdout:2/421: mknod d8/d19/d23/d25/d1a/c8c 0 2024-06-15T01:07:02.370 INFO:tasks.workunit.client.1.smithi142.stdout:5/476: mknod d2/de/d19/d93/c94 0 2024-06-15T01:07:02.370 INFO:tasks.workunit.client.0.smithi137.stdout:8/345: rename d9/d2a/d2f/f54 to d9/de/d1e/f75 0 2024-06-15T01:07:02.372 INFO:tasks.workunit.client.0.smithi137.stdout:0/392: dwrite d1/d2/f5e [0,4194304] 0 2024-06-15T01:07:02.373 INFO:tasks.workunit.client.1.smithi142.stdout:2/422: dwrite d8/d19/d23/d25/d1a/d63/f77 [0,4194304] 0 2024-06-15T01:07:02.374 INFO:tasks.workunit.client.1.smithi142.stdout:2/423: chown d8/d19/c60 493122 1 2024-06-15T01:07:02.375 INFO:tasks.workunit.client.0.smithi137.stdout:0/393: write d1/d2/f6 [4167578,89466] 0 2024-06-15T01:07:02.375 INFO:tasks.workunit.client.1.smithi142.stdout:7/452: rename d2/d8/da/d33/d25/d27/f5d to d2/d8/dd/d14/d2c/d37/d6b/f96 0 2024-06-15T01:07:02.381 INFO:tasks.workunit.client.1.smithi142.stdout:5/477: mknod d2/d18/d79/c95 0 2024-06-15T01:07:02.382 INFO:tasks.workunit.client.0.smithi137.stdout:0/394: mkdir d1/d7e/d87 0 2024-06-15T01:07:02.382 INFO:tasks.workunit.client.1.smithi142.stdout:0/456: creat d3/d39/d3e/d81/d44/d7c/faa x:0 0 0 2024-06-15T01:07:02.382 INFO:tasks.workunit.client.1.smithi142.stdout:5/478: chown d2/c53 145847 1 2024-06-15T01:07:02.383 INFO:tasks.workunit.client.0.smithi137.stdout:8/346: dwrite d9/de/d1e/f27 [0,4194304] 0 2024-06-15T01:07:02.384 INFO:tasks.workunit.client.0.smithi137.stdout:0/395: write d1/d2/dc/d10/d14/d15/d17/f69 [844110,5698] 0 2024-06-15T01:07:02.387 INFO:tasks.workunit.client.0.smithi137.stdout:0/396: mkdir d1/d2/dc/d6c/d88 0 2024-06-15T01:07:02.388 INFO:tasks.workunit.client.1.smithi142.stdout:2/424: unlink d8/df/d10/d1c/f8a 0 2024-06-15T01:07:02.388 INFO:tasks.workunit.client.0.smithi137.stdout:8/347: symlink d9/l76 0 2024-06-15T01:07:02.389 INFO:tasks.workunit.client.0.smithi137.stdout:0/397: write d1/d2/dc/d10/d14/d15/f35 [3820110,94248] 0 2024-06-15T01:07:02.390 INFO:tasks.workunit.client.1.smithi142.stdout:2/425: dread - d8/d19/d23/f5d zero size 2024-06-15T01:07:02.390 INFO:tasks.workunit.client.1.smithi142.stdout:2/426: fdatasync d8/f6b 0 2024-06-15T01:07:02.391 INFO:tasks.workunit.client.0.smithi137.stdout:0/398: stat d1/d2/d13 0 2024-06-15T01:07:02.393 INFO:tasks.workunit.client.0.smithi137.stdout:8/348: mknod d9/d55/c77 0 2024-06-15T01:07:02.393 INFO:tasks.workunit.client.1.smithi142.stdout:2/427: stat d8/d19/d23/f5d 0 2024-06-15T01:07:02.394 INFO:tasks.workunit.client.1.smithi142.stdout:0/457: write d3/d39/d3e/d81/d44/d7c/f19 [52407,131030] 0 2024-06-15T01:07:02.395 INFO:tasks.workunit.client.0.smithi137.stdout:0/399: symlink d1/d2/dc/d6c/d88/l89 0 2024-06-15T01:07:02.396 INFO:tasks.workunit.client.1.smithi142.stdout:5/479: dread d2/de/f50 [0,4194304] 0 2024-06-15T01:07:02.396 INFO:tasks.workunit.client.0.smithi137.stdout:8/349: write d9/de/d1e/f27 [42588,93870] 0 2024-06-15T01:07:02.397 INFO:tasks.workunit.client.1.smithi142.stdout:6/437: link d1a/d2d/d3b/c73 d1a/c9f 0 2024-06-15T01:07:02.397 INFO:tasks.workunit.client.1.smithi142.stdout:2/428: chown d8/df/d10/d1c/c1e 12862344 1 2024-06-15T01:07:02.398 INFO:tasks.workunit.client.1.smithi142.stdout:8/429: getdents d0/d17/d40/d86 0 2024-06-15T01:07:02.400 INFO:tasks.workunit.client.0.smithi137.stdout:8/350: read d9/de/d1e/d35/f63 [7804,94658] 0 2024-06-15T01:07:02.400 INFO:tasks.workunit.client.1.smithi142.stdout:3/540: dwrite d0/d3/da/f9b [0,4194304] 0 2024-06-15T01:07:02.401 INFO:tasks.workunit.client.0.smithi137.stdout:0/400: truncate d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d 4715579 0 2024-06-15T01:07:02.401 INFO:tasks.workunit.client.1.smithi142.stdout:0/458: symlink d3/d39/d3e/d81/d1b/d38/da3/lab 0 2024-06-15T01:07:02.402 INFO:tasks.workunit.client.1.smithi142.stdout:6/438: mkdir d1a/d2d/d68/da0 0 2024-06-15T01:07:02.403 INFO:tasks.workunit.client.0.smithi137.stdout:8/351: rmdir d9/de/d1e 39 2024-06-15T01:07:02.405 INFO:tasks.workunit.client.1.smithi142.stdout:2/429: mkdir d8/d19/d8d 0 2024-06-15T01:07:02.406 INFO:tasks.workunit.client.0.smithi137.stdout:0/401: mknod d1/d2/dc/c8a 0 2024-06-15T01:07:02.407 INFO:tasks.workunit.client.1.smithi142.stdout:5/480: dwrite d2/de/d19/d5f/d68/f73 [0,4194304] 0 2024-06-15T01:07:02.407 INFO:tasks.workunit.client.1.smithi142.stdout:2/430: read d8/d19/d23/d25/d1a/f1d [1902109,116380] 0 2024-06-15T01:07:02.409 INFO:tasks.workunit.client.1.smithi142.stdout:3/541: creat d0/d1f/d20/d59/fa6 x:0 0 0 2024-06-15T01:07:02.411 INFO:tasks.workunit.client.1.smithi142.stdout:2/431: chown d8/df/f59 8203 1 2024-06-15T01:07:02.412 INFO:tasks.workunit.client.0.smithi137.stdout:8/352: chown d9/de/d1e/d67/f6b 30694387 1 2024-06-15T01:07:02.412 INFO:tasks.workunit.client.1.smithi142.stdout:3/542: chown d0/d1f/f69 1939 1 2024-06-15T01:07:02.413 INFO:tasks.workunit.client.1.smithi142.stdout:6/439: read d1a/f38 [3587511,14961] 0 2024-06-15T01:07:02.413 INFO:tasks.workunit.client.1.smithi142.stdout:0/459: dwrite d3/d39/d3e/d81/d1e/f2b [0,4194304] 0 2024-06-15T01:07:02.414 INFO:tasks.workunit.client.1.smithi142.stdout:8/430: creat d0/d25/d5a/f8d x:0 0 0 2024-06-15T01:07:02.414 INFO:tasks.workunit.client.0.smithi137.stdout:0/402: read d1/d2/dc/d10/d14/d15/d17/f4c [26579,41892] 0 2024-06-15T01:07:02.416 INFO:tasks.workunit.client.1.smithi142.stdout:6/440: readlink d1a/d2d/l96 0 2024-06-15T01:07:02.417 INFO:tasks.workunit.client.1.smithi142.stdout:3/543: write d0/d1f/f38 [2467,103859] 0 2024-06-15T01:07:02.419 INFO:tasks.workunit.client.1.smithi142.stdout:3/544: write d0/d1f/d20/d59/f92 [316956,112379] 0 2024-06-15T01:07:02.422 INFO:tasks.workunit.client.1.smithi142.stdout:6/441: chown d1a/d2d/d68/c6e 2239275 1 2024-06-15T01:07:02.423 INFO:tasks.workunit.client.1.smithi142.stdout:0/460: rmdir d3/d39/d3e/d81/d44/d7c/d22 39 2024-06-15T01:07:02.423 INFO:tasks.workunit.client.0.smithi137.stdout:0/403: dread d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [0,4194304] 0 2024-06-15T01:07:02.424 INFO:tasks.workunit.client.0.smithi137.stdout:0/404: chown d1/d2/dc/d6c/d88 5 1 2024-06-15T01:07:02.425 INFO:tasks.workunit.client.1.smithi142.stdout:7/453: truncate d2/d8/dd/f76 1405785 0 2024-06-15T01:07:02.425 INFO:tasks.workunit.client.0.smithi137.stdout:0/405: write d1/d2/f47 [2994275,88538] 0 2024-06-15T01:07:02.427 INFO:tasks.workunit.client.0.smithi137.stdout:0/406: fdatasync d1/d2/dc/d10/d14/d15/f4b 0 2024-06-15T01:07:02.428 INFO:tasks.workunit.client.1.smithi142.stdout:7/454: read d2/d8/dd/d14/d2c/f62 [346430,64910] 0 2024-06-15T01:07:02.428 INFO:tasks.workunit.client.1.smithi142.stdout:8/431: creat d0/f8e x:0 0 0 2024-06-15T01:07:02.429 INFO:tasks.workunit.client.0.smithi137.stdout:8/353: dread d9/de/d1e/d35/f56 [0,4194304] 0 2024-06-15T01:07:02.430 INFO:tasks.workunit.client.0.smithi137.stdout:0/407: rmdir d1/d2/d13 39 2024-06-15T01:07:02.431 INFO:tasks.workunit.client.0.smithi137.stdout:0/408: readlink d1/d2/d55/l81 0 2024-06-15T01:07:02.432 INFO:tasks.workunit.client.1.smithi142.stdout:8/432: unlink d0/d25/d5a/f37 0 2024-06-15T01:07:02.433 INFO:tasks.workunit.client.1.smithi142.stdout:8/433: chown d0/d25/d53/f58 19446669 1 2024-06-15T01:07:02.433 INFO:tasks.workunit.client.1.smithi142.stdout:7/455: mkdir d2/d8/dd/d97 0 2024-06-15T01:07:02.434 INFO:tasks.workunit.client.0.smithi137.stdout:0/409: fsync d1/d2/d13/f40 0 2024-06-15T01:07:02.435 INFO:tasks.workunit.client.0.smithi137.stdout:8/354: link d9/de/d1e/d3f/f68 d9/de/d1e/d35/d5f/f78 0 2024-06-15T01:07:02.437 INFO:tasks.workunit.client.0.smithi137.stdout:8/355: mkdir d9/d2a/d79 0 2024-06-15T01:07:02.438 INFO:tasks.workunit.client.0.smithi137.stdout:8/356: chown d9/de/d1e/c32 80 1 2024-06-15T01:07:02.439 INFO:tasks.workunit.client.0.smithi137.stdout:0/410: mknod d1/d2/dc/d10/d63/d77/c8b 0 2024-06-15T01:07:02.444 INFO:tasks.workunit.client.0.smithi137.stdout:8/357: creat d9/d2a/f7a x:0 0 0 2024-06-15T01:07:02.445 INFO:tasks.workunit.client.1.smithi142.stdout:7/456: rmdir d2/d8/dd/d97 0 2024-06-15T01:07:02.449 INFO:tasks.workunit.client.1.smithi142.stdout:3/545: dread d0/f2c [0,4194304] 0 2024-06-15T01:07:02.449 INFO:tasks.workunit.client.1.smithi142.stdout:6/442: write d1a/f22 [1697270,122121] 0 2024-06-15T01:07:02.451 INFO:tasks.workunit.client.1.smithi142.stdout:7/457: write d2/d6/f69 [231599,123542] 0 2024-06-15T01:07:02.451 INFO:tasks.workunit.client.1.smithi142.stdout:5/481: dread d2/de/d27/f54 [0,4194304] 0 2024-06-15T01:07:02.455 INFO:tasks.workunit.client.1.smithi142.stdout:6/443: chown d1a/d83/f7c 0 1 2024-06-15T01:07:02.455 INFO:tasks.workunit.client.1.smithi142.stdout:0/461: dread d3/d39/d3e/d81/d1b/f35 [0,4194304] 0 2024-06-15T01:07:02.456 INFO:tasks.workunit.client.1.smithi142.stdout:7/458: mknod d2/d8/dd/d8a/c98 0 2024-06-15T01:07:02.457 INFO:tasks.workunit.client.1.smithi142.stdout:2/432: dread d8/d19/d33/f42 [0,4194304] 0 2024-06-15T01:07:02.457 INFO:tasks.workunit.client.1.smithi142.stdout:6/444: chown d1a/d2d/f32 28 1 2024-06-15T01:07:02.457 INFO:tasks.workunit.client.1.smithi142.stdout:3/546: creat d0/d3/da/d71/d75/fa7 x:0 0 0 2024-06-15T01:07:02.458 INFO:tasks.workunit.client.1.smithi142.stdout:5/482: mkdir d2/de/d19/d5f/d68/d76/d96 0 2024-06-15T01:07:02.459 INFO:tasks.workunit.client.1.smithi142.stdout:3/547: fsync d0/d3/f12 0 2024-06-15T01:07:02.459 INFO:tasks.workunit.client.1.smithi142.stdout:2/433: dread - d8/d19/d23/d25/d1a/d57/f68 zero size 2024-06-15T01:07:02.460 INFO:tasks.workunit.client.1.smithi142.stdout:0/462: fsync d3/d39/d3e/d81/d44/d7c/f3c 0 2024-06-15T01:07:02.461 INFO:tasks.workunit.client.1.smithi142.stdout:6/445: symlink d1a/d2d/d80/la1 0 2024-06-15T01:07:02.463 INFO:tasks.workunit.client.1.smithi142.stdout:6/446: fsync d1a/d39/d42/f45 0 2024-06-15T01:07:02.464 INFO:tasks.workunit.client.1.smithi142.stdout:7/459: symlink d2/d8/dd/d14/d2c/d37/l99 0 2024-06-15T01:07:02.465 INFO:tasks.workunit.client.1.smithi142.stdout:0/463: mknod d3/d39/d3e/d81/d4c/cac 0 2024-06-15T01:07:02.466 INFO:tasks.workunit.client.1.smithi142.stdout:5/483: symlink d2/de/d85/l97 0 2024-06-15T01:07:02.485 INFO:tasks.workunit.client.1.smithi142.stdout:4/571: sync 2024-06-15T01:07:02.486 INFO:tasks.workunit.client.1.smithi142.stdout:9/421: sync 2024-06-15T01:07:02.494 INFO:tasks.workunit.client.1.smithi142.stdout:7/460: readlink d2/d6/l64 0 2024-06-15T01:07:02.497 INFO:tasks.workunit.client.1.smithi142.stdout:9/422: write d6/dc/dd/f46 [2676561,106184] 0 2024-06-15T01:07:02.498 INFO:tasks.workunit.client.1.smithi142.stdout:3/548: dwrite d0/d3/da/d18/f98 [0,4194304] 0 2024-06-15T01:07:02.498 INFO:tasks.workunit.client.1.smithi142.stdout:8/434: dread d0/d25/d5a/f29 [4194304,4194304] 0 2024-06-15T01:07:02.498 INFO:tasks.workunit.client.1.smithi142.stdout:7/461: chown d2/d8/dd/d8a 0 1 2024-06-15T01:07:02.499 INFO:tasks.workunit.client.1.smithi142.stdout:4/572: dread - d5/d1c/d6f/f79 zero size 2024-06-15T01:07:02.500 INFO:tasks.workunit.client.1.smithi142.stdout:8/435: chown d0/d25 63601 1 2024-06-15T01:07:02.501 INFO:tasks.workunit.client.1.smithi142.stdout:4/573: stat d5/d9/d23/l88 0 2024-06-15T01:07:02.504 INFO:tasks.workunit.client.1.smithi142.stdout:9/423: symlink d6/dc/d1b/d25/d58/la8 0 2024-06-15T01:07:02.505 INFO:tasks.workunit.client.1.smithi142.stdout:7/462: write d2/f91 [795790,116309] 0 2024-06-15T01:07:02.508 INFO:tasks.workunit.client.1.smithi142.stdout:9/424: mknod d6/dc/d1b/ca9 0 2024-06-15T01:07:02.508 INFO:tasks.workunit.client.1.smithi142.stdout:8/436: read d0/f43 [53435,3265] 0 2024-06-15T01:07:02.510 INFO:tasks.workunit.client.1.smithi142.stdout:4/574: write d5/d9/da/f7f [3456808,57203] 0 2024-06-15T01:07:02.512 INFO:tasks.workunit.client.1.smithi142.stdout:8/437: chown d0/d25/d5a/l55 582991 1 2024-06-15T01:07:02.513 INFO:tasks.workunit.client.1.smithi142.stdout:6/447: dread fc [0,4194304] 0 2024-06-15T01:07:02.514 INFO:tasks.workunit.client.1.smithi142.stdout:3/549: dwrite d0/d3/f1a [4194304,4194304] 0 2024-06-15T01:07:02.514 INFO:tasks.workunit.client.1.smithi142.stdout:8/438: chown d0/f47 938392 1 2024-06-15T01:07:02.516 INFO:tasks.workunit.client.1.smithi142.stdout:0/464: dread d3/f4 [0,4194304] 0 2024-06-15T01:07:02.539 INFO:tasks.workunit.client.1.smithi142.stdout:6/448: dread d1a/d39/d42/d4c/d58/f5d [4194304,4194304] 0 2024-06-15T01:07:02.554 INFO:tasks.workunit.client.1.smithi142.stdout:9/425: mknod d6/d34/d8a/caa 0 2024-06-15T01:07:02.555 INFO:tasks.workunit.client.1.smithi142.stdout:7/463: fsync d2/d8/dd/d14/d2c/d37/d6b/d80/d85/f8e 0 2024-06-15T01:07:02.556 INFO:tasks.workunit.client.1.smithi142.stdout:3/550: symlink d0/d3/la8 0 2024-06-15T01:07:02.557 INFO:tasks.workunit.client.1.smithi142.stdout:7/464: chown d2/d6/f1d 86917541 1 2024-06-15T01:07:02.558 INFO:tasks.workunit.client.1.smithi142.stdout:0/465: readlink d3/d39/d3e/d8f/l93 0 2024-06-15T01:07:02.564 INFO:tasks.workunit.client.1.smithi142.stdout:8/439: fsync d0/d25/f6a 0 2024-06-15T01:07:02.566 INFO:tasks.workunit.client.1.smithi142.stdout:7/465: dread d2/d8/da/f55 [0,4194304] 0 2024-06-15T01:07:02.566 INFO:tasks.workunit.client.1.smithi142.stdout:6/449: symlink d1a/d39/d3a/la2 0 2024-06-15T01:07:02.569 INFO:tasks.workunit.client.1.smithi142.stdout:0/466: symlink d3/d39/d9a/da2/lad 0 2024-06-15T01:07:02.570 INFO:tasks.workunit.client.1.smithi142.stdout:8/440: chown d0/d25/d5a/l2e 244076 1 2024-06-15T01:07:02.570 INFO:tasks.workunit.client.1.smithi142.stdout:6/450: fdatasync f0 0 2024-06-15T01:07:02.573 INFO:tasks.workunit.client.1.smithi142.stdout:8/441: chown d0/d25/c36 468423723 1 2024-06-15T01:07:02.574 INFO:tasks.workunit.client.1.smithi142.stdout:3/551: symlink d0/d3/da/la9 0 2024-06-15T01:07:02.575 INFO:tasks.workunit.client.1.smithi142.stdout:9/426: dread d6/f21 [0,4194304] 0 2024-06-15T01:07:02.577 INFO:tasks.workunit.client.1.smithi142.stdout:6/451: unlink d1a/d39/d42/d4c/d58/d87/l9a 0 2024-06-15T01:07:02.579 INFO:tasks.workunit.client.1.smithi142.stdout:3/552: write d0/d3/f90 [1193630,119981] 0 2024-06-15T01:07:02.580 INFO:tasks.workunit.client.1.smithi142.stdout:9/427: fsync d6/dc/d1b/d25/f3f 0 2024-06-15T01:07:02.581 INFO:tasks.workunit.client.1.smithi142.stdout:7/466: dwrite d2/d8/da/d33/f35 [0,4194304] 0 2024-06-15T01:07:02.581 INFO:tasks.workunit.client.1.smithi142.stdout:6/452: fdatasync d1a/d39/d3a/f70 0 2024-06-15T01:07:02.585 INFO:tasks.workunit.client.1.smithi142.stdout:0/467: dwrite d3/d39/d3e/d81/d1e/f5d [0,4194304] 0 2024-06-15T01:07:02.587 INFO:tasks.workunit.client.1.smithi142.stdout:4/575: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f60 107503 0 2024-06-15T01:07:02.587 INFO:tasks.workunit.client.1.smithi142.stdout:7/467: chown d2/d6/f9 197914423 1 2024-06-15T01:07:02.588 INFO:tasks.workunit.client.1.smithi142.stdout:9/428: write d6/d34/fa0 [659932,10374] 0 2024-06-15T01:07:02.592 INFO:tasks.workunit.client.1.smithi142.stdout:6/453: creat d1a/d39/d3a/d51/fa3 x:0 0 0 2024-06-15T01:07:02.593 INFO:tasks.workunit.client.1.smithi142.stdout:6/454: fsync f0 0 2024-06-15T01:07:02.594 INFO:tasks.workunit.client.1.smithi142.stdout:7/468: creat d2/d8/da/d33/d25/d79/f9a x:0 0 0 2024-06-15T01:07:02.595 INFO:tasks.workunit.client.1.smithi142.stdout:0/468: mkdir d3/d39/d3e/d81/d1b/dae 0 2024-06-15T01:07:02.599 INFO:tasks.workunit.client.1.smithi142.stdout:7/469: write d2/d6/f8f [1945198,16122] 0 2024-06-15T01:07:02.600 INFO:tasks.workunit.client.1.smithi142.stdout:4/576: rename d5/d1c/d6f/dc7/d30/l4a to d5/d1c/d6f/dc7/d30/lcd 0 2024-06-15T01:07:02.601 INFO:tasks.workunit.client.1.smithi142.stdout:3/553: dwrite d0/d3/d15/f77 [0,4194304] 0 2024-06-15T01:07:02.601 INFO:tasks.workunit.client.1.smithi142.stdout:6/455: dread d1a/d39/d3a/f59 [0,4194304] 0 2024-06-15T01:07:02.602 INFO:tasks.workunit.client.1.smithi142.stdout:0/469: getdents d3/d39/d3e/d81/d1b/dae 0 2024-06-15T01:07:02.603 INFO:tasks.workunit.client.1.smithi142.stdout:7/470: fsync d2/d8/dd/d14/d2c/d37/f75 0 2024-06-15T01:07:02.603 INFO:tasks.workunit.client.1.smithi142.stdout:6/456: chown d1a/d39/d3a/l94 839983194 1 2024-06-15T01:07:02.605 INFO:tasks.workunit.client.1.smithi142.stdout:4/577: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f95 zero size 2024-06-15T01:07:02.607 INFO:tasks.workunit.client.1.smithi142.stdout:6/457: read d1a/d39/d3a/f5a [344014,108381] 0 2024-06-15T01:07:02.607 INFO:tasks.workunit.client.1.smithi142.stdout:3/554: truncate d0/d3/da/d18/f8a 253261 0 2024-06-15T01:07:02.608 INFO:tasks.workunit.client.1.smithi142.stdout:4/578: truncate d5/d9/da/f7f 4506119 0 2024-06-15T01:07:02.610 INFO:tasks.workunit.client.1.smithi142.stdout:0/470: rename d3/d39/d3e/d81/d1b/d38/f8e to d3/d39/d3e/d81/d4c/faf 0 2024-06-15T01:07:02.611 INFO:tasks.workunit.client.1.smithi142.stdout:8/442: dread d0/d25/d5a/f2c [0,4194304] 0 2024-06-15T01:07:02.612 INFO:tasks.workunit.client.1.smithi142.stdout:7/471: symlink d2/d8/dd/d14/d2c/d70/l9b 0 2024-06-15T01:07:02.612 INFO:tasks.workunit.client.1.smithi142.stdout:4/579: fdatasync d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/d99/f9a 0 2024-06-15T01:07:02.614 INFO:tasks.workunit.client.1.smithi142.stdout:6/458: creat d1a/d39/d42/d4c/d58/d87/fa4 x:0 0 0 2024-06-15T01:07:02.615 INFO:tasks.workunit.client.1.smithi142.stdout:0/471: write d3/d39/d3e/d81/fb [4006959,69224] 0 2024-06-15T01:07:02.619 INFO:tasks.workunit.client.1.smithi142.stdout:7/472: rmdir d2/d8/da/d33/d25/d79 39 2024-06-15T01:07:02.620 INFO:tasks.workunit.client.1.smithi142.stdout:3/555: dwrite d0/d1f/d20/d5d/d82/f9a [0,4194304] 0 2024-06-15T01:07:02.620 INFO:tasks.workunit.client.1.smithi142.stdout:0/472: chown d3/d39/d9a/d9d/f9e 77 1 2024-06-15T01:07:02.620 INFO:tasks.workunit.client.1.smithi142.stdout:1/518: sync 2024-06-15T01:07:02.620 INFO:tasks.workunit.client.1.smithi142.stdout:8/443: mkdir d0/d25/d5a/d8f 0 2024-06-15T01:07:02.621 INFO:tasks.workunit.client.1.smithi142.stdout:6/459: creat d1a/d2d/d68/da0/fa5 x:0 0 0 2024-06-15T01:07:02.624 INFO:tasks.workunit.client.1.smithi142.stdout:6/460: chown d1a/d39/d3a/d51/f5c 72031725 1 2024-06-15T01:07:02.626 INFO:tasks.workunit.client.1.smithi142.stdout:0/473: write d3/d39/d3e/d81/d44/d7c/f19 [1985231,61860] 0 2024-06-15T01:07:02.626 INFO:tasks.workunit.client.1.smithi142.stdout:6/461: dread - d1a/d39/d42/f74 zero size 2024-06-15T01:07:02.627 INFO:tasks.workunit.client.1.smithi142.stdout:6/462: read - d1a/d39/d3a/d51/fa3 zero size 2024-06-15T01:07:02.629 INFO:tasks.workunit.client.1.smithi142.stdout:7/473: creat d2/d8/dd/d14/d2c/d37/d6b/d80/f9c x:0 0 0 2024-06-15T01:07:02.631 INFO:tasks.workunit.client.1.smithi142.stdout:3/556: rename d0/d1b/f2a to d0/d1f/d20/d59/d61/d96/faa 0 2024-06-15T01:07:02.632 INFO:tasks.workunit.client.1.smithi142.stdout:3/557: write d0/d3/da/d71/f97 [747150,7942] 0 2024-06-15T01:07:02.633 INFO:tasks.workunit.client.1.smithi142.stdout:0/474: unlink d3/d39/d3e/d81/f6d 0 2024-06-15T01:07:02.634 INFO:tasks.workunit.client.1.smithi142.stdout:0/475: stat d3/d39/d3e/d81/d4c 0 2024-06-15T01:07:02.635 INFO:tasks.workunit.client.1.smithi142.stdout:6/463: fdatasync d1a/d39/f57 0 2024-06-15T01:07:02.637 INFO:tasks.workunit.client.1.smithi142.stdout:7/474: symlink d2/d8/da/l9d 0 2024-06-15T01:07:02.639 INFO:tasks.workunit.client.1.smithi142.stdout:6/464: read d1a/d39/d42/d4c/d58/f7a [2989854,117023] 0 2024-06-15T01:07:02.640 INFO:tasks.workunit.client.1.smithi142.stdout:0/476: mknod d3/d39/d3e/d81/d4c/d61/cb0 0 2024-06-15T01:07:02.640 INFO:tasks.workunit.client.1.smithi142.stdout:3/558: dread d0/d1f/d20/d5d/d82/f9a [0,4194304] 0 2024-06-15T01:07:02.640 INFO:tasks.workunit.client.1.smithi142.stdout:4/580: truncate d5/d9/d23/d32/f8f 723229 0 2024-06-15T01:07:02.645 INFO:tasks.workunit.client.1.smithi142.stdout:3/559: rmdir d0 39 2024-06-15T01:07:02.648 INFO:tasks.workunit.client.1.smithi142.stdout:8/444: truncate d0/f33 4134841 0 2024-06-15T01:07:02.648 INFO:tasks.workunit.client.1.smithi142.stdout:0/477: mknod d3/d39/d3e/d81/d1b/cb1 0 2024-06-15T01:07:02.649 INFO:tasks.workunit.client.1.smithi142.stdout:0/478: fsync d3/f23 0 2024-06-15T01:07:02.650 INFO:tasks.workunit.client.1.smithi142.stdout:6/465: unlink d1a/l27 0 2024-06-15T01:07:02.650 INFO:tasks.workunit.client.1.smithi142.stdout:0/479: dread - d3/d39/d3e/d81/d1b/d38/f96 zero size 2024-06-15T01:07:02.651 INFO:tasks.workunit.client.1.smithi142.stdout:4/581: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/fce x:0 0 0 2024-06-15T01:07:02.651 INFO:tasks.workunit.client.1.smithi142.stdout:8/445: dread d0/f43 [0,4194304] 0 2024-06-15T01:07:02.655 INFO:tasks.workunit.client.1.smithi142.stdout:7/475: truncate d2/d8/dd/f12 2867128 0 2024-06-15T01:07:02.656 INFO:tasks.workunit.client.1.smithi142.stdout:0/480: stat d3/d39/d3e/d81/d1b/l7a 0 2024-06-15T01:07:02.658 INFO:tasks.workunit.client.1.smithi142.stdout:7/476: dread d2/d8/f2d [0,4194304] 0 2024-06-15T01:07:02.658 INFO:tasks.workunit.client.1.smithi142.stdout:8/446: rmdir d0/d17 39 2024-06-15T01:07:02.661 INFO:tasks.workunit.client.1.smithi142.stdout:3/560: rename d0/d1f/f69 to d0/d3/da/d71/fab 0 2024-06-15T01:07:02.663 INFO:tasks.workunit.client.1.smithi142.stdout:4/582: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/ccf 0 2024-06-15T01:07:02.667 INFO:tasks.workunit.client.1.smithi142.stdout:7/477: chown d2/d8/da/d33/d25/d79/f92 5301 1 2024-06-15T01:07:02.670 INFO:tasks.workunit.client.1.smithi142.stdout:3/561: stat d0/d3/l4c 0 2024-06-15T01:07:02.673 INFO:tasks.workunit.client.1.smithi142.stdout:3/562: chown d0/d3/da/d71/d55/ca0 52 1 2024-06-15T01:07:02.675 INFO:tasks.workunit.client.1.smithi142.stdout:6/466: getdents d1a/d39/d42/d4c 0 2024-06-15T01:07:02.677 INFO:tasks.workunit.client.1.smithi142.stdout:7/478: dwrite d2/d6/d22/d47/f7a [0,4194304] 0 2024-06-15T01:07:02.677 INFO:tasks.workunit.client.1.smithi142.stdout:3/563: rmdir d0 39 2024-06-15T01:07:02.678 INFO:tasks.workunit.client.1.smithi142.stdout:0/481: truncate d3/d39/d3e/d81/fb 3099458 0 2024-06-15T01:07:02.680 INFO:tasks.workunit.client.1.smithi142.stdout:7/479: dread - d2/d6/d22/d47/f7c zero size 2024-06-15T01:07:02.680 INFO:tasks.workunit.client.1.smithi142.stdout:6/467: write d1a/d39/d42/d4c/d58/d87/fa4 [166051,90594] 0 2024-06-15T01:07:02.682 INFO:tasks.workunit.client.1.smithi142.stdout:3/564: fsync d0/d3/da/d71/d75/fa7 0 2024-06-15T01:07:02.684 INFO:tasks.workunit.client.1.smithi142.stdout:8/447: dread d0/d17/d40/f65 [0,4194304] 0 2024-06-15T01:07:02.684 INFO:tasks.workunit.client.1.smithi142.stdout:6/468: write d1a/f38 [3324209,91586] 0 2024-06-15T01:07:02.685 INFO:tasks.workunit.client.1.smithi142.stdout:7/480: rmdir d2/d8/dd/d14/d2c 39 2024-06-15T01:07:02.689 INFO:tasks.workunit.client.1.smithi142.stdout:3/565: write d0/d3/da/d71/f97 [1165559,46443] 0 2024-06-15T01:07:02.692 INFO:tasks.workunit.client.1.smithi142.stdout:0/482: dwrite d3/d39/d3e/fa5 [0,4194304] 0 2024-06-15T01:07:02.692 INFO:tasks.workunit.client.1.smithi142.stdout:8/448: rename d0/c59 to d0/d17/d40/d86/c90 0 2024-06-15T01:07:02.693 INFO:tasks.workunit.client.1.smithi142.stdout:6/469: read d1a/d39/d3a/d51/d62/f79 [420684,122127] 0 2024-06-15T01:07:02.695 INFO:tasks.workunit.client.1.smithi142.stdout:8/449: stat d0/d25/d5a/f8c 0 2024-06-15T01:07:02.696 INFO:tasks.workunit.client.1.smithi142.stdout:3/566: unlink d0/d1f/d20/d5d/d82/d6e/f93 0 2024-06-15T01:07:02.702 INFO:tasks.workunit.client.1.smithi142.stdout:8/450: symlink d0/d25/l91 0 2024-06-15T01:07:02.703 INFO:tasks.workunit.client.1.smithi142.stdout:8/451: chown d0/d25/d5a/l2e 27 1 2024-06-15T01:07:02.704 INFO:tasks.workunit.client.1.smithi142.stdout:6/470: mknod d1a/d39/d42/d4c/ca6 0 2024-06-15T01:07:02.706 INFO:tasks.workunit.client.1.smithi142.stdout:7/481: dread d2/d8/da/d33/d25/d27/f2f [0,4194304] 0 2024-06-15T01:07:02.707 INFO:tasks.workunit.client.1.smithi142.stdout:0/483: rename d3/f37 to d3/d39/d3e/d81/d4c/da0/fb2 0 2024-06-15T01:07:02.710 INFO:tasks.workunit.client.1.smithi142.stdout:6/471: creat d1a/d3c/fa7 x:0 0 0 2024-06-15T01:07:02.711 INFO:tasks.workunit.client.1.smithi142.stdout:3/567: symlink d0/d3/lac 0 2024-06-15T01:07:02.711 INFO:tasks.workunit.client.1.smithi142.stdout:7/482: read d2/d8/da/d33/d25/f16 [1756584,90170] 0 2024-06-15T01:07:02.713 INFO:tasks.workunit.client.1.smithi142.stdout:7/483: chown d2/d8/da/d33/d25/d27 0 1 2024-06-15T01:07:02.715 INFO:tasks.workunit.client.1.smithi142.stdout:8/452: write d0/d17/f56 [683430,111827] 0 2024-06-15T01:07:02.715 INFO:tasks.workunit.client.1.smithi142.stdout:6/472: chown d1a/l2b 63 1 2024-06-15T01:07:02.717 INFO:tasks.workunit.client.1.smithi142.stdout:0/484: dwrite d3/d39/d3e/d81/d44/d7c/faa [0,4194304] 0 2024-06-15T01:07:02.717 INFO:tasks.workunit.client.1.smithi142.stdout:8/453: rmdir d0/d17/d40 39 2024-06-15T01:07:02.718 INFO:tasks.workunit.client.1.smithi142.stdout:3/568: write d0/d3/da/f7b [986430,107270] 0 2024-06-15T01:07:02.719 INFO:tasks.workunit.client.1.smithi142.stdout:6/473: stat d1a/d39/d3a/f59 0 2024-06-15T01:07:02.721 INFO:tasks.workunit.client.1.smithi142.stdout:0/485: fdatasync d3/d39/d3e/d81/d44/f5a 0 2024-06-15T01:07:02.722 INFO:tasks.workunit.client.1.smithi142.stdout:3/569: rmdir d0/d3/da/d71/d55 39 2024-06-15T01:07:02.722 INFO:tasks.workunit.client.1.smithi142.stdout:8/454: getdents d0/d25/d53 0 2024-06-15T01:07:02.724 INFO:tasks.workunit.client.1.smithi142.stdout:0/486: stat d3/d39/d3e/d81/d1b/dae 0 2024-06-15T01:07:02.725 INFO:tasks.workunit.client.1.smithi142.stdout:7/484: dwrite d2/d6/d22/d3c/f56 [0,4194304] 0 2024-06-15T01:07:02.728 INFO:tasks.workunit.client.1.smithi142.stdout:6/474: truncate d1a/f28 351629 0 2024-06-15T01:07:02.728 INFO:tasks.workunit.client.1.smithi142.stdout:3/570: write d0/d3/d15/f9f [161492,74247] 0 2024-06-15T01:07:02.729 INFO:tasks.workunit.client.1.smithi142.stdout:0/487: mknod d3/d39/d3e/d81/d1b/d38/da3/cb3 0 2024-06-15T01:07:02.730 INFO:tasks.workunit.client.1.smithi142.stdout:8/455: creat d0/d25/d66/f92 x:0 0 0 2024-06-15T01:07:02.733 INFO:tasks.workunit.client.1.smithi142.stdout:0/488: mkdir d3/d39/d3e/db4 0 2024-06-15T01:07:02.733 INFO:tasks.workunit.client.1.smithi142.stdout:6/475: mkdir d1a/d39/d3a/d51/d62/da8 0 2024-06-15T01:07:02.733 INFO:tasks.workunit.client.1.smithi142.stdout:8/456: truncate d0/d25/d66/f92 226100 0 2024-06-15T01:07:02.736 INFO:tasks.workunit.client.1.smithi142.stdout:6/476: readlink d1a/l2b 0 2024-06-15T01:07:02.738 INFO:tasks.workunit.client.1.smithi142.stdout:3/571: rename d0/d1f/d20/d59/f80 to d0/d1f/d20/fad 0 2024-06-15T01:07:02.740 INFO:tasks.workunit.client.1.smithi142.stdout:6/477: chown d1a/l29 11953861 1 2024-06-15T01:07:02.741 INFO:tasks.workunit.client.1.smithi142.stdout:7/485: dwrite d2/d8/da/d33/d25/d27/f2f [0,4194304] 0 2024-06-15T01:07:02.743 INFO:tasks.workunit.client.1.smithi142.stdout:0/489: rename d3/d39/d3e/d81/d44/d7c/f19 to d3/d39/d3e/d81/d4c/d61/fb5 0 2024-06-15T01:07:02.744 INFO:tasks.workunit.client.1.smithi142.stdout:8/457: creat d0/d17/d40/d6e/f93 x:0 0 0 2024-06-15T01:07:02.744 INFO:tasks.workunit.client.1.smithi142.stdout:7/486: rmdir d2/d8/dd 39 2024-06-15T01:07:02.745 INFO:tasks.workunit.client.1.smithi142.stdout:8/458: chown d0/d17/c5c 2 1 2024-06-15T01:07:02.745 INFO:tasks.workunit.client.1.smithi142.stdout:3/572: mkdir d0/d3/da/d71/d55/dae 0 2024-06-15T01:07:02.747 INFO:tasks.workunit.client.1.smithi142.stdout:6/478: rename d1a/d39/d3a/f5a to d1a/d2d/d3b/fa9 0 2024-06-15T01:07:02.747 INFO:tasks.workunit.client.1.smithi142.stdout:6/479: stat d1a/d39/d42/f74 0 2024-06-15T01:07:02.748 INFO:tasks.workunit.client.1.smithi142.stdout:0/490: truncate d3/d39/d3e/d81/f73 51027 0 2024-06-15T01:07:02.749 INFO:tasks.workunit.client.1.smithi142.stdout:8/459: creat d0/d17/d40/d5e/f94 x:0 0 0 2024-06-15T01:07:02.751 INFO:tasks.workunit.client.1.smithi142.stdout:3/573: creat d0/d3/da/d71/d55/faf x:0 0 0 2024-06-15T01:07:02.752 INFO:tasks.workunit.client.1.smithi142.stdout:7/487: dwrite d2/d8/f42 [0,4194304] 0 2024-06-15T01:07:02.754 INFO:tasks.workunit.client.1.smithi142.stdout:0/491: mknod d3/d39/d3e/d81/d1e/d98/cb6 0 2024-06-15T01:07:02.755 INFO:tasks.workunit.client.1.smithi142.stdout:8/460: creat d0/d25/d66/f95 x:0 0 0 2024-06-15T01:07:02.755 INFO:tasks.workunit.client.1.smithi142.stdout:3/574: creat d0/d1f/d20/d5d/d60/fb0 x:0 0 0 2024-06-15T01:07:02.756 INFO:tasks.workunit.client.1.smithi142.stdout:8/461: fsync d0/d25/d5a/f69 0 2024-06-15T01:07:02.757 INFO:tasks.workunit.client.1.smithi142.stdout:0/492: truncate d3/d39/d3e/d81/f7e 947130 0 2024-06-15T01:07:02.758 INFO:tasks.workunit.client.1.smithi142.stdout:7/488: creat d2/d8/da/d33/f9e x:0 0 0 2024-06-15T01:07:02.759 INFO:tasks.workunit.client.1.smithi142.stdout:8/462: read d0/d25/d66/f92 [205815,99208] 0 2024-06-15T01:07:02.762 INFO:tasks.workunit.client.1.smithi142.stdout:0/493: symlink d3/d39/d9a/lb7 0 2024-06-15T01:07:02.766 INFO:tasks.workunit.client.1.smithi142.stdout:8/463: rename d0/d25/l91 to d0/d25/d66/l96 0 2024-06-15T01:07:02.766 INFO:tasks.workunit.client.1.smithi142.stdout:8/464: readlink d0/d25/l68 0 2024-06-15T01:07:02.771 INFO:tasks.workunit.client.1.smithi142.stdout:3/575: getdents d0/d1f/d20/d5d/d60/d72 0 2024-06-15T01:07:02.773 INFO:tasks.workunit.client.1.smithi142.stdout:0/494: dwrite d3/d39/d3e/d81/d44/d7c/d22/f65 [0,4194304] 0 2024-06-15T01:07:02.778 INFO:tasks.workunit.client.1.smithi142.stdout:6/480: write d1a/d2d/f33 [943961,87123] 0 2024-06-15T01:07:02.778 INFO:tasks.workunit.client.1.smithi142.stdout:2/434: sync 2024-06-15T01:07:02.780 INFO:tasks.workunit.client.1.smithi142.stdout:3/576: dwrite d0/d3/da/d71/d55/faf [0,4194304] 0 2024-06-15T01:07:02.782 INFO:tasks.workunit.client.1.smithi142.stdout:3/577: chown d0/d3/da/d71/d55/dae 4033359 1 2024-06-15T01:07:02.787 INFO:tasks.workunit.client.1.smithi142.stdout:0/495: link d3/d39/d3e/d81/d44/d7c/d22/f2d d3/d39/d9a/fb8 0 2024-06-15T01:07:02.789 INFO:tasks.workunit.client.1.smithi142.stdout:3/578: write d0/d1f/d20/d5d/d82/f9a [2104687,35378] 0 2024-06-15T01:07:02.790 INFO:tasks.workunit.client.1.smithi142.stdout:8/465: write d0/f33 [3924522,119967] 0 2024-06-15T01:07:02.791 INFO:tasks.workunit.client.1.smithi142.stdout:3/579: read - d0/d1f/d20/d59/f8e zero size 2024-06-15T01:07:02.792 INFO:tasks.workunit.client.1.smithi142.stdout:8/466: write d0/f43 [1259485,15250] 0 2024-06-15T01:07:02.792 INFO:tasks.workunit.client.1.smithi142.stdout:8/467: stat d0/f3f 0 2024-06-15T01:07:02.792 INFO:tasks.workunit.client.1.smithi142.stdout:3/580: stat d0/d3/d15/c67 0 2024-06-15T01:07:02.793 INFO:tasks.workunit.client.1.smithi142.stdout:2/435: unlink d8/d19/d23/d25/f49 0 2024-06-15T01:07:02.796 INFO:tasks.workunit.client.1.smithi142.stdout:2/436: readlink d8/df/d10/d1c/l2f 0 2024-06-15T01:07:02.798 INFO:tasks.workunit.client.1.smithi142.stdout:3/581: symlink d0/d1f/d20/d59/d61/d96/lb1 0 2024-06-15T01:07:02.799 INFO:tasks.workunit.client.1.smithi142.stdout:8/468: mkdir d0/d25/d5a/d8f/d97 0 2024-06-15T01:07:02.799 INFO:tasks.workunit.client.1.smithi142.stdout:3/582: chown d0/d3/da/c45 25641296 1 2024-06-15T01:07:02.800 INFO:tasks.workunit.client.1.smithi142.stdout:0/496: getdents d3 0 2024-06-15T01:07:02.800 INFO:tasks.workunit.client.1.smithi142.stdout:3/583: stat d0/d1f/d20/c7f 0 2024-06-15T01:07:02.801 INFO:tasks.workunit.client.1.smithi142.stdout:8/469: mkdir d0/d25/d5a/d8f/d98 0 2024-06-15T01:07:02.804 INFO:tasks.workunit.client.1.smithi142.stdout:3/584: mknod d0/d1f/d20/d5d/d60/d99/cb2 0 2024-06-15T01:07:02.805 INFO:tasks.workunit.client.1.smithi142.stdout:8/470: mknod d0/d17/d40/d86/c99 0 2024-06-15T01:07:02.806 INFO:tasks.workunit.client.1.smithi142.stdout:0/497: rename d3/d39/d3e/d81/d1e/l27 to d3/d39/d3e/d81/d44/lb9 0 2024-06-15T01:07:02.807 INFO:tasks.workunit.client.1.smithi142.stdout:2/437: dwrite d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:07:02.810 INFO:tasks.workunit.client.1.smithi142.stdout:0/498: unlink d3/d39/d3e/d81/d1b/d38/f8c 0 2024-06-15T01:07:02.814 INFO:tasks.workunit.client.1.smithi142.stdout:2/438: mknod d8/df/d10/c8e 0 2024-06-15T01:07:02.815 INFO:tasks.workunit.client.1.smithi142.stdout:6/481: dread d1a/d39/d3a/d51/f5c [0,4194304] 0 2024-06-15T01:07:02.816 INFO:tasks.workunit.client.1.smithi142.stdout:8/471: dread d0/d25/d5a/f4d [0,4194304] 0 2024-06-15T01:07:02.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:02 smithi142 ceph-mon[37379]: Adjusting osd_memory_target on smithi142 to 2240M 2024-06-15T01:07:02.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:02 smithi142 ceph-mon[37379]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:07:02.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:02 smithi142 ceph-mon[37379]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:07:02.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:02 smithi142 ceph-mon[37379]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:07:02.818 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:02 smithi142 ceph-mon[37379]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:07:02.819 INFO:tasks.workunit.client.1.smithi142.stdout:6/482: mkdir d1a/d83/daa 0 2024-06-15T01:07:02.819 INFO:tasks.workunit.client.1.smithi142.stdout:3/585: dwrite d0/d1f/d20/d5d/d60/f85 [0,4194304] 0 2024-06-15T01:07:02.819 INFO:tasks.workunit.client.1.smithi142.stdout:2/439: symlink d8/d19/d23/d25/d1a/d63/l8f 0 2024-06-15T01:07:02.824 INFO:tasks.workunit.client.1.smithi142.stdout:0/499: dwrite d3/f6e [0,4194304] 0 2024-06-15T01:07:02.825 INFO:tasks.workunit.client.1.smithi142.stdout:0/500: fsync d3/f23 0 2024-06-15T01:07:02.826 INFO:tasks.workunit.client.1.smithi142.stdout:0/501: readlink d3/d39/d3e/d81/d1b/l7a 0 2024-06-15T01:07:02.877 INFO:tasks.workunit.client.1.smithi142.stdout:6/483: mkdir d1a/d2d/d3b/dab 0 2024-06-15T01:07:02.877 INFO:tasks.workunit.client.1.smithi142.stdout:6/484: dread - d1a/d39/d3a/d51/fa3 zero size 2024-06-15T01:07:02.878 INFO:tasks.workunit.client.1.smithi142.stdout:3/586: creat d0/d1f/d20/d59/d61/d96/fb3 x:0 0 0 2024-06-15T01:07:02.879 INFO:tasks.workunit.client.1.smithi142.stdout:6/485: write d1a/d2d/d68/da0/fa5 [858554,108882] 0 2024-06-15T01:07:02.880 INFO:tasks.workunit.client.1.smithi142.stdout:0/502: creat d3/d39/d3e/d81/d44/d7c/d22/d59/fba x:0 0 0 2024-06-15T01:07:02.880 INFO:tasks.workunit.client.1.smithi142.stdout:3/587: write d0/d3/da/f7b [2090243,33943] 0 2024-06-15T01:07:02.882 INFO:tasks.workunit.client.1.smithi142.stdout:2/440: creat d8/df/d10/d1c/d7c/f90 x:0 0 0 2024-06-15T01:07:02.883 INFO:tasks.workunit.client.1.smithi142.stdout:8/472: getdents d0/d17/d40/d86 0 2024-06-15T01:07:02.884 INFO:tasks.workunit.client.1.smithi142.stdout:8/473: chown d0/d17/d40/d86 6261 1 2024-06-15T01:07:02.885 INFO:tasks.workunit.client.1.smithi142.stdout:6/486: creat d1a/d2d/d68/fac x:0 0 0 2024-06-15T01:07:02.886 INFO:tasks.workunit.client.1.smithi142.stdout:0/503: rmdir d3/d39/d9a/d9d 39 2024-06-15T01:07:02.888 INFO:tasks.workunit.client.1.smithi142.stdout:0/504: chown d3/l7b 1952854212 1 2024-06-15T01:07:02.888 INFO:tasks.workunit.client.1.smithi142.stdout:2/441: mknod d8/df/d10/d1c/c91 0 2024-06-15T01:07:02.889 INFO:tasks.workunit.client.1.smithi142.stdout:3/588: dwrite d0/d1f/f27 [0,4194304] 0 2024-06-15T01:07:02.889 INFO:tasks.workunit.client.1.smithi142.stdout:2/442: stat d8/d19/f56 0 2024-06-15T01:07:02.889 INFO:tasks.workunit.client.1.smithi142.stdout:6/487: creat d1a/d2d/d68/da0/fad x:0 0 0 2024-06-15T01:07:02.890 INFO:tasks.workunit.client.1.smithi142.stdout:8/474: getdents d0/d25/d5a/d8f/d98 0 2024-06-15T01:07:02.891 INFO:tasks.workunit.client.1.smithi142.stdout:3/589: write d0/d1f/d20/fa3 [405374,117423] 0 2024-06-15T01:07:02.893 INFO:tasks.workunit.client.1.smithi142.stdout:0/505: rmdir d3/d39/d3e/d81/d1e/d98 39 2024-06-15T01:07:02.895 INFO:tasks.workunit.client.1.smithi142.stdout:8/475: symlink d0/d25/d53/l9a 0 2024-06-15T01:07:02.896 INFO:tasks.workunit.client.1.smithi142.stdout:0/506: chown d3/d39/d3e/d81/d4c/l99 2596443 1 2024-06-15T01:07:02.900 INFO:tasks.workunit.client.1.smithi142.stdout:5/484: sync 2024-06-15T01:07:02.900 INFO:tasks.workunit.client.1.smithi142.stdout:9/429: sync 2024-06-15T01:07:02.907 INFO:tasks.workunit.client.0.smithi137.stdout:5/514: sync 2024-06-15T01:07:02.909 INFO:tasks.workunit.client.0.smithi137.stdout:5/515: mknod df/d11/d14/cac 0 2024-06-15T01:07:02.911 INFO:tasks.workunit.client.1.smithi142.stdout:3/590: creat d0/d95/fb4 x:0 0 0 2024-06-15T01:07:02.911 INFO:tasks.workunit.client.1.smithi142.stdout:8/476: unlink d0/c34 0 2024-06-15T01:07:02.915 INFO:tasks.workunit.client.1.smithi142.stdout:0/507: fdatasync d3/d39/d3e/d81/d44/f49 0 2024-06-15T01:07:02.915 INFO:tasks.workunit.client.1.smithi142.stdout:6/488: dwrite d1a/d3c/fa7 [0,4194304] 0 2024-06-15T01:07:02.916 INFO:tasks.workunit.client.1.smithi142.stdout:0/508: dread - d3/d39/d3e/d81/d1b/d38/f9b zero size 2024-06-15T01:07:02.917 INFO:tasks.workunit.client.1.smithi142.stdout:8/477: rmdir d0/d17/d40 39 2024-06-15T01:07:02.918 INFO:tasks.workunit.client.1.smithi142.stdout:3/591: write d0/d3/f43 [1305013,15086] 0 2024-06-15T01:07:02.918 INFO:tasks.workunit.client.0.smithi137.stdout:5/516: dwrite df/d11/d30/f82 [0,4194304] 0 2024-06-15T01:07:02.921 INFO:tasks.workunit.client.1.smithi142.stdout:9/430: rename d6/c38 to d6/dc/d4e/cab 0 2024-06-15T01:07:02.927 INFO:tasks.workunit.client.1.smithi142.stdout:8/478: rmdir d0/d17/d40/d86 39 2024-06-15T01:07:02.928 INFO:tasks.workunit.client.1.smithi142.stdout:3/592: mkdir d0/d3/da/db5 0 2024-06-15T01:07:02.928 INFO:tasks.workunit.client.0.smithi137.stdout:5/517: rmdir df/d76/d7c/d9c/da8/da4 0 2024-06-15T01:07:02.930 INFO:tasks.workunit.client.1.smithi142.stdout:0/509: creat d3/d39/d9a/fbb x:0 0 0 2024-06-15T01:07:02.930 INFO:tasks.workunit.client.1.smithi142.stdout:8/479: rmdir d0/d25 39 2024-06-15T01:07:02.932 INFO:tasks.workunit.client.1.smithi142.stdout:3/593: chown d0/d1f/d20/d5d/d82/l89 14422750 1 2024-06-15T01:07:02.932 INFO:tasks.workunit.client.1.smithi142.stdout:6/489: creat d1a/d39/d42/d4c/d58/fae x:0 0 0 2024-06-15T01:07:02.934 INFO:tasks.workunit.client.1.smithi142.stdout:5/485: dwrite d2/f9 [0,4194304] 0 2024-06-15T01:07:02.934 INFO:tasks.workunit.client.1.smithi142.stdout:8/480: dread - d0/d25/d5a/f88 zero size 2024-06-15T01:07:02.934 INFO:tasks.workunit.client.0.smithi137.stdout:5/518: creat df/d76/d7c/d9c/da8/fad x:0 0 0 2024-06-15T01:07:02.935 INFO:tasks.workunit.client.1.smithi142.stdout:6/490: chown d1a/d39/d42/d4c/d58/f5d 1 1 2024-06-15T01:07:02.935 INFO:tasks.workunit.client.1.smithi142.stdout:5/486: stat d2/de/d27/f77 0 2024-06-15T01:07:02.936 INFO:tasks.workunit.client.1.smithi142.stdout:3/594: write d0/d1f/d20/d5d/d60/fb0 [871666,3086] 0 2024-06-15T01:07:02.937 INFO:tasks.workunit.client.1.smithi142.stdout:8/481: chown d0/d25/d5a/l2e 557 1 2024-06-15T01:07:02.938 INFO:tasks.workunit.client.1.smithi142.stdout:5/487: mkdir d2/d3e/d98 0 2024-06-15T01:07:02.939 INFO:tasks.workunit.client.0.smithi137.stdout:5/519: getdents df/d44/d54/d95/d3b/d49 0 2024-06-15T01:07:02.939 INFO:tasks.workunit.client.1.smithi142.stdout:3/595: symlink d0/d3/d15/lb6 0 2024-06-15T01:07:02.941 INFO:tasks.workunit.client.1.smithi142.stdout:0/510: dwrite d3/d39/d3e/d81/d4c/faf [0,4194304] 0 2024-06-15T01:07:02.950 INFO:tasks.workunit.client.1.smithi142.stdout:0/511: dwrite d3/d39/d3e/d81/d44/d7c/d22/f82 [0,4194304] 0 2024-06-15T01:07:02.952 INFO:tasks.workunit.client.0.smithi137.stdout:5/520: write df/d44/d54/d95/d35/f9b [3360112,47363] 0 2024-06-15T01:07:02.952 INFO:tasks.workunit.client.1.smithi142.stdout:6/491: truncate d1a/f25 1397828 0 2024-06-15T01:07:02.955 INFO:tasks.workunit.client.1.smithi142.stdout:6/492: creat d1a/d2d/d3b/faf x:0 0 0 2024-06-15T01:07:02.956 INFO:tasks.workunit.client.1.smithi142.stdout:9/431: dread d6/dc/dd/f46 [0,4194304] 0 2024-06-15T01:07:02.959 INFO:tasks.workunit.client.1.smithi142.stdout:0/512: rename d3/d39/f45 to d3/d39/d9a/da2/fbc 0 2024-06-15T01:07:02.963 INFO:tasks.workunit.client.1.smithi142.stdout:9/432: unlink d6/d34/l9c 0 2024-06-15T01:07:02.963 INFO:tasks.workunit.client.1.smithi142.stdout:6/493: dread d1a/d2d/f2f [0,4194304] 0 2024-06-15T01:07:02.964 INFO:tasks.workunit.client.1.smithi142.stdout:6/494: mkdir d1a/d3c/db0 0 2024-06-15T01:07:02.967 INFO:tasks.workunit.client.1.smithi142.stdout:6/495: symlink d1a/d2d/d3b/lb1 0 2024-06-15T01:07:02.968 INFO:tasks.workunit.client.0.smithi137.stdout:5/521: dwrite df/d44/d54/d95/d3b/d4b/f63 [0,4194304] 0 2024-06-15T01:07:02.969 INFO:tasks.workunit.client.1.smithi142.stdout:6/496: mknod d1a/d2d/d68/cb2 0 2024-06-15T01:07:02.969 INFO:tasks.workunit.client.1.smithi142.stdout:6/497: chown d1a/d39/d42/f74 107 1 2024-06-15T01:07:02.970 INFO:tasks.workunit.client.1.smithi142.stdout:6/498: stat d1a/d2d/d80/d8e/l91 0 2024-06-15T01:07:02.971 INFO:tasks.workunit.client.1.smithi142.stdout:6/499: write d1a/f1e [1374292,33187] 0 2024-06-15T01:07:02.973 INFO:tasks.workunit.client.0.smithi137.stdout:5/522: mknod df/d11/d70/cae 0 2024-06-15T01:07:02.975 INFO:tasks.workunit.client.1.smithi142.stdout:6/500: fsync d1a/d2d/d3b/fa9 0 2024-06-15T01:07:02.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:02 smithi137 ceph-mon[29354]: Adjusting osd_memory_target on smithi142 to 2240M 2024-06-15T01:07:02.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:02 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:07:02.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:02 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:07:02.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:02 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:07:02.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:02 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:07:02.986 INFO:tasks.workunit.client.1.smithi142.stdout:9/433: dread d6/dc/d1b/d25/d58/f87 [0,4194304] 0 2024-06-15T01:07:02.987 INFO:tasks.workunit.client.1.smithi142.stdout:0/513: creat d3/d39/d3e/d81/d44/d7c/fbd x:0 0 0 2024-06-15T01:07:02.988 INFO:tasks.workunit.client.1.smithi142.stdout:0/514: fdatasync d3/d39/d3e/d81/d44/d7c/faa 0 2024-06-15T01:07:02.988 INFO:tasks.workunit.client.1.smithi142.stdout:9/434: creat d6/dc/dd/fac x:0 0 0 2024-06-15T01:07:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:5/523: dwrite df/d3d/f5f [0,4194304] 0 2024-06-15T01:07:02.993 INFO:tasks.workunit.client.0.smithi137.stdout:5/524: symlink df/d44/d54/laf 0 2024-06-15T01:07:02.996 INFO:tasks.workunit.client.0.smithi137.stdout:5/525: truncate df/d11/d70/fab 1280689 0 2024-06-15T01:07:02.996 INFO:tasks.workunit.client.1.smithi142.stdout:6/501: dwrite d1a/f25 [0,4194304] 0 2024-06-15T01:07:02.996 INFO:tasks.workunit.client.1.smithi142.stdout:0/515: rename d3/d39 to d3/d39/d9a/d9d/dbe 22 2024-06-15T01:07:02.997 INFO:tasks.workunit.client.1.smithi142.stdout:0/516: stat d3/d39/d3e/d81/d1b/d86 0 2024-06-15T01:07:02.998 INFO:tasks.workunit.client.1.smithi142.stdout:0/517: fsync d3/d39/d3e/f5f 0 2024-06-15T01:07:02.999 INFO:tasks.workunit.client.1.smithi142.stdout:0/518: dread - d3/d39/d3e/d81/d4c/d61/f66 zero size 2024-06-15T01:07:03.000 INFO:tasks.workunit.client.1.smithi142.stdout:9/435: creat d6/d34/fad x:0 0 0 2024-06-15T01:07:03.001 INFO:tasks.workunit.client.1.smithi142.stdout:6/502: creat d1a/d39/d42/d4c/fb3 x:0 0 0 2024-06-15T01:07:03.017 INFO:tasks.workunit.client.0.smithi137.stdout:7/368: sync 2024-06-15T01:07:03.017 INFO:tasks.workunit.client.0.smithi137.stdout:7/369: readlink df/d13/d2e/l3e 0 2024-06-15T01:07:03.018 INFO:tasks.workunit.client.0.smithi137.stdout:5/526: dread df/d11/f24 [0,4194304] 0 2024-06-15T01:07:03.018 INFO:tasks.workunit.client.0.smithi137.stdout:7/370: chown df/d13/d2e/d1a 1590 1 2024-06-15T01:07:03.018 INFO:tasks.workunit.client.0.smithi137.stdout:5/527: readlink df/d76/d8e/la6 0 2024-06-15T01:07:03.019 INFO:tasks.workunit.client.0.smithi137.stdout:5/528: chown df/l1b 4170 1 2024-06-15T01:07:03.021 INFO:tasks.workunit.client.0.smithi137.stdout:5/529: truncate df/d44/d54/d95/d3b/f59 1252633 0 2024-06-15T01:07:03.022 INFO:tasks.workunit.client.0.smithi137.stdout:7/371: creat df/d18/d1c/f74 x:0 0 0 2024-06-15T01:07:03.023 INFO:tasks.workunit.client.0.smithi137.stdout:7/372: fdatasync df/d13/d2e/f19 0 2024-06-15T01:07:03.023 INFO:tasks.workunit.client.0.smithi137.stdout:7/373: dread - df/d13/d2e/d1a/f68 zero size 2024-06-15T01:07:03.026 INFO:tasks.workunit.client.0.smithi137.stdout:5/530: link df/d44/d54/d95/l40 df/d44/d54/d95/lb0 0 2024-06-15T01:07:03.031 INFO:tasks.workunit.client.0.smithi137.stdout:7/374: dwrite df/d13/d2e/d1a/f68 [0,4194304] 0 2024-06-15T01:07:03.032 INFO:tasks.workunit.client.0.smithi137.stdout:7/375: fdatasync df/d13/d2e/d1a/f2c 0 2024-06-15T01:07:03.034 INFO:tasks.workunit.client.0.smithi137.stdout:7/376: chown df/d13/d2e/d1a/c46 0 1 2024-06-15T01:07:03.034 INFO:tasks.workunit.client.1.smithi142.stdout:7/489: sync 2024-06-15T01:07:03.034 INFO:tasks.workunit.client.1.smithi142.stdout:1/519: sync 2024-06-15T01:07:03.034 INFO:tasks.workunit.client.1.smithi142.stdout:4/583: sync 2024-06-15T01:07:03.035 INFO:tasks.workunit.client.0.smithi137.stdout:5/531: write df/d11/f6d [84025,113724] 0 2024-06-15T01:07:03.039 INFO:tasks.workunit.client.0.smithi137.stdout:5/532: dread - df/d44/f4d zero size 2024-06-15T01:07:03.041 INFO:tasks.workunit.client.0.smithi137.stdout:7/377: link df/d18/d1c/d1d/d27/c2a df/d13/d2e/c75 0 2024-06-15T01:07:03.042 INFO:tasks.workunit.client.0.smithi137.stdout:7/378: write df/d13/f20 [3347688,129766] 0 2024-06-15T01:07:03.042 INFO:tasks.workunit.client.1.smithi142.stdout:6/503: chown d1a/d39/l97 1 1 2024-06-15T01:07:03.043 INFO:tasks.workunit.client.1.smithi142.stdout:7/490: dread - d2/d8/da/d33/f52 zero size 2024-06-15T01:07:03.044 INFO:tasks.workunit.client.1.smithi142.stdout:1/520: unlink d6/d46/cab 0 2024-06-15T01:07:03.046 INFO:tasks.workunit.client.0.smithi137.stdout:7/379: rename df/d18/f5a to df/d13/d2e/d1a/d3f/f76 0 2024-06-15T01:07:03.047 INFO:tasks.workunit.client.1.smithi142.stdout:9/436: dwrite d6/f6a [0,4194304] 0 2024-06-15T01:07:03.049 INFO:tasks.workunit.client.1.smithi142.stdout:1/521: truncate d6/d12/d3c/f9c 212650 0 2024-06-15T01:07:03.049 INFO:tasks.workunit.client.1.smithi142.stdout:6/504: creat d1a/d39/d3a/d51/d62/fb4 x:0 0 0 2024-06-15T01:07:03.049 INFO:tasks.workunit.client.1.smithi142.stdout:7/491: mkdir d2/d8/da/d9f 0 2024-06-15T01:07:03.049 INFO:tasks.workunit.client.0.smithi137.stdout:7/380: creat df/d13/d2e/d1a/f77 x:0 0 0 2024-06-15T01:07:03.050 INFO:tasks.workunit.client.1.smithi142.stdout:0/519: dwrite d3/d39/d3e/d81/fb [0,4194304] 0 2024-06-15T01:07:03.051 INFO:tasks.workunit.client.1.smithi142.stdout:4/584: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fd0 x:0 0 0 2024-06-15T01:07:03.056 INFO:tasks.workunit.client.1.smithi142.stdout:7/492: dread - d2/d8/da/d33/f52 zero size 2024-06-15T01:07:03.056 INFO:tasks.workunit.client.1.smithi142.stdout:6/505: truncate d1a/d2d/d2e/f31 656693 0 2024-06-15T01:07:03.056 INFO:tasks.workunit.client.1.smithi142.stdout:9/437: mkdir d6/dc/d4e/dae 0 2024-06-15T01:07:03.056 INFO:tasks.workunit.client.1.smithi142.stdout:9/438: chown d6/dc/d1b/c1f 688468 1 2024-06-15T01:07:03.059 INFO:tasks.workunit.client.1.smithi142.stdout:0/520: dwrite f1 [0,4194304] 0 2024-06-15T01:07:03.062 INFO:tasks.workunit.client.1.smithi142.stdout:9/439: write d6/dc/dd/f77 [3286955,33971] 0 2024-06-15T01:07:03.063 INFO:tasks.workunit.client.1.smithi142.stdout:4/585: creat d5/fd1 x:0 0 0 2024-06-15T01:07:03.063 INFO:tasks.workunit.client.0.smithi137.stdout:7/381: truncate df/d13/d2e/d1a/f68 3299926 0 2024-06-15T01:07:03.065 INFO:tasks.workunit.client.1.smithi142.stdout:4/586: write f2 [1632895,79223] 0 2024-06-15T01:07:03.065 INFO:tasks.workunit.client.1.smithi142.stdout:7/493: mknod d2/d8/dd/d14/d2c/d37/d6b/ca0 0 2024-06-15T01:07:03.066 INFO:tasks.workunit.client.1.smithi142.stdout:6/506: getdents d1a/d39/d3a 0 2024-06-15T01:07:03.067 INFO:tasks.workunit.client.0.smithi137.stdout:5/533: dread df/d44/d54/d95/d35/f81 [0,4194304] 0 2024-06-15T01:07:03.068 INFO:tasks.workunit.client.1.smithi142.stdout:6/507: read fc [2943003,28991] 0 2024-06-15T01:07:03.071 INFO:tasks.workunit.client.0.smithi137.stdout:5/534: mkdir df/d11/d30/db1 0 2024-06-15T01:07:03.071 INFO:tasks.workunit.client.0.smithi137.stdout:7/382: write df/d18/d1c/d1d/d27/f49 [494876,85853] 0 2024-06-15T01:07:03.072 INFO:tasks.workunit.client.1.smithi142.stdout:0/521: dwrite d3/d39/d3e/d81/d1b/d38/f96 [0,4194304] 0 2024-06-15T01:07:03.072 INFO:tasks.workunit.client.1.smithi142.stdout:4/587: read - d5/d9/da/f5a zero size 2024-06-15T01:07:03.073 INFO:tasks.workunit.client.1.smithi142.stdout:4/588: chown d5/d1c/c70 121399657 1 2024-06-15T01:07:03.074 INFO:tasks.workunit.client.1.smithi142.stdout:4/589: write d5/d1c/fbb [766502,19879] 0 2024-06-15T01:07:03.075 INFO:tasks.workunit.client.0.smithi137.stdout:7/383: dread - df/d18/d3b/d5b/f5d zero size 2024-06-15T01:07:03.075 INFO:tasks.workunit.client.1.smithi142.stdout:9/440: rename d6/dc/d4e/cab to d6/dc/d1b/d6f/da1/caf 0 2024-06-15T01:07:03.082 INFO:tasks.workunit.client.1.smithi142.stdout:0/522: dread d3/d39/d3e/d81/d1b/f26 [0,4194304] 0 2024-06-15T01:07:03.083 INFO:tasks.workunit.client.0.smithi137.stdout:5/535: dwrite fe [4194304,4194304] 0 2024-06-15T01:07:03.084 INFO:tasks.workunit.client.0.smithi137.stdout:9/236: sync 2024-06-15T01:07:03.086 INFO:tasks.workunit.client.0.smithi137.stdout:3/435: sync 2024-06-15T01:07:03.087 INFO:tasks.workunit.client.0.smithi137.stdout:6/267: sync 2024-06-15T01:07:03.087 INFO:tasks.workunit.client.0.smithi137.stdout:2/332: sync 2024-06-15T01:07:03.087 INFO:tasks.workunit.client.0.smithi137.stdout:8/358: sync 2024-06-15T01:07:03.087 INFO:tasks.workunit.client.0.smithi137.stdout:0/411: sync 2024-06-15T01:07:03.088 INFO:tasks.workunit.client.0.smithi137.stdout:4/393: sync 2024-06-15T01:07:03.088 INFO:tasks.workunit.client.0.smithi137.stdout:1/453: sync 2024-06-15T01:07:03.088 INFO:tasks.workunit.client.1.smithi142.stdout:9/441: creat d6/dc/d4e/d78/fb0 x:0 0 0 2024-06-15T01:07:03.089 INFO:tasks.workunit.client.1.smithi142.stdout:1/522: truncate d6/f26 2184134 0 2024-06-15T01:07:03.091 INFO:tasks.workunit.client.0.smithi137.stdout:1/454: fdatasync d0/d1f/d76/f80 0 2024-06-15T01:07:03.091 INFO:tasks.workunit.client.1.smithi142.stdout:7/494: mknod d2/d8/dd/d8a/d94/ca1 0 2024-06-15T01:07:03.095 INFO:tasks.workunit.client.0.smithi137.stdout:4/394: fdatasync dd/d15/d36/d45/d55/d8d/f39 0 2024-06-15T01:07:03.095 INFO:tasks.workunit.client.1.smithi142.stdout:7/495: truncate d2/d6/d22/d47/f7c 528833 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.1.smithi142.stdout:1/523: mknod d6/d12/d3c/d63/d8d/cbc 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.1.smithi142.stdout:9/442: dwrite d6/dc/dd/fac [0,4194304] 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.0.smithi137.stdout:6/268: dread d0/d47/f24 [0,4194304] 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.0.smithi137.stdout:4/395: dread dd/d15/d30/f40 [0,4194304] 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.0.smithi137.stdout:6/269: truncate d0/d1a/f2f 761560 0 2024-06-15T01:07:03.098 INFO:tasks.workunit.client.0.smithi137.stdout:6/270: fdatasync d0/d1a/f2f 0 2024-06-15T01:07:03.099 INFO:tasks.workunit.client.0.smithi137.stdout:4/396: fsync dd/f12 0 2024-06-15T01:07:03.100 INFO:tasks.workunit.client.1.smithi142.stdout:4/590: read d5/d9/d18/f2f [3622556,73321] 0 2024-06-15T01:07:03.101 INFO:tasks.workunit.client.0.smithi137.stdout:6/271: write d0/d5/d9/d29/f4b [159491,15408] 0 2024-06-15T01:07:03.102 INFO:tasks.workunit.client.1.smithi142.stdout:4/591: chown d5/d1c/d6f/dc7/d30/d46/fc6 155199637 1 2024-06-15T01:07:03.106 INFO:tasks.workunit.client.1.smithi142.stdout:9/443: mknod d6/dc/d51/d53/d8d/cb1 0 2024-06-15T01:07:03.108 INFO:tasks.workunit.client.1.smithi142.stdout:4/592: symlink d5/d9/d23/d32/ld2 0 2024-06-15T01:07:03.110 INFO:tasks.workunit.client.1.smithi142.stdout:6/508: dread f6 [0,4194304] 0 2024-06-15T01:07:03.112 INFO:tasks.workunit.client.1.smithi142.stdout:1/524: dread d6/d12/d3c/d63/f6f [0,4194304] 0 2024-06-15T01:07:03.112 INFO:tasks.workunit.client.1.smithi142.stdout:4/593: dread - d5/d9/da/f8b zero size 2024-06-15T01:07:03.112 INFO:tasks.workunit.client.1.smithi142.stdout:9/444: mknod d6/dc/cb2 0 2024-06-15T01:07:03.116 INFO:tasks.workunit.client.1.smithi142.stdout:9/445: creat d6/dc/d4e/d78/fb3 x:0 0 0 2024-06-15T01:07:03.117 INFO:tasks.workunit.client.1.smithi142.stdout:4/594: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/ld3 0 2024-06-15T01:07:03.119 INFO:tasks.workunit.client.1.smithi142.stdout:4/595: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f43 370808227 1 2024-06-15T01:07:03.119 INFO:tasks.workunit.client.1.smithi142.stdout:6/509: link d1a/d39/d42/d4c/d58/d87/f89 d1a/d39/fb5 0 2024-06-15T01:07:03.120 INFO:tasks.workunit.client.1.smithi142.stdout:1/525: creat d6/d25/d9e/fbd x:0 0 0 2024-06-15T01:07:03.122 INFO:tasks.workunit.client.1.smithi142.stdout:4/596: symlink d5/d9/d41/ld4 0 2024-06-15T01:07:03.122 INFO:tasks.workunit.client.1.smithi142.stdout:6/510: write d1a/d39/d42/d4c/d58/d87/fa4 [1051907,112672] 0 2024-06-15T01:07:03.131 INFO:tasks.workunit.client.1.smithi142.stdout:1/526: rename d6/d12/d1c/l74 to d6/d12/d1c/d1e/lbe 0 2024-06-15T01:07:03.133 INFO:tasks.workunit.client.1.smithi142.stdout:6/511: creat d1a/d83/daa/fb6 x:0 0 0 2024-06-15T01:07:03.134 INFO:tasks.workunit.client.1.smithi142.stdout:4/597: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/dca/cd5 0 2024-06-15T01:07:03.135 INFO:tasks.workunit.client.1.smithi142.stdout:6/512: dread - d1a/d39/d3a/d51/f9b zero size 2024-06-15T01:07:03.136 INFO:tasks.workunit.client.1.smithi142.stdout:7/496: dread d2/d8/dd/d14/d2c/d37/d6b/d80/d85/f8e [0,4194304] 0 2024-06-15T01:07:03.138 INFO:tasks.workunit.client.1.smithi142.stdout:4/598: creat d5/d9/d41/fd6 x:0 0 0 2024-06-15T01:07:03.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/397: symlink dd/d15/d48/d72/l97 0 2024-06-15T01:07:03.139 INFO:tasks.workunit.client.1.smithi142.stdout:7/497: stat d2/d8/dd/d14/l68 0 2024-06-15T01:07:03.139 INFO:tasks.workunit.client.0.smithi137.stdout:6/272: fdatasync d0/f2a 0 2024-06-15T01:07:03.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/398: chown dd/d15/d30/l3d 21793008 1 2024-06-15T01:07:03.140 INFO:tasks.workunit.client.0.smithi137.stdout:6/273: chown d0/d1a/d3e 89 1 2024-06-15T01:07:03.140 INFO:tasks.workunit.client.1.smithi142.stdout:1/527: dwrite d6/d12/f2d [4194304,4194304] 0 2024-06-15T01:07:03.142 INFO:tasks.workunit.client.0.smithi137.stdout:5/536: symlink df/d11/d30/db1/lb2 0 2024-06-15T01:07:03.143 INFO:tasks.workunit.client.1.smithi142.stdout:7/498: readlink d2/d6/l5c 0 2024-06-15T01:07:03.143 INFO:tasks.workunit.client.1.smithi142.stdout:6/513: dread d1a/f20 [0,4194304] 0 2024-06-15T01:07:03.145 INFO:tasks.workunit.client.0.smithi137.stdout:2/333: link d0/d42/d4d/d56/f4f d0/d42/d4d/f67 0 2024-06-15T01:07:03.145 INFO:tasks.workunit.client.1.smithi142.stdout:7/499: write d2/d8/dd/d14/d2c/d37/d6b/f8c [19618,112925] 0 2024-06-15T01:07:03.146 INFO:tasks.workunit.client.1.smithi142.stdout:1/528: unlink d6/d12/d5b/l8b 0 2024-06-15T01:07:03.148 INFO:tasks.workunit.client.1.smithi142.stdout:4/599: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/fd7 x:0 0 0 2024-06-15T01:07:03.149 INFO:tasks.workunit.client.1.smithi142.stdout:4/600: dread - d5/d9/d23/d32/fb4 zero size 2024-06-15T01:07:03.150 INFO:tasks.workunit.client.1.smithi142.stdout:6/514: mkdir d1a/d39/d42/d4c/d58/d87/db7 0 2024-06-15T01:07:03.150 INFO:tasks.workunit.client.1.smithi142.stdout:4/601: chown d5/d1c/d6f/dc7/d30/d46/d50/f5e 210 1 2024-06-15T01:07:03.153 INFO:tasks.workunit.client.1.smithi142.stdout:1/529: mkdir d6/d12/d80/d9b/db2/dbf 0 2024-06-15T01:07:03.154 INFO:tasks.workunit.client.1.smithi142.stdout:6/515: readlink d1a/l26 0 2024-06-15T01:07:03.155 INFO:tasks.workunit.client.0.smithi137.stdout:5/537: creat df/d44/d54/d6b/fb3 x:0 0 0 2024-06-15T01:07:03.157 INFO:tasks.workunit.client.1.smithi142.stdout:7/500: mkdir d2/da2 0 2024-06-15T01:07:03.162 INFO:tasks.workunit.client.0.smithi137.stdout:7/384: getdents df/d13/d2e 0 2024-06-15T01:07:03.165 INFO:tasks.workunit.client.1.smithi142.stdout:1/530: getdents d6/d25/d9e 0 2024-06-15T01:07:03.165 INFO:tasks.workunit.client.0.smithi137.stdout:5/538: dwrite df/d44/d54/d95/d3b/d49/f96 [0,4194304] 0 2024-06-15T01:07:03.170 INFO:tasks.workunit.client.1.smithi142.stdout:1/531: fdatasync d6/f2a 0 2024-06-15T01:07:03.173 INFO:tasks.workunit.client.1.smithi142.stdout:4/602: truncate d5/d1c/d6f/dc7/d30/d46/d50/f91 2815379 0 2024-06-15T01:07:03.174 INFO:tasks.workunit.client.1.smithi142.stdout:7/501: truncate d2/d8/da/d33/d25/f59 2249715 0 2024-06-15T01:07:03.174 INFO:tasks.workunit.client.0.smithi137.stdout:6/274: rmdir d0/d5/d46/d48 0 2024-06-15T01:07:03.177 INFO:tasks.workunit.client.1.smithi142.stdout:4/603: dread d5/d9/d18/f6e [0,4194304] 0 2024-06-15T01:07:03.178 INFO:tasks.workunit.client.0.smithi137.stdout:6/275: dread d0/d1a/f35 [0,4194304] 0 2024-06-15T01:07:03.180 INFO:tasks.workunit.client.0.smithi137.stdout:6/276: truncate d0/d47/d1d/f40 681548 0 2024-06-15T01:07:03.181 INFO:tasks.workunit.client.1.smithi142.stdout:1/532: creat d6/d12/d1c/fc0 x:0 0 0 2024-06-15T01:07:03.183 INFO:tasks.workunit.client.1.smithi142.stdout:1/533: read - d6/d46/d61/faa zero size 2024-06-15T01:07:03.188 INFO:tasks.workunit.client.0.smithi137.stdout:6/277: dwrite d0/d5/d9/f41 [0,4194304] 0 2024-06-15T01:07:03.189 INFO:tasks.workunit.client.0.smithi137.stdout:9/237: creat df/d12/f3e x:0 0 0 2024-06-15T01:07:03.191 INFO:tasks.workunit.client.1.smithi142.stdout:1/534: rename d6/d12/d1c/l81 to d6/d12/d3c/lc1 0 2024-06-15T01:07:03.192 INFO:tasks.workunit.client.1.smithi142.stdout:2/443: sync 2024-06-15T01:07:03.196 INFO:tasks.workunit.client.0.smithi137.stdout:6/278: symlink d0/d5/d9/l54 0 2024-06-15T01:07:03.196 INFO:tasks.workunit.client.1.smithi142.stdout:7/502: write d2/d8/da/d33/d25/f59 [2875778,6597] 0 2024-06-15T01:07:03.197 INFO:tasks.workunit.client.1.smithi142.stdout:7/503: fsync d2/f2e 0 2024-06-15T01:07:03.199 INFO:tasks.workunit.client.0.smithi137.stdout:6/279: write d0/d47/d1d/d2e/f38 [957117,69419] 0 2024-06-15T01:07:03.205 INFO:tasks.workunit.client.1.smithi142.stdout:2/444: truncate d8/df/d10/f11 444417 0 2024-06-15T01:07:03.209 INFO:tasks.workunit.client.1.smithi142.stdout:2/445: rename d8/d19/d23/d25/d22/f6d to d8/d19/d23/d25/d22/d43/d52/d7a/f92 0 2024-06-15T01:07:03.209 INFO:tasks.workunit.client.0.smithi137.stdout:6/280: dwrite d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:03.211 INFO:tasks.workunit.client.0.smithi137.stdout:6/281: fsync d0/d47/f1e 0 2024-06-15T01:07:03.212 INFO:tasks.workunit.client.1.smithi142.stdout:2/446: creat d8/d19/d8d/f93 x:0 0 0 2024-06-15T01:07:03.213 INFO:tasks.workunit.client.0.smithi137.stdout:6/282: fdatasync d0/f6 0 2024-06-15T01:07:03.215 INFO:tasks.workunit.client.0.smithi137.stdout:0/412: rename d1/d2/dc/d10/d22/l80 to d1/d2/l8c 0 2024-06-15T01:07:03.217 INFO:tasks.workunit.client.1.smithi142.stdout:1/535: write d6/d46/f78 [269614,43783] 0 2024-06-15T01:07:03.219 INFO:tasks.workunit.client.0.smithi137.stdout:2/334: write d0/da/f18 [3672753,50661] 0 2024-06-15T01:07:03.220 INFO:tasks.workunit.client.0.smithi137.stdout:2/335: dread - d0/d42/d49/f62 zero size 2024-06-15T01:07:03.221 INFO:tasks.workunit.client.0.smithi137.stdout:2/336: chown d0/dc 383 1 2024-06-15T01:07:03.221 INFO:tasks.workunit.client.1.smithi142.stdout:7/504: truncate d2/d6/f8f 1610222 0 2024-06-15T01:07:03.222 INFO:tasks.workunit.client.0.smithi137.stdout:0/413: mkdir d1/d2/dc/d10/d63/d8d 0 2024-06-15T01:07:03.222 INFO:tasks.workunit.client.1.smithi142.stdout:7/505: write d2/d6/f83 [481822,43115] 0 2024-06-15T01:07:03.222 INFO:tasks.workunit.client.0.smithi137.stdout:7/385: write df/d18/d1c/d1d/d27/d33/f52 [544449,4311] 0 2024-06-15T01:07:03.223 INFO:tasks.workunit.client.1.smithi142.stdout:1/536: mkdir d6/d12/d80/d9b/db2/dc2 0 2024-06-15T01:07:03.225 INFO:tasks.workunit.client.0.smithi137.stdout:5/539: rename df/d44/d54/d95/c48 to df/d11/d14/cb4 0 2024-06-15T01:07:03.226 INFO:tasks.workunit.client.1.smithi142.stdout:2/447: rename d8/d19/d23 to d8/df/d10/d94 0 2024-06-15T01:07:03.228 INFO:tasks.workunit.client.1.smithi142.stdout:7/506: creat d2/d6/d22/fa3 x:0 0 0 2024-06-15T01:07:03.228 INFO:tasks.workunit.client.0.smithi137.stdout:0/414: mknod d1/d2/dc/d10/d63/c8e 0 2024-06-15T01:07:03.229 INFO:tasks.workunit.client.1.smithi142.stdout:7/507: write d2/d6/d22/d47/f4f [41588,23078] 0 2024-06-15T01:07:03.231 INFO:tasks.workunit.client.0.smithi137.stdout:5/540: fdatasync df/d44/d54/d6b/f8c 0 2024-06-15T01:07:03.233 INFO:tasks.workunit.client.1.smithi142.stdout:1/537: rename d6/d12/d1c/d1e to d6/d46/dc3 0 2024-06-15T01:07:03.234 INFO:tasks.workunit.client.1.smithi142.stdout:1/538: stat d6/d46/d61/f76 0 2024-06-15T01:07:03.235 INFO:tasks.workunit.client.1.smithi142.stdout:7/508: truncate d2/d8/dd/d14/d2c/f62 83594 0 2024-06-15T01:07:03.236 INFO:tasks.workunit.client.0.smithi137.stdout:4/399: creat dd/d15/f98 x:0 0 0 2024-06-15T01:07:03.236 INFO:tasks.workunit.client.1.smithi142.stdout:7/509: truncate d2/d8/f84 377595 0 2024-06-15T01:07:03.236 INFO:tasks.workunit.client.0.smithi137.stdout:0/415: symlink d1/d2/dc/d6c/d88/l8f 0 2024-06-15T01:07:03.237 INFO:tasks.workunit.client.1.smithi142.stdout:7/510: write d2/f3 [4788665,99366] 0 2024-06-15T01:07:03.238 INFO:tasks.workunit.client.1.smithi142.stdout:7/511: write d2/d8/da/d33/d25/f59 [1900478,17241] 0 2024-06-15T01:07:03.239 INFO:tasks.workunit.client.1.smithi142.stdout:2/448: rename d8/df/d10/d94/d25/d22/d43/d52/f72 to d8/df/d10/d94/d25/d1a/d63/f95 0 2024-06-15T01:07:03.240 INFO:tasks.workunit.client.1.smithi142.stdout:7/512: write d2/d8/dd/d14/d2c/d37/d6b/f8c [620109,128189] 0 2024-06-15T01:07:03.240 INFO:tasks.workunit.client.0.smithi137.stdout:0/416: mkdir d1/d2/dc/d6c/d90 0 2024-06-15T01:07:03.242 INFO:tasks.workunit.client.1.smithi142.stdout:7/513: write d2/d8/dd/d14/d2c/d37/d6b/d80/f9c [115588,70687] 0 2024-06-15T01:07:03.242 INFO:tasks.workunit.client.0.smithi137.stdout:4/400: mkdir dd/d15/d30/d49/d75/d99 0 2024-06-15T01:07:03.243 INFO:tasks.workunit.client.1.smithi142.stdout:7/514: stat d2/d8/da/d33/d25/f67 0 2024-06-15T01:07:03.243 INFO:tasks.workunit.client.0.smithi137.stdout:0/417: readlink d1/d2/dc/d10/d14/d15/d17/l1f 0 2024-06-15T01:07:03.244 INFO:tasks.workunit.client.1.smithi142.stdout:7/515: write d2/d8/da/d33/d25/d79/f95 [754720,72242] 0 2024-06-15T01:07:03.246 INFO:tasks.workunit.client.1.smithi142.stdout:2/449: rename d8/df/d1f/f6e to d8/df/d10/d94/d25/d22/d43/f96 0 2024-06-15T01:07:03.246 INFO:tasks.workunit.client.0.smithi137.stdout:4/401: creat dd/d15/d21/f9a x:0 0 0 2024-06-15T01:07:03.247 INFO:tasks.workunit.client.0.smithi137.stdout:9/238: write df/d12/d15/f1d [1120559,10133] 0 2024-06-15T01:07:03.247 INFO:tasks.workunit.client.0.smithi137.stdout:4/402: rename dd/d15/d21 to dd/d15/d21/d9b 22 2024-06-15T01:07:03.248 INFO:tasks.workunit.client.0.smithi137.stdout:4/403: dread - dd/d15/d36/d45/d55/f62 zero size 2024-06-15T01:07:03.249 INFO:tasks.workunit.client.1.smithi142.stdout:7/516: rename d2/d6/d22/fa3 to d2/d8/da/d33/d25/d81/fa4 0 2024-06-15T01:07:03.251 INFO:tasks.workunit.client.0.smithi137.stdout:9/239: truncate df/d12/d15/d16/f27 605153 0 2024-06-15T01:07:03.253 INFO:tasks.workunit.client.0.smithi137.stdout:9/240: write df/d12/d1b/d31/f33 [2432146,64140] 0 2024-06-15T01:07:03.253 INFO:tasks.workunit.client.0.smithi137.stdout:7/386: write df/f14 [128372,10179] 0 2024-06-15T01:07:03.255 INFO:tasks.workunit.client.1.smithi142.stdout:7/517: dwrite d2/f3 [0,4194304] 0 2024-06-15T01:07:03.256 INFO:tasks.workunit.client.0.smithi137.stdout:2/337: truncate d0/de/f1a 4140643 0 2024-06-15T01:07:03.260 INFO:tasks.workunit.client.0.smithi137.stdout:6/283: dwrite d0/d5/f2c [0,4194304] 0 2024-06-15T01:07:03.261 INFO:tasks.workunit.client.0.smithi137.stdout:2/338: mknod d0/de/d2b/c68 0 2024-06-15T01:07:03.267 INFO:tasks.workunit.client.0.smithi137.stdout:6/284: mknod d0/d5/d9/d29/c55 0 2024-06-15T01:07:03.269 INFO:tasks.workunit.client.0.smithi137.stdout:7/387: getdents df/d13/d2e/d1a 0 2024-06-15T01:07:03.269 INFO:tasks.workunit.client.0.smithi137.stdout:9/241: dwrite df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:03.274 INFO:tasks.workunit.client.0.smithi137.stdout:0/418: dread d1/d2/dc/d10/d14/d15/d17/f18 [0,4194304] 0 2024-06-15T01:07:03.274 INFO:tasks.workunit.client.1.smithi142.stdout:2/450: dread d8/f16 [0,4194304] 0 2024-06-15T01:07:03.274 INFO:tasks.workunit.client.1.smithi142.stdout:7/518: truncate d2/d8/f84 55235 0 2024-06-15T01:07:03.276 INFO:tasks.workunit.client.0.smithi137.stdout:9/242: write df/d12/f1f [1962552,37214] 0 2024-06-15T01:07:03.276 INFO:tasks.workunit.client.0.smithi137.stdout:0/419: fdatasync d1/d2/d13/d66/f86 0 2024-06-15T01:07:03.277 INFO:tasks.workunit.client.0.smithi137.stdout:6/285: unlink d0/d47/d32/c3b 0 2024-06-15T01:07:03.278 INFO:tasks.workunit.client.1.smithi142.stdout:2/451: creat d8/df/d10/d1c/f97 x:0 0 0 2024-06-15T01:07:03.279 INFO:tasks.workunit.client.0.smithi137.stdout:2/339: dwrite d0/d42/d49/d57/f58 [0,4194304] 0 2024-06-15T01:07:03.283 INFO:tasks.workunit.client.1.smithi142.stdout:2/452: read d8/df/f3f [1226731,86961] 0 2024-06-15T01:07:03.285 INFO:tasks.workunit.client.0.smithi137.stdout:9/243: mknod df/d13/c3f 0 2024-06-15T01:07:03.286 INFO:tasks.workunit.client.0.smithi137.stdout:0/420: mkdir d1/d2/dc/d6c/d88/d91 0 2024-06-15T01:07:03.287 INFO:tasks.workunit.client.0.smithi137.stdout:6/286: symlink d0/d5/d9/d29/l56 0 2024-06-15T01:07:03.289 INFO:tasks.workunit.client.0.smithi137.stdout:2/340: chown d0/de/d2b/d47/l5e 2570 1 2024-06-15T01:07:03.291 INFO:tasks.workunit.client.0.smithi137.stdout:9/244: symlink df/l40 0 2024-06-15T01:07:03.293 INFO:tasks.workunit.client.1.smithi142.stdout:2/453: dwrite d8/df/d10/d1c/f97 [0,4194304] 0 2024-06-15T01:07:03.294 INFO:tasks.workunit.client.0.smithi137.stdout:6/287: rename d0/d47 to d0/d1a/d3e/d57 0 2024-06-15T01:07:03.296 INFO:tasks.workunit.client.1.smithi142.stdout:7/519: dwrite d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:07:03.296 INFO:tasks.workunit.client.0.smithi137.stdout:2/341: unlink d0/f3 0 2024-06-15T01:07:03.298 INFO:tasks.workunit.client.0.smithi137.stdout:7/388: creat df/f78 x:0 0 0 2024-06-15T01:07:03.298 INFO:tasks.workunit.client.1.smithi142.stdout:7/520: write d2/f2e [1203462,12936] 0 2024-06-15T01:07:03.299 INFO:tasks.workunit.client.0.smithi137.stdout:0/421: symlink d1/d2/dc/d10/d63/d8d/l92 0 2024-06-15T01:07:03.299 INFO:tasks.workunit.client.1.smithi142.stdout:2/454: mkdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98 0 2024-06-15T01:07:03.300 INFO:tasks.workunit.client.1.smithi142.stdout:7/521: readlink d2/d8/dd/d14/d2c/d37/l99 0 2024-06-15T01:07:03.301 INFO:tasks.workunit.client.1.smithi142.stdout:7/522: write d2/f2e [2094407,46769] 0 2024-06-15T01:07:03.304 INFO:tasks.workunit.client.0.smithi137.stdout:0/422: write d1/d2/dc/d10/d14/d15/d17/f69 [736275,103355] 0 2024-06-15T01:07:03.304 INFO:tasks.workunit.client.0.smithi137.stdout:6/288: dwrite d0/d5/f2c [0,4194304] 0 2024-06-15T01:07:03.307 INFO:tasks.workunit.client.0.smithi137.stdout:9/245: mknod df/d12/d15/d16/d3c/c41 0 2024-06-15T01:07:03.312 INFO:tasks.workunit.client.0.smithi137.stdout:6/289: creat d0/d1a/f58 x:0 0 0 2024-06-15T01:07:03.313 INFO:tasks.workunit.client.0.smithi137.stdout:2/342: creat d0/f69 x:0 0 0 2024-06-15T01:07:03.313 INFO:tasks.workunit.client.0.smithi137.stdout:0/423: write d1/d2/dc/d10/d14/d15/d17/f4c [952101,25138] 0 2024-06-15T01:07:03.313 INFO:tasks.workunit.client.0.smithi137.stdout:7/389: dwrite df/d18/d1c/d1d/d27/d33/d43/d6c/f37 [0,4194304] 0 2024-06-15T01:07:03.314 INFO:tasks.workunit.client.0.smithi137.stdout:9/246: mkdir df/d12/d1b/d31/d42 0 2024-06-15T01:07:03.316 INFO:tasks.workunit.client.0.smithi137.stdout:9/247: truncate f5 4343693 0 2024-06-15T01:07:03.316 INFO:tasks.workunit.client.0.smithi137.stdout:6/290: unlink d0/d5/f13 0 2024-06-15T01:07:03.317 INFO:tasks.workunit.client.0.smithi137.stdout:2/343: chown d0/dc/l50 207 1 2024-06-15T01:07:03.317 INFO:tasks.workunit.client.0.smithi137.stdout:9/248: read f3 [2027574,68352] 0 2024-06-15T01:07:03.319 INFO:tasks.workunit.client.0.smithi137.stdout:7/390: write df/d13/d2e/f51 [1215527,13295] 0 2024-06-15T01:07:03.322 INFO:tasks.workunit.client.0.smithi137.stdout:9/249: write df/d12/d1b/f35 [1011515,89557] 0 2024-06-15T01:07:03.323 INFO:tasks.workunit.client.0.smithi137.stdout:6/291: write d0/d1a/d3e/d57/f1e [3229353,40391] 0 2024-06-15T01:07:03.324 INFO:tasks.workunit.client.0.smithi137.stdout:7/391: truncate df/d18/d3b/d5b/f5d 165759 0 2024-06-15T01:07:03.326 INFO:tasks.workunit.client.0.smithi137.stdout:0/424: rmdir d1/d2/dc/d6c/d84 0 2024-06-15T01:07:03.326 INFO:tasks.workunit.client.1.smithi142.stdout:7/523: write d2/d6/f8f [2258187,105805] 0 2024-06-15T01:07:03.328 INFO:tasks.workunit.client.0.smithi137.stdout:0/425: read d1/d2/dc/d10/f4f [1262035,129454] 0 2024-06-15T01:07:03.328 INFO:tasks.workunit.client.0.smithi137.stdout:9/250: creat df/d2a/f43 x:0 0 0 2024-06-15T01:07:03.332 INFO:tasks.workunit.client.0.smithi137.stdout:7/392: unlink df/d13/d2e/c69 0 2024-06-15T01:07:03.333 INFO:tasks.workunit.client.1.smithi142.stdout:7/524: dwrite d2/d8/dd/d14/d2c/d37/d6b/d80/f9c [0,4194304] 0 2024-06-15T01:07:03.334 INFO:tasks.workunit.client.1.smithi142.stdout:7/525: stat d2/d6/l64 0 2024-06-15T01:07:03.336 INFO:tasks.workunit.client.1.smithi142.stdout:2/455: dread d8/df/f3f [0,4194304] 0 2024-06-15T01:07:03.338 INFO:tasks.workunit.client.0.smithi137.stdout:9/251: dwrite df/d12/d15/f32 [0,4194304] 0 2024-06-15T01:07:03.342 INFO:tasks.workunit.client.1.smithi142.stdout:2/456: dwrite d8/df/d10/d94/d25/f74 [0,4194304] 0 2024-06-15T01:07:03.343 INFO:tasks.workunit.client.0.smithi137.stdout:7/393: unlink df/d13/d2e/d1a/d3f/f76 0 2024-06-15T01:07:03.345 INFO:tasks.workunit.client.1.smithi142.stdout:2/457: fsync d8/df/f59 0 2024-06-15T01:07:03.345 INFO:tasks.workunit.client.1.smithi142.stdout:5/488: sync 2024-06-15T01:07:03.345 INFO:tasks.workunit.client.1.smithi142.stdout:8/482: sync 2024-06-15T01:07:03.347 INFO:tasks.workunit.client.0.smithi137.stdout:0/426: symlink d1/d2/dc/d10/d14/d15/d17/l93 0 2024-06-15T01:07:03.348 INFO:tasks.workunit.client.0.smithi137.stdout:7/394: write df/d13/d3c/f6e [293908,102156] 0 2024-06-15T01:07:03.349 INFO:tasks.workunit.client.0.smithi137.stdout:0/427: write d1/d2/f5e [4137947,94478] 0 2024-06-15T01:07:03.355 INFO:tasks.workunit.client.0.smithi137.stdout:9/252: dwrite df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:03.356 INFO:tasks.workunit.client.0.smithi137.stdout:9/253: chown df/d12/f3e 1000034 1 2024-06-15T01:07:03.357 INFO:tasks.workunit.client.1.smithi142.stdout:2/458: creat d8/df/d10/d94/d25/d22/d43/d51/f99 x:0 0 0 2024-06-15T01:07:03.361 INFO:tasks.workunit.client.0.smithi137.stdout:6/292: getdents d0/d5 0 2024-06-15T01:07:03.361 INFO:tasks.workunit.client.1.smithi142.stdout:2/459: creat d8/df/d10/d94/d25/d1a/d63/f9a x:0 0 0 2024-06-15T01:07:03.361 INFO:tasks.workunit.client.0.smithi137.stdout:2/344: dread d0/de/d10/f19 [4194304,4194304] 0 2024-06-15T01:07:03.365 INFO:tasks.workunit.client.1.smithi142.stdout:2/460: dread d8/f16 [0,4194304] 0 2024-06-15T01:07:03.366 INFO:tasks.workunit.client.0.smithi137.stdout:9/254: write df/d12/d1b/f2f [468431,37784] 0 2024-06-15T01:07:03.367 INFO:tasks.workunit.client.0.smithi137.stdout:6/293: write d0/d5/d9/d3f/f4c [186091,63235] 0 2024-06-15T01:07:03.367 INFO:tasks.workunit.client.1.smithi142.stdout:2/461: fsync d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 0 2024-06-15T01:07:03.368 INFO:tasks.workunit.client.0.smithi137.stdout:7/395: dread df/d13/d2e/f51 [0,4194304] 0 2024-06-15T01:07:03.374 INFO:tasks.workunit.client.1.smithi142.stdout:2/462: dwrite d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [0,4194304] 0 2024-06-15T01:07:03.374 INFO:tasks.workunit.client.0.smithi137.stdout:0/428: dwrite d1/d2/dc/d10/d14/d15/f59 [0,4194304] 0 2024-06-15T01:07:03.375 INFO:tasks.workunit.client.0.smithi137.stdout:2/345: dwrite d0/dc/f3b [0,4194304] 0 2024-06-15T01:07:03.376 INFO:tasks.workunit.client.1.smithi142.stdout:8/483: write d0/d17/f2f [2542372,82893] 0 2024-06-15T01:07:03.376 INFO:tasks.workunit.client.0.smithi137.stdout:7/396: unlink df/d13/d2e/f19 0 2024-06-15T01:07:03.379 INFO:tasks.workunit.client.1.smithi142.stdout:8/484: write d0/d25/d53/f58 [1649717,14358] 0 2024-06-15T01:07:03.381 INFO:tasks.workunit.client.1.smithi142.stdout:2/463: symlink d8/df/d10/d94/d25/d1a/l9b 0 2024-06-15T01:07:03.381 INFO:tasks.workunit.client.1.smithi142.stdout:5/489: truncate d2/d18/f61 598706 0 2024-06-15T01:07:03.382 INFO:tasks.workunit.client.0.smithi137.stdout:9/255: truncate df/f28 565851 0 2024-06-15T01:07:03.385 INFO:tasks.workunit.client.0.smithi137.stdout:9/256: chown df/d2a/f34 2227 1 2024-06-15T01:07:03.388 INFO:tasks.workunit.client.1.smithi142.stdout:5/490: creat d2/d18/f99 x:0 0 0 2024-06-15T01:07:03.389 INFO:tasks.workunit.client.0.smithi137.stdout:0/429: rename d1/d2/dc/d10/d14/d15/d17/d54/l68 to d1/d2/dc/d10/d14/d56/l94 0 2024-06-15T01:07:03.389 INFO:tasks.workunit.client.0.smithi137.stdout:2/346: dwrite d0/dc/d22/f61 [0,4194304] 0 2024-06-15T01:07:03.390 INFO:tasks.workunit.client.1.smithi142.stdout:5/491: write d2/f25 [3005173,103970] 0 2024-06-15T01:07:03.391 INFO:tasks.workunit.client.1.smithi142.stdout:8/485: creat d0/d17/d40/d86/f9b x:0 0 0 2024-06-15T01:07:03.392 INFO:tasks.workunit.client.1.smithi142.stdout:8/486: chown d0/d25/d5a/l31 13746676 1 2024-06-15T01:07:03.392 INFO:tasks.workunit.client.1.smithi142.stdout:5/492: write d2/d3e/d31/f6d [547820,9507] 0 2024-06-15T01:07:03.392 INFO:tasks.workunit.client.0.smithi137.stdout:9/257: creat df/d12/d29/f44 x:0 0 0 2024-06-15T01:07:03.397 INFO:tasks.workunit.client.0.smithi137.stdout:6/294: dwrite d0/d5/d9/d29/f4d [0,4194304] 0 2024-06-15T01:07:03.397 INFO:tasks.workunit.client.1.smithi142.stdout:8/487: mkdir d0/d25/d53/d9c 0 2024-06-15T01:07:03.398 INFO:tasks.workunit.client.0.smithi137.stdout:6/295: chown d0/d5/d9/d29/l2b 64668553 1 2024-06-15T01:07:03.403 INFO:tasks.workunit.client.1.smithi142.stdout:8/488: unlink d0/d25/f6a 0 2024-06-15T01:07:03.405 INFO:tasks.workunit.client.0.smithi137.stdout:0/430: creat d1/d2/dc/d10/d14/d15/d17/f95 x:0 0 0 2024-06-15T01:07:03.410 INFO:tasks.workunit.client.1.smithi142.stdout:8/489: getdents d0/d17 0 2024-06-15T01:07:03.411 INFO:tasks.workunit.client.1.smithi142.stdout:8/490: readlink d0/d25/l68 0 2024-06-15T01:07:03.411 INFO:tasks.workunit.client.0.smithi137.stdout:6/296: dwrite d0/d5/f17 [0,4194304] 0 2024-06-15T01:07:03.412 INFO:tasks.workunit.client.0.smithi137.stdout:9/258: link df/d1a/c36 df/d12/d15/d16/c45 0 2024-06-15T01:07:03.413 INFO:tasks.workunit.client.1.smithi142.stdout:8/491: mkdir d0/d17/d40/d86/d9d 0 2024-06-15T01:07:03.414 INFO:tasks.workunit.client.0.smithi137.stdout:7/397: dread df/d18/d1c/d1d/d27/f48 [0,4194304] 0 2024-06-15T01:07:03.416 INFO:tasks.workunit.client.1.smithi142.stdout:8/492: creat d0/d25/f9e x:0 0 0 2024-06-15T01:07:03.417 INFO:tasks.workunit.client.0.smithi137.stdout:0/431: dread d1/d2/f5e [0,4194304] 0 2024-06-15T01:07:03.417 INFO:tasks.workunit.client.0.smithi137.stdout:7/398: dread - df/d18/d3b/f5f zero size 2024-06-15T01:07:03.418 INFO:tasks.workunit.client.1.smithi142.stdout:8/493: write d0/d25/d5a/f8d [481900,109311] 0 2024-06-15T01:07:03.418 INFO:tasks.workunit.client.1.smithi142.stdout:2/464: write d8/f16 [728109,8853] 0 2024-06-15T01:07:03.421 INFO:tasks.workunit.client.1.smithi142.stdout:8/494: creat d0/d17/d40/d86/f9f x:0 0 0 2024-06-15T01:07:03.422 INFO:tasks.workunit.client.1.smithi142.stdout:2/465: mknod d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/c9c 0 2024-06-15T01:07:03.423 INFO:tasks.workunit.client.1.smithi142.stdout:2/466: chown d8/df/d1f 3360763 1 2024-06-15T01:07:03.426 INFO:tasks.workunit.client.1.smithi142.stdout:2/467: rmdir d8/df/d10/d94/d3a 39 2024-06-15T01:07:03.426 INFO:tasks.workunit.client.1.smithi142.stdout:8/495: getdents d0 0 2024-06-15T01:07:03.427 INFO:tasks.workunit.client.1.smithi142.stdout:8/496: chown d0/d25/d5a/c71 5 1 2024-06-15T01:07:03.429 INFO:tasks.workunit.client.1.smithi142.stdout:8/497: write d0/d17/d40/d6e/f72 [529346,11876] 0 2024-06-15T01:07:03.429 INFO:tasks.workunit.client.1.smithi142.stdout:8/498: chown d0/f15 2312458 1 2024-06-15T01:07:03.436 INFO:tasks.workunit.client.1.smithi142.stdout:8/499: dwrite d0/d25/f7e [0,4194304] 0 2024-06-15T01:07:03.440 INFO:tasks.workunit.client.1.smithi142.stdout:8/500: creat d0/d25/d5a/d84/fa0 x:0 0 0 2024-06-15T01:07:03.442 INFO:tasks.workunit.client.1.smithi142.stdout:8/501: rmdir d0/d25/d53 39 2024-06-15T01:07:03.443 INFO:tasks.workunit.client.1.smithi142.stdout:8/502: chown d0/d17/l77 78 1 2024-06-15T01:07:03.444 INFO:tasks.workunit.client.1.smithi142.stdout:8/503: fdatasync d0/f3f 0 2024-06-15T01:07:03.444 INFO:tasks.workunit.client.1.smithi142.stdout:2/468: dwrite d8/df/d10/d94/d25/d1a/f1d [0,4194304] 0 2024-06-15T01:07:03.445 INFO:tasks.workunit.client.1.smithi142.stdout:2/469: chown d8/d19/d33/l47 1 1 2024-06-15T01:07:03.447 INFO:tasks.workunit.client.1.smithi142.stdout:2/470: chown d8/df/d10/d94/d25/d22/d43/d52/d7a/f92 129882063 1 2024-06-15T01:07:03.448 INFO:tasks.workunit.client.1.smithi142.stdout:8/504: getdents d0/d17/d40/d86/d9d 0 2024-06-15T01:07:03.451 INFO:tasks.workunit.client.1.smithi142.stdout:2/471: dread d8/df/d10/d94/d25/d1a/f1d [0,4194304] 0 2024-06-15T01:07:03.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/259: dread df/d12/f2c [0,4194304] 0 2024-06-15T01:07:03.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/260: fsync fd 0 2024-06-15T01:07:03.454 INFO:tasks.workunit.client.0.smithi137.stdout:9/261: dread - df/d2a/f43 zero size 2024-06-15T01:07:03.456 INFO:tasks.workunit.client.1.smithi142.stdout:2/472: dread d8/df/f3f [0,4194304] 0 2024-06-15T01:07:03.458 INFO:tasks.workunit.client.1.smithi142.stdout:2/473: readlink d8/df/d10/d94/d25/d1a/l9b 0 2024-06-15T01:07:03.460 INFO:tasks.workunit.client.1.smithi142.stdout:2/474: creat d8/df/d10/f9d x:0 0 0 2024-06-15T01:07:03.463 INFO:tasks.workunit.client.1.smithi142.stdout:8/505: dread d0/d25/d5a/f48 [0,4194304] 0 2024-06-15T01:07:03.465 INFO:tasks.workunit.client.1.smithi142.stdout:2/475: dread d8/f40 [0,4194304] 0 2024-06-15T01:07:03.468 INFO:tasks.workunit.client.1.smithi142.stdout:2/476: symlink d8/df/d10/d94/d25/d22/d43/d52/l9e 0 2024-06-15T01:07:03.474 INFO:tasks.workunit.client.1.smithi142.stdout:2/477: dwrite d8/df/d10/d94/d25/f27 [0,4194304] 0 2024-06-15T01:07:03.477 INFO:tasks.workunit.client.1.smithi142.stdout:8/506: dread d0/f4c [0,4194304] 0 2024-06-15T01:07:03.478 INFO:tasks.workunit.client.1.smithi142.stdout:8/507: readlink d0/d25/d5a/l31 0 2024-06-15T01:07:03.478 INFO:tasks.workunit.client.1.smithi142.stdout:2/478: dread d8/d19/d33/f66 [0,4194304] 0 2024-06-15T01:07:03.480 INFO:tasks.workunit.client.1.smithi142.stdout:2/479: dread - d8/df/d10/d94/d25/d22/d43/f96 zero size 2024-06-15T01:07:03.480 INFO:tasks.workunit.client.1.smithi142.stdout:8/508: creat d0/d17/d40/d86/fa1 x:0 0 0 2024-06-15T01:07:03.482 INFO:tasks.workunit.client.1.smithi142.stdout:2/480: truncate d8/d19/f70 900153 0 2024-06-15T01:07:03.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/509: creat d0/d17/d40/fa2 x:0 0 0 2024-06-15T01:07:03.483 INFO:tasks.workunit.client.1.smithi142.stdout:2/481: mknod d8/d19/c9f 0 2024-06-15T01:07:03.485 INFO:tasks.workunit.client.1.smithi142.stdout:8/510: truncate d0/f5f 47784 0 2024-06-15T01:07:03.485 INFO:tasks.workunit.client.1.smithi142.stdout:2/482: creat d8/d19/d33/fa0 x:0 0 0 2024-06-15T01:07:03.493 INFO:tasks.workunit.client.1.smithi142.stdout:0/523: sync 2024-06-15T01:07:03.493 INFO:tasks.workunit.client.1.smithi142.stdout:6/516: sync 2024-06-15T01:07:03.493 INFO:tasks.workunit.client.1.smithi142.stdout:9/446: sync 2024-06-15T01:07:03.493 INFO:tasks.workunit.client.1.smithi142.stdout:4/604: sync 2024-06-15T01:07:03.493 INFO:tasks.workunit.client.1.smithi142.stdout:3/596: sync 2024-06-15T01:07:03.494 INFO:tasks.workunit.client.1.smithi142.stdout:9/447: write d6/dc/dd/d22/f54 [1136345,96768] 0 2024-06-15T01:07:03.495 INFO:tasks.workunit.client.1.smithi142.stdout:6/517: write d1a/d39/d42/d4c/d58/f7f [255470,46070] 0 2024-06-15T01:07:03.499 INFO:tasks.workunit.client.1.smithi142.stdout:6/518: dread d1a/f1e [4194304,4194304] 0 2024-06-15T01:07:03.554 INFO:tasks.workunit.client.0.smithi137.stdout:3/436: sync 2024-06-15T01:07:03.561 INFO:tasks.workunit.client.0.smithi137.stdout:0/432: creat d1/d2/dc/d10/f96 x:0 0 0 2024-06-15T01:07:03.562 INFO:tasks.workunit.client.0.smithi137.stdout:0/433: write d1/d2/d55/f6e [1312743,38764] 0 2024-06-15T01:07:03.563 INFO:tasks.workunit.client.0.smithi137.stdout:7/399: mkdir df/d18/d1c/d1d/d79 0 2024-06-15T01:07:03.567 INFO:tasks.workunit.client.0.smithi137.stdout:0/434: dread d1/d2/dc/d10/d14/d56/f82 [0,4194304] 0 2024-06-15T01:07:03.569 INFO:tasks.workunit.client.0.smithi137.stdout:0/435: chown d1/d2/d13/d66 7302006 1 2024-06-15T01:07:03.571 INFO:tasks.workunit.client.0.smithi137.stdout:3/437: rename d0/d2/d22/d24/c75 to d0/d42/d79/c91 0 2024-06-15T01:07:03.578 INFO:tasks.workunit.client.0.smithi137.stdout:3/438: dwrite d0/f1a [0,4194304] 0 2024-06-15T01:07:03.579 INFO:tasks.workunit.client.0.smithi137.stdout:9/262: creat df/d12/d1b/d31/d42/f46 x:0 0 0 2024-06-15T01:07:03.582 INFO:tasks.workunit.client.1.smithi142.stdout:7/526: sync 2024-06-15T01:07:03.582 INFO:tasks.workunit.client.1.smithi142.stdout:1/539: sync 2024-06-15T01:07:03.582 INFO:tasks.workunit.client.1.smithi142.stdout:5/493: sync 2024-06-15T01:07:03.582 INFO:tasks.workunit.client.1.smithi142.stdout:1/540: fsync d6/fc 0 2024-06-15T01:07:03.583 INFO:tasks.workunit.client.1.smithi142.stdout:7/527: write d2/f2e [3423900,82510] 0 2024-06-15T01:07:03.584 INFO:tasks.workunit.client.0.smithi137.stdout:0/436: symlink d1/d2/dc/d10/d14/d15/d17/d54/l97 0 2024-06-15T01:07:03.585 INFO:tasks.workunit.client.0.smithi137.stdout:6/297: link d0/l37 d0/d1a/d3e/l59 0 2024-06-15T01:07:03.586 INFO:tasks.workunit.client.0.smithi137.stdout:6/298: chown d0/d5/d9/d29/c55 322641 1 2024-06-15T01:07:03.588 INFO:tasks.workunit.client.0.smithi137.stdout:7/400: mkdir df/d18/d1c/d1d/d27/d33/d43/d6c/d4a/d7a 0 2024-06-15T01:07:03.592 INFO:tasks.workunit.client.0.smithi137.stdout:9/263: symlink df/d1a/l47 0 2024-06-15T01:07:03.592 INFO:tasks.workunit.client.0.smithi137.stdout:3/439: mkdir d0/d2/d22/d24/d28/d2e/d92 0 2024-06-15T01:07:03.593 INFO:tasks.workunit.client.0.smithi137.stdout:0/437: mknod d1/d2/dc/d10/d63/d8d/c98 0 2024-06-15T01:07:03.598 INFO:tasks.workunit.client.0.smithi137.stdout:0/438: write d1/d2/dc/d10/d14/d15/f35 [3806269,22773] 0 2024-06-15T01:07:03.599 INFO:tasks.workunit.client.0.smithi137.stdout:6/299: mkdir d0/d5/d43/d5a 0 2024-06-15T01:07:03.605 INFO:tasks.workunit.client.0.smithi137.stdout:3/440: dwrite d0/d2/d23/f6f [0,4194304] 0 2024-06-15T01:07:03.608 INFO:tasks.workunit.client.0.smithi137.stdout:0/439: write d1/d2/dc/d10/d14/d15/d17/f4c [678686,116840] 0 2024-06-15T01:07:03.609 INFO:tasks.workunit.client.0.smithi137.stdout:6/300: rename d0/d5/d9/d3f/f4c to d0/d5/d9/d29/f5b 0 2024-06-15T01:07:03.616 INFO:tasks.workunit.client.1.smithi142.stdout:8/511: sync 2024-06-15T01:07:03.617 INFO:tasks.workunit.client.0.smithi137.stdout:7/401: dread df/d18/d1c/d1d/d27/d33/f52 [0,4194304] 0 2024-06-15T01:07:03.622 INFO:tasks.workunit.client.0.smithi137.stdout:0/440: mkdir d1/d2/dc/d10/d14/d56/d99 0 2024-06-15T01:07:03.623 INFO:tasks.workunit.client.1.smithi142.stdout:0/524: mkdir d3/d39/d9a/d9d/dbf 0 2024-06-15T01:07:03.624 INFO:tasks.workunit.client.0.smithi137.stdout:7/402: mknod df/d18/d3b/c7b 0 2024-06-15T01:07:03.627 INFO:tasks.workunit.client.0.smithi137.stdout:9/264: dread df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:03.627 INFO:tasks.workunit.client.1.smithi142.stdout:3/597: creat d0/d1f/d20/d59/d61/fb7 x:0 0 0 2024-06-15T01:07:03.628 INFO:tasks.workunit.client.0.smithi137.stdout:6/301: mknod d0/d5/d43/d5a/c5c 0 2024-06-15T01:07:03.628 INFO:tasks.workunit.client.0.smithi137.stdout:9/265: readlink df/l40 0 2024-06-15T01:07:03.631 INFO:tasks.workunit.client.0.smithi137.stdout:7/403: unlink df/d18/d1c/f74 0 2024-06-15T01:07:03.632 INFO:tasks.workunit.client.1.smithi142.stdout:4/605: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/d99/cd8 0 2024-06-15T01:07:03.633 INFO:tasks.workunit.client.0.smithi137.stdout:6/302: creat d0/d5/d46/f5d x:0 0 0 2024-06-15T01:07:03.638 INFO:tasks.workunit.client.1.smithi142.stdout:4/606: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d [0,4194304] 0 2024-06-15T01:07:03.638 INFO:tasks.workunit.client.0.smithi137.stdout:0/441: link d1/d2/dc/d10/d14/d15/f4b d1/d2/dc/d6c/d88/d91/f9a 0 2024-06-15T01:07:03.639 INFO:tasks.workunit.client.0.smithi137.stdout:7/404: write df/d13/d2e/f51 [2168885,117886] 0 2024-06-15T01:07:03.639 INFO:tasks.workunit.client.0.smithi137.stdout:6/303: write d0/d5/d9/d29/f4b [602313,35026] 0 2024-06-15T01:07:03.639 INFO:tasks.workunit.client.1.smithi142.stdout:9/448: rename d6/dc/l2e to d6/dc/d1b/d25/d35/lb4 0 2024-06-15T01:07:03.639 INFO:tasks.workunit.client.1.smithi142.stdout:4/607: chown d5/d1c/d6f/dc7/f3d 157676 1 2024-06-15T01:07:03.640 INFO:tasks.workunit.client.1.smithi142.stdout:2/483: rmdir d8 39 2024-06-15T01:07:03.642 INFO:tasks.workunit.client.0.smithi137.stdout:3/441: dwrite d0/d2/d1e/f6e [0,4194304] 0 2024-06-15T01:07:03.643 INFO:tasks.workunit.client.0.smithi137.stdout:6/304: write d0/d1a/d3e/d57/d1d/f45 [214832,2097] 0 2024-06-15T01:07:03.643 INFO:tasks.workunit.client.0.smithi137.stdout:7/405: truncate df/d13/d2e/d1a/f77 391116 0 2024-06-15T01:07:03.644 INFO:tasks.workunit.client.0.smithi137.stdout:0/442: mknod d1/d2/dc/d10/d14/d15/d17/d54/c9b 0 2024-06-15T01:07:03.645 INFO:tasks.workunit.client.0.smithi137.stdout:3/442: readlink d0/d2/l38 0 2024-06-15T01:07:03.647 INFO:tasks.workunit.client.1.smithi142.stdout:9/449: dwrite d6/dc/d4e/f94 [0,4194304] 0 2024-06-15T01:07:03.649 INFO:tasks.workunit.client.1.smithi142.stdout:9/450: chown d6/dc/d4e/d65/c66 296401 1 2024-06-15T01:07:03.649 INFO:tasks.workunit.client.0.smithi137.stdout:3/443: readlink d0/d2/d8/d67/d7e/l5b 0 2024-06-15T01:07:03.652 INFO:tasks.workunit.client.1.smithi142.stdout:5/494: symlink d2/de/d85/l9a 0 2024-06-15T01:07:03.655 INFO:tasks.workunit.client.0.smithi137.stdout:0/443: mknod d1/d2/dc/d10/d14/d52/c9c 0 2024-06-15T01:07:03.656 INFO:tasks.workunit.client.1.smithi142.stdout:0/525: symlink d3/d39/d3e/d81/d1b/d52/lc0 0 2024-06-15T01:07:03.657 INFO:tasks.workunit.client.0.smithi137.stdout:6/305: link d0/d1a/f35 d0/d1a/d3e/d57/f5e 0 2024-06-15T01:07:03.657 INFO:tasks.workunit.client.1.smithi142.stdout:3/598: rmdir d0/d1b/d6b 39 2024-06-15T01:07:03.658 INFO:tasks.workunit.client.1.smithi142.stdout:7/528: dread d2/d6/d22/d47/f50 [0,4194304] 0 2024-06-15T01:07:03.658 INFO:tasks.workunit.client.0.smithi137.stdout:0/444: fsync d1/d2/dc/d10/d14/d15/f59 0 2024-06-15T01:07:03.658 INFO:tasks.workunit.client.0.smithi137.stdout:6/306: stat d0/d1a/d3e/d57/f5e 0 2024-06-15T01:07:03.659 INFO:tasks.workunit.client.1.smithi142.stdout:3/599: write d0/d3/d15/f8c [800752,93353] 0 2024-06-15T01:07:03.663 INFO:tasks.workunit.client.1.smithi142.stdout:1/541: rename d6/d25/d9e to d6/d3a/dc4 0 2024-06-15T01:07:03.665 INFO:tasks.workunit.client.0.smithi137.stdout:0/445: rename d1/d2/dc/d10/d14/d15/d17/l1f to d1/d2/dc/d10/d63/d8d/l9d 0 2024-06-15T01:07:03.666 INFO:tasks.workunit.client.1.smithi142.stdout:7/529: dwrite d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:03.667 INFO:tasks.workunit.client.1.smithi142.stdout:2/484: fdatasync d8/df/d10/d94/d25/d22/d43/f46 0 2024-06-15T01:07:03.668 INFO:tasks.workunit.client.0.smithi137.stdout:6/307: link d0/d1a/f26 d0/d5/d9/d3f/f5f 0 2024-06-15T01:07:03.669 INFO:tasks.workunit.client.0.smithi137.stdout:7/406: dwrite df/d13/d2e/f4e [0,4194304] 0 2024-06-15T01:07:03.671 INFO:tasks.workunit.client.1.smithi142.stdout:9/451: fsync d6/dc/d1b/f76 0 2024-06-15T01:07:03.671 INFO:tasks.workunit.client.0.smithi137.stdout:3/444: write d0/d2f/f69 [186048,77665] 0 2024-06-15T01:07:03.673 INFO:tasks.workunit.client.1.smithi142.stdout:9/452: write d6/dc/d1b/d25/d58/f63 [1492139,20457] 0 2024-06-15T01:07:03.673 INFO:tasks.workunit.client.0.smithi137.stdout:3/445: chown d0/d13/f20 772 1 2024-06-15T01:07:03.674 INFO:tasks.workunit.client.0.smithi137.stdout:7/407: write df/d13/f20 [3061341,48238] 0 2024-06-15T01:07:03.675 INFO:tasks.workunit.client.1.smithi142.stdout:1/542: dread d6/d12/d28/f40 [0,4194304] 0 2024-06-15T01:07:03.675 INFO:tasks.workunit.client.1.smithi142.stdout:9/453: dread - d6/dc/d4e/d65/fa5 zero size 2024-06-15T01:07:03.677 INFO:tasks.workunit.client.1.smithi142.stdout:0/526: write d3/d39/d3e/d81/f34 [574122,56369] 0 2024-06-15T01:07:03.677 INFO:tasks.workunit.client.1.smithi142.stdout:1/543: chown d6/d12/d52/l86 31954 1 2024-06-15T01:07:03.679 INFO:tasks.workunit.client.1.smithi142.stdout:0/527: write d3/d39/d3e/d81/d1b/f40 [135870,120945] 0 2024-06-15T01:07:03.679 INFO:tasks.workunit.client.0.smithi137.stdout:0/446: dwrite d1/d2/dc/d10/d14/d15/d17/f95 [0,4194304] 0 2024-06-15T01:07:03.680 INFO:tasks.workunit.client.1.smithi142.stdout:1/544: write d6/fc [9075710,791] 0 2024-06-15T01:07:03.682 INFO:tasks.workunit.client.0.smithi137.stdout:3/446: symlink d0/d2f/l93 0 2024-06-15T01:07:03.682 INFO:tasks.workunit.client.1.smithi142.stdout:1/545: chown d6/d12/d1c/f34 44572 1 2024-06-15T01:07:03.683 INFO:tasks.workunit.client.1.smithi142.stdout:1/546: write d6/d12/d5b/f8a [1881565,67102] 0 2024-06-15T01:07:03.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/408: symlink df/d13/d2e/d1a/l7c 0 2024-06-15T01:07:03.687 INFO:tasks.workunit.client.0.smithi137.stdout:3/447: chown d0/d2/d8/d67/d7e/l5b 3288 1 2024-06-15T01:07:03.690 INFO:tasks.workunit.client.0.smithi137.stdout:3/448: truncate d0/d2/d23/f73 865932 0 2024-06-15T01:07:03.692 INFO:tasks.workunit.client.0.smithi137.stdout:0/447: dread d1/d2/dc/d6c/d88/d91/f9a [0,4194304] 0 2024-06-15T01:07:03.693 INFO:tasks.workunit.client.0.smithi137.stdout:7/409: chown df/d13/d2e/c75 1 1 2024-06-15T01:07:03.694 INFO:tasks.workunit.client.0.smithi137.stdout:0/448: write d1/d2/dc/d10/d14/d15/f50 [3226708,27623] 0 2024-06-15T01:07:03.695 INFO:tasks.workunit.client.1.smithi142.stdout:9/454: symlink d6/d34/d8a/lb5 0 2024-06-15T01:07:03.699 INFO:tasks.workunit.client.0.smithi137.stdout:7/410: fdatasync df/d13/d2e/f58 0 2024-06-15T01:07:03.702 INFO:tasks.workunit.client.0.smithi137.stdout:0/449: symlink d1/d2/d13/d66/l9e 0 2024-06-15T01:07:03.703 INFO:tasks.workunit.client.0.smithi137.stdout:7/411: fsync df/d18/d1c/d1d/d27/d33/d59/f60 0 2024-06-15T01:07:03.704 INFO:tasks.workunit.client.1.smithi142.stdout:2/485: read d8/df/d10/d94/d3a/f8b [508001,69317] 0 2024-06-15T01:07:03.705 INFO:tasks.workunit.client.0.smithi137.stdout:7/412: write df/f78 [855813,58972] 0 2024-06-15T01:07:03.710 INFO:tasks.workunit.client.0.smithi137.stdout:0/450: mknod d1/d2/dc/d10/d14/d15/d17/c9f 0 2024-06-15T01:07:03.710 INFO:tasks.workunit.client.0.smithi137.stdout:6/308: dread d0/f6 [0,4194304] 0 2024-06-15T01:07:03.710 INFO:tasks.workunit.client.1.smithi142.stdout:8/512: sync 2024-06-15T01:07:03.710 INFO:tasks.workunit.client.1.smithi142.stdout:3/600: rename d0/d3/da/db5 to d0/d3/da/db8 0 2024-06-15T01:07:03.710 INFO:tasks.workunit.client.1.smithi142.stdout:3/601: write d0/d1f/d20/fa3 [790534,12517] 0 2024-06-15T01:07:03.711 INFO:tasks.workunit.client.1.smithi142.stdout:3/602: stat d0/d3/da/d71/fab 0 2024-06-15T01:07:03.711 INFO:tasks.workunit.client.1.smithi142.stdout:9/455: mknod d6/dc/d51/d53/d8d/cb6 0 2024-06-15T01:07:03.711 INFO:tasks.workunit.client.1.smithi142.stdout:3/603: stat d0/d1f/d20/d5d/c73 0 2024-06-15T01:07:03.714 INFO:tasks.workunit.client.0.smithi137.stdout:7/413: rename df/d18 to df/d7d 0 2024-06-15T01:07:03.714 INFO:tasks.workunit.client.0.smithi137.stdout:3/449: dwrite d0/fd [0,4194304] 0 2024-06-15T01:07:03.715 INFO:tasks.workunit.client.1.smithi142.stdout:4/608: rename d5/d9/d41/fd6 to d5/d9/d23/fd9 0 2024-06-15T01:07:03.716 INFO:tasks.workunit.client.0.smithi137.stdout:6/309: mknod d0/d1a/c60 0 2024-06-15T01:07:03.717 INFO:tasks.workunit.client.1.smithi142.stdout:9/456: dwrite d6/dc/d1b/f4b [0,4194304] 0 2024-06-15T01:07:03.717 INFO:tasks.workunit.client.1.smithi142.stdout:4/609: readlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/ld3 0 2024-06-15T01:07:03.718 INFO:tasks.workunit.client.0.smithi137.stdout:7/414: readlink df/d13/d2e/d1a/d3f/l53 0 2024-06-15T01:07:03.718 INFO:tasks.workunit.client.1.smithi142.stdout:9/457: chown d6/dc/d51/f80 165812673 1 2024-06-15T01:07:03.718 INFO:tasks.workunit.client.1.smithi142.stdout:1/547: mknod d6/d12/d3c/d63/d8d/d97/cc5 0 2024-06-15T01:07:03.719 INFO:tasks.workunit.client.0.smithi137.stdout:0/451: truncate d1/d2/dc/d10/d14/d15/f4b 3766798 0 2024-06-15T01:07:03.720 INFO:tasks.workunit.client.0.smithi137.stdout:0/452: fsync d1/d2/dc/d10/d14/d56/f85 0 2024-06-15T01:07:03.722 INFO:tasks.workunit.client.0.smithi137.stdout:3/450: mknod d0/d2f/c94 0 2024-06-15T01:07:03.723 INFO:tasks.workunit.client.0.smithi137.stdout:3/451: chown d0/d2/d23/f48 1 1 2024-06-15T01:07:03.724 INFO:tasks.workunit.client.1.smithi142.stdout:2/486: chown d8/df/d10/c32 53 1 2024-06-15T01:07:03.724 INFO:tasks.workunit.client.0.smithi137.stdout:7/415: stat df/d7d/c23 0 2024-06-15T01:07:03.728 INFO:tasks.workunit.client.0.smithi137.stdout:3/452: write d0/d2/d22/d63/f6d [642547,110262] 0 2024-06-15T01:07:03.728 INFO:tasks.workunit.client.0.smithi137.stdout:6/310: getdents d0/d5/d46 0 2024-06-15T01:07:03.729 INFO:tasks.workunit.client.1.smithi142.stdout:6/519: creat d1a/d2d/fb8 x:0 0 0 2024-06-15T01:07:03.730 INFO:tasks.workunit.client.0.smithi137.stdout:6/311: chown d0/d5/d9/d3f/f5f 13 1 2024-06-15T01:07:03.731 INFO:tasks.workunit.client.0.smithi137.stdout:7/416: mkdir df/d7d/d1c/d7e 0 2024-06-15T01:07:03.732 INFO:tasks.workunit.client.0.smithi137.stdout:7/417: write df/d13/d3c/f67 [1368414,122188] 0 2024-06-15T01:07:03.733 INFO:tasks.workunit.client.0.smithi137.stdout:6/312: chown d0/d5/d9/c42 107 1 2024-06-15T01:07:03.734 INFO:tasks.workunit.client.1.smithi142.stdout:1/548: mkdir d6/d12/d3c/d63/d8d/d97/dc6 0 2024-06-15T01:07:03.734 INFO:tasks.workunit.client.0.smithi137.stdout:7/418: fdatasync df/d13/d2e/d1a/f2c 0 2024-06-15T01:07:03.735 INFO:tasks.workunit.client.1.smithi142.stdout:1/549: write d6/f9 [3061382,130083] 0 2024-06-15T01:07:03.736 INFO:tasks.workunit.client.0.smithi137.stdout:0/453: dread d1/d2/d13/f33 [0,4194304] 0 2024-06-15T01:07:03.736 INFO:tasks.workunit.client.1.smithi142.stdout:9/458: symlink d6/d34/d8a/lb7 0 2024-06-15T01:07:03.736 INFO:tasks.workunit.client.1.smithi142.stdout:1/550: dread - d6/d12/d1c/d2c/f62 zero size 2024-06-15T01:07:03.739 INFO:tasks.workunit.client.0.smithi137.stdout:0/454: chown d1/d2/dc/d10/d14/d15/d1c/c3c 137 1 2024-06-15T01:07:03.742 INFO:tasks.workunit.client.0.smithi137.stdout:7/419: rmdir df/d7d/d1c/d1d/d27/d33/d43/d6c 39 2024-06-15T01:07:03.742 INFO:tasks.workunit.client.0.smithi137.stdout:0/455: fdatasync d1/d2/dc/d10/d14/f1b 0 2024-06-15T01:07:03.743 INFO:tasks.workunit.client.1.smithi142.stdout:6/520: symlink d1a/d2d/d68/da0/lb9 0 2024-06-15T01:07:03.744 INFO:tasks.workunit.client.1.smithi142.stdout:1/551: dwrite d6/d12/d3c/d63/d8d/fb8 [0,4194304] 0 2024-06-15T01:07:03.745 INFO:tasks.workunit.client.1.smithi142.stdout:9/459: stat d6/dc/dd/ca6 0 2024-06-15T01:07:03.745 INFO:tasks.workunit.client.0.smithi137.stdout:7/420: mknod df/d13/d2e/d1a/c7f 0 2024-06-15T01:07:03.746 INFO:tasks.workunit.client.1.smithi142.stdout:9/460: fdatasync d6/dc/f19 0 2024-06-15T01:07:03.747 INFO:tasks.workunit.client.0.smithi137.stdout:7/421: truncate df/f14 1823169 0 2024-06-15T01:07:03.747 INFO:tasks.workunit.client.0.smithi137.stdout:0/456: mkdir d1/d7e/d87/da0 0 2024-06-15T01:07:03.748 INFO:tasks.workunit.client.1.smithi142.stdout:4/610: link d5/d1c/fbb d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fda 0 2024-06-15T01:07:03.752 INFO:tasks.workunit.client.1.smithi142.stdout:8/513: rename d0/d25/l68 to d0/d17/d40/la3 0 2024-06-15T01:07:03.752 INFO:tasks.workunit.client.1.smithi142.stdout:6/521: dwrite d1a/d39/f82 [0,4194304] 0 2024-06-15T01:07:03.753 INFO:tasks.workunit.client.1.smithi142.stdout:8/514: chown d0/d25/d5a/l55 533 1 2024-06-15T01:07:03.754 INFO:tasks.workunit.client.1.smithi142.stdout:6/522: truncate d1a/d39/d3a/d51/d62/fb4 181194 0 2024-06-15T01:07:03.755 INFO:tasks.workunit.client.0.smithi137.stdout:7/422: getdents df/d7d/d3b/d5b 0 2024-06-15T01:07:03.757 INFO:tasks.workunit.client.0.smithi137.stdout:1/455: sync 2024-06-15T01:07:03.758 INFO:tasks.workunit.client.0.smithi137.stdout:8/359: sync 2024-06-15T01:07:03.760 INFO:tasks.workunit.client.0.smithi137.stdout:7/423: creat df/d13/d2e/d1a/f80 x:0 0 0 2024-06-15T01:07:03.761 INFO:tasks.workunit.client.1.smithi142.stdout:2/487: dread f4 [0,4194304] 0 2024-06-15T01:07:03.761 INFO:tasks.workunit.client.1.smithi142.stdout:8/515: dwrite d0/f47 [0,4194304] 0 2024-06-15T01:07:03.763 INFO:tasks.workunit.client.1.smithi142.stdout:8/516: write d0/d25/d66/f95 [34073,74531] 0 2024-06-15T01:07:03.766 INFO:tasks.workunit.client.0.smithi137.stdout:1/456: mkdir d0/d10/daf 0 2024-06-15T01:07:03.767 INFO:tasks.workunit.client.1.smithi142.stdout:6/523: truncate f18 3941438 0 2024-06-15T01:07:03.768 INFO:tasks.workunit.client.0.smithi137.stdout:6/313: truncate d0/f3c 1471578 0 2024-06-15T01:07:03.769 INFO:tasks.workunit.client.1.smithi142.stdout:9/461: getdents d6/dc/d4e/dae 0 2024-06-15T01:07:03.770 INFO:tasks.workunit.client.1.smithi142.stdout:6/524: rename d1a to d1a/d39/d42/dba 22 2024-06-15T01:07:03.771 INFO:tasks.workunit.client.1.smithi142.stdout:8/517: dwrite d0/d17/f56 [0,4194304] 0 2024-06-15T01:07:03.772 INFO:tasks.workunit.client.0.smithi137.stdout:3/453: dread d0/d2/d8/f41 [0,4194304] 0 2024-06-15T01:07:03.799 INFO:tasks.workunit.client.0.smithi137.stdout:7/424: mknod df/d7d/d1c/c81 0 2024-06-15T01:07:03.800 INFO:tasks.workunit.client.0.smithi137.stdout:8/360: creat d9/de/f7b x:0 0 0 2024-06-15T01:07:03.800 INFO:tasks.workunit.client.1.smithi142.stdout:2/488: unlink d8/d19/d33/f42 0 2024-06-15T01:07:03.801 INFO:tasks.workunit.client.1.smithi142.stdout:2/489: write d8/df/d10/d1c/f2c [4990213,59355] 0 2024-06-15T01:07:03.804 INFO:tasks.workunit.client.0.smithi137.stdout:6/314: dwrite d0/d5/d9/fe [0,4194304] 0 2024-06-15T01:07:03.806 INFO:tasks.workunit.client.1.smithi142.stdout:6/525: fdatasync fc 0 2024-06-15T01:07:03.807 INFO:tasks.workunit.client.1.smithi142.stdout:9/462: symlink d6/dc/d51/lb8 0 2024-06-15T01:07:03.808 INFO:tasks.workunit.client.1.smithi142.stdout:6/526: read - d1a/d2d/d3b/faf zero size 2024-06-15T01:07:03.811 INFO:tasks.workunit.client.0.smithi137.stdout:3/454: dwrite d0/d2f/f69 [0,4194304] 0 2024-06-15T01:07:03.811 INFO:tasks.workunit.client.0.smithi137.stdout:0/457: write d1/d2/d13/d66/f3f [711425,110219] 0 2024-06-15T01:07:03.812 INFO:tasks.workunit.client.0.smithi137.stdout:0/458: readlink l0 0 2024-06-15T01:07:03.814 INFO:tasks.workunit.client.1.smithi142.stdout:1/552: getdents d6/d12/d5b 0 2024-06-15T01:07:03.815 INFO:tasks.workunit.client.1.smithi142.stdout:9/463: dwrite d6/dc/d4e/f94 [0,4194304] 0 2024-06-15T01:07:03.816 INFO:tasks.workunit.client.0.smithi137.stdout:1/457: dread d0/d1f/d76/f8f [0,4194304] 0 2024-06-15T01:07:03.819 INFO:tasks.workunit.client.0.smithi137.stdout:7/425: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82 0 2024-06-15T01:07:03.819 INFO:tasks.workunit.client.1.smithi142.stdout:9/464: write d6/dc/dd/d22/f49 [4845154,19462] 0 2024-06-15T01:07:03.821 INFO:tasks.workunit.client.0.smithi137.stdout:3/455: creat d0/f95 x:0 0 0 2024-06-15T01:07:03.823 INFO:tasks.workunit.client.1.smithi142.stdout:9/465: dread - d6/dc/d51/f80 zero size 2024-06-15T01:07:03.824 INFO:tasks.workunit.client.0.smithi137.stdout:8/361: getdents d9/de/d1e 0 2024-06-15T01:07:03.825 INFO:tasks.workunit.client.1.smithi142.stdout:1/553: dwrite d6/d12/d1c/fc0 [0,4194304] 0 2024-06-15T01:07:03.826 INFO:tasks.workunit.client.0.smithi137.stdout:6/315: dread d0/d1a/d3e/d57/d1d/d2e/f38 [0,4194304] 0 2024-06-15T01:07:03.828 INFO:tasks.workunit.client.0.smithi137.stdout:8/362: creat d9/de/d1e/d67/f7c x:0 0 0 2024-06-15T01:07:03.829 INFO:tasks.workunit.client.0.smithi137.stdout:1/458: getdents d0/d3/d1b/d4c/d59/d93 0 2024-06-15T01:07:03.832 INFO:tasks.workunit.client.0.smithi137.stdout:3/456: dread d0/fd [0,4194304] 0 2024-06-15T01:07:03.833 INFO:tasks.workunit.client.1.smithi142.stdout:2/490: fdatasync d8/df/d10/f11 0 2024-06-15T01:07:03.833 INFO:tasks.workunit.client.0.smithi137.stdout:6/316: rmdir d0/d1a/d3e/d57/d1d 39 2024-06-15T01:07:03.834 INFO:tasks.workunit.client.0.smithi137.stdout:1/459: chown d0/d3/d1b/d4c/ca8 269369969 1 2024-06-15T01:07:03.834 INFO:tasks.workunit.client.0.smithi137.stdout:3/457: stat d0/d2f/c8b 0 2024-06-15T01:07:03.835 INFO:tasks.workunit.client.0.smithi137.stdout:1/460: readlink d0/d2d/d3e/l5b 0 2024-06-15T01:07:03.835 INFO:tasks.workunit.client.0.smithi137.stdout:7/426: rename df/d7d/d1c/d1d/d27/d33/d43/d6c/c35 to df/d7d/d1c/d1d/d27/c83 0 2024-06-15T01:07:03.837 INFO:tasks.workunit.client.0.smithi137.stdout:7/427: write df/d13/d3c/f5e [941357,25123] 0 2024-06-15T01:07:03.837 INFO:tasks.workunit.client.0.smithi137.stdout:7/428: readlink df/d13/l44 0 2024-06-15T01:07:03.838 INFO:tasks.workunit.client.0.smithi137.stdout:6/317: creat d0/d5/d43/f61 x:0 0 0 2024-06-15T01:07:03.838 INFO:tasks.workunit.client.0.smithi137.stdout:8/363: write d9/de/d1e/d67/f6b [6375991,11025] 0 2024-06-15T01:07:03.839 INFO:tasks.workunit.client.0.smithi137.stdout:6/318: chown d0/f6 202013496 1 2024-06-15T01:07:03.840 INFO:tasks.workunit.client.0.smithi137.stdout:6/319: fsync d0/d5/f2c 0 2024-06-15T01:07:03.841 INFO:tasks.workunit.client.0.smithi137.stdout:3/458: creat d0/d56/d8f/d7b/f96 x:0 0 0 2024-06-15T01:07:03.846 INFO:tasks.workunit.client.1.smithi142.stdout:4/611: sync 2024-06-15T01:07:03.846 INFO:tasks.workunit.client.0.smithi137.stdout:7/429: write df/d7d/d3b/d5b/f5d [918465,127455] 0 2024-06-15T01:07:03.846 INFO:tasks.workunit.client.0.smithi137.stdout:7/430: fsync df/d13/d2e/f51 0 2024-06-15T01:07:03.847 INFO:tasks.workunit.client.1.smithi142.stdout:9/466: readlink d6/dc/d1b/d25/d35/lb4 0 2024-06-15T01:07:03.847 INFO:tasks.workunit.client.1.smithi142.stdout:3/604: symlink d0/d1f/lb9 0 2024-06-15T01:07:03.847 INFO:tasks.workunit.client.0.smithi137.stdout:3/459: write d0/fd [2463809,89855] 0 2024-06-15T01:07:03.848 INFO:tasks.workunit.client.0.smithi137.stdout:6/320: read d0/d1a/d3e/d57/d1d/f45 [3480,96286] 0 2024-06-15T01:07:03.851 INFO:tasks.workunit.client.1.smithi142.stdout:1/554: fdatasync d6/d12/d1c/d2c/d47/f5a 0 2024-06-15T01:07:03.854 INFO:tasks.workunit.client.1.smithi142.stdout:3/605: dwrite d0/d1b/f88 [0,4194304] 0 2024-06-15T01:07:03.854 INFO:tasks.workunit.client.1.smithi142.stdout:4/612: creat d5/d1c/d6f/dc7/d30/dcc/fdb x:0 0 0 2024-06-15T01:07:03.856 INFO:tasks.workunit.client.0.smithi137.stdout:0/459: dread d1/d2/dc/d10/d14/d15/f35 [0,4194304] 0 2024-06-15T01:07:03.856 INFO:tasks.workunit.client.1.smithi142.stdout:3/606: write d0/d1f/f6f [4898288,5773] 0 2024-06-15T01:07:03.856 INFO:tasks.workunit.client.1.smithi142.stdout:9/467: creat d6/dc/d51/d53/d8d/fb9 x:0 0 0 2024-06-15T01:07:03.858 INFO:tasks.workunit.client.1.smithi142.stdout:4/613: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/dca/cd5 198346 1 2024-06-15T01:07:03.859 INFO:tasks.workunit.client.0.smithi137.stdout:6/321: write d0/d1a/d3e/d57/d1d/d2e/f38 [1446027,97003] 0 2024-06-15T01:07:03.861 INFO:tasks.workunit.client.1.smithi142.stdout:9/468: write d6/dc/d4e/d78/f9d [301919,56971] 0 2024-06-15T01:07:03.861 INFO:tasks.workunit.client.1.smithi142.stdout:4/614: fdatasync d5/d9/d90/daf/dcb/f8a 0 2024-06-15T01:07:03.863 INFO:tasks.workunit.client.1.smithi142.stdout:9/469: write d6/dc/d4e/d65/fa5 [705932,102485] 0 2024-06-15T01:07:03.866 INFO:tasks.workunit.client.0.smithi137.stdout:7/431: link df/f32 df/d7d/d3b/d5b/f84 0 2024-06-15T01:07:03.867 INFO:tasks.workunit.client.1.smithi142.stdout:3/607: unlink d0/d1f/d20/d5d/d60/f84 0 2024-06-15T01:07:03.870 INFO:tasks.workunit.client.0.smithi137.stdout:6/322: mkdir d0/d1a/d3e/d57/d1d/d2e/d62 0 2024-06-15T01:07:03.870 INFO:tasks.workunit.client.1.smithi142.stdout:5/495: truncate d2/d18/f63 2231606 0 2024-06-15T01:07:03.872 INFO:tasks.workunit.client.1.smithi142.stdout:4/615: creat d5/d9/d90/daf/fdc x:0 0 0 2024-06-15T01:07:03.873 INFO:tasks.workunit.client.0.smithi137.stdout:0/460: unlink d1/d2/dc/d6c/d88/l8f 0 2024-06-15T01:07:03.874 INFO:tasks.workunit.client.1.smithi142.stdout:3/608: dwrite d0/f66 [0,4194304] 0 2024-06-15T01:07:03.875 INFO:tasks.workunit.client.1.smithi142.stdout:3/609: truncate d0/d1f/d20/d59/f8e 712952 0 2024-06-15T01:07:03.875 INFO:tasks.workunit.client.0.smithi137.stdout:0/461: write d1/d2/dc/d10/f75 [130217,73044] 0 2024-06-15T01:07:03.876 INFO:tasks.workunit.client.0.smithi137.stdout:7/432: mkdir df/d13/d2e/d1a/d3f/d85 0 2024-06-15T01:07:03.879 INFO:tasks.workunit.client.0.smithi137.stdout:3/460: dread d0/d2/d23/f44 [0,4194304] 0 2024-06-15T01:07:03.884 INFO:tasks.workunit.client.1.smithi142.stdout:3/610: creat d0/fba x:0 0 0 2024-06-15T01:07:03.884 INFO:tasks.workunit.client.1.smithi142.stdout:1/555: dread d6/d12/d1c/d2c/f59 [0,4194304] 0 2024-06-15T01:07:03.885 INFO:tasks.workunit.client.0.smithi137.stdout:0/462: unlink d1/d2/dc/d10/d14/d15/d1c/c4e 0 2024-06-15T01:07:03.886 INFO:tasks.workunit.client.1.smithi142.stdout:5/496: mknod d2/d3e/d98/c9b 0 2024-06-15T01:07:03.886 INFO:tasks.workunit.client.0.smithi137.stdout:6/323: creat d0/d5/d9/f63 x:0 0 0 2024-06-15T01:07:03.886 INFO:tasks.workunit.client.1.smithi142.stdout:5/497: stat d2/d3e/d31/l5b 0 2024-06-15T01:07:03.887 INFO:tasks.workunit.client.0.smithi137.stdout:8/364: truncate d9/de/d1e/d67/f6b 7613072 0 2024-06-15T01:07:03.888 INFO:tasks.workunit.client.1.smithi142.stdout:1/556: truncate d6/d12/f2f 83036 0 2024-06-15T01:07:03.890 INFO:tasks.workunit.client.0.smithi137.stdout:6/324: readlink d0/d1a/l21 0 2024-06-15T01:07:03.891 INFO:tasks.workunit.client.1.smithi142.stdout:7/530: mknod d2/d8/da/d33/ca5 0 2024-06-15T01:07:03.892 INFO:tasks.workunit.client.1.smithi142.stdout:3/611: dwrite d0/d1f/d20/d5d/d60/f85 [0,4194304] 0 2024-06-15T01:07:03.892 INFO:tasks.workunit.client.1.smithi142.stdout:7/531: chown d2/d8/da/d33 11 1 2024-06-15T01:07:03.892 INFO:tasks.workunit.client.1.smithi142.stdout:1/557: creat d6/d12/d1c/d2c/d93/fc7 x:0 0 0 2024-06-15T01:07:03.893 INFO:tasks.workunit.client.0.smithi137.stdout:3/461: dwrite d0/f95 [0,4194304] 0 2024-06-15T01:07:03.894 INFO:tasks.workunit.client.1.smithi142.stdout:5/498: dwrite d2/f25 [0,4194304] 0 2024-06-15T01:07:03.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/558: write d6/d3a/f43 [206140,71342] 0 2024-06-15T01:07:03.895 INFO:tasks.workunit.client.0.smithi137.stdout:1/461: dwrite d0/d1f/d60/f91 [0,4194304] 0 2024-06-15T01:07:03.896 INFO:tasks.workunit.client.1.smithi142.stdout:5/499: chown d2/de/f50 489153243 1 2024-06-15T01:07:03.896 INFO:tasks.workunit.client.0.smithi137.stdout:8/365: creat d9/d2a/d79/f7d x:0 0 0 2024-06-15T01:07:03.898 INFO:tasks.workunit.client.1.smithi142.stdout:6/527: rename d1a/f20 to d1a/d83/fbb 0 2024-06-15T01:07:03.899 INFO:tasks.workunit.client.0.smithi137.stdout:6/325: truncate d0/f6 1832839 0 2024-06-15T01:07:03.899 INFO:tasks.workunit.client.1.smithi142.stdout:6/528: dread - d1a/d39/d3a/d51/f9b zero size 2024-06-15T01:07:03.901 INFO:tasks.workunit.client.1.smithi142.stdout:7/532: creat d2/d8/da/d33/d25/d27/fa6 x:0 0 0 2024-06-15T01:07:03.902 INFO:tasks.workunit.client.0.smithi137.stdout:8/366: chown d9/d2a/d2f/l4a 23050548 1 2024-06-15T01:07:03.903 INFO:tasks.workunit.client.0.smithi137.stdout:6/326: write d0/d1a/f2f [40784,89744] 0 2024-06-15T01:07:03.903 INFO:tasks.workunit.client.1.smithi142.stdout:1/559: mkdir d6/d12/d80/dc8 0 2024-06-15T01:07:03.907 INFO:tasks.workunit.client.1.smithi142.stdout:0/528: unlink d3/d39/d3e/d81/d44/d7c/l3d 0 2024-06-15T01:07:03.908 INFO:tasks.workunit.client.1.smithi142.stdout:6/529: truncate d1a/d39/d42/d4c/d58/d87/f89 11015 0 2024-06-15T01:07:03.909 INFO:tasks.workunit.client.0.smithi137.stdout:8/367: mkdir d9/d2a/d79/d7e 0 2024-06-15T01:07:03.909 INFO:tasks.workunit.client.1.smithi142.stdout:5/500: write d2/de/d19/f22 [11340262,71987] 0 2024-06-15T01:07:03.910 INFO:tasks.workunit.client.1.smithi142.stdout:4/616: rename d5/d9/c84 to d5/d9/d41/cdd 0 2024-06-15T01:07:03.911 INFO:tasks.workunit.client.0.smithi137.stdout:6/327: mkdir d0/d5/d9/d29/d64 0 2024-06-15T01:07:03.911 INFO:tasks.workunit.client.1.smithi142.stdout:3/612: truncate d0/d1f/d20/d59/d61/d96/faa 1706606 0 2024-06-15T01:07:03.913 INFO:tasks.workunit.client.0.smithi137.stdout:1/462: dwrite d0/d10/d41/fa7 [0,4194304] 0 2024-06-15T01:07:03.914 INFO:tasks.workunit.client.0.smithi137.stdout:3/462: getdents d0/d2/d23/d68 0 2024-06-15T01:07:03.915 INFO:tasks.workunit.client.1.smithi142.stdout:1/560: creat d6/d12/d28/fc9 x:0 0 0 2024-06-15T01:07:03.915 INFO:tasks.workunit.client.1.smithi142.stdout:7/533: dwrite d2/d6/d22/d47/f4f [0,4194304] 0 2024-06-15T01:07:03.917 INFO:tasks.workunit.client.1.smithi142.stdout:7/534: chown d2/d8/dd/d14/d2c/d37/d6b/d80/d85 0 1 2024-06-15T01:07:03.918 INFO:tasks.workunit.client.0.smithi137.stdout:8/368: unlink d9/de/d1e/d67/f6b 0 2024-06-15T01:07:03.923 INFO:tasks.workunit.client.0.smithi137.stdout:0/463: dread d1/d2/dc/d10/d14/d15/f30 [0,4194304] 0 2024-06-15T01:07:03.924 INFO:tasks.workunit.client.1.smithi142.stdout:7/535: dwrite d2/d6/d22/d47/f7c [0,4194304] 0 2024-06-15T01:07:03.925 INFO:tasks.workunit.client.1.smithi142.stdout:1/561: unlink d6/c21 0 2024-06-15T01:07:03.926 INFO:tasks.workunit.client.0.smithi137.stdout:6/328: rename d0/d5/f17 to d0/d1a/f65 0 2024-06-15T01:07:03.928 INFO:tasks.workunit.client.0.smithi137.stdout:1/463: dwrite d0/d3/d1b/d4c/d59/f63 [0,4194304] 0 2024-06-15T01:07:03.935 INFO:tasks.workunit.client.0.smithi137.stdout:8/369: creat d9/de/d1e/f7f x:0 0 0 2024-06-15T01:07:03.940 INFO:tasks.workunit.client.1.smithi142.stdout:5/501: rmdir d2/de/d27/d86/d92 0 2024-06-15T01:07:03.942 INFO:tasks.workunit.client.1.smithi142.stdout:7/536: symlink d2/d8/dd/d14/la7 0 2024-06-15T01:07:03.945 INFO:tasks.workunit.client.0.smithi137.stdout:0/464: dwrite d1/d2/dc/d10/d14/f39 [4194304,4194304] 0 2024-06-15T01:07:03.946 INFO:tasks.workunit.client.1.smithi142.stdout:7/537: dread d2/d6/f83 [0,4194304] 0 2024-06-15T01:07:03.947 INFO:tasks.workunit.client.1.smithi142.stdout:3/613: link d0/d3/da/d71/d55/f57 d0/d1f/d20/d5d/fbb 0 2024-06-15T01:07:03.947 INFO:tasks.workunit.client.1.smithi142.stdout:5/502: dwrite d2/de/d19/d5f/d68/f73 [0,4194304] 0 2024-06-15T01:07:03.949 INFO:tasks.workunit.client.1.smithi142.stdout:7/538: write d2/d8/da/d33/f9e [711451,38245] 0 2024-06-15T01:07:03.950 INFO:tasks.workunit.client.1.smithi142.stdout:4/617: dread d5/d9/d90/daf/dcb/f8a [0,4194304] 0 2024-06-15T01:07:03.952 INFO:tasks.workunit.client.1.smithi142.stdout:8/518: truncate d0/f47 3686410 0 2024-06-15T01:07:03.952 INFO:tasks.workunit.client.0.smithi137.stdout:6/329: dwrite d0/d5/d9/f4f [0,4194304] 0 2024-06-15T01:07:03.954 INFO:tasks.workunit.client.0.smithi137.stdout:8/370: dwrite d9/d2a/d79/f7d [0,4194304] 0 2024-06-15T01:07:03.955 INFO:tasks.workunit.client.1.smithi142.stdout:2/491: write d8/f40 [781561,18802] 0 2024-06-15T01:07:03.960 INFO:tasks.workunit.client.0.smithi137.stdout:6/330: readlink d0/l37 0 2024-06-15T01:07:03.961 INFO:tasks.workunit.client.1.smithi142.stdout:2/492: read - d8/df/d10/d94/d25/d1a/d63/f95 zero size 2024-06-15T01:07:03.962 INFO:tasks.workunit.client.1.smithi142.stdout:3/614: mkdir d0/d95/dbc 0 2024-06-15T01:07:03.962 INFO:tasks.workunit.client.1.smithi142.stdout:5/503: dwrite d2/de/d19/d5f/d68/d76/f81 [0,4194304] 0 2024-06-15T01:07:03.965 INFO:tasks.workunit.client.1.smithi142.stdout:5/504: write d2/d18/f55 [2110710,110062] 0 2024-06-15T01:07:03.967 INFO:tasks.workunit.client.0.smithi137.stdout:6/331: symlink d0/d5/d43/d5a/l66 0 2024-06-15T01:07:03.968 INFO:tasks.workunit.client.0.smithi137.stdout:6/332: readlink d0/d5/l30 0 2024-06-15T01:07:03.970 INFO:tasks.workunit.client.1.smithi142.stdout:7/539: dread d2/d8/dd/d14/d2c/d37/d6b/f8c [0,4194304] 0 2024-06-15T01:07:03.980 INFO:tasks.workunit.client.0.smithi137.stdout:0/465: link d1/d2/dc/d10/d14/d15/d1c/c3c d1/d2/dc/d10/ca1 0 2024-06-15T01:07:03.984 INFO:tasks.workunit.client.0.smithi137.stdout:8/371: dwrite d9/de/d1e/d35/d5f/f74 [0,4194304] 0 2024-06-15T01:07:03.984 INFO:tasks.workunit.client.1.smithi142.stdout:8/519: stat d0/d25/d5a/c38 0 2024-06-15T01:07:03.986 INFO:tasks.workunit.client.1.smithi142.stdout:8/520: chown d0/d17/d40/d6e/f93 21171 1 2024-06-15T01:07:03.986 INFO:tasks.workunit.client.0.smithi137.stdout:3/463: dread d0/d13/d37/f5f [0,4194304] 0 2024-06-15T01:07:03.986 INFO:tasks.workunit.client.0.smithi137.stdout:0/466: stat d1/d2/d13/d66/f4a 0 2024-06-15T01:07:03.987 INFO:tasks.workunit.client.1.smithi142.stdout:8/521: dread - d0/d25/d5a/f78 zero size 2024-06-15T01:07:03.988 INFO:tasks.workunit.client.1.smithi142.stdout:4/618: mknod d5/d9/da/d92/d98/dc1/cde 0 2024-06-15T01:07:03.988 INFO:tasks.workunit.client.0.smithi137.stdout:3/464: symlink d0/d2f/l97 0 2024-06-15T01:07:03.990 INFO:tasks.workunit.client.1.smithi142.stdout:9/470: dwrite d6/d34/f3a [0,4194304] 0 2024-06-15T01:07:03.992 INFO:tasks.workunit.client.1.smithi142.stdout:9/471: dread - d6/dc/d4e/d78/f7f zero size 2024-06-15T01:07:03.993 INFO:tasks.workunit.client.0.smithi137.stdout:3/465: fdatasync d0/d2/d8/f7a 0 2024-06-15T01:07:03.993 INFO:tasks.workunit.client.0.smithi137.stdout:8/372: chown d9/d2a/d2f/l4a 423099 1 2024-06-15T01:07:03.993 INFO:tasks.workunit.client.0.smithi137.stdout:0/467: truncate d1/d2/d13/d66/f60 9418 0 2024-06-15T01:07:03.994 INFO:tasks.workunit.client.1.smithi142.stdout:5/505: symlink d2/de/d19/d5f/d68/d76/d96/l9c 0 2024-06-15T01:07:03.994 INFO:tasks.workunit.client.1.smithi142.stdout:3/615: truncate d0/d3/da/d18/f28 1789762 0 2024-06-15T01:07:03.995 INFO:tasks.workunit.client.0.smithi137.stdout:0/468: write d1/d2/dc/d10/d14/d15/d17/f4c [1734837,6667] 0 2024-06-15T01:07:03.996 INFO:tasks.workunit.client.0.smithi137.stdout:1/464: dread d0/d1f/d60/f74 [0,4194304] 0 2024-06-15T01:07:03.996 INFO:tasks.workunit.client.1.smithi142.stdout:7/540: mkdir d2/d8/da8 0 2024-06-15T01:07:03.997 INFO:tasks.workunit.client.1.smithi142.stdout:6/530: write d1a/d2d/d3b/fa9 [1241241,51510] 0 2024-06-15T01:07:03.998 INFO:tasks.workunit.client.0.smithi137.stdout:0/469: chown d1/d2/dc/d10 415 1 2024-06-15T01:07:03.999 INFO:tasks.workunit.client.0.smithi137.stdout:1/465: chown d0/d3/d14/l1a 3220 1 2024-06-15T01:07:03.999 INFO:tasks.workunit.client.1.smithi142.stdout:4/619: creat d5/d9/d90/daf/dcb/fdf x:0 0 0 2024-06-15T01:07:04.000 INFO:tasks.workunit.client.1.smithi142.stdout:9/472: mknod d6/dc/d1b/d25/d58/cba 0 2024-06-15T01:07:04.001 INFO:tasks.workunit.client.0.smithi137.stdout:3/466: rmdir d0/d56/d8f/d90 0 2024-06-15T01:07:04.001 INFO:tasks.workunit.client.1.smithi142.stdout:6/531: dread d1a/d39/d42/d4c/d58/d87/f89 [0,4194304] 0 2024-06-15T01:07:04.002 INFO:tasks.workunit.client.0.smithi137.stdout:8/373: symlink d9/de/l80 0 2024-06-15T01:07:04.002 INFO:tasks.workunit.client.1.smithi142.stdout:9/473: readlink d6/dc/l39 0 2024-06-15T01:07:04.003 INFO:tasks.workunit.client.1.smithi142.stdout:0/529: dwrite d3/d39/d3e/d81/d44/d7c/f3c [0,4194304] 0 2024-06-15T01:07:04.003 INFO:tasks.workunit.client.1.smithi142.stdout:5/506: rmdir d2/d18/d2e 39 2024-06-15T01:07:04.005 INFO:tasks.workunit.client.1.smithi142.stdout:0/530: readlink d3/l46 0 2024-06-15T01:07:04.005 INFO:tasks.workunit.client.1.smithi142.stdout:9/474: chown d6/dc/dd/d22/l90 0 1 2024-06-15T01:07:04.005 INFO:tasks.workunit.client.0.smithi137.stdout:1/466: chown d0/d3/d1b/d4c/f8e 12 1 2024-06-15T01:07:04.006 INFO:tasks.workunit.client.0.smithi137.stdout:1/467: chown d0/d10/d41 34161976 1 2024-06-15T01:07:04.007 INFO:tasks.workunit.client.0.smithi137.stdout:3/467: rename d0/d2/d23/f64 to d0/d2/d22/d24/d5e/f98 0 2024-06-15T01:07:04.008 INFO:tasks.workunit.client.0.smithi137.stdout:8/374: mknod d9/d2a/d2f/c81 0 2024-06-15T01:07:04.009 INFO:tasks.workunit.client.1.smithi142.stdout:0/531: chown d3/d39/d3e/d81/d4c/c97 5749303 1 2024-06-15T01:07:04.009 INFO:tasks.workunit.client.1.smithi142.stdout:8/522: mkdir d0/d25/d5a/d8f/d98/da4 0 2024-06-15T01:07:04.010 INFO:tasks.workunit.client.0.smithi137.stdout:8/375: chown d9/de/ff 15 1 2024-06-15T01:07:04.011 INFO:tasks.workunit.client.1.smithi142.stdout:0/532: write d3/f6e [4491779,109581] 0 2024-06-15T01:07:04.012 INFO:tasks.workunit.client.0.smithi137.stdout:8/376: fsync d9/de/f7b 0 2024-06-15T01:07:04.015 INFO:tasks.workunit.client.1.smithi142.stdout:6/532: dwrite d1a/d39/d42/d4c/d58/f7f [0,4194304] 0 2024-06-15T01:07:04.015 INFO:tasks.workunit.client.0.smithi137.stdout:3/468: mknod d0/d2/d1e/c99 0 2024-06-15T01:07:04.015 INFO:tasks.workunit.client.1.smithi142.stdout:0/533: read d3/d39/d3e/d81/f34 [731803,122332] 0 2024-06-15T01:07:04.016 INFO:tasks.workunit.client.0.smithi137.stdout:1/468: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-15T01:07:04.023 INFO:tasks.workunit.client.0.smithi137.stdout:8/377: rename d9/d2a/d2f/d4e to d9/d2a/d2f/d59/d82 0 2024-06-15T01:07:04.024 INFO:tasks.workunit.client.0.smithi137.stdout:8/378: dread - d9/d2a/f7a zero size 2024-06-15T01:07:04.025 INFO:tasks.workunit.client.1.smithi142.stdout:5/507: rename d2/d18/l6e to d2/d3e/d98/l9d 0 2024-06-15T01:07:04.025 INFO:tasks.workunit.client.1.smithi142.stdout:3/616: mkdir d0/d1b/d6b/dbd 0 2024-06-15T01:07:04.026 INFO:tasks.workunit.client.0.smithi137.stdout:6/333: truncate d0/d5/d9/f4f 88288 0 2024-06-15T01:07:04.027 INFO:tasks.workunit.client.1.smithi142.stdout:5/508: chown d2/de/d27/d86 1 1 2024-06-15T01:07:04.027 INFO:tasks.workunit.client.1.smithi142.stdout:8/523: stat d0/d25/d53/l75 0 2024-06-15T01:07:04.030 INFO:tasks.workunit.client.0.smithi137.stdout:1/469: write d0/d3/f22 [1202109,102074] 0 2024-06-15T01:07:04.030 INFO:tasks.workunit.client.1.smithi142.stdout:4/620: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f43 [0,4194304] 0 2024-06-15T01:07:04.031 INFO:tasks.workunit.client.1.smithi142.stdout:6/533: mknod d1a/d39/d42/cbc 0 2024-06-15T01:07:04.035 INFO:tasks.workunit.client.1.smithi142.stdout:0/534: rename d3/d39/d3e/d81/d44/d7c/f3c to d3/d39/d3e/d81/d4c/fc1 0 2024-06-15T01:07:04.035 INFO:tasks.workunit.client.0.smithi137.stdout:0/470: dwrite d1/d2/dc/d10/d14/d15/d17/f18 [0,4194304] 0 2024-06-15T01:07:04.037 INFO:tasks.workunit.client.0.smithi137.stdout:8/379: truncate d9/d2a/f2d 659826 0 2024-06-15T01:07:04.038 INFO:tasks.workunit.client.0.smithi137.stdout:6/334: link d0/d5/d46/f5d d0/d5/d9/d3f/f67 0 2024-06-15T01:07:04.038 INFO:tasks.workunit.client.0.smithi137.stdout:8/380: write d9/de/f36 [1456855,94564] 0 2024-06-15T01:07:04.040 INFO:tasks.workunit.client.0.smithi137.stdout:3/469: getdents d0/d2f 0 2024-06-15T01:07:04.040 INFO:tasks.workunit.client.0.smithi137.stdout:0/471: rmdir d1/d7e 39 2024-06-15T01:07:04.041 INFO:tasks.workunit.client.0.smithi137.stdout:6/335: fdatasync d0/d5/d9/d3f/f67 0 2024-06-15T01:07:04.043 INFO:tasks.workunit.client.1.smithi142.stdout:5/509: creat d2/de/d27/f9e x:0 0 0 2024-06-15T01:07:04.046 INFO:tasks.workunit.client.0.smithi137.stdout:1/470: creat d0/fb0 x:0 0 0 2024-06-15T01:07:04.048 INFO:tasks.workunit.client.0.smithi137.stdout:8/381: dwrite d9/d55/f5c [0,4194304] 0 2024-06-15T01:07:04.048 INFO:tasks.workunit.client.1.smithi142.stdout:4/621: rmdir d5/d9/d90/daf/dcb 39 2024-06-15T01:07:04.048 INFO:tasks.workunit.client.1.smithi142.stdout:5/510: dread d2/de/d19/d5f/d68/d76/f81 [0,4194304] 0 2024-06-15T01:07:04.051 INFO:tasks.workunit.client.0.smithi137.stdout:1/471: truncate d0/d3/d14/d34/f70 5225279 0 2024-06-15T01:07:04.052 INFO:tasks.workunit.client.0.smithi137.stdout:0/472: mkdir d1/d2/dc/d10/d53/da2 0 2024-06-15T01:07:04.056 INFO:tasks.workunit.client.1.smithi142.stdout:8/524: rename d0/d25/d53/f79 to d0/d17/d40/d86/d9d/fa5 0 2024-06-15T01:07:04.057 INFO:tasks.workunit.client.0.smithi137.stdout:0/473: fdatasync d1/d2/dc/d10/f96 0 2024-06-15T01:07:04.061 INFO:tasks.workunit.client.0.smithi137.stdout:8/382: dwrite d9/de/d1e/d35/d5f/f74 [0,4194304] 0 2024-06-15T01:07:04.063 INFO:tasks.workunit.client.1.smithi142.stdout:1/562: sync 2024-06-15T01:07:04.063 INFO:tasks.workunit.client.0.smithi137.stdout:6/336: unlink d0/d1a/d3e/d57/d32/c52 0 2024-06-15T01:07:04.065 INFO:tasks.workunit.client.0.smithi137.stdout:3/470: link d0/d2f/l93 d0/d2/d1e/d7c/l9a 0 2024-06-15T01:07:04.067 INFO:tasks.workunit.client.1.smithi142.stdout:9/475: getdents d6/dc/d82 0 2024-06-15T01:07:04.067 INFO:tasks.workunit.client.0.smithi137.stdout:8/383: chown d9/de/d1e/l49 132674365 1 2024-06-15T01:07:04.068 INFO:tasks.workunit.client.1.smithi142.stdout:6/534: symlink d1a/d2d/d3b/dab/lbd 0 2024-06-15T01:07:04.070 INFO:tasks.workunit.client.1.smithi142.stdout:3/617: dwrite d0/d1f/d20/d5d/fbb [0,4194304] 0 2024-06-15T01:07:04.070 INFO:tasks.workunit.client.0.smithi137.stdout:5/541: sync 2024-06-15T01:07:04.071 INFO:tasks.workunit.client.0.smithi137.stdout:5/542: fsync fe 0 2024-06-15T01:07:04.071 INFO:tasks.workunit.client.1.smithi142.stdout:3/618: stat d0/d1b/f91 0 2024-06-15T01:07:04.072 INFO:tasks.workunit.client.0.smithi137.stdout:0/474: symlink d1/d2/dc/d10/d53/da2/la3 0 2024-06-15T01:07:04.073 INFO:tasks.workunit.client.1.smithi142.stdout:0/535: dwrite d3/d39/d3e/d81/d4c/fc1 [0,4194304] 0 2024-06-15T01:07:04.073 INFO:tasks.workunit.client.0.smithi137.stdout:8/384: chown d9/l10 5 1 2024-06-15T01:07:04.077 INFO:tasks.workunit.client.1.smithi142.stdout:5/511: mknod d2/d18/c9f 0 2024-06-15T01:07:04.078 INFO:tasks.workunit.client.0.smithi137.stdout:3/471: symlink d0/d2/d1e/d7c/l9b 0 2024-06-15T01:07:04.078 INFO:tasks.workunit.client.1.smithi142.stdout:5/512: stat d2/de/f50 0 2024-06-15T01:07:04.079 INFO:tasks.workunit.client.1.smithi142.stdout:3/619: creat d0/d1f/d20/d59/d61/d96/fbe x:0 0 0 2024-06-15T01:07:04.080 INFO:tasks.workunit.client.0.smithi137.stdout:5/543: chown df/d11/c18 56285174 1 2024-06-15T01:07:04.080 INFO:tasks.workunit.client.1.smithi142.stdout:6/535: creat d1a/d39/fbe x:0 0 0 2024-06-15T01:07:04.081 INFO:tasks.workunit.client.0.smithi137.stdout:3/472: chown d0/d2/d1e/l3e 18767 1 2024-06-15T01:07:04.082 INFO:tasks.workunit.client.1.smithi142.stdout:1/563: link d6/d12/d1c/d2c/f91 d6/d46/dc3/fca 0 2024-06-15T01:07:04.082 INFO:tasks.workunit.client.0.smithi137.stdout:1/472: dread d0/d3/d14/fa5 [8388608,4194304] 0 2024-06-15T01:07:04.083 INFO:tasks.workunit.client.0.smithi137.stdout:1/473: chown d0/cd 1026520 1 2024-06-15T01:07:04.084 INFO:tasks.workunit.client.1.smithi142.stdout:5/513: mkdir d2/d3e/d31/da0 0 2024-06-15T01:07:04.084 INFO:tasks.workunit.client.0.smithi137.stdout:5/544: creat df/d7a/fb5 x:0 0 0 2024-06-15T01:07:04.085 INFO:tasks.workunit.client.1.smithi142.stdout:0/536: dread d3/d39/d3e/d81/f7e [0,4194304] 0 2024-06-15T01:07:04.085 INFO:tasks.workunit.client.1.smithi142.stdout:9/476: dwrite d6/dc/f19 [0,4194304] 0 2024-06-15T01:07:04.087 INFO:tasks.workunit.client.0.smithi137.stdout:0/475: mknod d1/d7e/d87/da0/ca4 0 2024-06-15T01:07:04.087 INFO:tasks.workunit.client.1.smithi142.stdout:9/477: chown d6 96291 1 2024-06-15T01:07:04.088 INFO:tasks.workunit.client.0.smithi137.stdout:8/385: dwrite d9/f1d [0,4194304] 0 2024-06-15T01:07:04.089 INFO:tasks.workunit.client.1.smithi142.stdout:5/514: rmdir d2/d3e/d31/d7e 39 2024-06-15T01:07:04.090 INFO:tasks.workunit.client.0.smithi137.stdout:0/476: write d1/d2/dc/d10/d14/f39 [5276897,65921] 0 2024-06-15T01:07:04.094 INFO:tasks.workunit.client.1.smithi142.stdout:9/478: creat d6/dc/d1b/d25/d58/fbb x:0 0 0 2024-06-15T01:07:04.097 INFO:tasks.workunit.client.0.smithi137.stdout:5/545: dwrite df/d44/d54/d95/d3b/d49/d5c/f91 [0,4194304] 0 2024-06-15T01:07:04.098 INFO:tasks.workunit.client.0.smithi137.stdout:0/477: creat d1/d2/dc/d10/d53/da2/fa5 x:0 0 0 2024-06-15T01:07:04.100 INFO:tasks.workunit.client.0.smithi137.stdout:0/478: read - d1/d2/dc/d10/d63/f6b zero size 2024-06-15T01:07:04.100 INFO:tasks.workunit.client.1.smithi142.stdout:3/620: dread d0/d3/da/d71/f97 [0,4194304] 0 2024-06-15T01:07:04.100 INFO:tasks.workunit.client.0.smithi137.stdout:6/337: write d0/d1a/d3e/d57/d1d/f45 [225241,56021] 0 2024-06-15T01:07:04.102 INFO:tasks.workunit.client.0.smithi137.stdout:5/546: dread - df/d3d/f97 zero size 2024-06-15T01:07:04.102 INFO:tasks.workunit.client.0.smithi137.stdout:0/479: chown d1/d2/dc/d10/d63/d8d/l92 43 1 2024-06-15T01:07:04.103 INFO:tasks.workunit.client.1.smithi142.stdout:5/515: rename d2/d3e/d74/l78 to d2/de/d85/la1 0 2024-06-15T01:07:04.104 INFO:tasks.workunit.client.1.smithi142.stdout:6/536: dwrite d1a/d39/d42/d4c/d58/f6a [0,4194304] 0 2024-06-15T01:07:04.106 INFO:tasks.workunit.client.0.smithi137.stdout:6/338: mkdir d0/d1a/d68 0 2024-06-15T01:07:04.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/339: write d0/f39 [3697167,126851] 0 2024-06-15T01:07:04.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/340: chown d0/d1a 58 1 2024-06-15T01:07:04.112 INFO:tasks.workunit.client.0.smithi137.stdout:6/341: chown d0/l37 5757 1 2024-06-15T01:07:04.113 INFO:tasks.workunit.client.0.smithi137.stdout:6/342: fsync d0/d1a/f58 0 2024-06-15T01:07:04.114 INFO:tasks.workunit.client.0.smithi137.stdout:0/480: dread d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [4194304,4194304] 0 2024-06-15T01:07:04.114 INFO:tasks.workunit.client.0.smithi137.stdout:5/547: creat df/fb6 x:0 0 0 2024-06-15T01:07:04.117 INFO:tasks.workunit.client.1.smithi142.stdout:3/621: symlink d0/d3/da/d71/d75/lbf 0 2024-06-15T01:07:04.118 INFO:tasks.workunit.client.1.smithi142.stdout:1/564: dwrite d6/d46/f56 [0,4194304] 0 2024-06-15T01:07:04.118 INFO:tasks.workunit.client.1.smithi142.stdout:0/537: dwrite d3/d39/d9a/fbb [0,4194304] 0 2024-06-15T01:07:04.118 INFO:tasks.workunit.client.0.smithi137.stdout:5/548: chown df/d44/d54/d6b/f8c 1010939268 1 2024-06-15T01:07:04.120 INFO:tasks.workunit.client.1.smithi142.stdout:4/622: truncate d5/d9/d18/f2f 1917145 0 2024-06-15T01:07:04.121 INFO:tasks.workunit.client.0.smithi137.stdout:1/474: dwrite d0/d10/d41/d7b/f90 [0,4194304] 0 2024-06-15T01:07:04.122 INFO:tasks.workunit.client.0.smithi137.stdout:3/473: dwrite d0/f16 [0,4194304] 0 2024-06-15T01:07:04.124 INFO:tasks.workunit.client.1.smithi142.stdout:8/525: dwrite d0/d25/d66/f92 [0,4194304] 0 2024-06-15T01:07:04.124 INFO:tasks.workunit.client.1.smithi142.stdout:3/622: creat d0/d1f/d20/d59/d61/fc0 x:0 0 0 2024-06-15T01:07:04.125 INFO:tasks.workunit.client.1.smithi142.stdout:9/479: dread d6/dc/dd/fac [0,4194304] 0 2024-06-15T01:07:04.127 INFO:tasks.workunit.client.1.smithi142.stdout:1/565: creat d6/d12/d80/d9b/fcb x:0 0 0 2024-06-15T01:07:04.127 INFO:tasks.workunit.client.1.smithi142.stdout:6/537: dwrite d1a/f2c [0,4194304] 0 2024-06-15T01:07:04.127 INFO:tasks.workunit.client.0.smithi137.stdout:0/481: dread d1/d2/dc/d10/d14/f39 [0,4194304] 0 2024-06-15T01:07:04.129 INFO:tasks.workunit.client.1.smithi142.stdout:8/526: truncate d0/d17/d40/d86/fa1 73654 0 2024-06-15T01:07:04.130 INFO:tasks.workunit.client.0.smithi137.stdout:3/474: mkdir d0/d56/d8f/d9c 0 2024-06-15T01:07:04.133 INFO:tasks.workunit.client.1.smithi142.stdout:8/527: dread d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:04.133 INFO:tasks.workunit.client.0.smithi137.stdout:1/475: dread d0/d3/d1b/d4c/f8e [0,4194304] 0 2024-06-15T01:07:04.134 INFO:tasks.workunit.client.1.smithi142.stdout:4/623: rmdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d 39 2024-06-15T01:07:04.135 INFO:tasks.workunit.client.1.smithi142.stdout:8/528: chown d0/d17/d40/d5e/f94 53972388 1 2024-06-15T01:07:04.136 INFO:tasks.workunit.client.0.smithi137.stdout:1/476: fsync d0/d1f/d76/da1/fa2 0 2024-06-15T01:07:04.137 INFO:tasks.workunit.client.0.smithi137.stdout:5/549: mknod df/d11/d14/cb7 0 2024-06-15T01:07:04.138 INFO:tasks.workunit.client.0.smithi137.stdout:5/550: dread - df/d3d/f97 zero size 2024-06-15T01:07:04.142 INFO:tasks.workunit.client.1.smithi142.stdout:1/566: creat d6/d12/da0/fcc x:0 0 0 2024-06-15T01:07:04.145 INFO:tasks.workunit.client.1.smithi142.stdout:8/529: read d0/d17/d40/d86/fa1 [39602,126769] 0 2024-06-15T01:07:04.146 INFO:tasks.workunit.client.1.smithi142.stdout:4/624: unlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/fb1 0 2024-06-15T01:07:04.146 INFO:tasks.workunit.client.0.smithi137.stdout:3/475: symlink d0/d2/d1e/l9d 0 2024-06-15T01:07:04.146 INFO:tasks.workunit.client.1.smithi142.stdout:8/530: write d0/d17/f1f [3203381,54332] 0 2024-06-15T01:07:04.147 INFO:tasks.workunit.client.1.smithi142.stdout:0/538: rename d3/d39/d3e/d81/d1b/d52/lc0 to d3/d39/d9a/lc2 0 2024-06-15T01:07:04.148 INFO:tasks.workunit.client.0.smithi137.stdout:0/482: symlink d1/la6 0 2024-06-15T01:07:04.149 INFO:tasks.workunit.client.1.smithi142.stdout:5/516: getdents d2/d18 0 2024-06-15T01:07:04.151 INFO:tasks.workunit.client.0.smithi137.stdout:1/477: dread d0/d3/d14/f38 [0,4194304] 0 2024-06-15T01:07:04.151 INFO:tasks.workunit.client.0.smithi137.stdout:3/476: truncate d0/f1f 662889 0 2024-06-15T01:07:04.151 INFO:tasks.workunit.client.1.smithi142.stdout:1/567: creat d6/d12/d1c/d2c/d47/fcd x:0 0 0 2024-06-15T01:07:04.152 INFO:tasks.workunit.client.1.smithi142.stdout:2/493: sync 2024-06-15T01:07:04.152 INFO:tasks.workunit.client.0.smithi137.stdout:0/483: mknod d1/d2/dc/d10/d53/da2/ca7 0 2024-06-15T01:07:04.157 INFO:tasks.workunit.client.0.smithi137.stdout:0/484: dwrite d1/d2/f6 [0,4194304] 0 2024-06-15T01:07:04.158 INFO:tasks.workunit.client.0.smithi137.stdout:0/485: chown d1/d2/dc/l64 25 1 2024-06-15T01:07:04.163 INFO:tasks.workunit.client.1.smithi142.stdout:6/538: dread f7 [0,4194304] 0 2024-06-15T01:07:04.163 INFO:tasks.workunit.client.1.smithi142.stdout:6/539: readlink d1a/d2d/d68/da0/lb9 0 2024-06-15T01:07:04.163 INFO:tasks.workunit.client.1.smithi142.stdout:6/540: chown cb 1422 1 2024-06-15T01:07:04.195 INFO:tasks.workunit.client.0.smithi137.stdout:5/551: creat df/d44/d54/fb8 x:0 0 0 2024-06-15T01:07:04.196 INFO:tasks.workunit.client.0.smithi137.stdout:3/477: symlink d0/d42/d79/l9e 0 2024-06-15T01:07:04.197 INFO:tasks.workunit.client.0.smithi137.stdout:1/478: unlink d0/d3/d14/fa5 0 2024-06-15T01:07:04.198 INFO:tasks.workunit.client.1.smithi142.stdout:4/625: chown l0 0 1 2024-06-15T01:07:04.198 INFO:tasks.workunit.client.1.smithi142.stdout:4/626: read - d5/d9/da/d92/d98/fac zero size 2024-06-15T01:07:04.199 INFO:tasks.workunit.client.0.smithi137.stdout:0/486: symlink d1/d7e/d87/la8 0 2024-06-15T01:07:04.201 INFO:tasks.workunit.client.1.smithi142.stdout:9/480: rename d6/dc/dd/d22/f49 to d6/dc/d1b/d6f/fbc 0 2024-06-15T01:07:04.202 INFO:tasks.workunit.client.1.smithi142.stdout:9/481: write d6/dc/d51/d53/d8d/fb9 [937963,56939] 0 2024-06-15T01:07:04.206 INFO:tasks.workunit.client.0.smithi137.stdout:6/343: link d0/d5/f2c d0/d5/d43/d5a/f69 0 2024-06-15T01:07:04.206 INFO:tasks.workunit.client.0.smithi137.stdout:3/478: dwrite d0/d2/d22/d24/f54 [0,4194304] 0 2024-06-15T01:07:04.207 INFO:tasks.workunit.client.1.smithi142.stdout:0/539: dwrite d3/d39/d3e/d81/d4c/da0/fb2 [0,4194304] 0 2024-06-15T01:07:04.208 INFO:tasks.workunit.client.0.smithi137.stdout:1/479: dwrite d0/d3/d1b/d4c/d59/f8c [0,4194304] 0 2024-06-15T01:07:04.214 INFO:tasks.workunit.client.0.smithi137.stdout:5/552: dwrite df/d11/d30/f61 [0,4194304] 0 2024-06-15T01:07:04.217 INFO:tasks.workunit.client.0.smithi137.stdout:6/344: creat d0/d1a/d68/f6a x:0 0 0 2024-06-15T01:07:04.218 INFO:tasks.workunit.client.0.smithi137.stdout:3/479: write d0/d4c/f82 [270777,11130] 0 2024-06-15T01:07:04.220 INFO:tasks.workunit.client.1.smithi142.stdout:3/623: dwrite d0/f2c [0,4194304] 0 2024-06-15T01:07:04.223 INFO:tasks.workunit.client.0.smithi137.stdout:5/553: creat df/d11/fb9 x:0 0 0 2024-06-15T01:07:04.225 INFO:tasks.workunit.client.1.smithi142.stdout:7/541: sync 2024-06-15T01:07:04.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:07:04.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:07:04.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: pgmap v17: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 231 MiB/s wr, 400 op/s 2024-06-15T01:07:04.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:07:04.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:07:04.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.227 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:03 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.227 INFO:tasks.workunit.client.0.smithi137.stdout:3/480: mkdir d0/d56/d8f/d9c/d9f 0 2024-06-15T01:07:04.228 INFO:tasks.workunit.client.0.smithi137.stdout:5/554: mknod df/d76/cba 0 2024-06-15T01:07:04.229 INFO:tasks.workunit.client.0.smithi137.stdout:6/345: getdents d0/d5 0 2024-06-15T01:07:04.231 INFO:tasks.workunit.client.1.smithi142.stdout:9/482: mknod d6/dc/dd/cbd 0 2024-06-15T01:07:04.232 INFO:tasks.workunit.client.1.smithi142.stdout:9/483: truncate d6/dc/d51/f80 948920 0 2024-06-15T01:07:04.232 INFO:tasks.workunit.client.0.smithi137.stdout:6/346: truncate d0/f10 1439143 0 2024-06-15T01:07:04.233 INFO:tasks.workunit.client.1.smithi142.stdout:1/568: mknod d6/d12/d1c/cce 0 2024-06-15T01:07:04.234 INFO:tasks.workunit.client.1.smithi142.stdout:1/569: read - d6/d12/d1c/d2c/d93/db9/fbb zero size 2024-06-15T01:07:04.234 INFO:tasks.workunit.client.0.smithi137.stdout:6/347: write d0/d1a/d3e/d57/d1d/f40 [1121300,91971] 0 2024-06-15T01:07:04.235 INFO:tasks.workunit.client.0.smithi137.stdout:5/555: link df/d3d/f5f df/d11/d30/db1/fbb 0 2024-06-15T01:07:04.236 INFO:tasks.workunit.client.1.smithi142.stdout:0/540: chown d3/d39/d3e/d81/d44/d7c/c4f 99605 1 2024-06-15T01:07:04.237 INFO:tasks.workunit.client.0.smithi137.stdout:6/348: fsync d0/d1a/f26 0 2024-06-15T01:07:04.237 INFO:tasks.workunit.client.1.smithi142.stdout:0/541: chown d3/d39/d3e/f5f 56604 1 2024-06-15T01:07:04.237 INFO:tasks.workunit.client.1.smithi142.stdout:3/624: mkdir d0/d1f/d20/d5d/d60/dc1 0 2024-06-15T01:07:04.238 INFO:tasks.workunit.client.0.smithi137.stdout:6/349: dread - d0/d1a/d68/f6a zero size 2024-06-15T01:07:04.238 INFO:tasks.workunit.client.1.smithi142.stdout:3/625: fsync d0/d3/da/d71/d75/fa7 0 2024-06-15T01:07:04.239 INFO:tasks.workunit.client.0.smithi137.stdout:5/556: symlink df/d44/d54/d95/d35/lbc 0 2024-06-15T01:07:04.240 INFO:tasks.workunit.client.1.smithi142.stdout:4/627: mkdir d5/d1c/d6f/dc7/de0 0 2024-06-15T01:07:04.240 INFO:tasks.workunit.client.0.smithi137.stdout:5/557: dread - df/d7a/fb5 zero size 2024-06-15T01:07:04.241 INFO:tasks.workunit.client.1.smithi142.stdout:9/484: rename d6/dc/d1b/d6f/fbc to d6/dc/dd/d22/fbe 0 2024-06-15T01:07:04.243 INFO:tasks.workunit.client.0.smithi137.stdout:6/350: mkdir d0/d5/d9/d29/d6b 0 2024-06-15T01:07:04.244 INFO:tasks.workunit.client.0.smithi137.stdout:5/558: chown df/la7 12 1 2024-06-15T01:07:04.247 INFO:tasks.workunit.client.0.smithi137.stdout:6/351: unlink d0/d5/d9/c42 0 2024-06-15T01:07:04.248 INFO:tasks.workunit.client.1.smithi142.stdout:8/531: getdents d0/d25 0 2024-06-15T01:07:04.250 INFO:tasks.workunit.client.0.smithi137.stdout:6/352: symlink d0/d1a/d3e/d57/d1d/l6c 0 2024-06-15T01:07:04.251 INFO:tasks.workunit.client.1.smithi142.stdout:9/485: creat d6/d34/d8a/fbf x:0 0 0 2024-06-15T01:07:04.251 INFO:tasks.workunit.client.0.smithi137.stdout:3/481: dread d0/d13/d37/f43 [4194304,4194304] 0 2024-06-15T01:07:04.252 INFO:tasks.workunit.client.1.smithi142.stdout:2/494: link d8/df/d10/d94/d3a/l5e d8/df/d10/d64/la1 0 2024-06-15T01:07:04.252 INFO:tasks.workunit.client.1.smithi142.stdout:2/495: dread - d8/f6b zero size 2024-06-15T01:07:04.254 INFO:tasks.workunit.client.1.smithi142.stdout:6/541: getdents d1a/d39/d42/d4c/d58 0 2024-06-15T01:07:04.255 INFO:tasks.workunit.client.0.smithi137.stdout:3/482: dread - d0/d2/d22/d24/d5e/f98 zero size 2024-06-15T01:07:04.255 INFO:tasks.workunit.client.1.smithi142.stdout:1/570: truncate d6/d12/d1c/d2c/f91 268879 0 2024-06-15T01:07:04.256 INFO:tasks.workunit.client.0.smithi137.stdout:1/480: write d0/d1f/f4d [1850220,12094] 0 2024-06-15T01:07:04.256 INFO:tasks.workunit.client.0.smithi137.stdout:1/481: chown d0/d10/daf 52 1 2024-06-15T01:07:04.257 INFO:tasks.workunit.client.0.smithi137.stdout:1/482: readlink d0/d1f/d60/l6a 0 2024-06-15T01:07:04.260 INFO:tasks.workunit.client.1.smithi142.stdout:5/517: sync 2024-06-15T01:07:04.260 INFO:tasks.workunit.client.1.smithi142.stdout:4/628: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f6d [0,4194304] 0 2024-06-15T01:07:04.260 INFO:tasks.workunit.client.1.smithi142.stdout:7/542: dwrite d2/d8/dd/f76 [0,4194304] 0 2024-06-15T01:07:04.260 INFO:tasks.workunit.client.0.smithi137.stdout:3/483: dread d0/d4c/f82 [0,4194304] 0 2024-06-15T01:07:04.263 INFO:tasks.workunit.client.1.smithi142.stdout:8/532: creat d0/d25/d5a/d8f/d98/da4/fa6 x:0 0 0 2024-06-15T01:07:04.263 INFO:tasks.workunit.client.0.smithi137.stdout:3/484: write d0/d13/d37/d66/f88 [665301,122073] 0 2024-06-15T01:07:04.264 INFO:tasks.workunit.client.1.smithi142.stdout:1/571: dwrite d6/d46/f56 [0,4194304] 0 2024-06-15T01:07:04.265 INFO:tasks.workunit.client.0.smithi137.stdout:6/353: write d0/fc [3313832,40500] 0 2024-06-15T01:07:04.265 INFO:tasks.workunit.client.1.smithi142.stdout:8/533: write d0/d25/d5a/f85 [512726,127456] 0 2024-06-15T01:07:04.266 INFO:tasks.workunit.client.0.smithi137.stdout:3/485: readlink d0/d2/d8/l45 0 2024-06-15T01:07:04.266 INFO:tasks.workunit.client.1.smithi142.stdout:6/542: truncate d1a/d39/d3a/d51/d62/f79 158308 0 2024-06-15T01:07:04.269 INFO:tasks.workunit.client.0.smithi137.stdout:1/483: dwrite d0/d3/d1b/f4f [0,4194304] 0 2024-06-15T01:07:04.271 INFO:tasks.workunit.client.1.smithi142.stdout:2/496: truncate d8/df/f48 1790517 0 2024-06-15T01:07:04.271 INFO:tasks.workunit.client.1.smithi142.stdout:8/534: chown d0/d17/d40/d86/d9d/fa5 13963669 1 2024-06-15T01:07:04.271 INFO:tasks.workunit.client.0.smithi137.stdout:6/354: creat d0/d5/f6d x:0 0 0 2024-06-15T01:07:04.272 INFO:tasks.workunit.client.1.smithi142.stdout:5/518: dwrite d2/d18/f37 [4194304,4194304] 0 2024-06-15T01:07:04.273 INFO:tasks.workunit.client.0.smithi137.stdout:3/486: chown d0/d42/d79/c91 802357 1 2024-06-15T01:07:04.274 INFO:tasks.workunit.client.0.smithi137.stdout:1/484: write d0/d1f/d60/f91 [3671190,22616] 0 2024-06-15T01:07:04.274 INFO:tasks.workunit.client.0.smithi137.stdout:3/487: write d0/d2/d22/d24/f47 [5576369,21466] 0 2024-06-15T01:07:04.275 INFO:tasks.workunit.client.1.smithi142.stdout:1/572: rename d6/d12/l66 to d6/d12/da0/lcf 0 2024-06-15T01:07:04.275 INFO:tasks.workunit.client.1.smithi142.stdout:9/486: link d6/dc/dd/f91 d6/d34/fc0 0 2024-06-15T01:07:04.276 INFO:tasks.workunit.client.1.smithi142.stdout:9/487: dread - d6/dc/f61 zero size 2024-06-15T01:07:04.277 INFO:tasks.workunit.client.1.smithi142.stdout:5/519: mkdir d2/d18/d79/da2 0 2024-06-15T01:07:04.277 INFO:tasks.workunit.client.0.smithi137.stdout:3/488: creat d0/d2/d22/d63/fa0 x:0 0 0 2024-06-15T01:07:04.278 INFO:tasks.workunit.client.0.smithi137.stdout:1/485: read d0/d10/f23 [1053933,114421] 0 2024-06-15T01:07:04.279 INFO:tasks.workunit.client.1.smithi142.stdout:6/543: readlink d1a/l40 0 2024-06-15T01:07:04.279 INFO:tasks.workunit.client.1.smithi142.stdout:2/497: rmdir d8/df/d10/d94/d25/d22/d43 39 2024-06-15T01:07:04.280 INFO:tasks.workunit.client.1.smithi142.stdout:5/520: write d2/de/d27/f9e [311078,16850] 0 2024-06-15T01:07:04.280 INFO:tasks.workunit.client.0.smithi137.stdout:3/489: mkdir d0/d2/d1e/d7c/da1 0 2024-06-15T01:07:04.281 INFO:tasks.workunit.client.0.smithi137.stdout:1/486: creat d0/d3/d1b/d4c/fb1 x:0 0 0 2024-06-15T01:07:04.283 INFO:tasks.workunit.client.0.smithi137.stdout:1/487: write d0/d1f/d76/f80 [3007246,33506] 0 2024-06-15T01:07:04.284 INFO:tasks.workunit.client.1.smithi142.stdout:6/544: creat d1a/d39/d42/fbf x:0 0 0 2024-06-15T01:07:04.284 INFO:tasks.workunit.client.0.smithi137.stdout:3/490: creat d0/d2/d8/fa2 x:0 0 0 2024-06-15T01:07:04.286 INFO:tasks.workunit.client.0.smithi137.stdout:3/491: fsync d0/d2/d22/d24/d5e/f85 0 2024-06-15T01:07:04.286 INFO:tasks.workunit.client.0.smithi137.stdout:1/488: mknod d0/d1f/d7d/cb2 0 2024-06-15T01:07:04.289 INFO:tasks.workunit.client.1.smithi142.stdout:5/521: symlink d2/d3e/d98/la3 0 2024-06-15T01:07:04.291 INFO:tasks.workunit.client.1.smithi142.stdout:2/498: unlink d8/df/d10/d94/d25/d22/d43/l5f 0 2024-06-15T01:07:04.291 INFO:tasks.workunit.client.1.smithi142.stdout:6/545: dread f6 [0,4194304] 0 2024-06-15T01:07:04.291 INFO:tasks.workunit.client.1.smithi142.stdout:8/535: getdents d0/d17/d40/d86/d9d 0 2024-06-15T01:07:04.292 INFO:tasks.workunit.client.1.smithi142.stdout:8/536: chown d0/d25/d5a/ld 0 1 2024-06-15T01:07:04.292 INFO:tasks.workunit.client.1.smithi142.stdout:5/522: fsync d2/de/d19/f2a 0 2024-06-15T01:07:04.295 INFO:tasks.workunit.client.1.smithi142.stdout:6/546: creat d1a/d3c/db0/fc0 x:0 0 0 2024-06-15T01:07:04.295 INFO:tasks.workunit.client.0.smithi137.stdout:6/355: dwrite d0/d5/d9/d3f/f67 [0,4194304] 0 2024-06-15T01:07:04.295 INFO:tasks.workunit.client.0.smithi137.stdout:1/489: dwrite d0/d3/f13 [0,4194304] 0 2024-06-15T01:07:04.296 INFO:tasks.workunit.client.1.smithi142.stdout:5/523: chown d2/de/d19/l80 3 1 2024-06-15T01:07:04.296 INFO:tasks.workunit.client.0.smithi137.stdout:1/490: chown d0/d3/d1b 0 1 2024-06-15T01:07:04.298 INFO:tasks.workunit.client.1.smithi142.stdout:8/537: mkdir d0/d25/d53/da7 0 2024-06-15T01:07:04.298 INFO:tasks.workunit.client.0.smithi137.stdout:6/356: write d0/d1a/d3e/d57/d1d/d2e/f36 [3099252,82604] 0 2024-06-15T01:07:04.300 INFO:tasks.workunit.client.1.smithi142.stdout:7/543: write d2/d6/f83 [648199,7239] 0 2024-06-15T01:07:04.301 INFO:tasks.workunit.client.1.smithi142.stdout:8/538: readlink d0/d25/d5a/l3b 0 2024-06-15T01:07:04.301 INFO:tasks.workunit.client.1.smithi142.stdout:2/499: dwrite d8/df/d1f/f3e [0,4194304] 0 2024-06-15T01:07:04.302 INFO:tasks.workunit.client.1.smithi142.stdout:8/539: read - d0/d25/d5a/f88 zero size 2024-06-15T01:07:04.303 INFO:tasks.workunit.client.0.smithi137.stdout:6/357: rename d0/d5/d9/l54 to d0/d1a/d3e/l6e 0 2024-06-15T01:07:04.304 INFO:tasks.workunit.client.1.smithi142.stdout:7/544: write d2/d8/f42 [623592,69818] 0 2024-06-15T01:07:04.304 INFO:tasks.workunit.client.1.smithi142.stdout:8/540: stat d0/f14 0 2024-06-15T01:07:04.304 INFO:tasks.workunit.client.0.smithi137.stdout:6/358: chown d0/d1a/d3e/d57/d1d/f40 1910658668 1 2024-06-15T01:07:04.305 INFO:tasks.workunit.client.0.smithi137.stdout:1/491: link d0/d2d/d3e/f77 d0/d3/d89/fb3 0 2024-06-15T01:07:04.306 INFO:tasks.workunit.client.1.smithi142.stdout:8/541: write d0/d17/d40/d86/f9b [554585,40620] 0 2024-06-15T01:07:04.309 INFO:tasks.workunit.client.0.smithi137.stdout:6/359: unlink d0/d5/c11 0 2024-06-15T01:07:04.309 INFO:tasks.workunit.client.1.smithi142.stdout:6/547: getdents d1a/d39/d42/d4c/d58/d87/db7 0 2024-06-15T01:07:04.311 INFO:tasks.workunit.client.1.smithi142.stdout:7/545: read d2/d6/f83 [326883,65176] 0 2024-06-15T01:07:04.314 INFO:tasks.workunit.client.1.smithi142.stdout:9/488: dread d6/f6a [0,4194304] 0 2024-06-15T01:07:04.314 INFO:tasks.workunit.client.1.smithi142.stdout:7/546: stat d2/d8/da/d33/d25/f67 0 2024-06-15T01:07:04.315 INFO:tasks.workunit.client.1.smithi142.stdout:9/489: readlink d6/l42 0 2024-06-15T01:07:04.315 INFO:tasks.workunit.client.0.smithi137.stdout:1/492: getdents d0/d1f 0 2024-06-15T01:07:04.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: pgmap v17: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 231 MiB/s wr, 400 op/s 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:03 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:04.318 INFO:tasks.workunit.client.1.smithi142.stdout:7/547: write d2/d6/d22/d47/f7c [3616193,77460] 0 2024-06-15T01:07:04.320 INFO:tasks.workunit.client.0.smithi137.stdout:1/493: truncate d0/d1f/d60/f55 599676 0 2024-06-15T01:07:04.320 INFO:tasks.workunit.client.0.smithi137.stdout:6/360: dwrite d0/d5/d43/f61 [0,4194304] 0 2024-06-15T01:07:04.321 INFO:tasks.workunit.client.1.smithi142.stdout:2/500: getdents d8/df/d10/d94/d25/d76 0 2024-06-15T01:07:04.322 INFO:tasks.workunit.client.0.smithi137.stdout:1/494: write d0/d3/d14/d34/f45 [2893109,122958] 0 2024-06-15T01:07:04.326 INFO:tasks.workunit.client.1.smithi142.stdout:9/490: symlink d6/dc/d4e/dae/lc1 0 2024-06-15T01:07:04.327 INFO:tasks.workunit.client.1.smithi142.stdout:8/542: rename d0/f33 to d0/d17/d40/fa8 0 2024-06-15T01:07:04.328 INFO:tasks.workunit.client.1.smithi142.stdout:8/543: readlink d0/d25/d5a/l3b 0 2024-06-15T01:07:04.329 INFO:tasks.workunit.client.0.smithi137.stdout:6/361: dwrite d0/d1a/f65 [0,4194304] 0 2024-06-15T01:07:04.331 INFO:tasks.workunit.client.0.smithi137.stdout:1/495: dread d0/d2d/f5e [0,4194304] 0 2024-06-15T01:07:04.334 INFO:tasks.workunit.client.1.smithi142.stdout:6/548: rename d1a/d2d/l60 to d1a/d39/d42/d4c/d58/d87/lc1 0 2024-06-15T01:07:04.335 INFO:tasks.workunit.client.0.smithi137.stdout:6/362: mknod d0/d5/d9/c6f 0 2024-06-15T01:07:04.336 INFO:tasks.workunit.client.1.smithi142.stdout:7/548: link d2/d8/dd/l60 d2/d8/dd/d14/d2c/d37/la9 0 2024-06-15T01:07:04.336 INFO:tasks.workunit.client.1.smithi142.stdout:7/549: fdatasync d2/f3 0 2024-06-15T01:07:04.337 INFO:tasks.workunit.client.1.smithi142.stdout:7/550: fsync d2/f49 0 2024-06-15T01:07:04.339 INFO:tasks.workunit.client.0.smithi137.stdout:6/363: mkdir d0/d1a/d3e/d57/d1d/d2e/d70 0 2024-06-15T01:07:04.339 INFO:tasks.workunit.client.1.smithi142.stdout:9/491: rename d6/dc/d1b/d25/d58/c9e to d6/dc/d1b/d25/cc2 0 2024-06-15T01:07:04.340 INFO:tasks.workunit.client.0.smithi137.stdout:1/496: dwrite d0/d10/d41/d7b/f90 [4194304,4194304] 0 2024-06-15T01:07:04.340 INFO:tasks.workunit.client.1.smithi142.stdout:6/549: dwrite d1a/d2d/f52 [0,4194304] 0 2024-06-15T01:07:04.342 INFO:tasks.workunit.client.0.smithi137.stdout:6/364: creat d0/d1a/d3e/f71 x:0 0 0 2024-06-15T01:07:04.342 INFO:tasks.workunit.client.1.smithi142.stdout:7/551: unlink d2/d8/dd/d14/la7 0 2024-06-15T01:07:04.343 INFO:tasks.workunit.client.1.smithi142.stdout:9/492: symlink d6/dc/d1b/d6f/lc3 0 2024-06-15T01:07:04.344 INFO:tasks.workunit.client.1.smithi142.stdout:2/501: getdents d8/df/d10/d94/d25/d22/d43/d52/d7a/d98 0 2024-06-15T01:07:04.344 INFO:tasks.workunit.client.0.smithi137.stdout:1/497: truncate d0/f3b 78402 0 2024-06-15T01:07:04.348 INFO:tasks.workunit.client.1.smithi142.stdout:6/550: dwrite d1a/d39/d3a/f84 [0,4194304] 0 2024-06-15T01:07:04.349 INFO:tasks.workunit.client.1.smithi142.stdout:6/551: stat d1a/d2d/d68 0 2024-06-15T01:07:04.350 INFO:tasks.workunit.client.1.smithi142.stdout:2/502: creat d8/d19/d33/fa2 x:0 0 0 2024-06-15T01:07:04.355 INFO:tasks.workunit.client.1.smithi142.stdout:1/573: truncate d6/d46/dc3/fca 1296903 0 2024-06-15T01:07:04.356 INFO:tasks.workunit.client.1.smithi142.stdout:6/552: dwrite d1a/d3c/fa7 [0,4194304] 0 2024-06-15T01:07:04.360 INFO:tasks.workunit.client.1.smithi142.stdout:3/626: dread d0/d1f/d20/fad [0,4194304] 0 2024-06-15T01:07:04.360 INFO:tasks.workunit.client.1.smithi142.stdout:0/542: read d3/d39/d9a/fb8 [196130,51256] 0 2024-06-15T01:07:04.363 INFO:tasks.workunit.client.1.smithi142.stdout:0/543: write d3/d39/d3e/d81/d44/d7c/d22/d59/f9c [40579,93369] 0 2024-06-15T01:07:04.366 INFO:tasks.workunit.client.1.smithi142.stdout:1/574: rmdir d6/d12/da0/db0 39 2024-06-15T01:07:04.367 INFO:tasks.workunit.client.1.smithi142.stdout:1/575: chown d6/d12/d28/l2e 630 1 2024-06-15T01:07:04.368 INFO:tasks.workunit.client.0.smithi137.stdout:1/498: dwrite d0/d2d/d3e/f77 [0,4194304] 0 2024-06-15T01:07:04.369 INFO:tasks.workunit.client.0.smithi137.stdout:1/499: readlink d0/d10/l37 0 2024-06-15T01:07:04.370 INFO:tasks.workunit.client.1.smithi142.stdout:0/544: dread d3/d39/d3e/d81/d4c/da0/fb2 [0,4194304] 0 2024-06-15T01:07:04.371 INFO:tasks.workunit.client.0.smithi137.stdout:1/500: fsync d0/d3/f22 0 2024-06-15T01:07:04.375 INFO:tasks.workunit.client.1.smithi142.stdout:8/544: truncate d0/d25/f35 2202440 0 2024-06-15T01:07:04.375 INFO:tasks.workunit.client.0.smithi137.stdout:1/501: creat d0/d3/d1b/d4c/d59/fb4 x:0 0 0 2024-06-15T01:07:04.378 INFO:tasks.workunit.client.0.smithi137.stdout:6/365: dread d0/d1a/d3e/d57/d1d/f40 [0,4194304] 0 2024-06-15T01:07:04.380 INFO:tasks.workunit.client.1.smithi142.stdout:3/627: unlink d0/d3/d15/c65 0 2024-06-15T01:07:04.382 INFO:tasks.workunit.client.1.smithi142.stdout:3/628: write d0/d1f/d20/d5d/d82/f9a [4923473,5268] 0 2024-06-15T01:07:04.383 INFO:tasks.workunit.client.1.smithi142.stdout:3/629: chown d0/d1f/d20/c7f 7651 1 2024-06-15T01:07:04.386 INFO:tasks.workunit.client.1.smithi142.stdout:6/553: symlink d1a/d39/d3a/d51/d62/da8/lc2 0 2024-06-15T01:07:04.386 INFO:tasks.workunit.client.1.smithi142.stdout:7/552: rmdir d2/d8 39 2024-06-15T01:07:04.388 INFO:tasks.workunit.client.1.smithi142.stdout:9/493: write d6/dc/d1b/d6f/da1/fa7 [40044,14682] 0 2024-06-15T01:07:04.388 INFO:tasks.workunit.client.0.smithi137.stdout:1/502: dwrite d0/d3/d14/d34/f70 [0,4194304] 0 2024-06-15T01:07:04.391 INFO:tasks.workunit.client.1.smithi142.stdout:2/503: write d8/d19/f70 [1093408,58521] 0 2024-06-15T01:07:04.393 INFO:tasks.workunit.client.0.smithi137.stdout:6/366: dwrite d0/f39 [0,4194304] 0 2024-06-15T01:07:04.394 INFO:tasks.workunit.client.1.smithi142.stdout:3/630: dwrite d0/d1f/d20/d59/d61/d96/fbe [0,4194304] 0 2024-06-15T01:07:04.395 INFO:tasks.workunit.client.1.smithi142.stdout:8/545: dread d0/f5f [0,4194304] 0 2024-06-15T01:07:04.396 INFO:tasks.workunit.client.1.smithi142.stdout:3/631: write d0/d95/fb4 [223338,56976] 0 2024-06-15T01:07:04.397 INFO:tasks.workunit.client.0.smithi137.stdout:6/367: write d0/d5/d9/d3f/f67 [5200000,90732] 0 2024-06-15T01:07:04.401 INFO:tasks.workunit.client.1.smithi142.stdout:4/629: sync 2024-06-15T01:07:04.401 INFO:tasks.workunit.client.1.smithi142.stdout:2/504: mkdir d8/d19/da3 0 2024-06-15T01:07:04.401 INFO:tasks.workunit.client.0.smithi137.stdout:1/503: dwrite d0/d3/d47/f5d [0,4194304] 0 2024-06-15T01:07:04.402 INFO:tasks.workunit.client.1.smithi142.stdout:9/494: dwrite d6/dc/d4e/d65/fa2 [0,4194304] 0 2024-06-15T01:07:04.403 INFO:tasks.workunit.client.1.smithi142.stdout:6/554: dread d1a/d2d/d2e/f49 [0,4194304] 0 2024-06-15T01:07:04.406 INFO:tasks.workunit.client.1.smithi142.stdout:7/553: creat d2/d8/da8/faa x:0 0 0 2024-06-15T01:07:04.406 INFO:tasks.workunit.client.1.smithi142.stdout:8/546: dread d0/f4c [0,4194304] 0 2024-06-15T01:07:04.406 INFO:tasks.workunit.client.1.smithi142.stdout:1/576: getdents d6/d12/da0 0 2024-06-15T01:07:04.407 INFO:tasks.workunit.client.0.smithi137.stdout:6/368: dwrite d0/d1a/f65 [0,4194304] 0 2024-06-15T01:07:04.408 INFO:tasks.workunit.client.1.smithi142.stdout:9/495: write d6/dc/f8e [566634,86992] 0 2024-06-15T01:07:04.408 INFO:tasks.workunit.client.1.smithi142.stdout:4/630: mkdir d5/d1c/d6f/dc7/de1 0 2024-06-15T01:07:04.409 INFO:tasks.workunit.client.1.smithi142.stdout:7/554: fdatasync d2/d8/dd/f12 0 2024-06-15T01:07:04.411 INFO:tasks.workunit.client.1.smithi142.stdout:2/505: write d8/df/d10/d94/d25/d1a/d57/f69 [771113,50708] 0 2024-06-15T01:07:04.412 INFO:tasks.workunit.client.0.smithi137.stdout:6/369: unlink d0/d1a/d3e/d57/d1d/f3d 0 2024-06-15T01:07:04.412 INFO:tasks.workunit.client.0.smithi137.stdout:6/370: dread - d0/d1a/d3e/f71 zero size 2024-06-15T01:07:04.412 INFO:tasks.workunit.client.1.smithi142.stdout:1/577: write d6/f90 [717770,43723] 0 2024-06-15T01:07:04.414 INFO:tasks.workunit.client.1.smithi142.stdout:9/496: write d6/d34/f3a [3330381,15634] 0 2024-06-15T01:07:04.414 INFO:tasks.workunit.client.1.smithi142.stdout:5/524: sync 2024-06-15T01:07:04.414 INFO:tasks.workunit.client.0.smithi137.stdout:6/371: symlink d0/d1a/d3e/d57/d1d/d2e/d62/l72 0 2024-06-15T01:07:04.415 INFO:tasks.workunit.client.1.smithi142.stdout:1/578: write d6/d46/d61/fac [956749,31428] 0 2024-06-15T01:07:04.416 INFO:tasks.workunit.client.1.smithi142.stdout:4/631: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f6d 4634450 0 2024-06-15T01:07:04.417 INFO:tasks.workunit.client.0.smithi137.stdout:6/372: dread d0/d5/d43/f61 [0,4194304] 0 2024-06-15T01:07:04.450 INFO:tasks.workunit.client.0.smithi137.stdout:6/373: chown d0/d1a/d3e/d57/c28 121453 1 2024-06-15T01:07:04.455 INFO:tasks.workunit.client.0.smithi137.stdout:6/374: dread d0/d5/d9/fe [0,4194304] 0 2024-06-15T01:07:04.462 INFO:tasks.workunit.client.1.smithi142.stdout:6/555: mknod d1a/d2d/d80/cc3 0 2024-06-15T01:07:04.467 INFO:tasks.workunit.client.1.smithi142.stdout:7/555: unlink d2/d8/f77 0 2024-06-15T01:07:04.469 INFO:tasks.workunit.client.1.smithi142.stdout:8/547: creat d0/d17/d40/d6e/fa9 x:0 0 0 2024-06-15T01:07:04.470 INFO:tasks.workunit.client.1.smithi142.stdout:9/497: rmdir d6/dc/d1b/d6f/da1 39 2024-06-15T01:07:04.470 INFO:tasks.workunit.client.1.smithi142.stdout:8/548: readlink d0/d25/d5a/l2e 0 2024-06-15T01:07:04.471 INFO:tasks.workunit.client.1.smithi142.stdout:5/525: symlink d2/de/d85/la4 0 2024-06-15T01:07:04.473 INFO:tasks.workunit.client.1.smithi142.stdout:9/498: chown d6/dc/dd/ca6 468897446 1 2024-06-15T01:07:04.475 INFO:tasks.workunit.client.1.smithi142.stdout:0/545: read d3/d39/d3e/d81/d44/d7c/d22/f2d [78057,43464] 0 2024-06-15T01:07:04.476 INFO:tasks.workunit.client.1.smithi142.stdout:3/632: getdents d0/d3/d15 0 2024-06-15T01:07:04.480 INFO:tasks.workunit.client.1.smithi142.stdout:7/556: dread d2/f1a [0,4194304] 0 2024-06-15T01:07:04.480 INFO:tasks.workunit.client.1.smithi142.stdout:3/633: write d0/d3/d15/f83 [804891,74524] 0 2024-06-15T01:07:04.483 INFO:tasks.workunit.client.1.smithi142.stdout:9/499: creat d6/d31/fc4 x:0 0 0 2024-06-15T01:07:04.484 INFO:tasks.workunit.client.1.smithi142.stdout:0/546: mknod d3/d39/d3e/d81/d4c/d61/cc3 0 2024-06-15T01:07:04.484 INFO:tasks.workunit.client.1.smithi142.stdout:4/632: link d5/d9/d23/l2c d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/le2 0 2024-06-15T01:07:04.485 INFO:tasks.workunit.client.1.smithi142.stdout:8/549: dwrite d0/f3f [0,4194304] 0 2024-06-15T01:07:04.486 INFO:tasks.workunit.client.1.smithi142.stdout:9/500: dread - d6/dc/f61 zero size 2024-06-15T01:07:04.486 INFO:tasks.workunit.client.1.smithi142.stdout:6/556: rmdir d1a/d39/d42/d4c/d58/d87/db7 0 2024-06-15T01:07:04.487 INFO:tasks.workunit.client.0.smithi137.stdout:3/492: dread d0/d2/d22/d24/f47 [4194304,4194304] 0 2024-06-15T01:07:04.488 INFO:tasks.workunit.client.1.smithi142.stdout:7/557: write d2/d8/da/d33/f20 [4228545,711] 0 2024-06-15T01:07:04.489 INFO:tasks.workunit.client.1.smithi142.stdout:5/526: read d2/d18/f55 [789520,63248] 0 2024-06-15T01:07:04.492 INFO:tasks.workunit.client.1.smithi142.stdout:4/633: write d5/d1c/d6f/dc7/f3d [1263534,12676] 0 2024-06-15T01:07:04.493 INFO:tasks.workunit.client.1.smithi142.stdout:3/634: creat d0/d3/da/d71/d55/dae/fc2 x:0 0 0 2024-06-15T01:07:04.493 INFO:tasks.workunit.client.0.smithi137.stdout:3/493: dwrite d0/f95 [0,4194304] 0 2024-06-15T01:07:04.494 INFO:tasks.workunit.client.1.smithi142.stdout:0/547: dread d3/d39/d3e/d81/d44/d7c/d22/d59/f9c [0,4194304] 0 2024-06-15T01:07:04.497 INFO:tasks.workunit.client.0.smithi137.stdout:5/559: dread f3 [0,4194304] 0 2024-06-15T01:07:04.497 INFO:tasks.workunit.client.1.smithi142.stdout:8/550: write d0/d17/d40/d5e/f81 [621831,122375] 0 2024-06-15T01:07:04.498 INFO:tasks.workunit.client.1.smithi142.stdout:7/558: chown d2/d8/dd/d14/d2c/d37/la9 340103 1 2024-06-15T01:07:04.498 INFO:tasks.workunit.client.0.smithi137.stdout:3/494: link d0/d4c/l5a d0/d42/la3 0 2024-06-15T01:07:04.499 INFO:tasks.workunit.client.1.smithi142.stdout:4/634: mknod d5/d9/da/ce3 0 2024-06-15T01:07:04.500 INFO:tasks.workunit.client.0.smithi137.stdout:3/495: truncate d0/d2/d1e/f81 534472 0 2024-06-15T01:07:04.500 INFO:tasks.workunit.client.1.smithi142.stdout:9/501: link d6/d34/l6b d6/dc/d1b/d25/d58/d74/lc5 0 2024-06-15T01:07:04.501 INFO:tasks.workunit.client.0.smithi137.stdout:5/560: mknod df/d44/d54/d95/d3b/cbd 0 2024-06-15T01:07:04.503 INFO:tasks.workunit.client.1.smithi142.stdout:8/551: creat d0/faa x:0 0 0 2024-06-15T01:07:04.506 INFO:tasks.workunit.client.1.smithi142.stdout:3/635: creat d0/d1f/d20/d5d/fc3 x:0 0 0 2024-06-15T01:07:04.507 INFO:tasks.workunit.client.1.smithi142.stdout:9/502: creat d6/dc/d1b/fc6 x:0 0 0 2024-06-15T01:07:04.509 INFO:tasks.workunit.client.1.smithi142.stdout:4/635: mknod d5/d9/d90/daf/ce4 0 2024-06-15T01:07:04.511 INFO:tasks.workunit.client.1.smithi142.stdout:8/552: rename d0/c5b to d0/d25/d5a/d8f/cab 0 2024-06-15T01:07:04.512 INFO:tasks.workunit.client.1.smithi142.stdout:8/553: write d0/d17/f1f [3458898,100612] 0 2024-06-15T01:07:04.514 INFO:tasks.workunit.client.1.smithi142.stdout:7/559: link d2/d6/l64 d2/d6/d22/lab 0 2024-06-15T01:07:04.514 INFO:tasks.workunit.client.1.smithi142.stdout:9/503: dwrite d6/dc/f19 [0,4194304] 0 2024-06-15T01:07:04.517 INFO:tasks.workunit.client.1.smithi142.stdout:1/579: truncate d6/fc 2096504 0 2024-06-15T01:07:04.519 INFO:tasks.workunit.client.0.smithi137.stdout:6/375: dwrite d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:04.519 INFO:tasks.workunit.client.1.smithi142.stdout:3/636: mknod d0/d3/da/d71/cc4 0 2024-06-15T01:07:04.520 INFO:tasks.workunit.client.1.smithi142.stdout:7/560: mknod d2/d6/d22/d3c/cac 0 2024-06-15T01:07:04.520 INFO:tasks.workunit.client.1.smithi142.stdout:9/504: rmdir d6/dc/d51 39 2024-06-15T01:07:04.522 INFO:tasks.workunit.client.1.smithi142.stdout:9/505: read d6/dc/d1b/f4b [941757,83064] 0 2024-06-15T01:07:04.522 INFO:tasks.workunit.client.1.smithi142.stdout:1/580: mknod d6/d25/cd0 0 2024-06-15T01:07:04.523 INFO:tasks.workunit.client.0.smithi137.stdout:6/376: write d0/d5/d46/f5d [2521251,93712] 0 2024-06-15T01:07:04.528 INFO:tasks.workunit.client.0.smithi137.stdout:6/377: link d0/d1a/f65 d0/d5/f73 0 2024-06-15T01:07:04.531 INFO:tasks.workunit.client.0.smithi137.stdout:6/378: unlink d0/d5/l30 0 2024-06-15T01:07:04.534 INFO:tasks.workunit.client.1.smithi142.stdout:6/557: dread d1a/d39/d42/d4c/d58/f7a [0,4194304] 0 2024-06-15T01:07:04.535 INFO:tasks.workunit.client.0.smithi137.stdout:6/379: rename d0/d5/d43 to d0/d1a/d3e/d57/d32/d74 0 2024-06-15T01:07:04.535 INFO:tasks.workunit.client.1.smithi142.stdout:1/581: dwrite d6/f9 [0,4194304] 0 2024-06-15T01:07:04.536 INFO:tasks.workunit.client.1.smithi142.stdout:9/506: dwrite d6/dc/d1b/d25/d58/f64 [0,4194304] 0 2024-06-15T01:07:04.539 INFO:tasks.workunit.client.1.smithi142.stdout:6/558: creat d1a/d3c/fc4 x:0 0 0 2024-06-15T01:07:04.541 INFO:tasks.workunit.client.1.smithi142.stdout:6/559: chown d1a/d39/d42 1 1 2024-06-15T01:07:04.543 INFO:tasks.workunit.client.1.smithi142.stdout:1/582: rename d6/d12/d1c/fc0 to d6/d12/d80/dc8/fd1 0 2024-06-15T01:07:04.544 INFO:tasks.workunit.client.1.smithi142.stdout:8/554: dread d0/d17/d40/f76 [0,4194304] 0 2024-06-15T01:07:04.545 INFO:tasks.workunit.client.1.smithi142.stdout:6/560: symlink d1a/d2d/d80/lc5 0 2024-06-15T01:07:04.546 INFO:tasks.workunit.client.1.smithi142.stdout:8/555: stat d0/d17/d40/d6e/f73 0 2024-06-15T01:07:04.549 INFO:tasks.workunit.client.1.smithi142.stdout:9/507: read d6/d34/fc0 [1774794,121643] 0 2024-06-15T01:07:04.549 INFO:tasks.workunit.client.1.smithi142.stdout:9/508: readlink d6/dc/l39 0 2024-06-15T01:07:04.551 INFO:tasks.workunit.client.0.smithi137.stdout:3/496: write d0/d2f/f35 [4840,101399] 0 2024-06-15T01:07:04.551 INFO:tasks.workunit.client.1.smithi142.stdout:1/583: rename d6/d46/d61 to d6/d12/da0/db0/dd2 0 2024-06-15T01:07:04.552 INFO:tasks.workunit.client.1.smithi142.stdout:8/556: fsync d0/d25/d5a/f29 0 2024-06-15T01:07:04.553 INFO:tasks.workunit.client.0.smithi137.stdout:6/380: dread d0/d1a/d3e/d57/d1d/d2e/f36 [0,4194304] 0 2024-06-15T01:07:04.553 INFO:tasks.workunit.client.1.smithi142.stdout:8/557: dread - d0/d17/d40/fa2 zero size 2024-06-15T01:07:04.554 INFO:tasks.workunit.client.1.smithi142.stdout:8/558: readlink d0/d25/d5a/l3b 0 2024-06-15T01:07:04.555 INFO:tasks.workunit.client.0.smithi137.stdout:6/381: creat d0/d1a/f75 x:0 0 0 2024-06-15T01:07:04.555 INFO:tasks.workunit.client.1.smithi142.stdout:3/637: dread d0/d3/da/d18/f28 [0,4194304] 0 2024-06-15T01:07:04.556 INFO:tasks.workunit.client.0.smithi137.stdout:3/497: rename d0/d2/d22/d24/d28/l4a to d0/d56/d8f/d9c/d9f/la4 0 2024-06-15T01:07:04.556 INFO:tasks.workunit.client.1.smithi142.stdout:9/509: fsync d6/d31/f84 0 2024-06-15T01:07:04.558 INFO:tasks.workunit.client.0.smithi137.stdout:3/498: creat d0/d56/d8f/d9c/d9f/fa5 x:0 0 0 2024-06-15T01:07:04.563 INFO:tasks.workunit.client.1.smithi142.stdout:0/548: dwrite d3/d39/d3e/d81/f34 [0,4194304] 0 2024-06-15T01:07:04.565 INFO:tasks.workunit.client.1.smithi142.stdout:8/559: creat d0/d25/d53/d9c/fac x:0 0 0 2024-06-15T01:07:04.565 INFO:tasks.workunit.client.0.smithi137.stdout:3/499: dwrite d0/d56/d8f/d7b/f96 [0,4194304] 0 2024-06-15T01:07:04.575 INFO:tasks.workunit.client.0.smithi137.stdout:3/500: rename d0/d2/d22/d63/fa0 to d0/d2/d22/d24/d28/fa6 0 2024-06-15T01:07:04.577 INFO:tasks.workunit.client.1.smithi142.stdout:2/506: sync 2024-06-15T01:07:04.579 INFO:tasks.workunit.client.1.smithi142.stdout:7/561: write d2/d8/f17 [4745618,43018] 0 2024-06-15T01:07:04.581 INFO:tasks.workunit.client.1.smithi142.stdout:9/510: dwrite d6/f6a [0,4194304] 0 2024-06-15T01:07:04.582 INFO:tasks.workunit.client.0.smithi137.stdout:3/501: dwrite d0/d2/d22/d24/d28/d2e/f3b [0,4194304] 0 2024-06-15T01:07:04.584 INFO:tasks.workunit.client.1.smithi142.stdout:1/584: rename d6/d12/da0/db0/dd2 to d6/d12/d3c/d63/d8d/d97/dd3 0 2024-06-15T01:07:04.584 INFO:tasks.workunit.client.1.smithi142.stdout:0/549: dwrite d3/d39/d3e/d81/d44/d7c/d22/d59/f94 [0,4194304] 0 2024-06-15T01:07:04.587 INFO:tasks.workunit.client.1.smithi142.stdout:6/561: truncate d1a/d83/fbb 917809 0 2024-06-15T01:07:04.590 INFO:tasks.workunit.client.0.smithi137.stdout:3/502: dwrite d0/d2/d22/d24/f54 [0,4194304] 0 2024-06-15T01:07:04.591 INFO:tasks.workunit.client.1.smithi142.stdout:1/585: mkdir d6/d3a/dd4 0 2024-06-15T01:07:04.591 INFO:tasks.workunit.client.1.smithi142.stdout:4/636: dwrite d5/d9/d18/f80 [4194304,4194304] 0 2024-06-15T01:07:04.591 INFO:tasks.workunit.client.1.smithi142.stdout:2/507: dread - d8/df/d10/d94/d3a/f6f zero size 2024-06-15T01:07:04.594 INFO:tasks.workunit.client.0.smithi137.stdout:3/503: write d0/d2f/f69 [264824,100076] 0 2024-06-15T01:07:04.595 INFO:tasks.workunit.client.0.smithi137.stdout:3/504: dread - d0/d2/d22/d24/d5e/f98 zero size 2024-06-15T01:07:04.596 INFO:tasks.workunit.client.1.smithi142.stdout:7/562: chown d2/d8/da/d33/d25/d27/l53 32863 1 2024-06-15T01:07:04.596 INFO:tasks.workunit.client.1.smithi142.stdout:2/508: write d8/d19/d33/fa0 [838961,105335] 0 2024-06-15T01:07:04.598 INFO:tasks.workunit.client.0.smithi137.stdout:3/505: mknod d0/d42/d79/ca7 0 2024-06-15T01:07:04.601 INFO:tasks.workunit.client.1.smithi142.stdout:8/560: dwrite d0/d25/d66/f92 [0,4194304] 0 2024-06-15T01:07:04.601 INFO:tasks.workunit.client.0.smithi137.stdout:3/506: creat d0/d56/d8f/fa8 x:0 0 0 2024-06-15T01:07:04.602 INFO:tasks.workunit.client.1.smithi142.stdout:9/511: dread d6/dc/d4e/d65/fa2 [0,4194304] 0 2024-06-15T01:07:04.602 INFO:tasks.workunit.client.0.smithi137.stdout:3/507: chown d0/d13/f20 15706 1 2024-06-15T01:07:04.603 INFO:tasks.workunit.client.0.smithi137.stdout:3/508: chown d0/d2/d22/d24/d28/d2e 363712 1 2024-06-15T01:07:04.608 INFO:tasks.workunit.client.1.smithi142.stdout:3/638: dread d0/d1f/d20/f26 [0,4194304] 0 2024-06-15T01:07:04.612 INFO:tasks.workunit.client.1.smithi142.stdout:8/561: creat d0/d17/d40/d5e/fad x:0 0 0 2024-06-15T01:07:04.613 INFO:tasks.workunit.client.0.smithi137.stdout:3/509: dwrite d0/d2f/f69 [0,4194304] 0 2024-06-15T01:07:04.615 INFO:tasks.workunit.client.0.smithi137.stdout:3/510: chown d0/d56/d8f/d9c 5365 1 2024-06-15T01:07:04.616 INFO:tasks.workunit.client.0.smithi137.stdout:3/511: chown d0/f1a 4399 1 2024-06-15T01:07:04.622 INFO:tasks.workunit.client.1.smithi142.stdout:7/563: link d2/d8/f17 d2/d8/dd/d14/d2c/d37/d6b/fad 0 2024-06-15T01:07:04.622 INFO:tasks.workunit.client.1.smithi142.stdout:8/562: dwrite d0/d17/d40/d5e/fad [0,4194304] 0 2024-06-15T01:07:04.625 INFO:tasks.workunit.client.1.smithi142.stdout:2/509: rename d8/df/d10/d94/d25/d22/d43/d52/d7a/f92 to d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa4 0 2024-06-15T01:07:04.626 INFO:tasks.workunit.client.1.smithi142.stdout:6/562: creat d1a/d39/d42/d4c/fc6 x:0 0 0 2024-06-15T01:07:04.628 INFO:tasks.workunit.client.1.smithi142.stdout:9/512: rename d6 to d6/dc/d51/d53/dc7 22 2024-06-15T01:07:04.628 INFO:tasks.workunit.client.1.smithi142.stdout:4/637: write d5/d1c/fbb [404185,89903] 0 2024-06-15T01:07:04.632 INFO:tasks.workunit.client.1.smithi142.stdout:8/563: chown d0/d17/f56 10389 1 2024-06-15T01:07:04.632 INFO:tasks.workunit.client.1.smithi142.stdout:3/639: dwrite d0/d3/da/d71/f97 [0,4194304] 0 2024-06-15T01:07:04.634 INFO:tasks.workunit.client.1.smithi142.stdout:7/564: rename d2/d6/f43 to d2/d8/da/fae 0 2024-06-15T01:07:04.639 INFO:tasks.workunit.client.1.smithi142.stdout:9/513: symlink d6/dc/d1b/d25/d58/d74/lc8 0 2024-06-15T01:07:04.640 INFO:tasks.workunit.client.1.smithi142.stdout:7/565: mkdir d2/d6/daf 0 2024-06-15T01:07:04.643 INFO:tasks.workunit.client.1.smithi142.stdout:8/564: creat d0/d25/d5a/d8f/d97/fae x:0 0 0 2024-06-15T01:07:04.648 INFO:tasks.workunit.client.1.smithi142.stdout:5/527: sync 2024-06-15T01:07:04.649 INFO:tasks.workunit.client.1.smithi142.stdout:8/565: fdatasync d0/d25/d53/f58 0 2024-06-15T01:07:04.649 INFO:tasks.workunit.client.1.smithi142.stdout:4/638: dwrite d5/d9/fd [0,4194304] 0 2024-06-15T01:07:04.652 INFO:tasks.workunit.client.1.smithi142.stdout:0/550: dwrite d3/d39/d3e/d81/f1d [4194304,4194304] 0 2024-06-15T01:07:04.653 INFO:tasks.workunit.client.1.smithi142.stdout:6/563: dread d1a/d39/d3a/f84 [0,4194304] 0 2024-06-15T01:07:04.653 INFO:tasks.workunit.client.1.smithi142.stdout:9/514: dread d6/dc/f8e [0,4194304] 0 2024-06-15T01:07:04.655 INFO:tasks.workunit.client.1.smithi142.stdout:8/566: fdatasync d0/d17/f61 0 2024-06-15T01:07:04.655 INFO:tasks.workunit.client.1.smithi142.stdout:3/640: dwrite d0/fba [0,4194304] 0 2024-06-15T01:07:04.655 INFO:tasks.workunit.client.1.smithi142.stdout:7/566: rename d2/d8/da/c26 to d2/d8/dd/d8a/d94/cb0 0 2024-06-15T01:07:04.656 INFO:tasks.workunit.client.1.smithi142.stdout:5/528: unlink d2/de/d85/l97 0 2024-06-15T01:07:04.660 INFO:tasks.workunit.client.1.smithi142.stdout:9/515: rmdir d6/d34 39 2024-06-15T01:07:04.662 INFO:tasks.workunit.client.0.smithi137.stdout:3/512: dread d0/d13/f20 [0,4194304] 0 2024-06-15T01:07:04.663 INFO:tasks.workunit.client.1.smithi142.stdout:4/639: dread d5/d1c/d6f/dc7/d30/dcc/f56 [0,4194304] 0 2024-06-15T01:07:04.663 INFO:tasks.workunit.client.1.smithi142.stdout:9/516: write d6/dc/dd/f4f [1352830,52629] 0 2024-06-15T01:07:04.663 INFO:tasks.workunit.client.1.smithi142.stdout:8/567: mknod d0/d17/d40/d6e/caf 0 2024-06-15T01:07:04.665 INFO:tasks.workunit.client.1.smithi142.stdout:4/640: write d5/fd1 [927302,121959] 0 2024-06-15T01:07:04.665 INFO:tasks.workunit.client.1.smithi142.stdout:3/641: mknod d0/d95/cc5 0 2024-06-15T01:07:04.667 INFO:tasks.workunit.client.1.smithi142.stdout:5/529: mkdir d2/d3e/d74/da5 0 2024-06-15T01:07:04.668 INFO:tasks.workunit.client.1.smithi142.stdout:5/530: chown d2/d3e/d98 3 1 2024-06-15T01:07:04.668 INFO:tasks.workunit.client.1.smithi142.stdout:6/564: mknod d1a/d39/d3a/d51/d81/cc7 0 2024-06-15T01:07:04.669 INFO:tasks.workunit.client.0.smithi137.stdout:3/513: link d0/d42/c4d d0/d2/d22/d24/d28/ca9 0 2024-06-15T01:07:04.669 INFO:tasks.workunit.client.1.smithi142.stdout:1/586: write d6/f10 [1044040,52431] 0 2024-06-15T01:07:04.673 INFO:tasks.workunit.client.1.smithi142.stdout:0/551: creat d3/d39/fc4 x:0 0 0 2024-06-15T01:07:04.674 INFO:tasks.workunit.client.1.smithi142.stdout:7/567: rename d2/d8/da/d33/f20 to d2/d8/da/d33/d25/d79/fb1 0 2024-06-15T01:07:04.675 INFO:tasks.workunit.client.0.smithi137.stdout:3/514: dwrite d0/d2/d22/d63/d76/f7d [0,4194304] 0 2024-06-15T01:07:04.676 INFO:tasks.workunit.client.1.smithi142.stdout:2/510: write f4 [3432073,113314] 0 2024-06-15T01:07:04.678 INFO:tasks.workunit.client.1.smithi142.stdout:4/641: dwrite d5/d9/d41/f78 [0,4194304] 0 2024-06-15T01:07:04.678 INFO:tasks.workunit.client.1.smithi142.stdout:8/568: creat d0/d25/fb0 x:0 0 0 2024-06-15T01:07:04.678 INFO:tasks.workunit.client.0.smithi137.stdout:3/515: dread d0/d2/d23/f73 [0,4194304] 0 2024-06-15T01:07:04.682 INFO:tasks.workunit.client.0.smithi137.stdout:3/516: creat d0/d2f/faa x:0 0 0 2024-06-15T01:07:04.690 INFO:tasks.workunit.client.0.smithi137.stdout:3/517: dwrite d0/d2/d22/d24/f89 [0,4194304] 0 2024-06-15T01:07:04.693 INFO:tasks.workunit.client.0.smithi137.stdout:3/518: creat d0/d42/fab x:0 0 0 2024-06-15T01:07:04.713 INFO:tasks.workunit.client.0.smithi137.stdout:3/519: dread d0/d2/f15 [4194304,4194304] 0 2024-06-15T01:07:04.714 INFO:tasks.workunit.client.0.smithi137.stdout:3/520: readlink d0/d2/d22/d24/d28/d2e/l4f 0 2024-06-15T01:07:04.715 INFO:tasks.workunit.client.0.smithi137.stdout:3/521: mknod d0/d2/d22/d63/cac 0 2024-06-15T01:07:04.718 INFO:tasks.workunit.client.0.smithi137.stdout:3/522: creat d0/d2/d22/d24/d28/fad x:0 0 0 2024-06-15T01:07:04.718 INFO:tasks.workunit.client.0.smithi137.stdout:3/523: write d0/d2/d8/f14 [1171964,86337] 0 2024-06-15T01:07:04.729 INFO:tasks.workunit.client.0.smithi137.stdout:3/524: dwrite d0/d2/d8/d67/d7e/f53 [0,4194304] 0 2024-06-15T01:07:04.731 INFO:tasks.workunit.client.0.smithi137.stdout:3/525: write d0/d2/d1e/f6e [2707281,104070] 0 2024-06-15T01:07:04.737 INFO:tasks.workunit.client.0.smithi137.stdout:3/526: dwrite d0/d2/d22/d24/d5e/f85 [0,4194304] 0 2024-06-15T01:07:04.740 INFO:tasks.workunit.client.0.smithi137.stdout:3/527: creat d0/d13/d37/d66/fae x:0 0 0 2024-06-15T01:07:04.741 INFO:tasks.workunit.client.0.smithi137.stdout:3/528: write d0/d2/d22/d24/d28/fad [493966,31034] 0 2024-06-15T01:07:04.758 INFO:tasks.workunit.client.0.smithi137.stdout:3/529: mkdir d0/d2/daf 0 2024-06-15T01:07:04.759 INFO:tasks.workunit.client.0.smithi137.stdout:3/530: truncate d0/d2/d8/f7f 387739 0 2024-06-15T01:07:04.760 INFO:tasks.workunit.client.0.smithi137.stdout:3/531: fsync d0/d2/d8/f41 0 2024-06-15T01:07:04.761 INFO:tasks.workunit.client.0.smithi137.stdout:3/532: chown d0/d2/d8/d67/d7e/l40 22509420 1 2024-06-15T01:07:04.762 INFO:tasks.workunit.client.0.smithi137.stdout:3/533: readlink d0/d4c/l5a 0 2024-06-15T01:07:04.762 INFO:tasks.workunit.client.0.smithi137.stdout:3/534: chown d0/d2f/c8b 96264664 1 2024-06-15T01:07:04.766 INFO:tasks.workunit.client.0.smithi137.stdout:3/535: dread d0/d2/d22/d24/f89 [0,4194304] 0 2024-06-15T01:07:04.796 INFO:tasks.workunit.client.1.smithi142.stdout:6/565: creat d1a/d3c/fc8 x:0 0 0 2024-06-15T01:07:04.797 INFO:tasks.workunit.client.1.smithi142.stdout:6/566: write d1a/d39/d3a/f5f [201509,90796] 0 2024-06-15T01:07:04.798 INFO:tasks.workunit.client.1.smithi142.stdout:1/587: write d6/d3a/f99 [1593186,85057] 0 2024-06-15T01:07:04.798 INFO:tasks.workunit.client.1.smithi142.stdout:5/531: mkdir d2/de/d19/d5f/d68/d76/da6 0 2024-06-15T01:07:04.799 INFO:tasks.workunit.client.1.smithi142.stdout:6/567: fsync d1a/d39/d3a/d51/fa3 0 2024-06-15T01:07:04.799 INFO:tasks.workunit.client.1.smithi142.stdout:1/588: chown d6/d12/d1c/d2c/d47/f5a 179168 1 2024-06-15T01:07:04.805 INFO:tasks.workunit.client.1.smithi142.stdout:5/532: dwrite d2/de/d19/f51 [0,4194304] 0 2024-06-15T01:07:04.806 INFO:tasks.workunit.client.1.smithi142.stdout:5/533: chown d2/d3e/d65/l6c 3600550 1 2024-06-15T01:07:04.807 INFO:tasks.workunit.client.1.smithi142.stdout:4/642: creat d5/d9/d1a/fe5 x:0 0 0 2024-06-15T01:07:04.809 INFO:tasks.workunit.client.1.smithi142.stdout:6/568: stat d1a/c9f 0 2024-06-15T01:07:04.811 INFO:tasks.workunit.client.1.smithi142.stdout:6/569: write d1a/d2d/d2e/f31 [476034,41491] 0 2024-06-15T01:07:04.813 INFO:tasks.workunit.client.1.smithi142.stdout:0/552: rmdir d3/d39/d3e/d81/d1b/d38 39 2024-06-15T01:07:04.816 INFO:tasks.workunit.client.1.smithi142.stdout:3/642: rename d0/d3/f12 to d0/d1f/d20/d59/fc6 0 2024-06-15T01:07:04.822 INFO:tasks.workunit.client.1.smithi142.stdout:1/589: dread d6/d12/d3c/d63/d8d/d97/dd3/f76 [0,4194304] 0 2024-06-15T01:07:04.822 INFO:tasks.workunit.client.1.smithi142.stdout:6/570: mknod d1a/d39/d3a/d51/d81/cc9 0 2024-06-15T01:07:04.826 INFO:tasks.workunit.client.1.smithi142.stdout:1/590: chown d6/d12/d1c/d2c/d93/db9 23871 1 2024-06-15T01:07:04.826 INFO:tasks.workunit.client.1.smithi142.stdout:9/517: dwrite d6/dc/d1b/f4b [0,4194304] 0 2024-06-15T01:07:04.826 INFO:tasks.workunit.client.1.smithi142.stdout:1/591: readlink d6/d25/l3b 0 2024-06-15T01:07:04.828 INFO:tasks.workunit.client.1.smithi142.stdout:5/534: creat d2/de/d19/d5f/d68/d76/d8c/fa7 x:0 0 0 2024-06-15T01:07:04.828 INFO:tasks.workunit.client.1.smithi142.stdout:9/518: chown d6/dc/d1b/f4b 193341 1 2024-06-15T01:07:04.829 INFO:tasks.workunit.client.1.smithi142.stdout:8/569: readlink d0/d25/d53/l9a 0 2024-06-15T01:07:04.830 INFO:tasks.workunit.client.1.smithi142.stdout:6/571: write f6 [3319729,105022] 0 2024-06-15T01:07:04.832 INFO:tasks.workunit.client.1.smithi142.stdout:6/572: stat d1a/d39/d3a 0 2024-06-15T01:07:04.832 INFO:tasks.workunit.client.1.smithi142.stdout:9/519: stat d6/dc/d1b/d25/d58/fbb 0 2024-06-15T01:07:04.832 INFO:tasks.workunit.client.1.smithi142.stdout:2/511: getdents d8/df/d10/d94/d25/d22/d43 0 2024-06-15T01:07:04.834 INFO:tasks.workunit.client.1.smithi142.stdout:2/512: chown d8/df/c17 141816 1 2024-06-15T01:07:04.835 INFO:tasks.workunit.client.1.smithi142.stdout:7/568: rename d2/d8/dd/d14/d2c/d37/d6b/fad to d2/d8/dd/fb2 0 2024-06-15T01:07:04.835 INFO:tasks.workunit.client.1.smithi142.stdout:8/570: mknod d0/d17/cb1 0 2024-06-15T01:07:04.838 INFO:tasks.workunit.client.1.smithi142.stdout:3/643: getdents d0/d3/da/d71 0 2024-06-15T01:07:04.839 INFO:tasks.workunit.client.1.smithi142.stdout:3/644: stat d0/d1b/f91 0 2024-06-15T01:07:04.839 INFO:tasks.workunit.client.1.smithi142.stdout:5/535: creat d2/d3e/d31/d7e/fa8 x:0 0 0 2024-06-15T01:07:04.840 INFO:tasks.workunit.client.1.smithi142.stdout:2/513: mknod d8/d19/d33/ca5 0 2024-06-15T01:07:04.841 INFO:tasks.workunit.client.1.smithi142.stdout:6/573: dwrite f0 [0,4194304] 0 2024-06-15T01:07:04.841 INFO:tasks.workunit.client.1.smithi142.stdout:3/645: write d0/d3/da/d71/f97 [4427183,38853] 0 2024-06-15T01:07:04.844 INFO:tasks.workunit.client.1.smithi142.stdout:9/520: link d6/f6a d6/dc/d51/d53/fc9 0 2024-06-15T01:07:04.846 INFO:tasks.workunit.client.1.smithi142.stdout:0/553: truncate d3/d39/d3e/d81/f34 1119460 0 2024-06-15T01:07:04.848 INFO:tasks.workunit.client.1.smithi142.stdout:4/643: rename d5/d9/l57 to d5/d1c/le6 0 2024-06-15T01:07:04.848 INFO:tasks.workunit.client.1.smithi142.stdout:0/554: chown d3/d39/d3e/d81/d4c/faf 9856 1 2024-06-15T01:07:04.849 INFO:tasks.workunit.client.1.smithi142.stdout:6/574: dwrite f0 [0,4194304] 0 2024-06-15T01:07:04.850 INFO:tasks.workunit.client.1.smithi142.stdout:2/514: rmdir d8/df/d10/d1c/d7c 39 2024-06-15T01:07:04.853 INFO:tasks.workunit.client.1.smithi142.stdout:1/592: truncate d6/fc 813999 0 2024-06-15T01:07:04.854 INFO:tasks.workunit.client.1.smithi142.stdout:5/536: symlink d2/de/la9 0 2024-06-15T01:07:04.857 INFO:tasks.workunit.client.1.smithi142.stdout:6/575: dwrite d1a/d83/f7c [0,4194304] 0 2024-06-15T01:07:04.859 INFO:tasks.workunit.client.1.smithi142.stdout:9/521: mkdir d6/dc/dd/dca 0 2024-06-15T01:07:04.862 INFO:tasks.workunit.client.1.smithi142.stdout:4/644: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/le7 0 2024-06-15T01:07:04.865 INFO:tasks.workunit.client.1.smithi142.stdout:1/593: mknod d6/d12/d1c/d2c/cd5 0 2024-06-15T01:07:04.869 INFO:tasks.workunit.client.1.smithi142.stdout:2/515: creat d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa6 x:0 0 0 2024-06-15T01:07:04.870 INFO:tasks.workunit.client.1.smithi142.stdout:7/569: truncate d2/d8/dd/d14/d2c/d37/d6b/d80/f36 2591662 0 2024-06-15T01:07:04.871 INFO:tasks.workunit.client.1.smithi142.stdout:8/571: dwrite d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:04.872 INFO:tasks.workunit.client.1.smithi142.stdout:8/572: readlink d0/d17/d40/l42 0 2024-06-15T01:07:04.872 INFO:tasks.workunit.client.1.smithi142.stdout:3/646: rename d0/d1b/c1c to d0/d1b/cc7 0 2024-06-15T01:07:04.873 INFO:tasks.workunit.client.1.smithi142.stdout:3/647: readlink d0/d1f/l86 0 2024-06-15T01:07:04.875 INFO:tasks.workunit.client.1.smithi142.stdout:1/594: creat d6/d12/d28/fd6 x:0 0 0 2024-06-15T01:07:04.877 INFO:tasks.workunit.client.1.smithi142.stdout:4/645: mkdir d5/d9/d23/d68/de8 0 2024-06-15T01:07:04.878 INFO:tasks.workunit.client.1.smithi142.stdout:4/646: dread - d5/d9/d41/f65 zero size 2024-06-15T01:07:04.886 INFO:tasks.workunit.client.1.smithi142.stdout:8/573: dwrite d0/d25/f7e [0,4194304] 0 2024-06-15T01:07:04.888 INFO:tasks.workunit.client.1.smithi142.stdout:0/555: dread d3/d39/d3e/d81/d44/d7c/d22/f65 [0,4194304] 0 2024-06-15T01:07:04.890 INFO:tasks.workunit.client.1.smithi142.stdout:4/647: mkdir d5/d9/da/da3/de9 0 2024-06-15T01:07:04.890 INFO:tasks.workunit.client.1.smithi142.stdout:5/537: dwrite d2/d18/f61 [0,4194304] 0 2024-06-15T01:07:04.894 INFO:tasks.workunit.client.1.smithi142.stdout:4/648: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/l14 0 2024-06-15T01:07:04.894 INFO:tasks.workunit.client.1.smithi142.stdout:4/649: readlink l0 0 2024-06-15T01:07:04.896 INFO:tasks.workunit.client.1.smithi142.stdout:9/522: getdents d6/d31 0 2024-06-15T01:07:04.896 INFO:tasks.workunit.client.1.smithi142.stdout:2/516: link l0 d8/df/d10/d94/d25/d1a/la7 0 2024-06-15T01:07:04.896 INFO:tasks.workunit.client.1.smithi142.stdout:2/517: fsync d8/df/d10/f9d 0 2024-06-15T01:07:04.899 INFO:tasks.workunit.client.1.smithi142.stdout:8/574: creat d0/d17/fb2 x:0 0 0 2024-06-15T01:07:04.900 INFO:tasks.workunit.client.1.smithi142.stdout:0/556: write d3/f4 [6339843,108258] 0 2024-06-15T01:07:04.903 INFO:tasks.workunit.client.1.smithi142.stdout:3/648: dread d0/d1b/f88 [0,4194304] 0 2024-06-15T01:07:04.904 INFO:tasks.workunit.client.1.smithi142.stdout:5/538: rename d2/d3e/d65/c8d to d2/d3e/caa 0 2024-06-15T01:07:04.904 INFO:tasks.workunit.client.1.smithi142.stdout:1/595: write d6/f26 [2605128,105775] 0 2024-06-15T01:07:04.905 INFO:tasks.workunit.client.1.smithi142.stdout:5/539: chown d2/de/d19/d5f/l6f 3081666 1 2024-06-15T01:07:04.906 INFO:tasks.workunit.client.1.smithi142.stdout:9/523: creat d6/dc/d4e/d65/fcb x:0 0 0 2024-06-15T01:07:04.906 INFO:tasks.workunit.client.1.smithi142.stdout:5/540: stat d2/d3e/d65/l6c 0 2024-06-15T01:07:04.907 INFO:tasks.workunit.client.1.smithi142.stdout:2/518: rmdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98 39 2024-06-15T01:07:04.908 INFO:tasks.workunit.client.1.smithi142.stdout:0/557: readlink d3/d39/d3e/d81/d44/lb9 0 2024-06-15T01:07:04.908 INFO:tasks.workunit.client.1.smithi142.stdout:2/519: chown d8/df/d10/d94/d25/d22/d43/c4f 30735 1 2024-06-15T01:07:04.910 INFO:tasks.workunit.client.1.smithi142.stdout:2/520: dread - d8/df/d10/d94/d25/d1a/d63/f95 zero size 2024-06-15T01:07:04.910 INFO:tasks.workunit.client.1.smithi142.stdout:9/524: creat d6/dc/dd/fcc x:0 0 0 2024-06-15T01:07:04.911 INFO:tasks.workunit.client.1.smithi142.stdout:8/575: symlink d0/d25/d66/lb3 0 2024-06-15T01:07:04.913 INFO:tasks.workunit.client.1.smithi142.stdout:8/576: truncate d0/d17/d40/d6e/fa9 100699 0 2024-06-15T01:07:04.914 INFO:tasks.workunit.client.1.smithi142.stdout:9/525: chown d6/dc/dd/l30 10114 1 2024-06-15T01:07:04.914 INFO:tasks.workunit.client.1.smithi142.stdout:2/521: fdatasync d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 0 2024-06-15T01:07:04.916 INFO:tasks.workunit.client.1.smithi142.stdout:4/650: getdents d5/d1c 0 2024-06-15T01:07:04.916 INFO:tasks.workunit.client.1.smithi142.stdout:3/649: getdents d0/d1f/d20/d5d/d60/d99 0 2024-06-15T01:07:04.920 INFO:tasks.workunit.client.1.smithi142.stdout:1/596: dread d6/d12/d3c/d63/fa8 [0,4194304] 0 2024-06-15T01:07:04.922 INFO:tasks.workunit.client.1.smithi142.stdout:4/651: unlink d5/d1c/d6f/dc7/d30/c77 0 2024-06-15T01:07:04.923 INFO:tasks.workunit.client.1.smithi142.stdout:0/558: dwrite d3/d39/d3e/d81/d1b/d38/f9b [0,4194304] 0 2024-06-15T01:07:04.924 INFO:tasks.workunit.client.1.smithi142.stdout:1/597: write d6/d3a/f44 [215867,58031] 0 2024-06-15T01:07:04.926 INFO:tasks.workunit.client.1.smithi142.stdout:2/522: rename d8/df/d10/c8e to d8/df/d10/ca8 0 2024-06-15T01:07:04.932 INFO:tasks.workunit.client.1.smithi142.stdout:7/570: dread d2/d8/dd/d14/d2c/d37/d6b/d80/f36 [0,4194304] 0 2024-06-15T01:07:04.932 INFO:tasks.workunit.client.1.smithi142.stdout:9/526: dwrite d6/dc/d4e/d65/fa2 [0,4194304] 0 2024-06-15T01:07:04.932 INFO:tasks.workunit.client.1.smithi142.stdout:1/598: rmdir d6/d12/d3c/d63/d8d 39 2024-06-15T01:07:04.933 INFO:tasks.workunit.client.1.smithi142.stdout:4/652: mknod d5/d1c/d6f/dc7/de0/cea 0 2024-06-15T01:07:04.934 INFO:tasks.workunit.client.1.smithi142.stdout:8/577: dwrite d0/d17/d40/fa8 [0,4194304] 0 2024-06-15T01:07:04.939 INFO:tasks.workunit.client.1.smithi142.stdout:3/650: truncate d0/d3/da/d71/f97 1298913 0 2024-06-15T01:07:04.940 INFO:tasks.workunit.client.1.smithi142.stdout:7/571: symlink d2/d8/da/d33/d25/d27/lb3 0 2024-06-15T01:07:04.944 INFO:tasks.workunit.client.1.smithi142.stdout:4/653: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/l58 94011250 1 2024-06-15T01:07:04.950 INFO:tasks.workunit.client.1.smithi142.stdout:4/654: write d5/d9/d90/daf/fdc [648740,124006] 0 2024-06-15T01:07:04.951 INFO:tasks.workunit.client.1.smithi142.stdout:0/559: write d3/d39/d3e/d81/d1b/f30 [1253384,50558] 0 2024-06-15T01:07:04.954 INFO:tasks.workunit.client.1.smithi142.stdout:9/527: dwrite d6/dc/d4e/d78/f7f [0,4194304] 0 2024-06-15T01:07:04.954 INFO:tasks.workunit.client.1.smithi142.stdout:8/578: dwrite d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:04.955 INFO:tasks.workunit.client.1.smithi142.stdout:7/572: link d2/d8/da/d33/d25/d27/l8d d2/d6/lb4 0 2024-06-15T01:07:04.959 INFO:tasks.workunit.client.1.smithi142.stdout:0/560: creat d3/d39/d3e/fc5 x:0 0 0 2024-06-15T01:07:04.959 INFO:tasks.workunit.client.1.smithi142.stdout:9/528: mknod d6/dc/d4e/d78/ccd 0 2024-06-15T01:07:04.960 INFO:tasks.workunit.client.1.smithi142.stdout:8/579: creat d0/d25/d66/fb4 x:0 0 0 2024-06-15T01:07:04.961 INFO:tasks.workunit.client.1.smithi142.stdout:3/651: dwrite d0/d3/da/d18/f98 [4194304,4194304] 0 2024-06-15T01:07:04.966 INFO:tasks.workunit.client.1.smithi142.stdout:2/523: truncate d8/df/d10/d94/d25/f6a 3651527 0 2024-06-15T01:07:04.970 INFO:tasks.workunit.client.1.smithi142.stdout:7/573: chown d2/d8/dd/l60 2 1 2024-06-15T01:07:04.973 INFO:tasks.workunit.client.1.smithi142.stdout:4/655: dwrite d5/d9/d18/fa8 [0,4194304] 0 2024-06-15T01:07:04.973 INFO:tasks.workunit.client.1.smithi142.stdout:3/652: symlink d0/d3/da/lc8 0 2024-06-15T01:07:04.973 INFO:tasks.workunit.client.1.smithi142.stdout:1/599: dread d6/f10 [0,4194304] 0 2024-06-15T01:07:04.974 INFO:tasks.workunit.client.1.smithi142.stdout:0/561: dread - d3/d39/d3e/d81/d44/d7c/d22/d59/f80 zero size 2024-06-15T01:07:04.978 INFO:tasks.workunit.client.1.smithi142.stdout:9/529: rename d6/dc/d51/d53/d8d/cb1 to d6/d31/cce 0 2024-06-15T01:07:04.978 INFO:tasks.workunit.client.1.smithi142.stdout:3/653: mkdir d0/d1f/d20/d5d/d82/dc9 0 2024-06-15T01:07:04.979 INFO:tasks.workunit.client.1.smithi142.stdout:7/574: truncate d2/d6/f1c 2022740 0 2024-06-15T01:07:04.982 INFO:tasks.workunit.client.1.smithi142.stdout:9/530: write d6/dc/d51/d53/d8d/fb9 [481941,79012] 0 2024-06-15T01:07:04.985 INFO:tasks.workunit.client.1.smithi142.stdout:0/562: dwrite d3/d39/d3e/d81/d44/d7c/f95 [0,4194304] 0 2024-06-15T01:07:04.986 INFO:tasks.workunit.client.1.smithi142.stdout:1/600: unlink d6/d12/d1c/d2c/c35 0 2024-06-15T01:07:04.991 INFO:tasks.workunit.client.1.smithi142.stdout:2/524: rename d8/df/d10/d94/d3a to d8/df/d10/d94/d25/da9 0 2024-06-15T01:07:04.993 INFO:tasks.workunit.client.1.smithi142.stdout:0/563: creat d3/d39/d3e/db4/fc6 x:0 0 0 2024-06-15T01:07:04.994 INFO:tasks.workunit.client.1.smithi142.stdout:9/531: dwrite d6/dc/d4e/d65/fa2 [0,4194304] 0 2024-06-15T01:07:04.995 INFO:tasks.workunit.client.1.smithi142.stdout:1/601: chown d6/d12/d3c/d63/d8d/d97/cc5 5129614 1 2024-06-15T01:07:04.996 INFO:tasks.workunit.client.1.smithi142.stdout:7/575: rename d2/d8/dd/d14/d2c/d37/d6b/f8c to d2/d8/dd/d8a/fb5 0 2024-06-15T01:07:04.996 INFO:tasks.workunit.client.1.smithi142.stdout:9/532: stat d6/dc/d1b/fc6 0 2024-06-15T01:07:04.998 INFO:tasks.workunit.client.1.smithi142.stdout:3/654: getdents d0/d3/da 0 2024-06-15T01:07:05.000 INFO:tasks.workunit.client.1.smithi142.stdout:1/602: creat d6/d3a/dc4/fd7 x:0 0 0 2024-06-15T01:07:05.000 INFO:tasks.workunit.client.1.smithi142.stdout:8/580: truncate d0/f47 167379 0 2024-06-15T01:07:05.000 INFO:tasks.workunit.client.1.smithi142.stdout:7/576: readlink d2/d6/l64 0 2024-06-15T01:07:05.002 INFO:tasks.workunit.client.1.smithi142.stdout:0/564: mknod d3/d39/d3e/d81/d1b/d52/cc7 0 2024-06-15T01:07:05.002 INFO:tasks.workunit.client.1.smithi142.stdout:2/525: dread d8/f40 [0,4194304] 0 2024-06-15T01:07:05.004 INFO:tasks.workunit.client.1.smithi142.stdout:3/655: write d0/d3/f54 [1430879,14575] 0 2024-06-15T01:07:05.005 INFO:tasks.workunit.client.1.smithi142.stdout:4/656: write d5/d9/d41/f4c [10378574,126022] 0 2024-06-15T01:07:05.006 INFO:tasks.workunit.client.1.smithi142.stdout:1/603: rename d6/c7f to d6/d12/d1c/d2c/cd8 0 2024-06-15T01:07:05.007 INFO:tasks.workunit.client.1.smithi142.stdout:8/581: creat d0/d25/d5a/d8f/d98/da4/fb5 x:0 0 0 2024-06-15T01:07:05.007 INFO:tasks.workunit.client.1.smithi142.stdout:4/657: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/c8c 6 1 2024-06-15T01:07:05.009 INFO:tasks.workunit.client.0.smithi137.stdout:4/404: sync 2024-06-15T01:07:05.010 INFO:tasks.workunit.client.1.smithi142.stdout:4/658: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f95 zero size 2024-06-15T01:07:05.011 INFO:tasks.workunit.client.1.smithi142.stdout:0/565: symlink d3/d39/d3e/d81/lc8 0 2024-06-15T01:07:05.011 INFO:tasks.workunit.client.1.smithi142.stdout:3/656: dread d0/d3/d15/f8c [0,4194304] 0 2024-06-15T01:07:05.015 INFO:tasks.workunit.client.1.smithi142.stdout:1/604: read d6/d25/f3d [233169,76796] 0 2024-06-15T01:07:05.016 INFO:tasks.workunit.client.0.smithi137.stdout:4/405: dwrite f8 [0,4194304] 0 2024-06-15T01:07:05.017 INFO:tasks.workunit.client.1.smithi142.stdout:8/582: truncate d0/d17/d40/d6e/f73 494255 0 2024-06-15T01:07:05.020 INFO:tasks.workunit.client.0.smithi137.stdout:4/406: unlink c7 0 2024-06-15T01:07:05.022 INFO:tasks.workunit.client.1.smithi142.stdout:4/659: fsync d5/d1c/d6f/dc7/fab 0 2024-06-15T01:07:05.023 INFO:tasks.workunit.client.1.smithi142.stdout:4/660: readlink d5/d9/d18/l83 0 2024-06-15T01:07:05.023 INFO:tasks.workunit.client.0.smithi137.stdout:4/407: fdatasync dd/f8e 0 2024-06-15T01:07:05.025 INFO:tasks.workunit.client.1.smithi142.stdout:4/661: fdatasync d5/d9/d90/daf/fdc 0 2024-06-15T01:07:05.028 INFO:tasks.workunit.client.0.smithi137.stdout:4/408: rename dd/d15/d30/d49 to dd/d15/d21/d9c 0 2024-06-15T01:07:05.029 INFO:tasks.workunit.client.0.smithi137.stdout:4/409: write dd/d15/d48/d67/f6c [194580,16892] 0 2024-06-15T01:07:05.029 INFO:tasks.workunit.client.1.smithi142.stdout:7/577: dread d2/d8/da/d33/d25/d27/f2f [0,4194304] 0 2024-06-15T01:07:05.032 INFO:tasks.workunit.client.1.smithi142.stdout:1/605: fdatasync d6/d12/d1c/d2c/d47/f5f 0 2024-06-15T01:07:05.032 INFO:tasks.workunit.client.0.smithi137.stdout:4/410: symlink dd/d15/d21/d9c/d75/d99/l9d 0 2024-06-15T01:07:05.033 INFO:tasks.workunit.client.0.smithi137.stdout:4/411: dread - dd/d15/d21/d3a/d51/f7f zero size 2024-06-15T01:07:05.034 INFO:tasks.workunit.client.1.smithi142.stdout:2/526: dread d8/df/d10/d94/d25/f74 [0,4194304] 0 2024-06-15T01:07:05.034 INFO:tasks.workunit.client.1.smithi142.stdout:3/657: symlink d0/d1b/d6b/dbd/lca 0 2024-06-15T01:07:05.034 INFO:tasks.workunit.client.0.smithi137.stdout:4/412: stat dd/d15/d48/d72/d81 0 2024-06-15T01:07:05.035 INFO:tasks.workunit.client.1.smithi142.stdout:2/527: write d8/d19/f70 [1660645,59975] 0 2024-06-15T01:07:05.035 INFO:tasks.workunit.client.1.smithi142.stdout:3/658: write d0/d3/d15/f64 [4819989,70998] 0 2024-06-15T01:07:05.035 INFO:tasks.workunit.client.1.smithi142.stdout:4/662: mknod d5/d9/da/d92/d98/ceb 0 2024-06-15T01:07:05.041 INFO:tasks.workunit.client.1.smithi142.stdout:8/583: symlink d0/d25/d53/da7/lb6 0 2024-06-15T01:07:05.042 INFO:tasks.workunit.client.1.smithi142.stdout:8/584: write d0/d17/f61 [5231355,108559] 0 2024-06-15T01:07:05.043 INFO:tasks.workunit.client.1.smithi142.stdout:1/606: unlink d6/d3a/l8c 0 2024-06-15T01:07:05.044 INFO:tasks.workunit.client.1.smithi142.stdout:1/607: chown d6/d12/d28/c41 149 1 2024-06-15T01:07:05.048 INFO:tasks.workunit.client.1.smithi142.stdout:2/528: creat d8/df/d10/d94/d25/d1a/faa x:0 0 0 2024-06-15T01:07:05.049 INFO:tasks.workunit.client.1.smithi142.stdout:8/585: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:07:05.049 INFO:tasks.workunit.client.1.smithi142.stdout:7/578: mkdir d2/db6 0 2024-06-15T01:07:05.049 INFO:tasks.workunit.client.0.smithi137.stdout:4/413: dwrite f5 [0,4194304] 0 2024-06-15T01:07:05.050 INFO:tasks.workunit.client.0.smithi137.stdout:4/414: chown dd/d15/d21/d3a/d51/d83 1240310 1 2024-06-15T01:07:05.052 INFO:tasks.workunit.client.1.smithi142.stdout:7/579: chown d2/d8/dd/d14/d2c/d70/l73 4125110 1 2024-06-15T01:07:05.052 INFO:tasks.workunit.client.0.smithi137.stdout:4/415: symlink dd/d15/d48/l9e 0 2024-06-15T01:07:05.052 INFO:tasks.workunit.client.0.smithi137.stdout:4/416: readlink l6 0 2024-06-15T01:07:05.055 INFO:tasks.workunit.client.1.smithi142.stdout:3/659: dwrite d0/d1f/d20/d59/fc6 [4194304,4194304] 0 2024-06-15T01:07:05.055 INFO:tasks.workunit.client.1.smithi142.stdout:2/529: read d8/d19/f70 [400652,95036] 0 2024-06-15T01:07:05.055 INFO:tasks.workunit.client.1.smithi142.stdout:7/580: write d2/d6/f69 [3745715,52377] 0 2024-06-15T01:07:05.056 INFO:tasks.workunit.client.1.smithi142.stdout:7/581: chown d2/d8/dd/d14/d2c/d37/d6b/d80 241 1 2024-06-15T01:07:05.058 INFO:tasks.workunit.client.1.smithi142.stdout:0/566: creat d3/d39/d9a/fc9 x:0 0 0 2024-06-15T01:07:05.058 INFO:tasks.workunit.client.0.smithi137.stdout:4/417: link dd/d15/d36/d45/d55/f7c dd/d15/d21/d3a/f9f 0 2024-06-15T01:07:05.058 INFO:tasks.workunit.client.1.smithi142.stdout:1/608: creat d6/d12/d28/fd9 x:0 0 0 2024-06-15T01:07:05.059 INFO:tasks.workunit.client.1.smithi142.stdout:4/663: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/d99/f9a 2939234 0 2024-06-15T01:07:05.060 INFO:tasks.workunit.client.0.smithi137.stdout:4/418: write dd/d15/d21/d3a/d51/f52 [4727914,96874] 0 2024-06-15T01:07:05.062 INFO:tasks.workunit.client.1.smithi142.stdout:8/586: symlink d0/d25/d5a/d8f/d98/lb7 0 2024-06-15T01:07:05.064 INFO:tasks.workunit.client.1.smithi142.stdout:1/609: creat d6/d12/d80/dc8/fda x:0 0 0 2024-06-15T01:07:05.066 INFO:tasks.workunit.client.1.smithi142.stdout:7/582: creat d2/d8/da/d33/d25/d79/fb7 x:0 0 0 2024-06-15T01:07:05.069 INFO:tasks.workunit.client.1.smithi142.stdout:3/660: dread d0/d1f/d20/fa3 [0,4194304] 0 2024-06-15T01:07:05.069 INFO:tasks.workunit.client.1.smithi142.stdout:7/583: write d2/d8/da/d33/f35 [4660598,109040] 0 2024-06-15T01:07:05.069 INFO:tasks.workunit.client.1.smithi142.stdout:2/530: symlink d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/lab 0 2024-06-15T01:07:05.069 INFO:tasks.workunit.client.1.smithi142.stdout:8/587: dread d0/f3f [0,4194304] 0 2024-06-15T01:07:05.070 INFO:tasks.workunit.client.1.smithi142.stdout:2/531: fdatasync f4 0 2024-06-15T01:07:05.070 INFO:tasks.workunit.client.1.smithi142.stdout:0/567: dread d3/d39/d3e/d81/d44/d7c/d22/f65 [0,4194304] 0 2024-06-15T01:07:05.071 INFO:tasks.workunit.client.1.smithi142.stdout:8/588: fdatasync d0/d17/f7c 0 2024-06-15T01:07:05.071 INFO:tasks.workunit.client.1.smithi142.stdout:4/664: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/lec 0 2024-06-15T01:07:05.072 INFO:tasks.workunit.client.1.smithi142.stdout:3/661: write d0/d1b/f91 [497663,39411] 0 2024-06-15T01:07:05.072 INFO:tasks.workunit.client.1.smithi142.stdout:0/568: fdatasync d3/f23 0 2024-06-15T01:07:05.073 INFO:tasks.workunit.client.1.smithi142.stdout:1/610: mkdir d6/d4d/da2/ddb 0 2024-06-15T01:07:05.076 INFO:tasks.workunit.client.1.smithi142.stdout:2/532: creat d8/df/d10/d94/d25/d22/d43/fac x:0 0 0 2024-06-15T01:07:05.077 INFO:tasks.workunit.client.1.smithi142.stdout:2/533: dread - d8/d19/d33/fa2 zero size 2024-06-15T01:07:05.078 INFO:tasks.workunit.client.1.smithi142.stdout:8/589: fdatasync d0/d25/d5a/f2c 0 2024-06-15T01:07:05.080 INFO:tasks.workunit.client.1.smithi142.stdout:3/662: fsync d0/d3/f3d 0 2024-06-15T01:07:05.081 INFO:tasks.workunit.client.1.smithi142.stdout:1/611: mknod d6/d12/d1c/d2c/d93/cdc 0 2024-06-15T01:07:05.082 INFO:tasks.workunit.client.1.smithi142.stdout:3/663: stat d0/d1f/d20/d5d/d60/fb0 0 2024-06-15T01:07:05.083 INFO:tasks.workunit.client.1.smithi142.stdout:2/534: dwrite d8/d19/d33/f83 [0,4194304] 0 2024-06-15T01:07:05.084 INFO:tasks.workunit.client.1.smithi142.stdout:3/664: dread - d0/d1f/d20/d59/d61/d96/fb3 zero size 2024-06-15T01:07:05.086 INFO:tasks.workunit.client.0.smithi137.stdout:4/419: dread f9 [0,4194304] 0 2024-06-15T01:07:05.088 INFO:tasks.workunit.client.1.smithi142.stdout:7/584: creat d2/d8/dd/d14/fb8 x:0 0 0 2024-06-15T01:07:05.089 INFO:tasks.workunit.client.0.smithi137.stdout:2/347: sync 2024-06-15T01:07:05.090 INFO:tasks.workunit.client.0.smithi137.stdout:2/348: write d0/d42/d4d/f59 [6081404,95547] 0 2024-06-15T01:07:05.091 INFO:tasks.workunit.client.0.smithi137.stdout:4/420: rename dd/d15/d36/d45/d55/d8d/c11 to dd/d15/d48/d72/ca0 0 2024-06-15T01:07:05.092 INFO:tasks.workunit.client.0.smithi137.stdout:2/349: write d0/d42/d49/f62 [530700,37851] 0 2024-06-15T01:07:05.093 INFO:tasks.workunit.client.1.smithi142.stdout:8/590: mkdir d0/d17/d40/db8 0 2024-06-15T01:07:05.093 INFO:tasks.workunit.client.1.smithi142.stdout:4/665: creat d5/d9/d90/daf/fed x:0 0 0 2024-06-15T01:07:05.094 INFO:tasks.workunit.client.1.smithi142.stdout:0/569: link d3/d39/d3e/d81/d44/l67 d3/d39/d9a/d9d/dbf/lca 0 2024-06-15T01:07:05.094 INFO:tasks.workunit.client.1.smithi142.stdout:7/585: rename d2/d8 to d2/d8/da/d33/db9 22 2024-06-15T01:07:05.094 INFO:tasks.workunit.client.0.smithi137.stdout:4/421: mkdir dd/d15/d48/da1 0 2024-06-15T01:07:05.095 INFO:tasks.workunit.client.1.smithi142.stdout:8/591: fsync d0/d17/d40/d5e/fad 0 2024-06-15T01:07:05.096 INFO:tasks.workunit.client.1.smithi142.stdout:1/612: getdents d6/d12/db5 0 2024-06-15T01:07:05.096 INFO:tasks.workunit.client.0.smithi137.stdout:4/422: dread - dd/d15/d36/d45/d55/d8d/f70 zero size 2024-06-15T01:07:05.096 INFO:tasks.workunit.client.1.smithi142.stdout:1/613: chown d6/d12/d1c/d2c/d47/f5a 1065577416 1 2024-06-15T01:07:05.097 INFO:tasks.workunit.client.1.smithi142.stdout:7/586: fdatasync d2/d8/f2d 0 2024-06-15T01:07:05.098 INFO:tasks.workunit.client.0.smithi137.stdout:2/350: getdents d0/d42/d4d 0 2024-06-15T01:07:05.099 INFO:tasks.workunit.client.0.smithi137.stdout:2/351: write d0/f69 [750941,9955] 0 2024-06-15T01:07:05.100 INFO:tasks.workunit.client.1.smithi142.stdout:7/587: write d2/d8/dd/d14/f58 [26056,94847] 0 2024-06-15T01:07:05.101 INFO:tasks.workunit.client.0.smithi137.stdout:4/423: getdents dd/d15/d21/d3a/d6e/d7b 0 2024-06-15T01:07:05.101 INFO:tasks.workunit.client.1.smithi142.stdout:0/570: symlink d3/d39/lcb 0 2024-06-15T01:07:05.103 INFO:tasks.workunit.client.0.smithi137.stdout:4/424: mknod dd/d15/d36/d45/ca2 0 2024-06-15T01:07:05.103 INFO:tasks.workunit.client.1.smithi142.stdout:4/666: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fda [0,4194304] 0 2024-06-15T01:07:05.104 INFO:tasks.workunit.client.1.smithi142.stdout:0/571: write d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [270658,88741] 0 2024-06-15T01:07:05.105 INFO:tasks.workunit.client.0.smithi137.stdout:4/425: symlink dd/la3 0 2024-06-15T01:07:05.105 INFO:tasks.workunit.client.1.smithi142.stdout:4/667: dread - d5/d9/d23/fd9 zero size 2024-06-15T01:07:05.106 INFO:tasks.workunit.client.0.smithi137.stdout:4/426: chown dd/f14 287 1 2024-06-15T01:07:05.106 INFO:tasks.workunit.client.1.smithi142.stdout:8/592: mkdir d0/d25/db9 0 2024-06-15T01:07:05.108 INFO:tasks.workunit.client.1.smithi142.stdout:4/668: read d5/d9/da/f54 [1173192,105971] 0 2024-06-15T01:07:05.113 INFO:tasks.workunit.client.1.smithi142.stdout:0/572: mknod d3/d39/d3e/d81/d1b/d38/ccc 0 2024-06-15T01:07:05.117 INFO:tasks.workunit.client.1.smithi142.stdout:7/588: mknod d2/d8/cba 0 2024-06-15T01:07:05.117 INFO:tasks.workunit.client.0.smithi137.stdout:2/352: dwrite d0/d42/d4d/f67 [0,4194304] 0 2024-06-15T01:07:05.118 INFO:tasks.workunit.client.0.smithi137.stdout:4/427: dwrite dd/f8e [0,4194304] 0 2024-06-15T01:07:05.119 INFO:tasks.workunit.client.1.smithi142.stdout:7/589: truncate d2/d6/d22/d47/f5a 947424 0 2024-06-15T01:07:05.119 INFO:tasks.workunit.client.1.smithi142.stdout:3/665: dread d0/d3/d15/f2d [0,4194304] 0 2024-06-15T01:07:05.119 INFO:tasks.workunit.client.1.smithi142.stdout:4/669: unlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/cae 0 2024-06-15T01:07:05.123 INFO:tasks.workunit.client.0.smithi137.stdout:2/353: rename d0/f5 to d0/dc/d22/d23/f6a 0 2024-06-15T01:07:05.123 INFO:tasks.workunit.client.1.smithi142.stdout:1/614: link d6/d12/d3c/d63/d8d/fb8 d6/d12/d3c/d63/fdd 0 2024-06-15T01:07:05.124 INFO:tasks.workunit.client.0.smithi137.stdout:2/354: chown d0/d42/d49/d57 675982 1 2024-06-15T01:07:05.126 INFO:tasks.workunit.client.1.smithi142.stdout:0/573: write d3/d39/d3e/d81/d4c/d61/f77 [481489,91388] 0 2024-06-15T01:07:05.127 INFO:tasks.workunit.client.1.smithi142.stdout:8/593: mknod d0/d25/d53/cba 0 2024-06-15T01:07:05.127 INFO:tasks.workunit.client.1.smithi142.stdout:7/590: dwrite d2/d6/d22/d47/f7c [0,4194304] 0 2024-06-15T01:07:05.129 INFO:tasks.workunit.client.1.smithi142.stdout:8/594: truncate d0/f43 1727138 0 2024-06-15T01:07:05.130 INFO:tasks.workunit.client.0.smithi137.stdout:4/428: dwrite dd/d15/d30/f6b [0,4194304] 0 2024-06-15T01:07:05.134 INFO:tasks.workunit.client.1.smithi142.stdout:4/670: fdatasync d5/d1c/d6f/dc7/d30/f64 0 2024-06-15T01:07:05.135 INFO:tasks.workunit.client.1.smithi142.stdout:1/615: truncate d6/d12/d1c/d2c/d47/f5a 1206198 0 2024-06-15T01:07:05.136 INFO:tasks.workunit.client.1.smithi142.stdout:4/671: chown d5/d1c/d6f/dc7 22236 1 2024-06-15T01:07:05.136 INFO:tasks.workunit.client.0.smithi137.stdout:4/429: write dd/d15/d36/d45/d55/d8d/f1c [4466130,116595] 0 2024-06-15T01:07:05.138 INFO:tasks.workunit.client.1.smithi142.stdout:0/574: mkdir d3/d39/d9a/d9d/dcd 0 2024-06-15T01:07:05.138 INFO:tasks.workunit.client.0.smithi137.stdout:2/355: dwrite d0/dc/d22/f4b [0,4194304] 0 2024-06-15T01:07:05.138 INFO:tasks.workunit.client.0.smithi137.stdout:4/430: chown dd/d15/d21/f32 21 1 2024-06-15T01:07:05.142 INFO:tasks.workunit.client.1.smithi142.stdout:7/591: creat d2/d8/da/d33/d25/d27/fbb x:0 0 0 2024-06-15T01:07:05.144 INFO:tasks.workunit.client.1.smithi142.stdout:3/666: dwrite d0/d3/f1a [4194304,4194304] 0 2024-06-15T01:07:05.147 INFO:tasks.workunit.client.1.smithi142.stdout:8/595: dwrite d0/d17/d40/d5e/f81 [0,4194304] 0 2024-06-15T01:07:05.148 INFO:tasks.workunit.client.1.smithi142.stdout:1/616: rename d6/d12/d1c/c7a to d6/d12/d1c/d2c/cde 0 2024-06-15T01:07:05.154 INFO:tasks.workunit.client.1.smithi142.stdout:7/592: mkdir d2/d8/da/d33/d25/d79/dbc 0 2024-06-15T01:07:05.155 INFO:tasks.workunit.client.1.smithi142.stdout:8/596: creat d0/d25/d53/fbb x:0 0 0 2024-06-15T01:07:05.156 INFO:tasks.workunit.client.1.smithi142.stdout:1/617: creat d6/d12/d80/fdf x:0 0 0 2024-06-15T01:07:05.158 INFO:tasks.workunit.client.1.smithi142.stdout:4/672: getdents d5/d1c/d6f 0 2024-06-15T01:07:05.159 INFO:tasks.workunit.client.1.smithi142.stdout:8/597: chown d0/d17/d40/d86/d9d/fa5 1234917 1 2024-06-15T01:07:05.160 INFO:tasks.workunit.client.1.smithi142.stdout:1/618: write d6/d12/d52/f89 [894100,5471] 0 2024-06-15T01:07:05.162 INFO:tasks.workunit.client.1.smithi142.stdout:0/575: dwrite d3/d39/d3e/db4/fc6 [0,4194304] 0 2024-06-15T01:07:05.164 INFO:tasks.workunit.client.1.smithi142.stdout:3/667: dwrite d0/d3/f90 [0,4194304] 0 2024-06-15T01:07:05.166 INFO:tasks.workunit.client.1.smithi142.stdout:8/598: creat d0/d17/d40/d6e/fbc x:0 0 0 2024-06-15T01:07:05.169 INFO:tasks.workunit.client.1.smithi142.stdout:1/619: creat d6/d12/d1c/fe0 x:0 0 0 2024-06-15T01:07:05.173 INFO:tasks.workunit.client.1.smithi142.stdout:3/668: truncate d0/d3/f3d 3831574 0 2024-06-15T01:07:05.174 INFO:tasks.workunit.client.1.smithi142.stdout:8/599: dwrite d0/f14 [0,4194304] 0 2024-06-15T01:07:05.175 INFO:tasks.workunit.client.1.smithi142.stdout:0/576: link d3/d39/d3e/d81/d1b/d52/l60 d3/d39/d9a/da2/lce 0 2024-06-15T01:07:05.180 INFO:tasks.workunit.client.1.smithi142.stdout:8/600: creat d0/d25/d53/d9c/fbd x:0 0 0 2024-06-15T01:07:05.180 INFO:tasks.workunit.client.1.smithi142.stdout:9/533: sync 2024-06-15T01:07:05.180 INFO:tasks.workunit.client.1.smithi142.stdout:1/620: mkdir d6/d12/da0/db0/de1 0 2024-06-15T01:07:05.181 INFO:tasks.workunit.client.0.smithi137.stdout:4/431: dwrite dd/d15/d36/d45/d55/f7c [0,4194304] 0 2024-06-15T01:07:05.184 INFO:tasks.workunit.client.1.smithi142.stdout:8/601: fsync d0/d17/f27 0 2024-06-15T01:07:05.185 INFO:tasks.workunit.client.1.smithi142.stdout:5/541: sync 2024-06-15T01:07:05.185 INFO:tasks.workunit.client.1.smithi142.stdout:6/576: sync 2024-06-15T01:07:05.186 INFO:tasks.workunit.client.1.smithi142.stdout:2/535: sync 2024-06-15T01:07:05.187 INFO:tasks.workunit.client.1.smithi142.stdout:0/577: link d3/d39/d3e/d81/d4c/d61/f91 d3/d39/d9a/d9d/dcd/fcf 0 2024-06-15T01:07:05.192 INFO:tasks.workunit.client.1.smithi142.stdout:0/578: write d3/f4 [2731767,6390] 0 2024-06-15T01:07:05.192 INFO:tasks.workunit.client.1.smithi142.stdout:9/534: dread - d6/d34/d8a/fbf zero size 2024-06-15T01:07:05.192 INFO:tasks.workunit.client.1.smithi142.stdout:4/673: truncate d5/d1c/d6f/dc7/d30/f64 433656 0 2024-06-15T01:07:05.194 INFO:tasks.workunit.client.1.smithi142.stdout:2/536: read - d8/df/d10/d94/d25/d1a/faa zero size 2024-06-15T01:07:05.199 INFO:tasks.workunit.client.1.smithi142.stdout:8/602: rmdir d0/d17/d40/d86 39 2024-06-15T01:07:05.201 INFO:tasks.workunit.client.1.smithi142.stdout:7/593: dread d2/d6/f1c [0,4194304] 0 2024-06-15T01:07:05.201 INFO:tasks.workunit.client.1.smithi142.stdout:1/621: dwrite d6/d12/d28/f33 [4194304,4194304] 0 2024-06-15T01:07:05.201 INFO:tasks.workunit.client.1.smithi142.stdout:5/542: creat d2/de/d27/fab x:0 0 0 2024-06-15T01:07:05.202 INFO:tasks.workunit.client.0.smithi137.stdout:4/432: write dd/d15/d30/f46 [55770,114455] 0 2024-06-15T01:07:05.206 INFO:tasks.workunit.client.1.smithi142.stdout:6/577: dwrite d1a/f1f [0,4194304] 0 2024-06-15T01:07:05.213 INFO:tasks.workunit.client.1.smithi142.stdout:0/579: symlink d3/d39/d3e/d81/ld0 0 2024-06-15T01:07:05.214 INFO:tasks.workunit.client.1.smithi142.stdout:9/535: mkdir d6/dc/d1b/d25/dcf 0 2024-06-15T01:07:05.215 INFO:tasks.workunit.client.1.smithi142.stdout:7/594: mknod d2/d8/dd/d14/d2c/d37/cbd 0 2024-06-15T01:07:05.216 INFO:tasks.workunit.client.1.smithi142.stdout:8/603: dread d0/d17/d40/f4a [0,4194304] 0 2024-06-15T01:07:05.217 INFO:tasks.workunit.client.1.smithi142.stdout:5/543: mkdir d2/de/d19/d5f/d68/d76/d96/dac 0 2024-06-15T01:07:05.217 INFO:tasks.workunit.client.1.smithi142.stdout:0/580: fdatasync d3/d39/d3e/d81/d44/fa7 0 2024-06-15T01:07:05.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/433: write dd/d15/d48/f61 [428909,121335] 0 2024-06-15T01:07:05.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/434: stat f8 0 2024-06-15T01:07:05.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/435: stat fa 0 2024-06-15T01:07:05.225 INFO:tasks.workunit.client.0.smithi137.stdout:4/436: mkdir dd/d15/d21/d3a/d6e/da4 0 2024-06-15T01:07:05.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:04 smithi137 ceph-mon[29354]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:07:05.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:04 smithi137 ceph-mon[29354]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:07:05.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:04 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:05.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/437: dwrite dd/f13 [0,4194304] 0 2024-06-15T01:07:05.230 INFO:tasks.workunit.client.0.smithi137.stdout:4/438: fdatasync dd/d15/d30/f6b 0 2024-06-15T01:07:05.231 INFO:tasks.workunit.client.0.smithi137.stdout:4/439: fdatasync f5 0 2024-06-15T01:07:05.240 INFO:tasks.workunit.client.0.smithi137.stdout:4/440: link dd/d15/d36/d45/d55/d8d/l4d dd/d15/d21/d9c/d75/la5 0 2024-06-15T01:07:05.242 INFO:tasks.workunit.client.0.smithi137.stdout:4/441: read dd/d15/d30/f40 [478,116038] 0 2024-06-15T01:07:05.243 INFO:tasks.workunit.client.0.smithi137.stdout:4/442: rmdir dd/d15/d48/d67 39 2024-06-15T01:07:05.249 INFO:tasks.workunit.client.0.smithi137.stdout:4/443: dwrite dd/d15/d48/d67/f6c [0,4194304] 0 2024-06-15T01:07:05.251 INFO:tasks.workunit.client.0.smithi137.stdout:4/444: truncate dd/d15/d30/f8c 80412 0 2024-06-15T01:07:05.254 INFO:tasks.workunit.client.1.smithi142.stdout:1/622: getdents d6/d12/d3c/d63/d65 0 2024-06-15T01:07:05.255 INFO:tasks.workunit.client.1.smithi142.stdout:2/537: dwrite d8/d19/d33/fa0 [0,4194304] 0 2024-06-15T01:07:05.255 INFO:tasks.workunit.client.1.smithi142.stdout:9/536: read d6/d34/fc0 [2124099,45894] 0 2024-06-15T01:07:05.256 INFO:tasks.workunit.client.0.smithi137.stdout:4/445: dwrite dd/f18 [0,4194304] 0 2024-06-15T01:07:05.257 INFO:tasks.workunit.client.0.smithi137.stdout:4/446: write dd/d15/d48/f61 [1355533,26191] 0 2024-06-15T01:07:05.258 INFO:tasks.workunit.client.1.smithi142.stdout:1/623: fsync d6/d12/fa1 0 2024-06-15T01:07:05.259 INFO:tasks.workunit.client.0.smithi137.stdout:4/447: rename dd/d15/d21/d3a to dd/d15/d21/d3a/d6e/d7b/da6 22 2024-06-15T01:07:05.259 INFO:tasks.workunit.client.1.smithi142.stdout:6/578: dwrite d1a/d3c/fc4 [0,4194304] 0 2024-06-15T01:07:05.262 INFO:tasks.workunit.client.0.smithi137.stdout:4/448: creat dd/d15/d30/fa7 x:0 0 0 2024-06-15T01:07:05.262 INFO:tasks.workunit.client.1.smithi142.stdout:9/537: readlink d6/dc/d1b/d25/d58/d74/l9b 0 2024-06-15T01:07:05.266 INFO:tasks.workunit.client.0.smithi137.stdout:4/449: rename dd/d15/d30/f8c to dd/d15/d36/d45/d55/fa8 0 2024-06-15T01:07:05.266 INFO:tasks.workunit.client.1.smithi142.stdout:4/674: write d5/d9/d23/d32/f8f [1537081,94529] 0 2024-06-15T01:07:05.268 INFO:tasks.workunit.client.1.smithi142.stdout:8/604: rename d0/d25/d5a/d8f/d97/fae to d0/d25/d5a/d8f/d97/fbe 0 2024-06-15T01:07:05.269 INFO:tasks.workunit.client.1.smithi142.stdout:8/605: stat d0/d17/d40/f76 0 2024-06-15T01:07:05.270 INFO:tasks.workunit.client.0.smithi137.stdout:4/450: creat dd/d15/d21/d9c/d75/d99/fa9 x:0 0 0 2024-06-15T01:07:05.272 INFO:tasks.workunit.client.0.smithi137.stdout:4/451: symlink dd/d15/d21/d3a/laa 0 2024-06-15T01:07:05.272 INFO:tasks.workunit.client.1.smithi142.stdout:6/579: truncate d1a/f25 615971 0 2024-06-15T01:07:05.272 INFO:tasks.workunit.client.1.smithi142.stdout:4/675: mkdir d5/d9/d1a/dee 0 2024-06-15T01:07:05.273 INFO:tasks.workunit.client.1.smithi142.stdout:9/538: mknod d6/dc/d1b/d25/dcf/cd0 0 2024-06-15T01:07:05.275 INFO:tasks.workunit.client.1.smithi142.stdout:6/580: stat d1a/d3c/db0 0 2024-06-15T01:07:05.275 INFO:tasks.workunit.client.1.smithi142.stdout:2/538: creat d8/df/d10/d1c/fad x:0 0 0 2024-06-15T01:07:05.276 INFO:tasks.workunit.client.1.smithi142.stdout:0/581: rename d3/d39/d3e/d81/d1e/c78 to d3/d39/d3e/db4/cd1 0 2024-06-15T01:07:05.276 INFO:tasks.workunit.client.0.smithi137.stdout:4/452: creat dd/d15/d36/d45/d55/d8d/d1f/d86/fab x:0 0 0 2024-06-15T01:07:05.278 INFO:tasks.workunit.client.0.smithi137.stdout:4/453: mkdir dd/d15/d36/d45/d8f/dac 0 2024-06-15T01:07:05.280 INFO:tasks.workunit.client.1.smithi142.stdout:0/582: readlink d3/d39/d3e/d81/d44/d7c/l25 0 2024-06-15T01:07:05.280 INFO:tasks.workunit.client.1.smithi142.stdout:9/539: creat d6/dc/d1b/d25/d35/fd1 x:0 0 0 2024-06-15T01:07:05.280 INFO:tasks.workunit.client.1.smithi142.stdout:1/624: dwrite d6/d12/d3c/d63/d8d/d97/dd3/f76 [0,4194304] 0 2024-06-15T01:07:05.282 INFO:tasks.workunit.client.1.smithi142.stdout:8/606: dwrite d0/d25/d53/f58 [0,4194304] 0 2024-06-15T01:07:05.282 INFO:tasks.workunit.client.1.smithi142.stdout:1/625: fdatasync d6/d12/d28/f9a 0 2024-06-15T01:07:05.283 INFO:tasks.workunit.client.1.smithi142.stdout:1/626: stat d6/d12/db5 0 2024-06-15T01:07:05.284 INFO:tasks.workunit.client.1.smithi142.stdout:0/583: creat d3/d39/d3e/d8f/fd2 x:0 0 0 2024-06-15T01:07:05.285 INFO:tasks.workunit.client.0.smithi137.stdout:4/454: dwrite dd/f22 [4194304,4194304] 0 2024-06-15T01:07:05.285 INFO:tasks.workunit.client.1.smithi142.stdout:9/540: creat d6/dc/d4e/d78/fd2 x:0 0 0 2024-06-15T01:07:05.286 INFO:tasks.workunit.client.1.smithi142.stdout:8/607: mkdir d0/d25/d53/d9c/dbf 0 2024-06-15T01:07:05.287 INFO:tasks.workunit.client.1.smithi142.stdout:8/608: write d0/d25/d5a/f69 [583576,111510] 0 2024-06-15T01:07:05.288 INFO:tasks.workunit.client.0.smithi137.stdout:4/455: write dd/f18 [4802524,46128] 0 2024-06-15T01:07:05.288 INFO:tasks.workunit.client.1.smithi142.stdout:8/609: stat d0/f43 0 2024-06-15T01:07:05.290 INFO:tasks.workunit.client.1.smithi142.stdout:1/627: unlink d6/d12/d3c/d63/d8d/d97/cc5 0 2024-06-15T01:07:05.292 INFO:tasks.workunit.client.1.smithi142.stdout:0/584: dwrite d3/d39/d3e/d81/d4c/fc1 [0,4194304] 0 2024-06-15T01:07:05.293 INFO:tasks.workunit.client.1.smithi142.stdout:1/628: write d6/d12/d1c/f34 [9000642,89511] 0 2024-06-15T01:07:05.295 INFO:tasks.workunit.client.1.smithi142.stdout:8/610: dread d0/d17/d40/d6e/fa9 [0,4194304] 0 2024-06-15T01:07:05.296 INFO:tasks.workunit.client.1.smithi142.stdout:9/541: truncate d6/d34/fc0 5226170 0 2024-06-15T01:07:05.299 INFO:tasks.workunit.client.1.smithi142.stdout:8/611: chown d0/d25/d5a 34041 1 2024-06-15T01:07:05.301 INFO:tasks.workunit.client.1.smithi142.stdout:5/544: truncate d2/d18/f61 2626894 0 2024-06-15T01:07:05.303 INFO:tasks.workunit.client.1.smithi142.stdout:5/545: write d2/d18/d79/f82 [708095,57135] 0 2024-06-15T01:07:05.304 INFO:tasks.workunit.client.1.smithi142.stdout:0/585: mknod d3/d39/d9a/da2/cd3 0 2024-06-15T01:07:05.304 INFO:tasks.workunit.client.1.smithi142.stdout:5/546: stat d2/d18/c39 0 2024-06-15T01:07:05.305 INFO:tasks.workunit.client.1.smithi142.stdout:8/612: dread d0/d17/d40/d5e/fad [0,4194304] 0 2024-06-15T01:07:05.305 INFO:tasks.workunit.client.1.smithi142.stdout:3/669: sync 2024-06-15T01:07:05.305 INFO:tasks.workunit.client.1.smithi142.stdout:5/547: chown d2/d18/c9f 854 1 2024-06-15T01:07:05.306 INFO:tasks.workunit.client.1.smithi142.stdout:3/670: read d0/d1f/d20/f26 [2520661,18634] 0 2024-06-15T01:07:05.307 INFO:tasks.workunit.client.1.smithi142.stdout:2/539: write d8/df/f85 [124948,103329] 0 2024-06-15T01:07:05.308 INFO:tasks.workunit.client.1.smithi142.stdout:5/548: unlink d2/d18/f69 0 2024-06-15T01:07:05.312 INFO:tasks.workunit.client.1.smithi142.stdout:1/629: dwrite d6/d12/d3c/d63/fa8 [0,4194304] 0 2024-06-15T01:07:05.313 INFO:tasks.workunit.client.1.smithi142.stdout:3/671: rmdir d0 39 2024-06-15T01:07:05.316 INFO:tasks.workunit.client.1.smithi142.stdout:5/549: write d2/d18/f99 [426879,39848] 0 2024-06-15T01:07:05.316 INFO:tasks.workunit.client.1.smithi142.stdout:2/540: dwrite d8/d19/d8d/f93 [0,4194304] 0 2024-06-15T01:07:05.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:04 smithi142 ceph-mon[37379]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:07:05.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:04 smithi142 ceph-mon[37379]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:07:05.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:04 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:05.319 INFO:tasks.workunit.client.0.smithi137.stdout:8/386: sync 2024-06-15T01:07:05.319 INFO:tasks.workunit.client.0.smithi137.stdout:7/433: sync 2024-06-15T01:07:05.319 INFO:tasks.workunit.client.0.smithi137.stdout:1/504: sync 2024-06-15T01:07:05.319 INFO:tasks.workunit.client.0.smithi137.stdout:3/536: sync 2024-06-15T01:07:05.320 INFO:tasks.workunit.client.0.smithi137.stdout:5/561: sync 2024-06-15T01:07:05.320 INFO:tasks.workunit.client.0.smithi137.stdout:9/266: sync 2024-06-15T01:07:05.320 INFO:tasks.workunit.client.0.smithi137.stdout:6/382: sync 2024-06-15T01:07:05.320 INFO:tasks.workunit.client.0.smithi137.stdout:0/487: sync 2024-06-15T01:07:05.320 INFO:tasks.workunit.client.1.smithi142.stdout:8/613: creat d0/d25/fc0 x:0 0 0 2024-06-15T01:07:05.321 INFO:tasks.workunit.client.1.smithi142.stdout:1/630: creat d6/d25/fe2 x:0 0 0 2024-06-15T01:07:05.322 INFO:tasks.workunit.client.1.smithi142.stdout:9/542: link d6/dc/d1b/d6f/da1/fa7 d6/dc/d1b/d25/fd3 0 2024-06-15T01:07:05.322 INFO:tasks.workunit.client.0.smithi137.stdout:5/562: write df/f41 [2599178,21086] 0 2024-06-15T01:07:05.323 INFO:tasks.workunit.client.1.smithi142.stdout:9/543: chown d6/d31/l3c 48 1 2024-06-15T01:07:05.324 INFO:tasks.workunit.client.0.smithi137.stdout:1/505: fdatasync d0/d1f/d60/f91 0 2024-06-15T01:07:05.324 INFO:tasks.workunit.client.1.smithi142.stdout:0/586: creat d3/d39/d3e/d81/d44/d7c/fd4 x:0 0 0 2024-06-15T01:07:05.327 INFO:tasks.workunit.client.0.smithi137.stdout:5/563: fsync f3 0 2024-06-15T01:07:05.330 INFO:tasks.workunit.client.1.smithi142.stdout:0/587: truncate d3/d39/d3e/d81/d44/d7c/d22/d59/fba 529793 0 2024-06-15T01:07:05.330 INFO:tasks.workunit.client.1.smithi142.stdout:0/588: chown f1 3 1 2024-06-15T01:07:05.331 INFO:tasks.workunit.client.0.smithi137.stdout:9/267: mkdir df/d12/d1b/d31/d42/d48 0 2024-06-15T01:07:05.331 INFO:tasks.workunit.client.0.smithi137.stdout:7/434: chown df/d7d/d3b/d5b/f6d 190085646 1 2024-06-15T01:07:05.332 INFO:tasks.workunit.client.1.smithi142.stdout:0/589: read d3/d39/d3e/d81/d1b/d38/f9b [1253211,33473] 0 2024-06-15T01:07:05.332 INFO:tasks.workunit.client.0.smithi137.stdout:1/506: dread - d0/d3/d1b/d4c/d59/fb4 zero size 2024-06-15T01:07:05.333 INFO:tasks.workunit.client.1.smithi142.stdout:0/590: chown d3/d39/d3e/d81/d44/d7c/c4f 444 1 2024-06-15T01:07:05.334 INFO:tasks.workunit.client.0.smithi137.stdout:7/435: chown c8 0 1 2024-06-15T01:07:05.334 INFO:tasks.workunit.client.1.smithi142.stdout:8/614: unlink d0/d17/f2f 0 2024-06-15T01:07:05.334 INFO:tasks.workunit.client.0.smithi137.stdout:4/456: truncate dd/d15/d36/d45/d55/f7c 3388149 0 2024-06-15T01:07:05.335 INFO:tasks.workunit.client.1.smithi142.stdout:8/615: dread - d0/d17/d40/d5e/f94 zero size 2024-06-15T01:07:05.335 INFO:tasks.workunit.client.1.smithi142.stdout:4/676: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d 207998 0 2024-06-15T01:07:05.336 INFO:tasks.workunit.client.0.smithi137.stdout:3/537: dwrite d0/d2f/faa [0,4194304] 0 2024-06-15T01:07:05.337 INFO:tasks.workunit.client.1.smithi142.stdout:1/631: chown d6/d12/d3c/d63/d8d/d97/dd3/l7d 83801753 1 2024-06-15T01:07:05.337 INFO:tasks.workunit.client.1.smithi142.stdout:3/672: dread d0/d3/da/d71/d55/f57 [0,4194304] 0 2024-06-15T01:07:05.338 INFO:tasks.workunit.client.0.smithi137.stdout:4/457: stat dd/f12 0 2024-06-15T01:07:05.339 INFO:tasks.workunit.client.1.smithi142.stdout:3/673: truncate d0/d1f/d20/d5d/d82/f9a 5928245 0 2024-06-15T01:07:05.340 INFO:tasks.workunit.client.1.smithi142.stdout:5/550: mknod d2/d3e/d31/cad 0 2024-06-15T01:07:05.340 INFO:tasks.workunit.client.1.smithi142.stdout:4/677: read d5/d9/da/f3c [2488782,77610] 0 2024-06-15T01:07:05.341 INFO:tasks.workunit.client.1.smithi142.stdout:5/551: chown d2/de/d19/d5f/d68/d76/f81 0 1 2024-06-15T01:07:05.342 INFO:tasks.workunit.client.1.smithi142.stdout:5/552: readlink d2/de/d19/l80 0 2024-06-15T01:07:05.342 INFO:tasks.workunit.client.1.smithi142.stdout:4/678: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/l9f 7741 1 2024-06-15T01:07:05.343 INFO:tasks.workunit.client.0.smithi137.stdout:2/356: sync 2024-06-15T01:07:05.343 INFO:tasks.workunit.client.0.smithi137.stdout:3/538: write d0/d2f/f35 [1448840,120339] 0 2024-06-15T01:07:05.343 INFO:tasks.workunit.client.0.smithi137.stdout:6/383: creat d0/d1a/d3e/d57/d1d/d2e/d62/f76 x:0 0 0 2024-06-15T01:07:05.344 INFO:tasks.workunit.client.0.smithi137.stdout:3/539: chown d0/d2/d1e/c27 50 1 2024-06-15T01:07:05.345 INFO:tasks.workunit.client.0.smithi137.stdout:9/268: rename df/d12/d1b/d31/d42/d48 to df/d49 0 2024-06-15T01:07:05.346 INFO:tasks.workunit.client.1.smithi142.stdout:3/674: dwrite d0/d3/f43 [0,4194304] 0 2024-06-15T01:07:05.347 INFO:tasks.workunit.client.0.smithi137.stdout:7/436: creat df/d13/d2e/f86 x:0 0 0 2024-06-15T01:07:05.349 INFO:tasks.workunit.client.1.smithi142.stdout:2/541: mknod d8/df/d10/d94/d25/d22/d43/d52/d7a/cae 0 2024-06-15T01:07:05.351 INFO:tasks.workunit.client.1.smithi142.stdout:2/542: read d8/df/d10/f11 [187341,44223] 0 2024-06-15T01:07:05.351 INFO:tasks.workunit.client.1.smithi142.stdout:9/544: link d6/d34/fa0 d6/dc/d82/fd4 0 2024-06-15T01:07:05.353 INFO:tasks.workunit.client.1.smithi142.stdout:2/543: dread d8/df/f59 [0,4194304] 0 2024-06-15T01:07:05.356 INFO:tasks.workunit.client.1.smithi142.stdout:2/544: truncate d8/df/d10/d1c/f97 4770988 0 2024-06-15T01:07:05.357 INFO:tasks.workunit.client.1.smithi142.stdout:8/616: rename d0/d17/f7c to d0/d25/d5a/d8f/d98/da4/fc1 0 2024-06-15T01:07:05.357 INFO:tasks.workunit.client.0.smithi137.stdout:8/387: dread d9/de/f36 [0,4194304] 0 2024-06-15T01:07:05.360 INFO:tasks.workunit.client.0.smithi137.stdout:8/388: write d9/de/d1e/d35/f5b [1025129,25066] 0 2024-06-15T01:07:05.360 INFO:tasks.workunit.client.1.smithi142.stdout:5/553: fsync d2/d18/f37 0 2024-06-15T01:07:05.361 INFO:tasks.workunit.client.0.smithi137.stdout:7/437: dwrite df/d13/d2e/f51 [0,4194304] 0 2024-06-15T01:07:05.361 INFO:tasks.workunit.client.1.smithi142.stdout:4/679: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/cef 0 2024-06-15T01:07:05.362 INFO:tasks.workunit.client.1.smithi142.stdout:7/595: sync 2024-06-15T01:07:05.362 INFO:tasks.workunit.client.1.smithi142.stdout:2/545: dwrite d8/df/d10/d94/d25/d22/d43/d51/f99 [0,4194304] 0 2024-06-15T01:07:05.363 INFO:tasks.workunit.client.1.smithi142.stdout:5/554: chown d2/de/d19/d5f/d68/f73 10 1 2024-06-15T01:07:05.365 INFO:tasks.workunit.client.1.smithi142.stdout:7/596: truncate d2/d8/da/d33/f52 908218 0 2024-06-15T01:07:05.366 INFO:tasks.workunit.client.1.smithi142.stdout:0/591: getdents d3/d39/d3e/d81/d1b/d86 0 2024-06-15T01:07:05.367 INFO:tasks.workunit.client.0.smithi137.stdout:5/564: dread df/d11/d14/d3c/f3e [0,4194304] 0 2024-06-15T01:07:05.370 INFO:tasks.workunit.client.1.smithi142.stdout:2/546: fsync d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa6 0 2024-06-15T01:07:05.371 INFO:tasks.workunit.client.1.smithi142.stdout:9/545: rmdir d6/dc/d1b 39 2024-06-15T01:07:05.372 INFO:tasks.workunit.client.0.smithi137.stdout:5/565: chown df/d76/d7c/d9c/da8/fad 0 1 2024-06-15T01:07:05.373 INFO:tasks.workunit.client.1.smithi142.stdout:4/680: dwrite d5/d9/d41/f78 [4194304,4194304] 0 2024-06-15T01:07:05.373 INFO:tasks.workunit.client.0.smithi137.stdout:9/269: truncate df/d12/d15/f1d 376809 0 2024-06-15T01:07:05.374 INFO:tasks.workunit.client.0.smithi137.stdout:0/488: dwrite d1/d2/dc/d10/d14/d56/f82 [0,4194304] 0 2024-06-15T01:07:05.374 INFO:tasks.workunit.client.0.smithi137.stdout:9/270: readlink df/d12/l3b 0 2024-06-15T01:07:05.375 INFO:tasks.workunit.client.1.smithi142.stdout:4/681: write d5/d1c/d6f/dc7/fab [432244,98269] 0 2024-06-15T01:07:05.377 INFO:tasks.workunit.client.1.smithi142.stdout:5/555: dread d2/d3e/d31/f6d [0,4194304] 0 2024-06-15T01:07:05.378 INFO:tasks.workunit.client.1.smithi142.stdout:1/632: link d6/d12/f22 d6/d3a/fe3 0 2024-06-15T01:07:05.379 INFO:tasks.workunit.client.0.smithi137.stdout:8/389: chown d9/de/f18 31447 1 2024-06-15T01:07:05.379 INFO:tasks.workunit.client.0.smithi137.stdout:4/458: symlink dd/d15/d36/d45/d8f/dac/lad 0 2024-06-15T01:07:05.381 INFO:tasks.workunit.client.0.smithi137.stdout:2/357: mknod d0/dc/d22/d23/d5c/c6b 0 2024-06-15T01:07:05.383 INFO:tasks.workunit.client.1.smithi142.stdout:8/617: unlink d0/d25/fb0 0 2024-06-15T01:07:05.389 INFO:tasks.workunit.client.0.smithi137.stdout:5/566: creat df/d44/d54/d95/d35/fbe x:0 0 0 2024-06-15T01:07:05.390 INFO:tasks.workunit.client.0.smithi137.stdout:9/271: stat df/d12/d15/d16/f27 0 2024-06-15T01:07:05.391 INFO:tasks.workunit.client.0.smithi137.stdout:5/567: read - df/d44/d54/d95/d3b/d4b/fa0 zero size 2024-06-15T01:07:05.392 INFO:tasks.workunit.client.0.smithi137.stdout:0/489: rename d1/d2/dc/d10/d14/d15/l37 to d1/d2/dc/d10/d14/d52/la9 0 2024-06-15T01:07:05.393 INFO:tasks.workunit.client.0.smithi137.stdout:7/438: chown df/d7d/d1c/d1d/d27/d33/d59 1 1 2024-06-15T01:07:05.394 INFO:tasks.workunit.client.0.smithi137.stdout:4/459: creat dd/d15/d21/d9c/d75/fae x:0 0 0 2024-06-15T01:07:05.395 INFO:tasks.workunit.client.1.smithi142.stdout:7/597: mknod d2/d8/da/d33/d25/d79/cbe 0 2024-06-15T01:07:05.395 INFO:tasks.workunit.client.0.smithi137.stdout:8/390: write d9/de/d1e/f42 [270255,123811] 0 2024-06-15T01:07:05.395 INFO:tasks.workunit.client.0.smithi137.stdout:0/490: read d1/d2/dc/d10/d14/f1b [1392691,81634] 0 2024-06-15T01:07:05.396 INFO:tasks.workunit.client.0.smithi137.stdout:2/358: readlink d0/d1c/l51 0 2024-06-15T01:07:05.397 INFO:tasks.workunit.client.0.smithi137.stdout:1/507: getdents d0/d1f/d60 0 2024-06-15T01:07:05.399 INFO:tasks.workunit.client.0.smithi137.stdout:9/272: fsync df/d12/d1b/d31/f33 0 2024-06-15T01:07:05.400 INFO:tasks.workunit.client.0.smithi137.stdout:1/508: chown d0/d2d/f5e 18548569 1 2024-06-15T01:07:05.401 INFO:tasks.workunit.client.1.smithi142.stdout:3/675: creat d0/d3/fcb x:0 0 0 2024-06-15T01:07:05.401 INFO:tasks.workunit.client.0.smithi137.stdout:3/540: getdents d0/d2/d22/d63 0 2024-06-15T01:07:05.401 INFO:tasks.workunit.client.0.smithi137.stdout:0/491: chown d1/d2/dc/d10/d53/l6f 12152539 1 2024-06-15T01:07:05.403 INFO:tasks.workunit.client.1.smithi142.stdout:2/547: mkdir d8/df/daf 0 2024-06-15T01:07:05.404 INFO:tasks.workunit.client.0.smithi137.stdout:2/359: creat d0/dc/d22/f6c x:0 0 0 2024-06-15T01:07:05.405 INFO:tasks.workunit.client.1.smithi142.stdout:3/676: dread d0/d1f/d20/d59/f92 [0,4194304] 0 2024-06-15T01:07:05.406 INFO:tasks.workunit.client.1.smithi142.stdout:7/598: symlink d2/d8/da/d33/d25/d27/lbf 0 2024-06-15T01:07:05.406 INFO:tasks.workunit.client.1.smithi142.stdout:3/677: write d0/d3/f54 [1945365,1891] 0 2024-06-15T01:07:05.407 INFO:tasks.workunit.client.0.smithi137.stdout:7/439: dread df/d7d/d1c/d1d/d27/f49 [0,4194304] 0 2024-06-15T01:07:05.407 INFO:tasks.workunit.client.0.smithi137.stdout:4/460: rename dd/d15/d21/f78 to dd/d15/d21/d3a/d6e/d7b/faf 0 2024-06-15T01:07:05.409 INFO:tasks.workunit.client.1.smithi142.stdout:5/556: mkdir d2/d18/d2e/dae 0 2024-06-15T01:07:05.410 INFO:tasks.workunit.client.0.smithi137.stdout:1/509: creat d0/d10/fb5 x:0 0 0 2024-06-15T01:07:05.411 INFO:tasks.workunit.client.1.smithi142.stdout:1/633: mkdir d6/de4 0 2024-06-15T01:07:05.412 INFO:tasks.workunit.client.0.smithi137.stdout:4/461: fsync dd/d15/d30/fa7 0 2024-06-15T01:07:05.412 INFO:tasks.workunit.client.0.smithi137.stdout:4/462: fdatasync dd/d15/d48/d67/f6c 0 2024-06-15T01:07:05.414 INFO:tasks.workunit.client.0.smithi137.stdout:8/391: fsync d9/de/d1e/d67/f6a 0 2024-06-15T01:07:05.414 INFO:tasks.workunit.client.1.smithi142.stdout:8/618: link d0/f18 d0/d17/d40/d6e/fc2 0 2024-06-15T01:07:05.415 INFO:tasks.workunit.client.1.smithi142.stdout:7/599: dwrite d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:07:05.415 INFO:tasks.workunit.client.0.smithi137.stdout:9/273: dwrite df/d12/d29/f44 [0,4194304] 0 2024-06-15T01:07:05.416 INFO:tasks.workunit.client.1.smithi142.stdout:3/678: dwrite d0/d3/f1a [4194304,4194304] 0 2024-06-15T01:07:05.416 INFO:tasks.workunit.client.0.smithi137.stdout:3/541: mknod d0/d2/d22/d24/cb0 0 2024-06-15T01:07:05.417 INFO:tasks.workunit.client.0.smithi137.stdout:0/492: symlink d1/d2/dc/d6c/laa 0 2024-06-15T01:07:05.417 INFO:tasks.workunit.client.0.smithi137.stdout:9/274: truncate df/f18 5115433 0 2024-06-15T01:07:05.419 INFO:tasks.workunit.client.0.smithi137.stdout:3/542: readlink d0/d13/l72 0 2024-06-15T01:07:05.422 INFO:tasks.workunit.client.0.smithi137.stdout:4/463: truncate dd/d15/d21/d3a/d6e/d7b/faf 421237 0 2024-06-15T01:07:05.426 INFO:tasks.workunit.client.0.smithi137.stdout:7/440: mkdir df/d13/d2e/d87 0 2024-06-15T01:07:05.426 INFO:tasks.workunit.client.1.smithi142.stdout:4/682: dread d5/d9/d18/f80 [4194304,4194304] 0 2024-06-15T01:07:05.426 INFO:tasks.workunit.client.1.smithi142.stdout:0/592: dread d3/d39/d3e/d81/fb [0,4194304] 0 2024-06-15T01:07:05.429 INFO:tasks.workunit.client.1.smithi142.stdout:8/619: mknod d0/cc3 0 2024-06-15T01:07:05.429 INFO:tasks.workunit.client.1.smithi142.stdout:2/548: unlink d8/df/d10/d94/d25/d22/d43/d51/f99 0 2024-06-15T01:07:05.433 INFO:tasks.workunit.client.0.smithi137.stdout:5/568: getdents df/d7a 0 2024-06-15T01:07:05.434 INFO:tasks.workunit.client.1.smithi142.stdout:7/600: dwrite d2/d8/dd/d14/d2c/d37/d6b/d88/f7d [0,4194304] 0 2024-06-15T01:07:05.434 INFO:tasks.workunit.client.1.smithi142.stdout:8/620: write d0/d25/d66/f92 [1739708,113729] 0 2024-06-15T01:07:05.435 INFO:tasks.workunit.client.0.smithi137.stdout:3/543: mknod d0/d13/cb1 0 2024-06-15T01:07:05.435 INFO:tasks.workunit.client.1.smithi142.stdout:8/621: stat d0/c7b 0 2024-06-15T01:07:05.436 INFO:tasks.workunit.client.1.smithi142.stdout:7/601: chown d2/d6/f1c 15124 1 2024-06-15T01:07:05.436 INFO:tasks.workunit.client.1.smithi142.stdout:4/683: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f6d [5232293,74755] 0 2024-06-15T01:07:05.436 INFO:tasks.workunit.client.0.smithi137.stdout:9/275: mkdir df/d12/d29/d4a 0 2024-06-15T01:07:05.436 INFO:tasks.workunit.client.1.smithi142.stdout:2/549: symlink d8/df/d10/d94/d25/d22/d43/lb0 0 2024-06-15T01:07:05.437 INFO:tasks.workunit.client.0.smithi137.stdout:2/360: truncate d0/f6 3888785 0 2024-06-15T01:07:05.438 INFO:tasks.workunit.client.0.smithi137.stdout:2/361: stat d0/de/f16 0 2024-06-15T01:07:05.439 INFO:tasks.workunit.client.1.smithi142.stdout:8/622: write d0/d17/d40/fa2 [3194,108025] 0 2024-06-15T01:07:05.440 INFO:tasks.workunit.client.0.smithi137.stdout:7/441: mknod df/d7d/d3b/d5b/c88 0 2024-06-15T01:07:05.440 INFO:tasks.workunit.client.1.smithi142.stdout:4/684: readlink d5/d9/d23/d32/ld2 0 2024-06-15T01:07:05.440 INFO:tasks.workunit.client.0.smithi137.stdout:7/442: fdatasync df/d13/d2e/f86 0 2024-06-15T01:07:05.441 INFO:tasks.workunit.client.0.smithi137.stdout:7/443: fsync df/d13/d2e/f58 0 2024-06-15T01:07:05.442 INFO:tasks.workunit.client.1.smithi142.stdout:7/602: dread d2/d6/d22/d47/f5a [0,4194304] 0 2024-06-15T01:07:05.442 INFO:tasks.workunit.client.0.smithi137.stdout:7/444: stat df/d7d/d3b/d5b 0 2024-06-15T01:07:05.443 INFO:tasks.workunit.client.0.smithi137.stdout:5/569: mkdir df/d11/d70/dbf 0 2024-06-15T01:07:05.444 INFO:tasks.workunit.client.0.smithi137.stdout:0/493: symlink d1/d2/dc/d10/d14/d15/d17/lab 0 2024-06-15T01:07:05.445 INFO:tasks.workunit.client.0.smithi137.stdout:9/276: rmdir df/d12/d15/d16/d3c 39 2024-06-15T01:07:05.446 INFO:tasks.workunit.client.0.smithi137.stdout:0/494: chown d1/d2/dc/d10/c51 723 1 2024-06-15T01:07:05.446 INFO:tasks.workunit.client.1.smithi142.stdout:7/603: mkdir d2/d8/dd/dc0 0 2024-06-15T01:07:05.447 INFO:tasks.workunit.client.0.smithi137.stdout:9/277: fdatasync df/d12/d1b/f2f 0 2024-06-15T01:07:05.447 INFO:tasks.workunit.client.0.smithi137.stdout:0/495: readlink d1/d2/dc/d6c/laa 0 2024-06-15T01:07:05.447 INFO:tasks.workunit.client.0.smithi137.stdout:2/362: mknod d0/d42/d4d/c6d 0 2024-06-15T01:07:05.449 INFO:tasks.workunit.client.0.smithi137.stdout:8/392: creat d9/de/f83 x:0 0 0 2024-06-15T01:07:05.449 INFO:tasks.workunit.client.0.smithi137.stdout:2/363: write d0/d42/d49/f62 [1330806,89373] 0 2024-06-15T01:07:05.449 INFO:tasks.workunit.client.1.smithi142.stdout:2/550: dwrite d8/df/d10/d94/d25/d22/d43/f46 [4194304,4194304] 0 2024-06-15T01:07:05.450 INFO:tasks.workunit.client.0.smithi137.stdout:3/544: symlink d0/d77/lb2 0 2024-06-15T01:07:05.450 INFO:tasks.workunit.client.1.smithi142.stdout:8/623: dwrite d0/d25/d53/d9c/fbd [0,4194304] 0 2024-06-15T01:07:05.451 INFO:tasks.workunit.client.1.smithi142.stdout:7/604: write d2/d8/da/d33/f9e [410217,37479] 0 2024-06-15T01:07:05.452 INFO:tasks.workunit.client.0.smithi137.stdout:4/464: rmdir dd/d15/d21/d3a/d6e/da4 0 2024-06-15T01:07:05.453 INFO:tasks.workunit.client.1.smithi142.stdout:2/551: fsync d8/df/d10/d94/d25/d1a/d57/f67 0 2024-06-15T01:07:05.454 INFO:tasks.workunit.client.0.smithi137.stdout:9/278: mkdir df/d49/d4b 0 2024-06-15T01:07:05.455 INFO:tasks.workunit.client.0.smithi137.stdout:1/510: getdents d0/d2d/d3e 0 2024-06-15T01:07:05.456 INFO:tasks.workunit.client.1.smithi142.stdout:2/552: fdatasync d8/df/d10/d94/d25/d1a/d63/f9a 0 2024-06-15T01:07:05.457 INFO:tasks.workunit.client.1.smithi142.stdout:7/605: creat d2/d6/d22/d3c/fc1 x:0 0 0 2024-06-15T01:07:05.458 INFO:tasks.workunit.client.0.smithi137.stdout:0/496: mknod d1/d2/d55/cac 0 2024-06-15T01:07:05.461 INFO:tasks.workunit.client.0.smithi137.stdout:8/393: dread d9/d2a/f6f [0,4194304] 0 2024-06-15T01:07:05.463 INFO:tasks.workunit.client.0.smithi137.stdout:9/279: dread df/d12/d15/d16/f27 [0,4194304] 0 2024-06-15T01:07:05.464 INFO:tasks.workunit.client.1.smithi142.stdout:7/606: rename d2/d8/dd/d14/d2c/d37/c45 to d2/d8/dd/dc0/cc2 0 2024-06-15T01:07:05.465 INFO:tasks.workunit.client.1.smithi142.stdout:7/607: chown d2/d8/dd/d14/d2c 2164488 1 2024-06-15T01:07:05.467 INFO:tasks.workunit.client.0.smithi137.stdout:3/545: read d0/f1f [594374,8268] 0 2024-06-15T01:07:05.468 INFO:tasks.workunit.client.0.smithi137.stdout:9/280: fsync df/d12/d1b/d31/f33 0 2024-06-15T01:07:05.468 INFO:tasks.workunit.client.0.smithi137.stdout:8/394: truncate d9/d2a/f7a 432668 0 2024-06-15T01:07:05.469 INFO:tasks.workunit.client.1.smithi142.stdout:7/608: unlink d2/d8/dd/fb2 0 2024-06-15T01:07:05.469 INFO:tasks.workunit.client.1.smithi142.stdout:7/609: chown d2/d8/dd/l60 208084 1 2024-06-15T01:07:05.472 INFO:tasks.workunit.client.1.smithi142.stdout:4/685: dread d5/d9/da/f7f [0,4194304] 0 2024-06-15T01:07:05.472 INFO:tasks.workunit.client.1.smithi142.stdout:7/610: creat d2/d8/dd/dc0/fc3 x:0 0 0 2024-06-15T01:07:05.473 INFO:tasks.workunit.client.0.smithi137.stdout:0/497: dwrite d1/d2/dc/d10/f96 [0,4194304] 0 2024-06-15T01:07:05.474 INFO:tasks.workunit.client.0.smithi137.stdout:5/570: creat df/d44/d54/d95/d3b/d4b/fc0 x:0 0 0 2024-06-15T01:07:05.474 INFO:tasks.workunit.client.1.smithi142.stdout:4/686: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f6d 0 2024-06-15T01:07:05.474 INFO:tasks.workunit.client.1.smithi142.stdout:7/611: write d2/d6/d22/d47/f4f [2024589,2849] 0 2024-06-15T01:07:05.475 INFO:tasks.workunit.client.0.smithi137.stdout:5/571: readlink df/d11/d14/d3c/l88 0 2024-06-15T01:07:05.475 INFO:tasks.workunit.client.0.smithi137.stdout:5/572: chown df/c58 3104002 1 2024-06-15T01:07:05.477 INFO:tasks.workunit.client.0.smithi137.stdout:1/511: creat d0/d10/daf/fb6 x:0 0 0 2024-06-15T01:07:05.479 INFO:tasks.workunit.client.1.smithi142.stdout:7/612: unlink d2/d8/da/d33/d25/d79/f9a 0 2024-06-15T01:07:05.480 INFO:tasks.workunit.client.0.smithi137.stdout:3/546: unlink d0/d2f/f35 0 2024-06-15T01:07:05.481 INFO:tasks.workunit.client.1.smithi142.stdout:7/613: write d2/d6/d22/d3c/f56 [3552953,58210] 0 2024-06-15T01:07:05.481 INFO:tasks.workunit.client.1.smithi142.stdout:6/581: sync 2024-06-15T01:07:05.483 INFO:tasks.workunit.client.0.smithi137.stdout:9/281: unlink df/d2a/f43 0 2024-06-15T01:07:05.483 INFO:tasks.workunit.client.1.smithi142.stdout:4/687: symlink d5/d1c/d6f/dc7/lf0 0 2024-06-15T01:07:05.484 INFO:tasks.workunit.client.1.smithi142.stdout:8/624: dread d0/d17/f61 [0,4194304] 0 2024-06-15T01:07:05.485 INFO:tasks.workunit.client.1.smithi142.stdout:7/614: write d2/d8/dd/d14/fb8 [668685,43454] 0 2024-06-15T01:07:05.486 INFO:tasks.workunit.client.0.smithi137.stdout:9/282: chown df/d12/d15/f26 1 1 2024-06-15T01:07:05.486 INFO:tasks.workunit.client.1.smithi142.stdout:2/553: dread d8/df/d1f/f3e [0,4194304] 0 2024-06-15T01:07:05.486 INFO:tasks.workunit.client.1.smithi142.stdout:8/625: chown d0/f18 1094 1 2024-06-15T01:07:05.486 INFO:tasks.workunit.client.1.smithi142.stdout:6/582: mknod d1a/d39/cca 0 2024-06-15T01:07:05.487 INFO:tasks.workunit.client.0.smithi137.stdout:2/364: write d0/de/f1a [1266822,105804] 0 2024-06-15T01:07:05.488 INFO:tasks.workunit.client.1.smithi142.stdout:7/615: truncate d2/d6/d22/d47/f7c 4351097 0 2024-06-15T01:07:05.488 INFO:tasks.workunit.client.1.smithi142.stdout:7/616: readlink d2/d8/da/d33/d25/l82 0 2024-06-15T01:07:05.489 INFO:tasks.workunit.client.0.smithi137.stdout:3/547: dwrite d0/d2f/f69 [0,4194304] 0 2024-06-15T01:07:05.490 INFO:tasks.workunit.client.1.smithi142.stdout:4/688: unlink d5/d9/d23/f4e 0 2024-06-15T01:07:05.491 INFO:tasks.workunit.client.1.smithi142.stdout:1/634: truncate d6/d12/d3c/d63/d8d/d97/dd3/f76 1107702 0 2024-06-15T01:07:05.492 INFO:tasks.workunit.client.1.smithi142.stdout:2/554: rmdir d8/df/d10 39 2024-06-15T01:07:05.492 INFO:tasks.workunit.client.0.smithi137.stdout:0/498: symlink d1/d7e/lad 0 2024-06-15T01:07:05.493 INFO:tasks.workunit.client.0.smithi137.stdout:7/445: getdents df/d7d/d1c/d1d 0 2024-06-15T01:07:05.495 INFO:tasks.workunit.client.1.smithi142.stdout:8/626: mknod d0/d25/d66/cc4 0 2024-06-15T01:07:05.497 INFO:tasks.workunit.client.0.smithi137.stdout:1/512: rename d0/d3/d1b/d4c/fb1 to d0/d3/d1b/d4c/d59/fb7 0 2024-06-15T01:07:05.498 INFO:tasks.workunit.client.1.smithi142.stdout:6/583: creat d1a/d2d/d3b/dab/fcb x:0 0 0 2024-06-15T01:07:05.499 INFO:tasks.workunit.client.1.smithi142.stdout:4/689: unlink d5/d9/da/f54 0 2024-06-15T01:07:05.501 INFO:tasks.workunit.client.1.smithi142.stdout:7/617: write d2/d8/dd/d14/d2c/d37/d6b/d80/f36 [1147503,112904] 0 2024-06-15T01:07:05.502 INFO:tasks.workunit.client.1.smithi142.stdout:0/593: dwrite d3/d39/d3e/d81/f88 [0,4194304] 0 2024-06-15T01:07:05.503 INFO:tasks.workunit.client.1.smithi142.stdout:6/584: write d1a/d39/d3a/f70 [313385,50298] 0 2024-06-15T01:07:05.504 INFO:tasks.workunit.client.0.smithi137.stdout:9/283: creat df/d2a/f4c x:0 0 0 2024-06-15T01:07:05.504 INFO:tasks.workunit.client.1.smithi142.stdout:8/627: unlink d0/faa 0 2024-06-15T01:07:05.505 INFO:tasks.workunit.client.0.smithi137.stdout:5/573: write df/d11/d14/d6c/f99 [1563136,114776] 0 2024-06-15T01:07:05.506 INFO:tasks.workunit.client.0.smithi137.stdout:2/365: mkdir d0/d42/d4d/d56/d6e 0 2024-06-15T01:07:05.506 INFO:tasks.workunit.client.1.smithi142.stdout:2/555: dwrite d8/d19/f56 [0,4194304] 0 2024-06-15T01:07:05.506 INFO:tasks.workunit.client.1.smithi142.stdout:7/618: write d2/d8/da/d33/d25/f40 [1421686,104156] 0 2024-06-15T01:07:05.507 INFO:tasks.workunit.client.0.smithi137.stdout:8/395: dwrite d9/de/d1e/d67/f6a [0,4194304] 0 2024-06-15T01:07:05.509 INFO:tasks.workunit.client.1.smithi142.stdout:2/556: chown d8/df/c38 0 1 2024-06-15T01:07:05.511 INFO:tasks.workunit.client.0.smithi137.stdout:4/465: getdents dd/d15/d36/d45/d55/d8d/d1f/d86 0 2024-06-15T01:07:05.512 INFO:tasks.workunit.client.0.smithi137.stdout:2/366: write d0/d42/d4d/d56/f4f [213085,2430] 0 2024-06-15T01:07:05.513 INFO:tasks.workunit.client.0.smithi137.stdout:0/499: creat d1/d7e/fae x:0 0 0 2024-06-15T01:07:05.514 INFO:tasks.workunit.client.1.smithi142.stdout:0/594: dwrite d3/d39/d3e/d81/d1b/d52/f85 [0,4194304] 0 2024-06-15T01:07:05.515 INFO:tasks.workunit.client.1.smithi142.stdout:0/595: read - d3/f48 zero size 2024-06-15T01:07:05.517 INFO:tasks.workunit.client.0.smithi137.stdout:7/446: mkdir df/d13/d2e/d1a/d3f/d89 0 2024-06-15T01:07:05.517 INFO:tasks.workunit.client.0.smithi137.stdout:3/548: write d0/d2/d22/d24/f89 [2216211,52879] 0 2024-06-15T01:07:05.519 INFO:tasks.workunit.client.0.smithi137.stdout:2/367: dread - d0/f5b zero size 2024-06-15T01:07:05.519 INFO:tasks.workunit.client.0.smithi137.stdout:5/574: write df/d44/d54/d6b/fb3 [187193,14267] 0 2024-06-15T01:07:05.519 INFO:tasks.workunit.client.0.smithi137.stdout:1/513: rmdir d0/d1f/d76/da1 39 2024-06-15T01:07:05.523 INFO:tasks.workunit.client.1.smithi142.stdout:1/635: creat d6/d12/d80/d9b/db2/dbf/fe5 x:0 0 0 2024-06-15T01:07:05.524 INFO:tasks.workunit.client.0.smithi137.stdout:9/284: creat df/f4d x:0 0 0 2024-06-15T01:07:05.525 INFO:tasks.workunit.client.1.smithi142.stdout:4/690: symlink d5/d1c/d6f/dc7/d30/d46/lf1 0 2024-06-15T01:07:05.526 INFO:tasks.workunit.client.0.smithi137.stdout:4/466: creat dd/d15/d21/d3a/d51/d5b/fb0 x:0 0 0 2024-06-15T01:07:05.527 INFO:tasks.workunit.client.0.smithi137.stdout:4/467: chown dd/d15/d36/d45/d55/l5c 12 1 2024-06-15T01:07:05.528 INFO:tasks.workunit.client.1.smithi142.stdout:6/585: mknod d1a/d39/d3a/d51/ccc 0 2024-06-15T01:07:05.529 INFO:tasks.workunit.client.1.smithi142.stdout:9/546: sync 2024-06-15T01:07:05.529 INFO:tasks.workunit.client.0.smithi137.stdout:2/368: mknod d0/d42/d49/c6f 0 2024-06-15T01:07:05.532 INFO:tasks.workunit.client.0.smithi137.stdout:1/514: symlink d0/d10/daf/lb8 0 2024-06-15T01:07:05.533 INFO:tasks.workunit.client.0.smithi137.stdout:3/549: rename d0/d2/d1e/l3e to d0/d56/d8f/d9c/d9f/lb3 0 2024-06-15T01:07:05.533 INFO:tasks.workunit.client.1.smithi142.stdout:2/557: write d8/df/d10/d94/d25/d22/d43/fac [579247,47091] 0 2024-06-15T01:07:05.534 INFO:tasks.workunit.client.1.smithi142.stdout:0/596: rmdir d3/d39/d3e/d81/d44/d7c/d22/d59 39 2024-06-15T01:07:05.535 INFO:tasks.workunit.client.0.smithi137.stdout:7/447: creat df/d13/d2e/d1a/d3f/d89/f8a x:0 0 0 2024-06-15T01:07:05.536 INFO:tasks.workunit.client.0.smithi137.stdout:5/575: mknod df/d44/d54/cc1 0 2024-06-15T01:07:05.536 INFO:tasks.workunit.client.0.smithi137.stdout:2/369: write d0/f69 [467706,35070] 0 2024-06-15T01:07:05.537 INFO:tasks.workunit.client.1.smithi142.stdout:4/691: readlink d5/d9/d18/l53 0 2024-06-15T01:07:05.540 INFO:tasks.workunit.client.0.smithi137.stdout:1/515: creat d0/d3/d47/d73/fb9 x:0 0 0 2024-06-15T01:07:05.541 INFO:tasks.workunit.client.1.smithi142.stdout:6/586: symlink d1a/d2d/lcd 0 2024-06-15T01:07:05.542 INFO:tasks.workunit.client.1.smithi142.stdout:6/587: truncate d1a/d2d/fb8 542724 0 2024-06-15T01:07:05.543 INFO:tasks.workunit.client.1.smithi142.stdout:0/597: rename d3/d39/d3e/d81/d44/d7c/c5e to d3/d39/d3e/d81/d4c/da0/cd5 0 2024-06-15T01:07:05.544 INFO:tasks.workunit.client.0.smithi137.stdout:0/500: dwrite d1/d2/dc/d10/d14/d15/f35 [0,4194304] 0 2024-06-15T01:07:05.544 INFO:tasks.workunit.client.1.smithi142.stdout:2/558: dwrite d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [0,4194304] 0 2024-06-15T01:07:05.544 INFO:tasks.workunit.client.1.smithi142.stdout:6/588: readlink d1a/d39/d3a/d51/l53 0 2024-06-15T01:07:05.546 INFO:tasks.workunit.client.1.smithi142.stdout:8/628: getdents d0/d17/d40/d5e 0 2024-06-15T01:07:05.546 INFO:tasks.workunit.client.1.smithi142.stdout:6/589: write d1a/d39/d42/d4c/fc6 [324084,63530] 0 2024-06-15T01:07:05.549 INFO:tasks.workunit.client.0.smithi137.stdout:2/370: creat d0/d1c/f70 x:0 0 0 2024-06-15T01:07:05.554 INFO:tasks.workunit.client.1.smithi142.stdout:8/629: read d0/d25/d66/f92 [3141904,62777] 0 2024-06-15T01:07:05.554 INFO:tasks.workunit.client.0.smithi137.stdout:5/576: stat df/d44/d54/d95/d3b/d49/f77 0 2024-06-15T01:07:05.554 INFO:tasks.workunit.client.0.smithi137.stdout:1/516: write d0/d3/d47/f5d [2318528,46550] 0 2024-06-15T01:07:05.554 INFO:tasks.workunit.client.0.smithi137.stdout:4/468: dwrite dd/d15/d21/d3a/d51/f93 [0,4194304] 0 2024-06-15T01:07:05.557 INFO:tasks.workunit.client.0.smithi137.stdout:7/448: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/f8b x:0 0 0 2024-06-15T01:07:05.558 INFO:tasks.workunit.client.1.smithi142.stdout:8/630: dwrite d0/d17/fb2 [0,4194304] 0 2024-06-15T01:07:05.558 INFO:tasks.workunit.client.0.smithi137.stdout:9/285: dwrite df/d12/d1b/d31/d42/f46 [0,4194304] 0 2024-06-15T01:07:05.559 INFO:tasks.workunit.client.0.smithi137.stdout:0/501: fsync d1/d2/dc/d10/d14/d15/f57 0 2024-06-15T01:07:05.559 INFO:tasks.workunit.client.1.smithi142.stdout:8/631: truncate d0/f87 171253 0 2024-06-15T01:07:05.560 INFO:tasks.workunit.client.0.smithi137.stdout:9/286: write df/d2a/f4c [642558,18312] 0 2024-06-15T01:07:05.561 INFO:tasks.workunit.client.1.smithi142.stdout:6/590: mkdir d1a/d2d/d3b/dab/dce 0 2024-06-15T01:07:05.561 INFO:tasks.workunit.client.0.smithi137.stdout:5/577: fdatasync df/d11/f13 0 2024-06-15T01:07:05.568 INFO:tasks.workunit.client.1.smithi142.stdout:6/591: read d1a/f38 [3214355,76363] 0 2024-06-15T01:07:05.568 INFO:tasks.workunit.client.1.smithi142.stdout:1/636: link d6/d12/da0/db0/c4f d6/d3a/dc4/ce6 0 2024-06-15T01:07:05.570 INFO:tasks.workunit.client.1.smithi142.stdout:1/637: truncate d6/d12/d3c/f9c 1114386 0 2024-06-15T01:07:05.572 INFO:tasks.workunit.client.0.smithi137.stdout:9/287: dwrite df/d2a/f34 [0,4194304] 0 2024-06-15T01:07:05.572 INFO:tasks.workunit.client.0.smithi137.stdout:4/469: creat dd/d15/d36/d45/fb1 x:0 0 0 2024-06-15T01:07:05.572 INFO:tasks.workunit.client.1.smithi142.stdout:7/619: write d2/d6/f9 [4383425,46350] 0 2024-06-15T01:07:05.574 INFO:tasks.workunit.client.1.smithi142.stdout:6/592: mknod d1a/d2d/d68/da0/ccf 0 2024-06-15T01:07:05.574 INFO:tasks.workunit.client.1.smithi142.stdout:7/620: stat d2/d6/d22/d3c/f56 0 2024-06-15T01:07:05.575 INFO:tasks.workunit.client.0.smithi137.stdout:0/502: rmdir d1 39 2024-06-15T01:07:05.584 INFO:tasks.workunit.client.1.smithi142.stdout:8/632: getdents d0/d17/d40/d86/d9d 0 2024-06-15T01:07:05.584 INFO:tasks.workunit.client.1.smithi142.stdout:2/559: link d8/df/d10/c21 d8/df/d10/d1c/d7c/cb1 0 2024-06-15T01:07:05.584 INFO:tasks.workunit.client.1.smithi142.stdout:1/638: dwrite d6/d12/d28/f39 [0,4194304] 0 2024-06-15T01:07:05.585 INFO:tasks.workunit.client.0.smithi137.stdout:9/288: mkdir df/d12/d4e 0 2024-06-15T01:07:05.586 INFO:tasks.workunit.client.1.smithi142.stdout:9/547: dwrite d6/f6a [0,4194304] 0 2024-06-15T01:07:05.588 INFO:tasks.workunit.client.1.smithi142.stdout:7/621: getdents d2/d8/da/d33 0 2024-06-15T01:07:05.589 INFO:tasks.workunit.client.0.smithi137.stdout:4/470: rename dd/d15/d48 to dd/d15/d21/d3a/d51/d83/db2 0 2024-06-15T01:07:05.589 INFO:tasks.workunit.client.1.smithi142.stdout:0/598: dread d3/d39/d3e/d81/d1b/f30 [0,4194304] 0 2024-06-15T01:07:05.591 INFO:tasks.workunit.client.1.smithi142.stdout:0/599: read - d3/d39/d3e/fc5 zero size 2024-06-15T01:07:05.592 INFO:tasks.workunit.client.1.smithi142.stdout:8/633: write d0/d25/d5a/f4d [1080225,79740] 0 2024-06-15T01:07:05.592 INFO:tasks.workunit.client.0.smithi137.stdout:2/371: link d0/dc/l50 d0/de/d10/d3a/l71 0 2024-06-15T01:07:05.593 INFO:tasks.workunit.client.1.smithi142.stdout:2/560: truncate d8/df/f48 305481 0 2024-06-15T01:07:05.594 INFO:tasks.workunit.client.0.smithi137.stdout:2/372: fdatasync d0/de/f52 0 2024-06-15T01:07:05.596 INFO:tasks.workunit.client.0.smithi137.stdout:5/578: symlink df/d44/d54/d95/lc2 0 2024-06-15T01:07:05.597 INFO:tasks.workunit.client.1.smithi142.stdout:0/600: readlink d3/d39/d9a/la6 0 2024-06-15T01:07:05.599 INFO:tasks.workunit.client.1.smithi142.stdout:2/561: write d8/d19/d33/fa2 [564238,45525] 0 2024-06-15T01:07:05.601 INFO:tasks.workunit.client.0.smithi137.stdout:4/471: dwrite dd/d15/d36/d45/d55/d8d/f1c [0,4194304] 0 2024-06-15T01:07:05.601 INFO:tasks.workunit.client.1.smithi142.stdout:2/562: chown d8/df/d10/d94/d25/d1a/f1d 1 1 2024-06-15T01:07:05.601 INFO:tasks.workunit.client.0.smithi137.stdout:5/579: write df/d44/d54/d95/d3b/f59 [1919314,92406] 0 2024-06-15T01:07:05.602 INFO:tasks.workunit.client.1.smithi142.stdout:0/601: creat d3/d39/d3e/d81/d44/fd6 x:0 0 0 2024-06-15T01:07:05.605 INFO:tasks.workunit.client.0.smithi137.stdout:4/472: dread - dd/d15/d36/d45/d55/d8d/d1f/d86/fab zero size 2024-06-15T01:07:05.606 INFO:tasks.workunit.client.1.smithi142.stdout:6/593: dread d1a/d83/fbb [0,4194304] 0 2024-06-15T01:07:05.606 INFO:tasks.workunit.client.1.smithi142.stdout:7/622: link d2/d8/da/d33/d25/d27/c7f d2/d8/da8/cc4 0 2024-06-15T01:07:05.607 INFO:tasks.workunit.client.1.smithi142.stdout:6/594: chown d1a/d2d/d80 10287 1 2024-06-15T01:07:05.607 INFO:tasks.workunit.client.1.smithi142.stdout:2/563: symlink d8/d19/d8d/lb2 0 2024-06-15T01:07:05.608 INFO:tasks.workunit.client.0.smithi137.stdout:5/580: symlink df/d11/d70/lc3 0 2024-06-15T01:07:05.610 INFO:tasks.workunit.client.1.smithi142.stdout:7/623: creat d2/d6/fc5 x:0 0 0 2024-06-15T01:07:05.610 INFO:tasks.workunit.client.1.smithi142.stdout:0/602: dwrite d3/d39/fc4 [0,4194304] 0 2024-06-15T01:07:05.611 INFO:tasks.workunit.client.0.smithi137.stdout:2/373: getdents d0/de/d10 0 2024-06-15T01:07:05.611 INFO:tasks.workunit.client.0.smithi137.stdout:2/374: chown d0/de/d2b/d47/l5e 19999497 1 2024-06-15T01:07:05.612 INFO:tasks.workunit.client.0.smithi137.stdout:5/581: symlink df/d11/d70/dbf/lc4 0 2024-06-15T01:07:05.614 INFO:tasks.workunit.client.0.smithi137.stdout:4/473: dread dd/d15/d36/d45/d55/f7c [0,4194304] 0 2024-06-15T01:07:05.615 INFO:tasks.workunit.client.1.smithi142.stdout:7/624: fdatasync d2/d8/f2d 0 2024-06-15T01:07:05.616 INFO:tasks.workunit.client.1.smithi142.stdout:7/625: dread - d2/d8/dd/dc0/fc3 zero size 2024-06-15T01:07:05.616 INFO:tasks.workunit.client.1.smithi142.stdout:2/564: creat d8/df/d10/d94/d25/d22/d43/d52/fb3 x:0 0 0 2024-06-15T01:07:05.617 INFO:tasks.workunit.client.1.smithi142.stdout:0/603: rmdir d3/d39/d9a/d9d/dcd 39 2024-06-15T01:07:05.618 INFO:tasks.workunit.client.0.smithi137.stdout:2/375: rename d0/d42/d4d to d0/dc/d2e/d72 0 2024-06-15T01:07:05.618 INFO:tasks.workunit.client.1.smithi142.stdout:0/604: dread - d3/d39/d9a/d9d/f9f zero size 2024-06-15T01:07:05.618 INFO:tasks.workunit.client.0.smithi137.stdout:2/376: stat d0/de/f16 0 2024-06-15T01:07:05.619 INFO:tasks.workunit.client.1.smithi142.stdout:2/565: chown d8/df/d10/d94/d25/d22/d43/d52/d7a/cae 0 1 2024-06-15T01:07:05.619 INFO:tasks.workunit.client.0.smithi137.stdout:2/377: write d0/f69 [686360,45765] 0 2024-06-15T01:07:05.620 INFO:tasks.workunit.client.1.smithi142.stdout:7/626: symlink d2/d8/dd/d8a/d94/lc6 0 2024-06-15T01:07:05.622 INFO:tasks.workunit.client.1.smithi142.stdout:0/605: mknod d3/d39/d3e/db4/cd7 0 2024-06-15T01:07:05.623 INFO:tasks.workunit.client.1.smithi142.stdout:7/627: stat d2/d8/da/fae 0 2024-06-15T01:07:05.624 INFO:tasks.workunit.client.0.smithi137.stdout:5/582: getdents df/d44/d54/d80 0 2024-06-15T01:07:05.625 INFO:tasks.workunit.client.1.smithi142.stdout:2/566: unlink d8/df/d10/c21 0 2024-06-15T01:07:05.625 INFO:tasks.workunit.client.0.smithi137.stdout:4/474: link dd/d15/d36/d45/d8f/dac/lad dd/d15/d21/d3a/d6e/d7b/d96/lb3 0 2024-06-15T01:07:05.627 INFO:tasks.workunit.client.1.smithi142.stdout:3/679: sync 2024-06-15T01:07:05.627 INFO:tasks.workunit.client.1.smithi142.stdout:5/557: sync 2024-06-15T01:07:05.628 INFO:tasks.workunit.client.0.smithi137.stdout:5/583: creat df/d76/d7c/d9c/da1/fc5 x:0 0 0 2024-06-15T01:07:05.628 INFO:tasks.workunit.client.1.smithi142.stdout:0/606: mknod d3/d39/d3e/d81/d1e/cd8 0 2024-06-15T01:07:05.631 INFO:tasks.workunit.client.0.smithi137.stdout:2/378: dwrite d0/f69 [0,4194304] 0 2024-06-15T01:07:05.632 INFO:tasks.workunit.client.1.smithi142.stdout:7/628: read d2/d6/d22/d3c/f56 [3931096,69737] 0 2024-06-15T01:07:05.633 INFO:tasks.workunit.client.1.smithi142.stdout:2/567: symlink d8/df/daf/lb4 0 2024-06-15T01:07:05.636 INFO:tasks.workunit.client.1.smithi142.stdout:5/558: mknod d2/de/d19/caf 0 2024-06-15T01:07:05.636 INFO:tasks.workunit.client.1.smithi142.stdout:1/639: dread d6/d12/d3c/d63/f85 [0,4194304] 0 2024-06-15T01:07:05.637 INFO:tasks.workunit.client.0.smithi137.stdout:3/550: truncate d0/d2/d22/d63/d76/f7d 3488411 0 2024-06-15T01:07:05.647 INFO:tasks.workunit.client.0.smithi137.stdout:4/475: dwrite dd/d15/d36/d45/d55/f95 [0,4194304] 0 2024-06-15T01:07:05.648 INFO:tasks.workunit.client.0.smithi137.stdout:0/503: dread d1/d2/f6 [0,4194304] 0 2024-06-15T01:07:05.648 INFO:tasks.workunit.client.0.smithi137.stdout:1/517: truncate d0/d2d/f62 3161042 0 2024-06-15T01:07:05.648 INFO:tasks.workunit.client.1.smithi142.stdout:9/548: dread d6/dc/d1b/d25/d35/f5c [0,4194304] 0 2024-06-15T01:07:05.648 INFO:tasks.workunit.client.1.smithi142.stdout:1/640: chown d6/d3a/fe3 41175471 1 2024-06-15T01:07:05.648 INFO:tasks.workunit.client.1.smithi142.stdout:2/568: chown d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 31297145 1 2024-06-15T01:07:05.649 INFO:tasks.workunit.client.1.smithi142.stdout:7/629: unlink d2/d6/d22/d47/f5a 0 2024-06-15T01:07:05.649 INFO:tasks.workunit.client.1.smithi142.stdout:3/680: dwrite d0/d1f/d20/d59/d61/fc0 [0,4194304] 0 2024-06-15T01:07:05.653 INFO:tasks.workunit.client.0.smithi137.stdout:5/584: dread df/d11/f31 [0,4194304] 0 2024-06-15T01:07:05.654 INFO:tasks.workunit.client.0.smithi137.stdout:1/518: mkdir d0/d3/d14/dba 0 2024-06-15T01:07:05.654 INFO:tasks.workunit.client.1.smithi142.stdout:9/549: readlink d6/dc/l29 0 2024-06-15T01:07:05.655 INFO:tasks.workunit.client.0.smithi137.stdout:4/476: mknod dd/d15/d30/cb4 0 2024-06-15T01:07:05.656 INFO:tasks.workunit.client.1.smithi142.stdout:5/559: truncate d2/de/f66 80285 0 2024-06-15T01:07:05.658 INFO:tasks.workunit.client.0.smithi137.stdout:4/477: write dd/d15/d30/f6b [3538033,74841] 0 2024-06-15T01:07:05.660 INFO:tasks.workunit.client.0.smithi137.stdout:9/289: truncate df/d12/d29/f44 3775300 0 2024-06-15T01:07:05.660 INFO:tasks.workunit.client.1.smithi142.stdout:9/550: mknod d6/dc/d51/d53/d8d/cd5 0 2024-06-15T01:07:05.661 INFO:tasks.workunit.client.1.smithi142.stdout:3/681: write d0/d3/da/d71/d55/f57 [4627909,85855] 0 2024-06-15T01:07:05.661 INFO:tasks.workunit.client.1.smithi142.stdout:2/569: mkdir d8/df/d10/d94/db5 0 2024-06-15T01:07:05.662 INFO:tasks.workunit.client.0.smithi137.stdout:1/519: mknod d0/d3/d1b/cbb 0 2024-06-15T01:07:05.662 INFO:tasks.workunit.client.0.smithi137.stdout:0/504: getdents d1/d2/d13 0 2024-06-15T01:07:05.663 INFO:tasks.workunit.client.1.smithi142.stdout:6/595: dread d1a/f1e [0,4194304] 0 2024-06-15T01:07:05.663 INFO:tasks.workunit.client.1.smithi142.stdout:1/641: dwrite d6/d12/d80/d9b/fcb [0,4194304] 0 2024-06-15T01:07:05.664 INFO:tasks.workunit.client.0.smithi137.stdout:4/478: symlink dd/d15/d36/lb5 0 2024-06-15T01:07:05.665 INFO:tasks.workunit.client.1.smithi142.stdout:5/560: creat d2/de/fb0 x:0 0 0 2024-06-15T01:07:05.665 INFO:tasks.workunit.client.1.smithi142.stdout:7/630: dwrite d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:05.670 INFO:tasks.workunit.client.1.smithi142.stdout:2/570: mkdir d8/df/d10/d94/d25/d1a/d57/db6 0 2024-06-15T01:07:05.671 INFO:tasks.workunit.client.0.smithi137.stdout:9/290: creat df/d49/d4b/f4f x:0 0 0 2024-06-15T01:07:05.671 INFO:tasks.workunit.client.0.smithi137.stdout:3/551: dread d0/d2/d1e/f6e [0,4194304] 0 2024-06-15T01:07:05.673 INFO:tasks.workunit.client.1.smithi142.stdout:1/642: creat d6/d12/d80/dc8/fe7 x:0 0 0 2024-06-15T01:07:05.674 INFO:tasks.workunit.client.0.smithi137.stdout:4/479: rmdir dd/d15/d21/d3a/d51/d5b 39 2024-06-15T01:07:05.675 INFO:tasks.workunit.client.0.smithi137.stdout:1/520: fdatasync d0/d2d/f62 0 2024-06-15T01:07:05.675 INFO:tasks.workunit.client.1.smithi142.stdout:6/596: rename d1a/d2d/f32 to d1a/d2d/d68/da0/fd0 0 2024-06-15T01:07:05.676 INFO:tasks.workunit.client.0.smithi137.stdout:4/480: write dd/f14 [3891812,83478] 0 2024-06-15T01:07:05.676 INFO:tasks.workunit.client.1.smithi142.stdout:9/551: link d6/dc/f8e d6/dc/d1b/d25/d58/fd6 0 2024-06-15T01:07:05.676 INFO:tasks.workunit.client.1.smithi142.stdout:7/631: rmdir d2/d6/d22 39 2024-06-15T01:07:05.677 INFO:tasks.workunit.client.0.smithi137.stdout:1/521: fsync d0/d3/d47/f5d 0 2024-06-15T01:07:05.678 INFO:tasks.workunit.client.1.smithi142.stdout:0/607: dread d3/d39/d3e/d81/d4c/faf [0,4194304] 0 2024-06-15T01:07:05.680 INFO:tasks.workunit.client.1.smithi142.stdout:6/597: read d1a/d2d/f33 [405024,118341] 0 2024-06-15T01:07:05.680 INFO:tasks.workunit.client.0.smithi137.stdout:9/291: mknod df/d12/d1b/c50 0 2024-06-15T01:07:05.680 INFO:tasks.workunit.client.1.smithi142.stdout:3/682: getdents d0/d3/da/d71/d75 0 2024-06-15T01:07:05.681 INFO:tasks.workunit.client.1.smithi142.stdout:5/561: dwrite d2/de/f1f [4194304,4194304] 0 2024-06-15T01:07:05.684 INFO:tasks.workunit.client.0.smithi137.stdout:4/481: creat dd/d15/d36/d45/d55/d8d/d1f/fb6 x:0 0 0 2024-06-15T01:07:05.684 INFO:tasks.workunit.client.1.smithi142.stdout:9/552: write d6/dc/d51/d53/f9f [1182344,18927] 0 2024-06-15T01:07:05.684 INFO:tasks.workunit.client.1.smithi142.stdout:0/608: truncate d3/d39/d3e/d81/d4c/d61/f66 162310 0 2024-06-15T01:07:05.685 INFO:tasks.workunit.client.0.smithi137.stdout:1/522: creat d0/d3/d89/fbc x:0 0 0 2024-06-15T01:07:05.685 INFO:tasks.workunit.client.1.smithi142.stdout:6/598: creat d1a/d39/d3a/d51/d62/fd1 x:0 0 0 2024-06-15T01:07:05.686 INFO:tasks.workunit.client.1.smithi142.stdout:3/683: creat d0/d1f/d20/d59/d61/d96/fcc x:0 0 0 2024-06-15T01:07:05.686 INFO:tasks.workunit.client.0.smithi137.stdout:9/292: truncate df/d12/d15/d16/f27 703217 0 2024-06-15T01:07:05.688 INFO:tasks.workunit.client.1.smithi142.stdout:6/599: chown d1a/d2d/d2e/f31 293446 1 2024-06-15T01:07:05.689 INFO:tasks.workunit.client.0.smithi137.stdout:3/552: rename d0/d2/d22/d24/d5e/f57 to d0/d2/d22/d24/d28/fb4 0 2024-06-15T01:07:05.689 INFO:tasks.workunit.client.0.smithi137.stdout:9/293: truncate df/d12/d1b/f35 1281528 0 2024-06-15T01:07:05.689 INFO:tasks.workunit.client.1.smithi142.stdout:3/684: chown d0/d3/da/c45 4098041 1 2024-06-15T01:07:05.690 INFO:tasks.workunit.client.0.smithi137.stdout:1/523: dread - d0/d3/d1b/f50 zero size 2024-06-15T01:07:05.690 INFO:tasks.workunit.client.1.smithi142.stdout:9/553: creat d6/dc/d4e/d65/fd7 x:0 0 0 2024-06-15T01:07:05.691 INFO:tasks.workunit.client.1.smithi142.stdout:3/685: readlink d0/d1b/l50 0 2024-06-15T01:07:05.692 INFO:tasks.workunit.client.1.smithi142.stdout:7/632: dwrite d2/d8/da/d33/d25/f67 [0,4194304] 0 2024-06-15T01:07:05.694 INFO:tasks.workunit.client.1.smithi142.stdout:4/692: sync 2024-06-15T01:07:05.694 INFO:tasks.workunit.client.0.smithi137.stdout:3/553: symlink d0/d42/d79/lb5 0 2024-06-15T01:07:05.696 INFO:tasks.workunit.client.1.smithi142.stdout:2/571: dread d8/fe [0,4194304] 0 2024-06-15T01:07:05.696 INFO:tasks.workunit.client.1.smithi142.stdout:9/554: chown d6/dc/d51/d53/d8d/cb6 15924560 1 2024-06-15T01:07:05.696 INFO:tasks.workunit.client.1.smithi142.stdout:5/562: dwrite d2/de/d19/f51 [0,4194304] 0 2024-06-15T01:07:05.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/554: readlink d0/d2/d22/d63/l80 0 2024-06-15T01:07:05.703 INFO:tasks.workunit.client.0.smithi137.stdout:1/524: dwrite d0/d2d/f79 [0,4194304] 0 2024-06-15T01:07:05.705 INFO:tasks.workunit.client.0.smithi137.stdout:9/294: dwrite df/d12/f17 [8388608,4194304] 0 2024-06-15T01:07:05.706 INFO:tasks.workunit.client.1.smithi142.stdout:0/609: dwrite d3/d39/d3e/d81/d44/d7c/fbd [0,4194304] 0 2024-06-15T01:07:05.707 INFO:tasks.workunit.client.0.smithi137.stdout:4/482: link cc dd/d15/d21/d3a/d51/d83/db2/d72/d81/cb7 0 2024-06-15T01:07:05.709 INFO:tasks.workunit.client.1.smithi142.stdout:3/686: dwrite d0/d3/f43 [0,4194304] 0 2024-06-15T01:07:05.711 INFO:tasks.workunit.client.1.smithi142.stdout:3/687: fsync d0/d3/da/d71/d55/f94 0 2024-06-15T01:07:05.711 INFO:tasks.workunit.client.1.smithi142.stdout:7/633: creat d2/d8/dd/d14/d2c/d37/fc7 x:0 0 0 2024-06-15T01:07:05.712 INFO:tasks.workunit.client.1.smithi142.stdout:3/688: chown d0/d1f/d20/d5d/d60 2 1 2024-06-15T01:07:05.713 INFO:tasks.workunit.client.0.smithi137.stdout:2/379: dwrite d0/dc/d22/d23/f6a [0,4194304] 0 2024-06-15T01:07:05.713 INFO:tasks.workunit.client.0.smithi137.stdout:3/555: rmdir d0/d2/d22/d24/d28/d2e 39 2024-06-15T01:07:05.716 INFO:tasks.workunit.client.0.smithi137.stdout:1/525: mknod d0/d1f/d60/cbd 0 2024-06-15T01:07:05.717 INFO:tasks.workunit.client.0.smithi137.stdout:0/505: write d1/d2/dc/d10/d14/d15/f4b [4779228,36664] 0 2024-06-15T01:07:05.718 INFO:tasks.workunit.client.1.smithi142.stdout:5/563: symlink d2/d3e/d65/lb1 0 2024-06-15T01:07:05.719 INFO:tasks.workunit.client.0.smithi137.stdout:9/295: rmdir df/d12/d15/d16 39 2024-06-15T01:07:05.720 INFO:tasks.workunit.client.1.smithi142.stdout:5/564: dread - d2/d3e/d31/f64 zero size 2024-06-15T01:07:05.722 INFO:tasks.workunit.client.1.smithi142.stdout:3/689: dwrite d0/d1f/d20/d59/fa6 [0,4194304] 0 2024-06-15T01:07:05.729 INFO:tasks.workunit.client.1.smithi142.stdout:5/565: dread d2/de/d27/f9e [0,4194304] 0 2024-06-15T01:07:05.731 INFO:tasks.workunit.client.1.smithi142.stdout:5/566: write d2/de/d19/d5f/d68/f73 [4157242,80170] 0 2024-06-15T01:07:05.733 INFO:tasks.workunit.client.1.smithi142.stdout:5/567: dread d2/de/d27/f9e [0,4194304] 0 2024-06-15T01:07:05.760 INFO:tasks.workunit.client.0.smithi137.stdout:1/526: read d0/d3/d14/d34/f45 [417238,76089] 0 2024-06-15T01:07:05.764 INFO:tasks.workunit.client.0.smithi137.stdout:1/527: dread - d0/d3/d47/f92 zero size 2024-06-15T01:07:05.764 INFO:tasks.workunit.client.0.smithi137.stdout:3/556: mknod d0/d13/d37/d83/cb6 0 2024-06-15T01:07:05.764 INFO:tasks.workunit.client.0.smithi137.stdout:2/380: rmdir d0/d42/d49/d57 39 2024-06-15T01:07:05.772 INFO:tasks.workunit.client.0.smithi137.stdout:2/381: dwrite d0/dc/f65 [0,4194304] 0 2024-06-15T01:07:05.774 INFO:tasks.workunit.client.1.smithi142.stdout:6/600: getdents d1a/d39 0 2024-06-15T01:07:05.774 INFO:tasks.workunit.client.0.smithi137.stdout:4/483: mkdir dd/d15/d21/d3a/d51/d83/db2/db8 0 2024-06-15T01:07:05.777 INFO:tasks.workunit.client.1.smithi142.stdout:2/572: unlink d8/df/d10/d94/d25/da9/l4e 0 2024-06-15T01:07:05.778 INFO:tasks.workunit.client.1.smithi142.stdout:0/610: chown d3/d39/d3e/d81/d1e/c3b 92628 1 2024-06-15T01:07:05.780 INFO:tasks.workunit.client.0.smithi137.stdout:0/506: rename d1/d2/dc/d10/d14/d15/d17/f4c to d1/d2/d55/faf 0 2024-06-15T01:07:05.782 INFO:tasks.workunit.client.0.smithi137.stdout:4/484: dread dd/d15/d36/d45/d55/d8d/f1c [0,4194304] 0 2024-06-15T01:07:05.783 INFO:tasks.workunit.client.1.smithi142.stdout:7/634: mknod d2/d8/dd/d14/d2c/d37/d6b/d80/d85/cc8 0 2024-06-15T01:07:05.783 INFO:tasks.workunit.client.0.smithi137.stdout:1/528: mknod d0/d1f/cbe 0 2024-06-15T01:07:05.783 INFO:tasks.workunit.client.1.smithi142.stdout:7/635: chown d2/d8/dd/d14/d2c/d37/d6b/d80/c3d 1810738 1 2024-06-15T01:07:05.784 INFO:tasks.workunit.client.0.smithi137.stdout:1/529: chown d0/f2 43354 1 2024-06-15T01:07:05.784 INFO:tasks.workunit.client.1.smithi142.stdout:9/555: rename d6/dc/d4e/d65 to d6/dc/d4e/dae/dd8 0 2024-06-15T01:07:05.785 INFO:tasks.workunit.client.1.smithi142.stdout:9/556: write d6/d34/f3a [5031518,124901] 0 2024-06-15T01:07:05.787 INFO:tasks.workunit.client.0.smithi137.stdout:2/382: chown d0/c28 46243310 1 2024-06-15T01:07:05.790 INFO:tasks.workunit.client.1.smithi142.stdout:2/573: mknod d8/d19/d33/cb7 0 2024-06-15T01:07:05.791 INFO:tasks.workunit.client.0.smithi137.stdout:4/485: dread dd/d15/f42 [0,4194304] 0 2024-06-15T01:07:05.791 INFO:tasks.workunit.client.1.smithi142.stdout:0/611: rmdir d3/d39/d3e/d81/d1b/d38/da3 39 2024-06-15T01:07:05.792 INFO:tasks.workunit.client.0.smithi137.stdout:1/530: creat d0/d3/d89/fbf x:0 0 0 2024-06-15T01:07:05.793 INFO:tasks.workunit.client.1.smithi142.stdout:7/636: fdatasync d2/d6/f1c 0 2024-06-15T01:07:05.793 INFO:tasks.workunit.client.0.smithi137.stdout:4/486: dread - dd/d15/d21/d3a/d51/d83/db2/d67/f7d zero size 2024-06-15T01:07:05.794 INFO:tasks.workunit.client.0.smithi137.stdout:1/531: dread - d0/d3/d47/d73/fb9 zero size 2024-06-15T01:07:05.794 INFO:tasks.workunit.client.1.smithi142.stdout:2/574: truncate d8/df/d10/d94/d25/d22/d43/f96 530396 0 2024-06-15T01:07:05.794 INFO:tasks.workunit.client.1.smithi142.stdout:2/575: readlink d8/df/daf/lb4 0 2024-06-15T01:07:05.795 INFO:tasks.workunit.client.0.smithi137.stdout:4/487: dread - dd/d15/d21/d3a/f73 zero size 2024-06-15T01:07:05.795 INFO:tasks.workunit.client.0.smithi137.stdout:1/532: chown d0/d1f/d60/l6a 214 1 2024-06-15T01:07:05.796 INFO:tasks.workunit.client.1.smithi142.stdout:8/634: sync 2024-06-15T01:07:05.796 INFO:tasks.workunit.client.0.smithi137.stdout:2/383: mkdir d0/da/d73 0 2024-06-15T01:07:05.798 INFO:tasks.workunit.client.0.smithi137.stdout:2/384: read d0/dc/d22/f4b [734555,4094] 0 2024-06-15T01:07:05.800 INFO:tasks.workunit.client.0.smithi137.stdout:0/507: rename d1/d2/dc/d10/d14/d15/d1c/c3c to d1/d2/d55/cb0 0 2024-06-15T01:07:05.801 INFO:tasks.workunit.client.0.smithi137.stdout:4/488: write dd/d15/f42 [1861672,31369] 0 2024-06-15T01:07:05.802 INFO:tasks.workunit.client.1.smithi142.stdout:5/568: dread d2/d18/f61 [0,4194304] 0 2024-06-15T01:07:05.802 INFO:tasks.workunit.client.1.smithi142.stdout:3/690: dread d0/d3/da/d71/fab [4194304,4194304] 0 2024-06-15T01:07:05.803 INFO:tasks.workunit.client.1.smithi142.stdout:5/569: fdatasync d2/de/d19/f6a 0 2024-06-15T01:07:05.806 INFO:tasks.workunit.client.1.smithi142.stdout:9/557: dread d6/dc/dd/d22/fbe [0,4194304] 0 2024-06-15T01:07:05.807 INFO:tasks.workunit.client.1.smithi142.stdout:9/558: fsync d6/dc/dd/d22/f54 0 2024-06-15T01:07:05.808 INFO:tasks.workunit.client.0.smithi137.stdout:3/557: getdents d0 0 2024-06-15T01:07:05.808 INFO:tasks.workunit.client.0.smithi137.stdout:9/296: write df/d13/f22 [700826,75733] 0 2024-06-15T01:07:05.809 INFO:tasks.workunit.client.1.smithi142.stdout:2/576: unlink d8/d19/c60 0 2024-06-15T01:07:05.809 INFO:tasks.workunit.client.0.smithi137.stdout:2/385: mknod d0/de/d2b/c74 0 2024-06-15T01:07:05.812 INFO:tasks.workunit.client.1.smithi142.stdout:7/637: dread d2/d8/da/fae [0,4194304] 0 2024-06-15T01:07:05.813 INFO:tasks.workunit.client.1.smithi142.stdout:3/691: read d0/d3/f3d [3083199,115149] 0 2024-06-15T01:07:05.816 INFO:tasks.workunit.client.1.smithi142.stdout:5/570: mkdir d2/de/d27/db2 0 2024-06-15T01:07:05.818 INFO:tasks.workunit.client.1.smithi142.stdout:0/612: creat d3/d39/d3e/d81/d1e/d98/fd9 x:0 0 0 2024-06-15T01:07:05.819 INFO:tasks.workunit.client.1.smithi142.stdout:0/613: write d3/d39/d3e/d81/d1e/f63 [463816,124447] 0 2024-06-15T01:07:05.821 INFO:tasks.workunit.client.1.smithi142.stdout:9/559: unlink d6/dc/dd/d22/l90 0 2024-06-15T01:07:05.822 INFO:tasks.workunit.client.1.smithi142.stdout:1/643: sync 2024-06-15T01:07:05.823 INFO:tasks.workunit.client.1.smithi142.stdout:4/693: sync 2024-06-15T01:07:05.823 INFO:tasks.workunit.client.1.smithi142.stdout:1/644: chown d6/d4d/c7c 15 1 2024-06-15T01:07:05.823 INFO:tasks.workunit.client.1.smithi142.stdout:6/601: truncate d1a/d39/d42/d4c/d58/f7f 1197369 0 2024-06-15T01:07:05.824 INFO:tasks.workunit.client.1.smithi142.stdout:4/694: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/f75 14335 1 2024-06-15T01:07:05.825 INFO:tasks.workunit.client.0.smithi137.stdout:1/533: dwrite d0/d3/d14/d34/f45 [0,4194304] 0 2024-06-15T01:07:05.826 INFO:tasks.workunit.client.0.smithi137.stdout:1/534: fdatasync d0/d3/d14/d34/f70 0 2024-06-15T01:07:05.827 INFO:tasks.workunit.client.0.smithi137.stdout:1/535: fsync d0/d3/d89/fbc 0 2024-06-15T01:07:05.828 INFO:tasks.workunit.client.1.smithi142.stdout:8/635: write d0/d17/d40/f4a [23021,130987] 0 2024-06-15T01:07:05.828 INFO:tasks.workunit.client.1.smithi142.stdout:8/636: readlink d0/d25/d53/da7/lb6 0 2024-06-15T01:07:05.829 INFO:tasks.workunit.client.0.smithi137.stdout:4/489: dread dd/d15/d36/d45/d55/d8d/d1f/f56 [0,4194304] 0 2024-06-15T01:07:05.829 INFO:tasks.workunit.client.1.smithi142.stdout:8/637: readlink d0/d25/d53/l9a 0 2024-06-15T01:07:05.831 INFO:tasks.workunit.client.1.smithi142.stdout:2/577: symlink d8/df/d10/d94/d25/d22/lb8 0 2024-06-15T01:07:05.833 INFO:tasks.workunit.client.1.smithi142.stdout:6/602: mkdir d1a/d39/d3a/dd2 0 2024-06-15T01:07:05.834 INFO:tasks.workunit.client.1.smithi142.stdout:7/638: symlink d2/db6/lc9 0 2024-06-15T01:07:05.834 INFO:tasks.workunit.client.0.smithi137.stdout:2/386: rmdir d0/dc 39 2024-06-15T01:07:05.837 INFO:tasks.workunit.client.1.smithi142.stdout:4/695: read d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f60 [45106,50449] 0 2024-06-15T01:07:05.837 INFO:tasks.workunit.client.0.smithi137.stdout:0/508: rename d1/d2/l8c to d1/d2/dc/d10/lb1 0 2024-06-15T01:07:05.838 INFO:tasks.workunit.client.1.smithi142.stdout:8/638: mknod d0/d25/d5a/d8f/cc5 0 2024-06-15T01:07:05.838 INFO:tasks.workunit.client.1.smithi142.stdout:3/692: dread d0/d3/d15/f2d [0,4194304] 0 2024-06-15T01:07:05.839 INFO:tasks.workunit.client.0.smithi137.stdout:0/509: write d1/f79 [146056,14594] 0 2024-06-15T01:07:05.842 INFO:tasks.workunit.client.1.smithi142.stdout:9/560: creat d6/dc/fd9 x:0 0 0 2024-06-15T01:07:05.843 INFO:tasks.workunit.client.1.smithi142.stdout:2/578: dwrite d8/df/d10/d94/d25/f27 [8388608,4194304] 0 2024-06-15T01:07:05.843 INFO:tasks.workunit.client.0.smithi137.stdout:4/490: mknod dd/d15/d36/d45/d55/cb9 0 2024-06-15T01:07:05.844 INFO:tasks.workunit.client.1.smithi142.stdout:7/639: mknod d2/d8/da/d33/cca 0 2024-06-15T01:07:05.845 INFO:tasks.workunit.client.1.smithi142.stdout:3/693: write d0/d1f/d20/d59/d61/d96/fcc [841705,83343] 0 2024-06-15T01:07:05.849 INFO:tasks.workunit.client.1.smithi142.stdout:1/645: creat d6/d12/da0/db0/fe8 x:0 0 0 2024-06-15T01:07:05.851 INFO:tasks.workunit.client.1.smithi142.stdout:3/694: chown d0/d1f/d20/d59/d61/d96 1460 1 2024-06-15T01:07:05.852 INFO:tasks.workunit.client.1.smithi142.stdout:0/614: dwrite d3/d39/d3e/d81/d44/d7c/d22/d59/f9c [0,4194304] 0 2024-06-15T01:07:05.852 INFO:tasks.workunit.client.1.smithi142.stdout:2/579: chown d8/df/d10/d94/d25/d1a/d63/f95 58117 1 2024-06-15T01:07:05.852 INFO:tasks.workunit.client.0.smithi137.stdout:9/297: mkdir df/d12/d15/d16/d3c/d51 0 2024-06-15T01:07:05.854 INFO:tasks.workunit.client.0.smithi137.stdout:2/387: dwrite d0/dc/f3b [0,4194304] 0 2024-06-15T01:07:05.857 INFO:tasks.workunit.client.1.smithi142.stdout:3/695: chown d0/d1f/d20/d5d/d82/d6e 1633 1 2024-06-15T01:07:05.859 INFO:tasks.workunit.client.1.smithi142.stdout:9/561: dwrite d6/dc/f19 [0,4194304] 0 2024-06-15T01:07:05.862 INFO:tasks.workunit.client.0.smithi137.stdout:4/491: dwrite dd/d15/d21/d3a/d51/d83/db2/f61 [0,4194304] 0 2024-06-15T01:07:05.862 INFO:tasks.workunit.client.0.smithi137.stdout:0/510: dwrite d1/d2/dc/d10/d14/d56/f85 [0,4194304] 0 2024-06-15T01:07:05.863 INFO:tasks.workunit.client.0.smithi137.stdout:4/492: stat dd/d15/d21/d3a/d51/d83/db2/c94 0 2024-06-15T01:07:05.866 INFO:tasks.workunit.client.1.smithi142.stdout:5/571: truncate d2/d3e/d31/f3c 44456 0 2024-06-15T01:07:05.869 INFO:tasks.workunit.client.1.smithi142.stdout:7/640: dwrite d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:05.869 INFO:tasks.workunit.client.0.smithi137.stdout:2/388: link d0/dc/d22/f39 d0/dc/d2e/f75 0 2024-06-15T01:07:05.870 INFO:tasks.workunit.client.1.smithi142.stdout:8/639: dwrite d0/d17/d40/f65 [0,4194304] 0 2024-06-15T01:07:05.871 INFO:tasks.workunit.client.1.smithi142.stdout:4/696: symlink d5/d1c/d6f/dc7/d30/lf2 0 2024-06-15T01:07:05.873 INFO:tasks.workunit.client.1.smithi142.stdout:8/640: fsync d0/f87 0 2024-06-15T01:07:05.873 INFO:tasks.workunit.client.0.smithi137.stdout:4/493: truncate f8 3219281 0 2024-06-15T01:07:05.875 INFO:tasks.workunit.client.0.smithi137.stdout:0/511: dwrite d1/d2/dc/d10/f75 [0,4194304] 0 2024-06-15T01:07:05.876 INFO:tasks.workunit.client.0.smithi137.stdout:0/512: chown l0 737256 1 2024-06-15T01:07:05.876 INFO:tasks.workunit.client.1.smithi142.stdout:6/603: rename l15 to d1a/d39/d42/d4c/ld3 0 2024-06-15T01:07:05.877 INFO:tasks.workunit.client.0.smithi137.stdout:2/389: creat d0/dc/d22/d23/f76 x:0 0 0 2024-06-15T01:07:05.880 INFO:tasks.workunit.client.1.smithi142.stdout:9/562: chown d6/dc/d1b/d25/d58/fd6 119314 1 2024-06-15T01:07:05.880 INFO:tasks.workunit.client.1.smithi142.stdout:0/615: dread d3/d39/d3e/d81/d44/d7c/d22/d59/f9c [0,4194304] 0 2024-06-15T01:07:05.880 INFO:tasks.workunit.client.1.smithi142.stdout:7/641: rename d2/d8/dd/d14/d2c/d37 to d2/d8/dd/d14/d2c/d37/d6b/d88/dcb 22 2024-06-15T01:07:05.880 INFO:tasks.workunit.client.0.smithi137.stdout:4/494: mknod dd/d15/d36/d45/d8f/dac/cba 0 2024-06-15T01:07:05.881 INFO:tasks.workunit.client.0.smithi137.stdout:0/513: fdatasync d1/d2/dc/d10/d53/da2/fa5 0 2024-06-15T01:07:05.882 INFO:tasks.workunit.client.0.smithi137.stdout:9/298: link df/f4d df/d12/d15/f52 0 2024-06-15T01:07:05.883 INFO:tasks.workunit.client.1.smithi142.stdout:8/641: chown d0/d25/d5a/c4e 3 1 2024-06-15T01:07:05.883 INFO:tasks.workunit.client.1.smithi142.stdout:2/580: dwrite d8/df/d10/d1c/f2c [0,4194304] 0 2024-06-15T01:07:05.884 INFO:tasks.workunit.client.1.smithi142.stdout:7/642: write d2/d8/da8/faa [258880,32711] 0 2024-06-15T01:07:05.885 INFO:tasks.workunit.client.1.smithi142.stdout:0/616: creat d3/d39/d3e/db4/fda x:0 0 0 2024-06-15T01:07:05.886 INFO:tasks.workunit.client.1.smithi142.stdout:5/572: mknod d2/d18/d2e/dae/cb3 0 2024-06-15T01:07:05.887 INFO:tasks.workunit.client.1.smithi142.stdout:6/604: creat d1a/d2d/d80/d8e/fd4 x:0 0 0 2024-06-15T01:07:05.890 INFO:tasks.workunit.client.0.smithi137.stdout:2/390: dwrite d0/da/f18 [0,4194304] 0 2024-06-15T01:07:05.890 INFO:tasks.workunit.client.1.smithi142.stdout:6/605: readlink d1a/d2d/l96 0 2024-06-15T01:07:05.892 INFO:tasks.workunit.client.0.smithi137.stdout:2/391: chown d0/de/d10/d3a 2088 1 2024-06-15T01:07:05.893 INFO:tasks.workunit.client.1.smithi142.stdout:9/563: read d6/dc/d82/fd4 [292304,130636] 0 2024-06-15T01:07:05.895 INFO:tasks.workunit.client.0.smithi137.stdout:2/392: mkdir d0/dc/d22/d23/d77 0 2024-06-15T01:07:05.896 INFO:tasks.workunit.client.0.smithi137.stdout:4/495: link dd/d15/d21/c58 dd/d15/d36/d45/d55/cbb 0 2024-06-15T01:07:05.896 INFO:tasks.workunit.client.1.smithi142.stdout:8/642: creat d0/d17/d40/d86/fc6 x:0 0 0 2024-06-15T01:07:05.896 INFO:tasks.workunit.client.1.smithi142.stdout:0/617: symlink d3/d39/d3e/d81/d4c/da0/ldb 0 2024-06-15T01:07:05.896 INFO:tasks.workunit.client.0.smithi137.stdout:4/496: read - dd/d15/d36/d45/fb1 zero size 2024-06-15T01:07:05.897 INFO:tasks.workunit.client.1.smithi142.stdout:2/581: creat d8/df/daf/fb9 x:0 0 0 2024-06-15T01:07:05.899 INFO:tasks.workunit.client.0.smithi137.stdout:9/299: link df/d1a/c20 df/d12/c53 0 2024-06-15T01:07:05.901 INFO:tasks.workunit.client.0.smithi137.stdout:0/514: link d1/d2/dc/d10/d14/d15/d17/lab d1/d2/d13/d66/lb2 0 2024-06-15T01:07:05.902 INFO:tasks.workunit.client.0.smithi137.stdout:4/497: symlink dd/d15/d21/d9c/d75/d99/lbc 0 2024-06-15T01:07:05.903 INFO:tasks.workunit.client.0.smithi137.stdout:2/393: dread d0/dc/d2e/d72/d56/f4f [0,4194304] 0 2024-06-15T01:07:05.904 INFO:tasks.workunit.client.0.smithi137.stdout:4/498: chown dd/d15/d36 64733 1 2024-06-15T01:07:05.906 INFO:tasks.workunit.client.1.smithi142.stdout:6/606: dwrite d1a/d39/fbe [0,4194304] 0 2024-06-15T01:07:05.906 INFO:tasks.workunit.client.1.smithi142.stdout:8/643: truncate d0/d17/d40/d86/fa1 823477 0 2024-06-15T01:07:05.908 INFO:tasks.workunit.client.0.smithi137.stdout:3/558: truncate d0/d56/f6a 1941320 0 2024-06-15T01:07:05.909 INFO:tasks.workunit.client.1.smithi142.stdout:6/607: dread - d1a/d39/d42/d4c/fb3 zero size 2024-06-15T01:07:05.909 INFO:tasks.workunit.client.1.smithi142.stdout:8/644: dread - d0/d25/d5a/d8f/d98/da4/fa6 zero size 2024-06-15T01:07:05.909 INFO:tasks.workunit.client.1.smithi142.stdout:0/618: dread d3/d39/d3e/d81/d44/d7c/fbd [0,4194304] 0 2024-06-15T01:07:05.909 INFO:tasks.workunit.client.0.smithi137.stdout:9/300: dread df/d12/d29/f44 [0,4194304] 0 2024-06-15T01:07:05.910 INFO:tasks.workunit.client.0.smithi137.stdout:4/499: creat dd/d15/d21/d9c/d75/d99/fbd x:0 0 0 2024-06-15T01:07:05.911 INFO:tasks.workunit.client.0.smithi137.stdout:0/515: symlink d1/d2/dc/d6c/d88/d91/lb3 0 2024-06-15T01:07:05.912 INFO:tasks.workunit.client.1.smithi142.stdout:0/619: mknod d3/d39/d3e/d81/d1e/cdc 0 2024-06-15T01:07:05.927 INFO:tasks.workunit.client.1.smithi142.stdout:7/643: dread d2/d8/da8/faa [0,4194304] 0 2024-06-15T01:07:05.935 INFO:tasks.workunit.client.1.smithi142.stdout:2/582: dread d8/df/d10/d94/d25/d22/d43/f46 [0,4194304] 0 2024-06-15T01:07:05.939 INFO:tasks.workunit.client.1.smithi142.stdout:6/608: dread d1a/d39/d3a/f84 [0,4194304] 0 2024-06-15T01:07:05.940 INFO:tasks.workunit.client.1.smithi142.stdout:3/696: sync 2024-06-15T01:07:05.940 INFO:tasks.workunit.client.1.smithi142.stdout:1/646: sync 2024-06-15T01:07:05.945 INFO:tasks.workunit.client.0.smithi137.stdout:2/394: symlink d0/da/d73/l78 0 2024-06-15T01:07:05.945 INFO:tasks.workunit.client.0.smithi137.stdout:3/559: dwrite d0/d2f/faa [0,4194304] 0 2024-06-15T01:07:05.945 INFO:tasks.workunit.client.1.smithi142.stdout:3/697: dwrite d0/d1f/d20/d59/fc6 [4194304,4194304] 0 2024-06-15T01:07:05.948 INFO:tasks.workunit.client.1.smithi142.stdout:7/644: unlink d2/d8/dd/d14/d2c/d37/d6b/d88/f7d 0 2024-06-15T01:07:05.949 INFO:tasks.workunit.client.1.smithi142.stdout:4/697: write d5/d1c/d6f/dc7/d30/d46/d50/f91 [3586177,84003] 0 2024-06-15T01:07:05.951 INFO:tasks.workunit.client.0.smithi137.stdout:4/500: mkdir dd/d15/d21/d3a/d51/d83/db2/d67/dbe 0 2024-06-15T01:07:05.951 INFO:tasks.workunit.client.0.smithi137.stdout:3/560: write d0/d2f/faa [4737500,111512] 0 2024-06-15T01:07:05.951 INFO:tasks.workunit.client.0.smithi137.stdout:2/395: mkdir d0/dc/d22/d23/d5c/d79 0 2024-06-15T01:07:05.952 INFO:tasks.workunit.client.1.smithi142.stdout:2/583: fsync d8/df/d10/d94/d25/f53 0 2024-06-15T01:07:05.954 INFO:tasks.workunit.client.1.smithi142.stdout:5/573: truncate d2/d3e/d65/f70 147032 0 2024-06-15T01:07:05.954 INFO:tasks.workunit.client.0.smithi137.stdout:0/516: mknod d1/cb4 0 2024-06-15T01:07:05.956 INFO:tasks.workunit.client.0.smithi137.stdout:4/501: mkdir dd/dbf 0 2024-06-15T01:07:05.957 INFO:tasks.workunit.client.0.smithi137.stdout:2/396: fdatasync d0/dc/d2e/d72/d56/f4f 0 2024-06-15T01:07:05.957 INFO:tasks.workunit.client.0.smithi137.stdout:0/517: fdatasync d1/d2/d13/d66/f3f 0 2024-06-15T01:07:05.957 INFO:tasks.workunit.client.1.smithi142.stdout:0/620: dread d3/d39/d3e/d81/d1b/f26 [0,4194304] 0 2024-06-15T01:07:05.957 INFO:tasks.workunit.client.0.smithi137.stdout:9/301: truncate df/d12/d15/d16/f27 412889 0 2024-06-15T01:07:05.958 INFO:tasks.workunit.client.1.smithi142.stdout:1/647: creat d6/d12/da0/fe9 x:0 0 0 2024-06-15T01:07:05.961 INFO:tasks.workunit.client.1.smithi142.stdout:3/698: creat d0/d3/da/d71/d55/dae/fcd x:0 0 0 2024-06-15T01:07:05.962 INFO:tasks.workunit.client.0.smithi137.stdout:4/502: fsync dd/d15/d36/d45/d55/f82 0 2024-06-15T01:07:05.962 INFO:tasks.workunit.client.0.smithi137.stdout:0/518: chown d1/f79 44526912 1 2024-06-15T01:07:05.962 INFO:tasks.workunit.client.0.smithi137.stdout:4/503: dread - dd/d15/d21/d3a/f73 zero size 2024-06-15T01:07:05.963 INFO:tasks.workunit.client.0.smithi137.stdout:3/561: link d0/d2/d22/d24/f89 d0/d56/d8f/d9c/d9f/fb7 0 2024-06-15T01:07:05.964 INFO:tasks.workunit.client.1.smithi142.stdout:9/564: dwrite d6/dc/d1b/d25/d58/fd6 [0,4194304] 0 2024-06-15T01:07:05.964 INFO:tasks.workunit.client.1.smithi142.stdout:4/698: readlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/l1e 0 2024-06-15T01:07:05.966 INFO:tasks.workunit.client.0.smithi137.stdout:4/504: symlink dd/d15/lc0 0 2024-06-15T01:07:05.966 INFO:tasks.workunit.client.1.smithi142.stdout:0/621: dwrite d3/f6e [0,4194304] 0 2024-06-15T01:07:05.966 INFO:tasks.workunit.client.1.smithi142.stdout:9/565: chown d6/dc/d4e/d78/fd2 1 1 2024-06-15T01:07:05.968 INFO:tasks.workunit.client.1.smithi142.stdout:2/584: rename d8/df/d10/d94/d25/d1a/d57/f67 to d8/d19/d33/fba 0 2024-06-15T01:07:05.968 INFO:tasks.workunit.client.0.smithi137.stdout:9/302: write df/f4d [440130,126154] 0 2024-06-15T01:07:05.969 INFO:tasks.workunit.client.0.smithi137.stdout:9/303: write f2 [2582733,20665] 0 2024-06-15T01:07:05.971 INFO:tasks.workunit.client.0.smithi137.stdout:6/384: sync 2024-06-15T01:07:05.972 INFO:tasks.workunit.client.0.smithi137.stdout:3/562: creat d0/d2f/fb8 x:0 0 0 2024-06-15T01:07:05.974 INFO:tasks.workunit.client.0.smithi137.stdout:0/519: dwrite d1/d7e/fae [0,4194304] 0 2024-06-15T01:07:05.975 INFO:tasks.workunit.client.1.smithi142.stdout:9/566: dread d6/dc/d4e/d78/f9d [0,4194304] 0 2024-06-15T01:07:05.975 INFO:tasks.workunit.client.1.smithi142.stdout:3/699: creat d0/d1f/d20/d5d/d60/fce x:0 0 0 2024-06-15T01:07:05.976 INFO:tasks.workunit.client.1.smithi142.stdout:9/567: readlink d6/dc/d1b/d25/l6c 0 2024-06-15T01:07:05.976 INFO:tasks.workunit.client.1.smithi142.stdout:5/574: mkdir d2/d3e/d31/db4 0 2024-06-15T01:07:05.978 INFO:tasks.workunit.client.1.smithi142.stdout:3/700: write d0/d3/da/d71/d75/fa7 [172230,126355] 0 2024-06-15T01:07:05.980 INFO:tasks.workunit.client.1.smithi142.stdout:8/645: write d0/d17/f56 [4959823,106697] 0 2024-06-15T01:07:05.981 INFO:tasks.workunit.client.0.smithi137.stdout:3/563: stat d0/d2/d22/d24/d28/c55 0 2024-06-15T01:07:05.982 INFO:tasks.workunit.client.1.smithi142.stdout:8/646: fsync d0/d25/d5a/d8f/d98/da4/fa6 0 2024-06-15T01:07:05.982 INFO:tasks.workunit.client.1.smithi142.stdout:8/647: chown d0/f4c 1003 1 2024-06-15T01:07:05.983 INFO:tasks.workunit.client.0.smithi137.stdout:2/397: dwrite d0/f1 [4194304,4194304] 0 2024-06-15T01:07:05.983 INFO:tasks.workunit.client.0.smithi137.stdout:2/398: dread - d0/de/d10/f3c zero size 2024-06-15T01:07:05.990 INFO:tasks.workunit.client.1.smithi142.stdout:7/645: creat d2/d8/da/d33/d25/d79/fcc x:0 0 0 2024-06-15T01:07:05.990 INFO:tasks.workunit.client.1.smithi142.stdout:8/648: dwrite d0/d17/d40/fa2 [0,4194304] 0 2024-06-15T01:07:05.991 INFO:tasks.workunit.client.0.smithi137.stdout:0/520: mknod d1/d2/dc/d10/d14/cb5 0 2024-06-15T01:07:05.996 INFO:tasks.workunit.client.0.smithi137.stdout:0/521: write d1/f32 [3524320,112639] 0 2024-06-15T01:07:05.999 INFO:tasks.workunit.client.0.smithi137.stdout:4/505: dwrite dd/d15/d36/d45/d55/d8d/d1f/fb6 [0,4194304] 0 2024-06-15T01:07:06.000 INFO:tasks.workunit.client.1.smithi142.stdout:6/609: dwrite d1a/f76 [0,4194304] 0 2024-06-15T01:07:06.002 INFO:tasks.workunit.client.1.smithi142.stdout:6/610: write d1a/d39/d42/d4c/fb3 [221394,104817] 0 2024-06-15T01:07:06.002 INFO:tasks.workunit.client.1.smithi142.stdout:5/575: symlink d2/de/d19/d5f/lb5 0 2024-06-15T01:07:06.003 INFO:tasks.workunit.client.1.smithi142.stdout:3/701: symlink d0/d3/da/d18/lcf 0 2024-06-15T01:07:06.004 INFO:tasks.workunit.client.1.smithi142.stdout:6/611: fsync d1a/f2c 0 2024-06-15T01:07:06.005 INFO:tasks.workunit.client.0.smithi137.stdout:3/564: rename d0/d13/d37/d83/cb6 to d0/d13/d37/d83/cb9 0 2024-06-15T01:07:06.007 INFO:tasks.workunit.client.0.smithi137.stdout:0/522: read - d1/d2/dc/f73 zero size 2024-06-15T01:07:06.011 INFO:tasks.workunit.client.1.smithi142.stdout:2/585: mkdir d8/df/d10/d1c/dbb 0 2024-06-15T01:07:06.012 INFO:tasks.workunit.client.0.smithi137.stdout:3/565: creat d0/d13/fba x:0 0 0 2024-06-15T01:07:06.012 INFO:tasks.workunit.client.0.smithi137.stdout:4/506: getdents dd/d15/d21/d3a/d51/d83/db2/db8 0 2024-06-15T01:07:06.012 INFO:tasks.workunit.client.1.smithi142.stdout:2/586: chown d8/d19/d33/l47 361 1 2024-06-15T01:07:06.014 INFO:tasks.workunit.client.1.smithi142.stdout:9/568: unlink d6/d34/ca4 0 2024-06-15T01:07:06.014 INFO:tasks.workunit.client.1.smithi142.stdout:3/702: dwrite d0/d1f/d20/d59/fc6 [0,4194304] 0 2024-06-15T01:07:06.016 INFO:tasks.workunit.client.1.smithi142.stdout:3/703: dread - d0/d1f/d20/d5d/d60/fce zero size 2024-06-15T01:07:06.017 INFO:tasks.workunit.client.1.smithi142.stdout:3/704: chown d0/d1f/d20/d5d/fbb 3 1 2024-06-15T01:07:06.018 INFO:tasks.workunit.client.0.smithi137.stdout:3/566: write d0/d2/d23/d68/f3c [3478509,108405] 0 2024-06-15T01:07:06.018 INFO:tasks.workunit.client.1.smithi142.stdout:0/622: creat d3/d39/d3e/d81/d1e/fdd x:0 0 0 2024-06-15T01:07:06.019 INFO:tasks.workunit.client.0.smithi137.stdout:3/567: stat d0/fd 0 2024-06-15T01:07:06.020 INFO:tasks.workunit.client.1.smithi142.stdout:0/623: chown d3/d39/d3e/d81/d44/d7c/f14 0 1 2024-06-15T01:07:06.020 INFO:tasks.workunit.client.0.smithi137.stdout:3/568: chown d0/d2/d1e/d7c/da1 8075 1 2024-06-15T01:07:06.022 INFO:tasks.workunit.client.0.smithi137.stdout:3/569: fsync d0/d2/d8/f7f 0 2024-06-15T01:07:06.024 INFO:tasks.workunit.client.1.smithi142.stdout:1/648: getdents d6/d12/d1c 0 2024-06-15T01:07:06.025 INFO:tasks.workunit.client.1.smithi142.stdout:1/649: chown d6/d12/d1c/d2c/d47/c49 3 1 2024-06-15T01:07:06.027 INFO:tasks.workunit.client.1.smithi142.stdout:7/646: mknod d2/d8/dd/ccd 0 2024-06-15T01:07:06.027 INFO:tasks.workunit.client.0.smithi137.stdout:0/523: link d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d d1/fb6 0 2024-06-15T01:07:06.028 INFO:tasks.workunit.client.0.smithi137.stdout:0/524: write d1/d2/d13/f40 [733060,32426] 0 2024-06-15T01:07:06.029 INFO:tasks.workunit.client.1.smithi142.stdout:0/624: write d3/d39/d3e/d81/d44/d7c/fbd [4479068,90492] 0 2024-06-15T01:07:06.030 INFO:tasks.workunit.client.1.smithi142.stdout:6/612: mknod d1a/d39/d3a/dd2/cd5 0 2024-06-15T01:07:06.030 INFO:tasks.workunit.client.0.smithi137.stdout:3/570: symlink d0/d2/d1e/d7c/da1/lbb 0 2024-06-15T01:07:06.030 INFO:tasks.workunit.client.1.smithi142.stdout:6/613: chown d1a/d2d/d2e/c77 133857 1 2024-06-15T01:07:06.030 INFO:tasks.workunit.client.1.smithi142.stdout:1/650: symlink d6/d25/lea 0 2024-06-15T01:07:06.032 INFO:tasks.workunit.client.1.smithi142.stdout:9/569: link d6/dc/d4e/dae/dd8/f73 d6/dc/d1b/d25/d58/fda 0 2024-06-15T01:07:06.033 INFO:tasks.workunit.client.1.smithi142.stdout:9/570: readlink d6/dc/l29 0 2024-06-15T01:07:06.035 INFO:tasks.workunit.client.1.smithi142.stdout:5/576: rmdir d2/d18/d2e/d84 0 2024-06-15T01:07:06.037 INFO:tasks.workunit.client.1.smithi142.stdout:7/647: mkdir d2/d6/daf/dce 0 2024-06-15T01:07:06.038 INFO:tasks.workunit.client.1.smithi142.stdout:1/651: fsync d6/d12/d80/dc8/fd1 0 2024-06-15T01:07:06.038 INFO:tasks.workunit.client.0.smithi137.stdout:9/304: truncate df/d12/d1b/d31/d42/f46 3769626 0 2024-06-15T01:07:06.038 INFO:tasks.workunit.client.0.smithi137.stdout:6/385: dwrite d0/d5/f73 [4194304,4194304] 0 2024-06-15T01:07:06.039 INFO:tasks.workunit.client.0.smithi137.stdout:2/399: truncate d0/de/d10/f19 5815339 0 2024-06-15T01:07:06.042 INFO:tasks.workunit.client.1.smithi142.stdout:9/571: truncate d6/dc/d82/f81 403328 0 2024-06-15T01:07:06.043 INFO:tasks.workunit.client.1.smithi142.stdout:9/572: fdatasync d6/d34/f3a 0 2024-06-15T01:07:06.044 INFO:tasks.workunit.client.1.smithi142.stdout:9/573: fsync d6/d34/f3a 0 2024-06-15T01:07:06.046 INFO:tasks.workunit.client.1.smithi142.stdout:0/625: mknod d3/d39/d3e/d81/d1b/d38/da3/cde 0 2024-06-15T01:07:06.046 INFO:tasks.workunit.client.1.smithi142.stdout:9/574: chown d6/dc/d4e/dae/dd8/fcb 33017 1 2024-06-15T01:07:06.049 INFO:tasks.workunit.client.1.smithi142.stdout:2/587: dread d8/df/d10/d94/d25/d1a/d63/f77 [0,4194304] 0 2024-06-15T01:07:06.049 INFO:tasks.workunit.client.1.smithi142.stdout:2/588: write d8/d19/d33/fa2 [1243940,120515] 0 2024-06-15T01:07:06.051 INFO:tasks.workunit.client.1.smithi142.stdout:1/652: rename d6/d12/d1c/d2c/d47/fcd to d6/d12/d1c/d2c/d93/dba/feb 0 2024-06-15T01:07:06.051 INFO:tasks.workunit.client.1.smithi142.stdout:4/699: truncate d5/d1c/d6f/dc7/d30/d46/d50/f5e 5069143 0 2024-06-15T01:07:06.052 INFO:tasks.workunit.client.0.smithi137.stdout:4/507: dwrite dd/f17 [4194304,4194304] 0 2024-06-15T01:07:06.052 INFO:tasks.workunit.client.1.smithi142.stdout:8/649: write d0/d17/d40/d5e/f81 [4576087,64532] 0 2024-06-15T01:07:06.053 INFO:tasks.workunit.client.1.smithi142.stdout:1/653: truncate d6/d12/d1c/d2c/d93/db9/fbb 671553 0 2024-06-15T01:07:06.053 INFO:tasks.workunit.client.0.smithi137.stdout:9/305: getdents df/d12/d15 0 2024-06-15T01:07:06.057 INFO:tasks.workunit.client.0.smithi137.stdout:6/386: dwrite d0/d1a/f58 [0,4194304] 0 2024-06-15T01:07:06.060 INFO:tasks.workunit.client.1.smithi142.stdout:5/577: symlink d2/d18/lb6 0 2024-06-15T01:07:06.061 INFO:tasks.workunit.client.1.smithi142.stdout:5/578: chown d2/d18/f99 3797629 1 2024-06-15T01:07:06.062 INFO:tasks.workunit.client.0.smithi137.stdout:0/525: dwrite d1/fb6 [0,4194304] 0 2024-06-15T01:07:06.062 INFO:tasks.workunit.client.0.smithi137.stdout:3/571: write d0/d56/f6a [220660,85088] 0 2024-06-15T01:07:06.065 INFO:tasks.workunit.client.1.smithi142.stdout:3/705: truncate d0/d3/f1a 5499732 0 2024-06-15T01:07:06.066 INFO:tasks.workunit.client.0.smithi137.stdout:0/526: write d1/d2/dc/d10/d14/d15/f57 [984475,3379] 0 2024-06-15T01:07:06.067 INFO:tasks.workunit.client.0.smithi137.stdout:6/387: truncate d0/f10 2136421 0 2024-06-15T01:07:06.069 INFO:tasks.workunit.client.0.smithi137.stdout:0/527: dread - d1/d2/d13/d66/f86 zero size 2024-06-15T01:07:06.071 INFO:tasks.workunit.client.1.smithi142.stdout:8/650: rmdir d0/d17/d40 39 2024-06-15T01:07:06.072 INFO:tasks.workunit.client.0.smithi137.stdout:3/572: fsync d0/d56/d8f/d9c/d9f/fb7 0 2024-06-15T01:07:06.073 INFO:tasks.workunit.client.1.smithi142.stdout:6/614: write d1a/d39/d42/d4c/d58/d87/f89 [957208,92359] 0 2024-06-15T01:07:06.074 INFO:tasks.workunit.client.0.smithi137.stdout:3/573: chown d0/d2/d22/d24/d28/c55 329496721 1 2024-06-15T01:07:06.074 INFO:tasks.workunit.client.1.smithi142.stdout:4/700: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/lf3 0 2024-06-15T01:07:06.076 INFO:tasks.workunit.client.1.smithi142.stdout:1/654: chown d6/d46/f78 2 1 2024-06-15T01:07:06.076 INFO:tasks.workunit.client.0.smithi137.stdout:0/528: mkdir d1/d2/dc/d10/d22/db7 0 2024-06-15T01:07:06.076 INFO:tasks.workunit.client.1.smithi142.stdout:6/615: dread d1a/d39/d3a/d51/d62/fb4 [0,4194304] 0 2024-06-15T01:07:06.078 INFO:tasks.workunit.client.1.smithi142.stdout:3/706: mkdir d0/d1f/d20/d5d/d60/dd0 0 2024-06-15T01:07:06.079 INFO:tasks.workunit.client.1.smithi142.stdout:3/707: fdatasync d0/d3/da/f7b 0 2024-06-15T01:07:06.080 INFO:tasks.workunit.client.1.smithi142.stdout:9/575: rename d6/dc/d1b/d25/d58/d74/lc5 to d6/dc/d82/ldb 0 2024-06-15T01:07:06.080 INFO:tasks.workunit.client.0.smithi137.stdout:3/574: link d0/d13/cb1 d0/d2/d8/cbc 0 2024-06-15T01:07:06.081 INFO:tasks.workunit.client.1.smithi142.stdout:8/651: mknod d0/d25/d5a/d8f/d98/da4/cc7 0 2024-06-15T01:07:06.082 INFO:tasks.workunit.client.1.smithi142.stdout:7/648: dwrite d2/d6/f1c [0,4194304] 0 2024-06-15T01:07:06.082 INFO:tasks.workunit.client.0.smithi137.stdout:9/306: write f3 [1282710,23680] 0 2024-06-15T01:07:06.083 INFO:tasks.workunit.client.1.smithi142.stdout:0/626: dread d3/d39/d3e/d81/d44/d7c/d22/f65 [0,4194304] 0 2024-06-15T01:07:06.083 INFO:tasks.workunit.client.0.smithi137.stdout:6/388: dwrite d0/d1a/d3e/d57/d1d/f40 [0,4194304] 0 2024-06-15T01:07:06.084 INFO:tasks.workunit.client.1.smithi142.stdout:4/701: fsync d5/d9/d23/fd9 0 2024-06-15T01:07:06.084 INFO:tasks.workunit.client.0.smithi137.stdout:9/307: readlink df/d12/l19 0 2024-06-15T01:07:06.088 INFO:tasks.workunit.client.0.smithi137.stdout:4/508: truncate dd/d15/d21/f29 2934157 0 2024-06-15T01:07:06.088 INFO:tasks.workunit.client.1.smithi142.stdout:6/616: symlink d1a/d39/d3a/dd2/ld6 0 2024-06-15T01:07:06.090 INFO:tasks.workunit.client.0.smithi137.stdout:9/308: dread df/f4d [0,4194304] 0 2024-06-15T01:07:06.092 INFO:tasks.workunit.client.1.smithi142.stdout:3/708: mknod d0/d1b/d6b/dbd/cd1 0 2024-06-15T01:07:06.092 INFO:tasks.workunit.client.0.smithi137.stdout:9/309: write fd [6832507,24210] 0 2024-06-15T01:07:06.094 INFO:tasks.workunit.client.1.smithi142.stdout:8/652: fdatasync d0/d17/d40/d5e/f81 0 2024-06-15T01:07:06.094 INFO:tasks.workunit.client.1.smithi142.stdout:9/576: mknod d6/dc/d1b/d25/dcf/cdc 0 2024-06-15T01:07:06.094 INFO:tasks.workunit.client.0.smithi137.stdout:4/509: creat dd/d15/d21/d9c/d75/fc1 x:0 0 0 2024-06-15T01:07:06.095 INFO:tasks.workunit.client.0.smithi137.stdout:9/310: chown df/d12/d1b/f35 0 1 2024-06-15T01:07:06.098 INFO:tasks.workunit.client.0.smithi137.stdout:6/389: link d0/d1a/d3e/d57/d1d/d2e/f36 d0/d1a/d3e/f77 0 2024-06-15T01:07:06.099 INFO:tasks.workunit.client.1.smithi142.stdout:2/589: getdents d8 0 2024-06-15T01:07:06.100 INFO:tasks.workunit.client.1.smithi142.stdout:2/590: write d8/df/d10/f87 [111499,23082] 0 2024-06-15T01:07:06.103 INFO:tasks.workunit.client.1.smithi142.stdout:5/579: dread d2/de/d27/f54 [4194304,4194304] 0 2024-06-15T01:07:06.104 INFO:tasks.workunit.client.0.smithi137.stdout:9/311: symlink df/d12/d1b/d31/d42/l54 0 2024-06-15T01:07:06.106 INFO:tasks.workunit.client.0.smithi137.stdout:6/390: creat d0/d1a/d3e/d57/d32/d74/f78 x:0 0 0 2024-06-15T01:07:06.110 INFO:tasks.workunit.client.1.smithi142.stdout:4/702: mknod d5/d9/d23/d68/de8/cf4 0 2024-06-15T01:07:06.112 INFO:tasks.workunit.client.0.smithi137.stdout:3/575: dread d0/d56/d8f/d7b/f96 [0,4194304] 0 2024-06-15T01:07:06.113 INFO:tasks.workunit.client.0.smithi137.stdout:3/576: chown d0/d42/d79/lb5 2604573 1 2024-06-15T01:07:06.115 INFO:tasks.workunit.client.0.smithi137.stdout:6/391: link d0/d1a/d3e/l6e d0/d1a/d3e/d57/d1d/d2e/l79 0 2024-06-15T01:07:06.116 INFO:tasks.workunit.client.1.smithi142.stdout:5/580: rmdir d2/de/d19/d5f/d68/d76/d8c 39 2024-06-15T01:07:06.116 INFO:tasks.workunit.client.1.smithi142.stdout:0/627: dread d3/d39/d3e/d81/d44/d7c/d22/f70 [0,4194304] 0 2024-06-15T01:07:06.117 INFO:tasks.workunit.client.1.smithi142.stdout:8/653: getdents d0/d17/d40/d5e 0 2024-06-15T01:07:06.117 INFO:tasks.workunit.client.1.smithi142.stdout:2/591: dwrite d8/df/f59 [0,4194304] 0 2024-06-15T01:07:06.118 INFO:tasks.workunit.client.1.smithi142.stdout:0/628: stat d3/d39/d3e/d81/c1a 0 2024-06-15T01:07:06.120 INFO:tasks.workunit.client.1.smithi142.stdout:7/649: rename d2/d8/da/d33/d25/d27 to d2/d8/dd/dcf 0 2024-06-15T01:07:06.122 INFO:tasks.workunit.client.1.smithi142.stdout:4/703: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/df5 0 2024-06-15T01:07:06.123 INFO:tasks.workunit.client.1.smithi142.stdout:4/704: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/c6c 0 2024-06-15T01:07:06.125 INFO:tasks.workunit.client.0.smithi137.stdout:9/312: truncate f3 866094 0 2024-06-15T01:07:06.126 INFO:tasks.workunit.client.1.smithi142.stdout:8/654: creat d0/d25/d5a/d8f/d98/fc8 x:0 0 0 2024-06-15T01:07:06.127 INFO:tasks.workunit.client.0.smithi137.stdout:6/392: dwrite d0/d5/d9/f63 [0,4194304] 0 2024-06-15T01:07:06.130 INFO:tasks.workunit.client.1.smithi142.stdout:2/592: creat d8/df/d10/d94/d25/d1a/d63/fbc x:0 0 0 2024-06-15T01:07:06.132 INFO:tasks.workunit.client.1.smithi142.stdout:7/650: creat d2/d8/dd/d8a/d94/fd0 x:0 0 0 2024-06-15T01:07:06.133 INFO:tasks.workunit.client.1.smithi142.stdout:8/655: dwrite d0/d25/d5a/f88 [0,4194304] 0 2024-06-15T01:07:06.133 INFO:tasks.workunit.client.1.smithi142.stdout:2/593: dread - d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa6 zero size 2024-06-15T01:07:06.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/393: mknod d0/d5/d9/c7a 0 2024-06-15T01:07:06.136 INFO:tasks.workunit.client.1.smithi142.stdout:5/581: fsync d2/de/f66 0 2024-06-15T01:07:06.136 INFO:tasks.workunit.client.1.smithi142.stdout:2/594: readlink l1 0 2024-06-15T01:07:06.136 INFO:tasks.workunit.client.0.smithi137.stdout:3/577: truncate d0/d2/d22/d24/d5e/f85 1443713 0 2024-06-15T01:07:06.136 INFO:tasks.workunit.client.0.smithi137.stdout:4/510: dread dd/f3b [0,4194304] 0 2024-06-15T01:07:06.138 INFO:tasks.workunit.client.1.smithi142.stdout:7/651: chown d2/d8/dd/d14/d2c/d37/d6b/f96 1 1 2024-06-15T01:07:06.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/511: dread - dd/d15/d21/d3a/d51/f7f zero size 2024-06-15T01:07:06.140 INFO:tasks.workunit.client.0.smithi137.stdout:6/394: mknod d0/d1a/d3e/d57/d32/c7b 0 2024-06-15T01:07:06.142 INFO:tasks.workunit.client.1.smithi142.stdout:3/709: rename d0/d3/d15/f9f to d0/d3/fd2 0 2024-06-15T01:07:06.143 INFO:tasks.workunit.client.0.smithi137.stdout:3/578: mkdir d0/d2/d8/dbd 0 2024-06-15T01:07:06.144 INFO:tasks.workunit.client.1.smithi142.stdout:1/655: dwrite d6/f13 [0,4194304] 0 2024-06-15T01:07:06.146 INFO:tasks.workunit.client.0.smithi137.stdout:4/512: creat dd/d15/d21/d9c/d75/fc2 x:0 0 0 2024-06-15T01:07:06.146 INFO:tasks.workunit.client.1.smithi142.stdout:2/595: dwrite d8/df/d10/d1c/f2c [0,4194304] 0 2024-06-15T01:07:06.147 INFO:tasks.workunit.client.1.smithi142.stdout:7/652: dread d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:06.174 INFO:tasks.workunit.client.1.smithi142.stdout:2/596: dread f4 [0,4194304] 0 2024-06-15T01:07:06.175 INFO:tasks.workunit.client.1.smithi142.stdout:2/597: dread - d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa6 zero size 2024-06-15T01:07:06.176 INFO:tasks.workunit.client.1.smithi142.stdout:2/598: write d8/df/d10/d94/d25/d1a/d63/fbc [862307,35307] 0 2024-06-15T01:07:06.178 INFO:tasks.workunit.client.1.smithi142.stdout:2/599: dread d8/df/d10/f11 [0,4194304] 0 2024-06-15T01:07:06.181 INFO:tasks.workunit.client.1.smithi142.stdout:6/617: sync 2024-06-15T01:07:06.183 INFO:tasks.workunit.client.1.smithi142.stdout:2/600: dwrite d8/d19/f56 [0,4194304] 0 2024-06-15T01:07:06.188 INFO:tasks.workunit.client.1.smithi142.stdout:2/601: dread d8/d19/f70 [0,4194304] 0 2024-06-15T01:07:06.207 INFO:tasks.workunit.client.0.smithi137.stdout:6/395: creat d0/d5/f7c x:0 0 0 2024-06-15T01:07:06.207 INFO:tasks.workunit.client.0.smithi137.stdout:4/513: chown dd/d15/d21/d3a/d51/d83/db2/d67/f7d 718 1 2024-06-15T01:07:06.208 INFO:tasks.workunit.client.0.smithi137.stdout:8/396: sync 2024-06-15T01:07:06.208 INFO:tasks.workunit.client.0.smithi137.stdout:3/579: dwrite d0/d42/fab [0,4194304] 0 2024-06-15T01:07:06.209 INFO:tasks.workunit.client.0.smithi137.stdout:4/514: read - dd/d15/d36/d45/d55/f62 zero size 2024-06-15T01:07:06.212 INFO:tasks.workunit.client.0.smithi137.stdout:3/580: mkdir d0/d2f/dbe 0 2024-06-15T01:07:06.212 INFO:tasks.workunit.client.0.smithi137.stdout:9/313: truncate df/d12/d15/f1d 287894 0 2024-06-15T01:07:06.273 INFO:tasks.workunit.client.1.smithi142.stdout:2/602: sync 2024-06-15T01:07:06.277 INFO:tasks.workunit.client.0.smithi137.stdout:8/397: mkdir d9/de/d1e/d3f/d84 0 2024-06-15T01:07:06.280 INFO:tasks.workunit.client.0.smithi137.stdout:4/515: creat dd/d15/d21/d3a/d51/d5b/fc3 x:0 0 0 2024-06-15T01:07:06.280 INFO:tasks.workunit.client.0.smithi137.stdout:9/314: dwrite df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:06.280 INFO:tasks.workunit.client.0.smithi137.stdout:9/315: readlink l6 0 2024-06-15T01:07:06.281 INFO:tasks.workunit.client.1.smithi142.stdout:8/656: mknod d0/d17/d40/cc9 0 2024-06-15T01:07:06.282 INFO:tasks.workunit.client.0.smithi137.stdout:6/396: getdents d0/d1a/d3e/d57/d1d 0 2024-06-15T01:07:06.282 INFO:tasks.workunit.client.0.smithi137.stdout:3/581: chown d0/d2/d22/d24/d28/fb4 22212439 1 2024-06-15T01:07:06.282 INFO:tasks.workunit.client.0.smithi137.stdout:3/582: fdatasync d0/d2f/fb8 0 2024-06-15T01:07:06.283 INFO:tasks.workunit.client.0.smithi137.stdout:8/398: mknod d9/d2a/d2f/d59/d82/c85 0 2024-06-15T01:07:06.284 INFO:tasks.workunit.client.1.smithi142.stdout:7/653: mknod d2/d8/dd/d8a/d94/cd1 0 2024-06-15T01:07:06.287 INFO:tasks.workunit.client.1.smithi142.stdout:8/657: dwrite d0/d17/d40/d5e/f81 [0,4194304] 0 2024-06-15T01:07:06.287 INFO:tasks.workunit.client.0.smithi137.stdout:9/316: symlink df/d12/d1b/l55 0 2024-06-15T01:07:06.288 INFO:tasks.workunit.client.1.smithi142.stdout:7/654: fsync d2/d8/da/d33/d25/d79/fcc 0 2024-06-15T01:07:06.292 INFO:tasks.workunit.client.0.smithi137.stdout:8/399: fsync d9/d2a/f6f 0 2024-06-15T01:07:06.294 INFO:tasks.workunit.client.1.smithi142.stdout:8/658: dread d0/d17/f1f [4194304,4194304] 0 2024-06-15T01:07:06.299 INFO:tasks.workunit.client.0.smithi137.stdout:6/397: creat d0/d5/d9/d3f/f7d x:0 0 0 2024-06-15T01:07:06.299 INFO:tasks.workunit.client.0.smithi137.stdout:9/317: dread df/d12/d15/d16/f27 [0,4194304] 0 2024-06-15T01:07:06.302 INFO:tasks.workunit.client.0.smithi137.stdout:8/400: dwrite d9/f1d [0,4194304] 0 2024-06-15T01:07:06.304 INFO:tasks.workunit.client.0.smithi137.stdout:3/583: truncate d0/d2/d22/d63/d76/f7d 4336075 0 2024-06-15T01:07:06.305 INFO:tasks.workunit.client.0.smithi137.stdout:9/318: creat df/d12/d1b/d31/d42/f56 x:0 0 0 2024-06-15T01:07:06.307 INFO:tasks.workunit.client.0.smithi137.stdout:8/401: readlink d9/de/l46 0 2024-06-15T01:07:06.313 INFO:tasks.workunit.client.0.smithi137.stdout:6/398: getdents d0/d1a/d3e/d57/d1d/d2e/d62 0 2024-06-15T01:07:06.317 INFO:tasks.workunit.client.1.smithi142.stdout:6/618: fdatasync d1a/f3e 0 2024-06-15T01:07:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:05 smithi142 ceph-mon[37379]: pgmap v18: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 214 MiB/s wr, 343 op/s 2024-06-15T01:07:06.317 INFO:tasks.workunit.client.0.smithi137.stdout:6/399: getdents d0/d1a 0 2024-06-15T01:07:06.319 INFO:tasks.workunit.client.0.smithi137.stdout:3/584: dread d0/d2/d8/f14 [0,4194304] 0 2024-06-15T01:07:06.322 INFO:tasks.workunit.client.1.smithi142.stdout:2/603: unlink d8/df/d10/d94/d25/d1a/d63/fbc 0 2024-06-15T01:07:06.322 INFO:tasks.workunit.client.0.smithi137.stdout:3/585: fsync d0/d2/d22/d24/d28/fb4 0 2024-06-15T01:07:06.325 INFO:tasks.workunit.client.1.smithi142.stdout:9/577: rename d6/dc/d4e/dae/dd8 to d6/d34/ddd 0 2024-06-15T01:07:06.327 INFO:tasks.workunit.client.1.smithi142.stdout:3/710: creat d0/d1f/fd3 x:0 0 0 2024-06-15T01:07:06.328 INFO:tasks.workunit.client.0.smithi137.stdout:9/319: truncate df/d12/d1b/f35 116182 0 2024-06-15T01:07:06.328 INFO:tasks.workunit.client.1.smithi142.stdout:2/604: dwrite d8/df/d10/d94/d25/da9/f6f [0,4194304] 0 2024-06-15T01:07:06.329 INFO:tasks.workunit.client.1.smithi142.stdout:5/582: getdents d2/de 0 2024-06-15T01:07:06.330 INFO:tasks.workunit.client.0.smithi137.stdout:9/320: fdatasync df/d12/f1f 0 2024-06-15T01:07:06.330 INFO:tasks.workunit.client.0.smithi137.stdout:6/400: dwrite d0/d5/d9/f4f [0,4194304] 0 2024-06-15T01:07:06.332 INFO:tasks.workunit.client.1.smithi142.stdout:6/619: creat d1a/d2d/d80/d8e/fd7 x:0 0 0 2024-06-15T01:07:06.333 INFO:tasks.workunit.client.0.smithi137.stdout:6/401: readlink d0/lb 0 2024-06-15T01:07:06.335 INFO:tasks.workunit.client.1.smithi142.stdout:8/659: symlink d0/d25/d53/d9c/dbf/lca 0 2024-06-15T01:07:06.335 INFO:tasks.workunit.client.0.smithi137.stdout:8/402: dwrite d9/de/d1e/d3f/f68 [0,4194304] 0 2024-06-15T01:07:06.336 INFO:tasks.workunit.client.1.smithi142.stdout:8/660: write d0/d17/d40/d86/fc6 [575244,64013] 0 2024-06-15T01:07:06.337 INFO:tasks.workunit.client.0.smithi137.stdout:8/403: readlink d9/de/d1e/l49 0 2024-06-15T01:07:06.338 INFO:tasks.workunit.client.0.smithi137.stdout:8/404: chown d9/de/d1e/f42 4735 1 2024-06-15T01:07:06.341 INFO:tasks.workunit.client.1.smithi142.stdout:0/629: rename d3/d39/d3e/d81/f87 to d3/d39/d3e/d81/d1b/dae/fdf 0 2024-06-15T01:07:06.342 INFO:tasks.workunit.client.1.smithi142.stdout:7/655: dread d2/d6/d22/d47/f4f [0,4194304] 0 2024-06-15T01:07:06.343 INFO:tasks.workunit.client.0.smithi137.stdout:6/402: mknod d0/d1a/d3e/d57/d1d/d2e/d62/c7e 0 2024-06-15T01:07:06.343 INFO:tasks.workunit.client.1.smithi142.stdout:9/578: mknod d6/d34/d8a/cde 0 2024-06-15T01:07:06.346 INFO:tasks.workunit.client.1.smithi142.stdout:9/579: dread - d6/dc/d4e/d78/fd2 zero size 2024-06-15T01:07:06.346 INFO:tasks.workunit.client.0.smithi137.stdout:9/321: getdents df/d13 0 2024-06-15T01:07:06.347 INFO:tasks.workunit.client.1.smithi142.stdout:3/711: creat d0/d3/da/d18/fd4 x:0 0 0 2024-06-15T01:07:06.349 INFO:tasks.workunit.client.1.smithi142.stdout:3/712: chown d0/d3/f54 34092448 1 2024-06-15T01:07:06.350 INFO:tasks.workunit.client.0.smithi137.stdout:8/405: creat d9/de/d1e/d3f/d84/f86 x:0 0 0 2024-06-15T01:07:06.351 INFO:tasks.workunit.client.0.smithi137.stdout:9/322: creat df/d49/d4b/f57 x:0 0 0 2024-06-15T01:07:06.352 INFO:tasks.workunit.client.0.smithi137.stdout:8/406: truncate f6 4875850 0 2024-06-15T01:07:06.352 INFO:tasks.workunit.client.1.smithi142.stdout:0/630: dwrite d3/d39/d3e/d81/f88 [0,4194304] 0 2024-06-15T01:07:06.352 INFO:tasks.workunit.client.1.smithi142.stdout:6/620: mkdir d1a/d3c/db0/dd8 0 2024-06-15T01:07:06.353 INFO:tasks.workunit.client.0.smithi137.stdout:6/403: dwrite d0/d1a/f58 [0,4194304] 0 2024-06-15T01:07:06.353 INFO:tasks.workunit.client.1.smithi142.stdout:0/631: truncate d3/d39/d3e/d8f/fd2 160309 0 2024-06-15T01:07:06.357 INFO:tasks.workunit.client.0.smithi137.stdout:9/323: rename df/d12/f2c to df/d12/d15/f58 0 2024-06-15T01:07:06.359 INFO:tasks.workunit.client.1.smithi142.stdout:4/705: rename d5/d9/da/f7c to d5/d9/d23/dbe/ff6 0 2024-06-15T01:07:06.359 INFO:tasks.workunit.client.0.smithi137.stdout:3/586: dread d0/d2/d22/d24/f54 [0,4194304] 0 2024-06-15T01:07:06.359 INFO:tasks.workunit.client.0.smithi137.stdout:3/587: stat d0/d42 0 2024-06-15T01:07:06.361 INFO:tasks.workunit.client.0.smithi137.stdout:9/324: creat df/d12/f59 x:0 0 0 2024-06-15T01:07:06.362 INFO:tasks.workunit.client.1.smithi142.stdout:7/656: mknod d2/d8/dd/d8a/d94/cd2 0 2024-06-15T01:07:06.363 INFO:tasks.workunit.client.1.smithi142.stdout:7/657: readlink d2/d8/dd/dcf/lb3 0 2024-06-15T01:07:06.363 INFO:tasks.workunit.client.0.smithi137.stdout:6/404: rename d0/ld to d0/d1a/d3e/d57/d1d/d2e/l7f 0 2024-06-15T01:07:06.363 INFO:tasks.workunit.client.0.smithi137.stdout:8/407: getdents d9/de/d1e/d67 0 2024-06-15T01:07:06.365 INFO:tasks.workunit.client.0.smithi137.stdout:3/588: chown d0/d2/d22/d63/d76/f7d 17097 1 2024-06-15T01:07:06.367 INFO:tasks.workunit.client.1.smithi142.stdout:1/656: link d6/d12/d3c/lc1 d6/lec 0 2024-06-15T01:07:06.367 INFO:tasks.workunit.client.0.smithi137.stdout:8/408: creat d9/d2a/d2f/d59/f87 x:0 0 0 2024-06-15T01:07:06.368 INFO:tasks.workunit.client.1.smithi142.stdout:6/621: creat d1a/d2d/d2e/fd9 x:0 0 0 2024-06-15T01:07:06.368 INFO:tasks.workunit.client.1.smithi142.stdout:8/661: unlink d0/d17/d40/d86/fa1 0 2024-06-15T01:07:06.368 INFO:tasks.workunit.client.1.smithi142.stdout:3/713: stat d0/d1f/d20/d5d/c7d 0 2024-06-15T01:07:06.368 INFO:tasks.workunit.client.0.smithi137.stdout:8/409: chown d9/d22/f33 50 1 2024-06-15T01:07:06.369 INFO:tasks.workunit.client.1.smithi142.stdout:8/662: chown d0/d17/c5d 7530 1 2024-06-15T01:07:06.370 INFO:tasks.workunit.client.1.smithi142.stdout:3/714: write d0/d3/fcb [126699,113476] 0 2024-06-15T01:07:06.372 INFO:tasks.workunit.client.1.smithi142.stdout:7/658: symlink d2/d8/da/d33/d25/ld3 0 2024-06-15T01:07:06.373 INFO:tasks.workunit.client.1.smithi142.stdout:2/605: getdents d8/df 0 2024-06-15T01:07:06.373 INFO:tasks.workunit.client.1.smithi142.stdout:5/583: getdents d2/d3e/d74 0 2024-06-15T01:07:06.374 INFO:tasks.workunit.client.1.smithi142.stdout:8/663: dread - d0/d25/d5a/d8f/d97/fbe zero size 2024-06-15T01:07:06.375 INFO:tasks.workunit.client.0.smithi137.stdout:6/405: dread d0/d5/d9/d3f/f67 [4194304,4194304] 0 2024-06-15T01:07:06.375 INFO:tasks.workunit.client.1.smithi142.stdout:0/632: getdents d3/d39/d9a/d9d/dbf 0 2024-06-15T01:07:06.377 INFO:tasks.workunit.client.0.smithi137.stdout:6/406: unlink d0/d1a/d3e/d57/f1e 0 2024-06-15T01:07:06.377 INFO:tasks.workunit.client.1.smithi142.stdout:2/606: stat d8/df/d10/d94/d25/d1a/faa 0 2024-06-15T01:07:06.378 INFO:tasks.workunit.client.0.smithi137.stdout:6/407: write d0/d1a/f75 [250010,14023] 0 2024-06-15T01:07:06.378 INFO:tasks.workunit.client.1.smithi142.stdout:1/657: dwrite d6/d12/f2d [0,4194304] 0 2024-06-15T01:07:06.379 INFO:tasks.workunit.client.1.smithi142.stdout:6/622: link d1a/d3c/c4f d1a/d39/d3a/dd2/cda 0 2024-06-15T01:07:06.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/408: readlink d0/d1a/d3e/l59 0 2024-06-15T01:07:06.380 INFO:tasks.workunit.client.1.smithi142.stdout:7/659: dread d2/d8/f84 [0,4194304] 0 2024-06-15T01:07:06.381 INFO:tasks.workunit.client.1.smithi142.stdout:3/715: truncate d0/d3/da/d71/d55/f94 350639 0 2024-06-15T01:07:06.381 INFO:tasks.workunit.client.1.smithi142.stdout:7/660: readlink d2/d8/da/l9d 0 2024-06-15T01:07:06.382 INFO:tasks.workunit.client.1.smithi142.stdout:1/658: chown d6/d12/d3c/d63/d8d/d97/dd3 86908 1 2024-06-15T01:07:06.383 INFO:tasks.workunit.client.1.smithi142.stdout:9/580: dread d6/dc/d4e/f94 [0,4194304] 0 2024-06-15T01:07:06.383 INFO:tasks.workunit.client.1.smithi142.stdout:7/661: write d2/d8/dd/f76 [3841364,273] 0 2024-06-15T01:07:06.384 INFO:tasks.workunit.client.0.smithi137.stdout:9/325: write df/d12/d1b/f35 [199320,8227] 0 2024-06-15T01:07:06.384 INFO:tasks.workunit.client.1.smithi142.stdout:9/581: chown d6/d34/ddd/c89 199913 1 2024-06-15T01:07:06.387 INFO:tasks.workunit.client.0.smithi137.stdout:6/409: unlink d0/d1a/d3e/l6e 0 2024-06-15T01:07:06.392 INFO:tasks.workunit.client.0.smithi137.stdout:6/410: dread d0/d5/d9/f63 [0,4194304] 0 2024-06-15T01:07:06.393 INFO:tasks.workunit.client.0.smithi137.stdout:3/589: write d0/d2/d22/d24/d28/fb4 [273359,40149] 0 2024-06-15T01:07:06.395 INFO:tasks.workunit.client.0.smithi137.stdout:3/590: mkdir d0/d77/dbf 0 2024-06-15T01:07:06.396 INFO:tasks.workunit.client.1.smithi142.stdout:8/664: dread d0/d25/f7e [0,4194304] 0 2024-06-15T01:07:06.401 INFO:tasks.workunit.client.0.smithi137.stdout:6/411: dwrite d0/d5/d9/f63 [4194304,4194304] 0 2024-06-15T01:07:06.403 INFO:tasks.workunit.client.0.smithi137.stdout:3/591: link d0/d56/d8f/d7b/l8c d0/d56/d8f/d9c/lc0 0 2024-06-15T01:07:06.405 INFO:tasks.workunit.client.1.smithi142.stdout:1/659: dread d6/d12/f22 [0,4194304] 0 2024-06-15T01:07:06.406 INFO:tasks.workunit.client.0.smithi137.stdout:6/412: read d0/d1a/d3e/d57/f5e [39301,126839] 0 2024-06-15T01:07:06.411 INFO:tasks.workunit.client.0.smithi137.stdout:9/326: dwrite f3 [0,4194304] 0 2024-06-15T01:07:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:9/327: symlink df/d49/d4b/l5a 0 2024-06-15T01:07:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:3/592: rmdir d0/d77/dbf 0 2024-06-15T01:07:06.422 INFO:tasks.workunit.client.0.smithi137.stdout:3/593: unlink d0/d42/fab 0 2024-06-15T01:07:06.423 INFO:tasks.workunit.client.0.smithi137.stdout:9/328: mkdir df/d12/d1b/d31/d42/d5b 0 2024-06-15T01:07:06.424 INFO:tasks.workunit.client.0.smithi137.stdout:9/329: write df/d12/f3e [879682,7812] 0 2024-06-15T01:07:06.426 INFO:tasks.workunit.client.0.smithi137.stdout:3/594: truncate d0/d2/d22/d24/d5e/f98 449089 0 2024-06-15T01:07:06.426 INFO:tasks.workunit.client.0.smithi137.stdout:6/413: dread d0/fc [0,4194304] 0 2024-06-15T01:07:06.427 INFO:tasks.workunit.client.0.smithi137.stdout:9/330: creat df/d12/d4e/f5c x:0 0 0 2024-06-15T01:07:06.430 INFO:tasks.workunit.client.0.smithi137.stdout:9/331: unlink df/f18 0 2024-06-15T01:07:06.432 INFO:tasks.workunit.client.0.smithi137.stdout:3/595: unlink d0/d56/d8f/d9c/d9f/la4 0 2024-06-15T01:07:06.433 INFO:tasks.workunit.client.0.smithi137.stdout:3/596: chown d0/d2/d22/d63/cac 103953794 1 2024-06-15T01:07:06.433 INFO:tasks.workunit.client.1.smithi142.stdout:4/706: sync 2024-06-15T01:07:06.434 INFO:tasks.workunit.client.0.smithi137.stdout:3/597: read d0/d2/d23/f44 [1337307,30655] 0 2024-06-15T01:07:06.435 INFO:tasks.workunit.client.0.smithi137.stdout:6/414: creat d0/d1a/d3e/d57/f80 x:0 0 0 2024-06-15T01:07:06.438 INFO:tasks.workunit.client.0.smithi137.stdout:3/598: dread d0/d2/d23/f73 [0,4194304] 0 2024-06-15T01:07:06.439 INFO:tasks.workunit.client.0.smithi137.stdout:6/415: mkdir d0/d5/d9/d29/d81 0 2024-06-15T01:07:06.440 INFO:tasks.workunit.client.0.smithi137.stdout:3/599: write d0/d2/d1e/f6e [728974,68169] 0 2024-06-15T01:07:06.441 INFO:tasks.workunit.client.1.smithi142.stdout:0/633: creat d3/d39/d3e/fe0 x:0 0 0 2024-06-15T01:07:06.442 INFO:tasks.workunit.client.0.smithi137.stdout:6/416: creat d0/d1a/d68/f82 x:0 0 0 2024-06-15T01:07:06.444 INFO:tasks.workunit.client.1.smithi142.stdout:2/607: dread d8/df/f85 [0,4194304] 0 2024-06-15T01:07:06.444 INFO:tasks.workunit.client.1.smithi142.stdout:6/623: symlink d1a/d83/ldb 0 2024-06-15T01:07:06.445 INFO:tasks.workunit.client.0.smithi137.stdout:3/600: rmdir d0/d2f 39 2024-06-15T01:07:06.445 INFO:tasks.workunit.client.1.smithi142.stdout:2/608: stat d8/df/d10/d94/d25/c73 0 2024-06-15T01:07:06.445 INFO:tasks.workunit.client.1.smithi142.stdout:6/624: truncate d1a/d2d/d68/da0/fa5 1326124 0 2024-06-15T01:07:06.450 INFO:tasks.workunit.client.1.smithi142.stdout:6/625: dwrite d1a/d3c/fc8 [0,4194304] 0 2024-06-15T01:07:06.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/332: write df/f28 [1181355,6394] 0 2024-06-15T01:07:06.454 INFO:tasks.workunit.client.0.smithi137.stdout:3/601: dread d0/d2/d22/d24/d28/fb4 [0,4194304] 0 2024-06-15T01:07:06.463 INFO:tasks.workunit.client.0.smithi137.stdout:6/417: dwrite d0/d5/f16 [0,4194304] 0 2024-06-15T01:07:06.469 INFO:tasks.workunit.client.1.smithi142.stdout:3/716: mknod d0/d1f/d20/d59/cd5 0 2024-06-15T01:07:06.470 INFO:tasks.workunit.client.1.smithi142.stdout:3/717: dread - d0/d3/da/d71/d55/dae/fcd zero size 2024-06-15T01:07:06.471 INFO:tasks.workunit.client.0.smithi137.stdout:9/333: creat df/d12/d29/d4a/f5d x:0 0 0 2024-06-15T01:07:06.472 INFO:tasks.workunit.client.0.smithi137.stdout:6/418: creat d0/d5/f83 x:0 0 0 2024-06-15T01:07:06.473 INFO:tasks.workunit.client.0.smithi137.stdout:3/602: link d0/d56/d8f/d9c/lc0 d0/d2/d8/d67/lc1 0 2024-06-15T01:07:06.473 INFO:tasks.workunit.client.1.smithi142.stdout:7/662: mkdir d2/d8/da/d33/d25/d79/dd4 0 2024-06-15T01:07:06.474 INFO:tasks.workunit.client.0.smithi137.stdout:9/334: mknod df/d1a/c5e 0 2024-06-15T01:07:06.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:05 smithi137 ceph-mon[29354]: pgmap v18: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 214 MiB/s wr, 343 op/s 2024-06-15T01:07:06.475 INFO:tasks.workunit.client.1.smithi142.stdout:8/665: mkdir d0/d25/d53/da7/dcb 0 2024-06-15T01:07:06.476 INFO:tasks.workunit.client.1.smithi142.stdout:8/666: truncate d0/d25/d66/fb4 679202 0 2024-06-15T01:07:06.477 INFO:tasks.workunit.client.0.smithi137.stdout:3/603: creat d0/d77/fc2 x:0 0 0 2024-06-15T01:07:06.478 INFO:tasks.workunit.client.0.smithi137.stdout:3/604: fdatasync d0/d2/d22/d24/f54 0 2024-06-15T01:07:06.479 INFO:tasks.workunit.client.1.smithi142.stdout:1/660: mknod d6/d12/d80/dae/ced 0 2024-06-15T01:07:06.481 INFO:tasks.workunit.client.1.smithi142.stdout:4/707: creat d5/d1c/d6f/dc7/de0/ff7 x:0 0 0 2024-06-15T01:07:06.482 INFO:tasks.workunit.client.0.smithi137.stdout:9/335: rmdir df/d12/d1b/d31/d42/d5b 0 2024-06-15T01:07:06.482 INFO:tasks.workunit.client.1.smithi142.stdout:0/634: mknod d3/d39/d3e/d81/d4c/d61/ce1 0 2024-06-15T01:07:06.483 INFO:tasks.workunit.client.1.smithi142.stdout:5/584: link d2/f28 d2/fb7 0 2024-06-15T01:07:06.485 INFO:tasks.workunit.client.0.smithi137.stdout:9/336: chown df/d12/d15/f58 1690315 1 2024-06-15T01:07:06.487 INFO:tasks.workunit.client.0.smithi137.stdout:9/337: read df/d12/d29/f44 [2666302,74128] 0 2024-06-15T01:07:06.488 INFO:tasks.workunit.client.0.smithi137.stdout:3/605: dwrite d0/d2/d22/d24/d28/fa6 [0,4194304] 0 2024-06-15T01:07:06.488 INFO:tasks.workunit.client.1.smithi142.stdout:3/718: mknod d0/d1f/d20/d5d/d60/cd6 0 2024-06-15T01:07:06.491 INFO:tasks.workunit.client.0.smithi137.stdout:3/606: write d0/d56/d8f/d7b/f96 [1664190,7978] 0 2024-06-15T01:07:06.493 INFO:tasks.workunit.client.0.smithi137.stdout:9/338: rmdir df/d12/d4e 39 2024-06-15T01:07:06.497 INFO:tasks.workunit.client.1.smithi142.stdout:3/719: dwrite d0/d3/da/d71/d55/dae/fcd [0,4194304] 0 2024-06-15T01:07:06.498 INFO:tasks.workunit.client.1.smithi142.stdout:4/708: write d5/d9/d90/daf/dcb/f7a [834727,97111] 0 2024-06-15T01:07:06.498 INFO:tasks.workunit.client.0.smithi137.stdout:9/339: write df/d12/f59 [65726,97058] 0 2024-06-15T01:07:06.499 INFO:tasks.workunit.client.0.smithi137.stdout:3/607: dwrite d0/d56/d8f/d7b/f96 [0,4194304] 0 2024-06-15T01:07:06.500 INFO:tasks.workunit.client.1.smithi142.stdout:7/663: dread d2/d6/d22/d47/f7a [0,4194304] 0 2024-06-15T01:07:06.500 INFO:tasks.workunit.client.1.smithi142.stdout:9/582: creat d6/dc/fdf x:0 0 0 2024-06-15T01:07:06.501 INFO:tasks.workunit.client.1.smithi142.stdout:5/585: dread d2/de/d27/f38 [0,4194304] 0 2024-06-15T01:07:06.505 INFO:tasks.workunit.client.1.smithi142.stdout:8/667: link d0/d17/d40/d5e/f81 d0/d25/d5a/d84/fcc 0 2024-06-15T01:07:06.511 INFO:tasks.workunit.client.1.smithi142.stdout:1/661: link d6/d12/d3c/d63/d8d/d97/dd3/faa d6/d3a/fee 0 2024-06-15T01:07:06.511 INFO:tasks.workunit.client.0.smithi137.stdout:3/608: dwrite d0/d2/d8/fa2 [0,4194304] 0 2024-06-15T01:07:06.515 INFO:tasks.workunit.client.1.smithi142.stdout:0/635: symlink d3/d39/d3e/d81/d44/d7c/d22/le2 0 2024-06-15T01:07:06.516 INFO:tasks.workunit.client.0.smithi137.stdout:3/609: rename d0/d2/d8/d67/d7e/l40 to d0/d2/d8/d67/d7e/lc3 0 2024-06-15T01:07:06.516 INFO:tasks.workunit.client.1.smithi142.stdout:2/609: sync 2024-06-15T01:07:06.517 INFO:tasks.workunit.client.1.smithi142.stdout:6/626: creat d1a/fdc x:0 0 0 2024-06-15T01:07:06.517 INFO:tasks.workunit.client.1.smithi142.stdout:3/720: symlink d0/ld7 0 2024-06-15T01:07:06.519 INFO:tasks.workunit.client.0.smithi137.stdout:3/610: write d0/d2/d8/f14 [935827,86037] 0 2024-06-15T01:07:06.520 INFO:tasks.workunit.client.1.smithi142.stdout:0/636: dread d3/d39/d3e/d81/d1e/f63 [0,4194304] 0 2024-06-15T01:07:06.522 INFO:tasks.workunit.client.1.smithi142.stdout:7/664: truncate d2/d6/f83 1648178 0 2024-06-15T01:07:06.523 INFO:tasks.workunit.client.0.smithi137.stdout:3/611: mknod d0/d2/d1e/d7c/cc4 0 2024-06-15T01:07:06.524 INFO:tasks.workunit.client.1.smithi142.stdout:1/662: mkdir d6/d12/d3c/d63/d8d/def 0 2024-06-15T01:07:06.524 INFO:tasks.workunit.client.0.smithi137.stdout:3/612: mknod d0/d2/daf/cc5 0 2024-06-15T01:07:06.526 INFO:tasks.workunit.client.0.smithi137.stdout:3/613: mknod d0/d2/d22/d24/d28/cc6 0 2024-06-15T01:07:06.527 INFO:tasks.workunit.client.1.smithi142.stdout:4/709: unlink d5/d9/d18/f2f 0 2024-06-15T01:07:06.528 INFO:tasks.workunit.client.0.smithi137.stdout:3/614: chown d0/d2f/dbe 124526454 1 2024-06-15T01:07:06.528 INFO:tasks.workunit.client.1.smithi142.stdout:9/583: symlink d6/le0 0 2024-06-15T01:07:06.528 INFO:tasks.workunit.client.1.smithi142.stdout:9/584: readlink d6/d31/l4c 0 2024-06-15T01:07:06.529 INFO:tasks.workunit.client.1.smithi142.stdout:9/585: write d6/dc/f19 [4883825,107366] 0 2024-06-15T01:07:06.530 INFO:tasks.workunit.client.1.smithi142.stdout:0/637: rename d3/d39/d9a/la6 to d3/d39/d3e/d81/d1e/d98/le3 0 2024-06-15T01:07:06.531 INFO:tasks.workunit.client.0.smithi137.stdout:3/615: creat d0/d2/d22/d24/d28/fc7 x:0 0 0 2024-06-15T01:07:06.532 INFO:tasks.workunit.client.1.smithi142.stdout:8/668: link d0/d17/f56 d0/d17/d40/d5e/fcd 0 2024-06-15T01:07:06.533 INFO:tasks.workunit.client.1.smithi142.stdout:7/665: truncate d2/d8/da/fae 5518319 0 2024-06-15T01:07:06.534 INFO:tasks.workunit.client.0.smithi137.stdout:3/616: symlink d0/d2f/lc8 0 2024-06-15T01:07:06.535 INFO:tasks.workunit.client.1.smithi142.stdout:1/663: mknod d6/d12/da0/cf0 0 2024-06-15T01:07:06.536 INFO:tasks.workunit.client.0.smithi137.stdout:3/617: creat d0/d42/fc9 x:0 0 0 2024-06-15T01:07:06.536 INFO:tasks.workunit.client.1.smithi142.stdout:1/664: write d6/d12/da0/fe9 [320630,51181] 0 2024-06-15T01:07:06.538 INFO:tasks.workunit.client.0.smithi137.stdout:3/618: rmdir d0/d2/d22/d24/d28/d2e 39 2024-06-15T01:07:06.539 INFO:tasks.workunit.client.0.smithi137.stdout:3/619: dread - d0/d56/d8f/fa8 zero size 2024-06-15T01:07:06.543 INFO:tasks.workunit.client.0.smithi137.stdout:3/620: chown d0/d2/d8/d67/d7e/lc3 24236 1 2024-06-15T01:07:06.543 INFO:tasks.workunit.client.1.smithi142.stdout:5/586: write d2/f14 [643693,64845] 0 2024-06-15T01:07:06.545 INFO:tasks.workunit.client.0.smithi137.stdout:3/621: fdatasync d0/d2/d23/f73 0 2024-06-15T01:07:06.546 INFO:tasks.workunit.client.1.smithi142.stdout:8/669: rename d0/d17/d40/d5e/fcd to d0/d25/d53/d9c/fce 0 2024-06-15T01:07:06.548 INFO:tasks.workunit.client.1.smithi142.stdout:3/721: write d0/d3/f3d [1731791,64878] 0 2024-06-15T01:07:06.548 INFO:tasks.workunit.client.1.smithi142.stdout:0/638: dwrite d3/d39/d9a/d9d/f9e [4194304,4194304] 0 2024-06-15T01:07:06.549 INFO:tasks.workunit.client.1.smithi142.stdout:1/665: creat d6/d12/da0/ff1 x:0 0 0 2024-06-15T01:07:06.549 INFO:tasks.workunit.client.0.smithi137.stdout:3/622: fdatasync d0/d13/f20 0 2024-06-15T01:07:06.551 INFO:tasks.workunit.client.1.smithi142.stdout:2/610: creat d8/df/d10/d94/d25/d22/fbd x:0 0 0 2024-06-15T01:07:06.553 INFO:tasks.workunit.client.1.smithi142.stdout:2/611: readlink d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/lab 0 2024-06-15T01:07:06.555 INFO:tasks.workunit.client.0.smithi137.stdout:3/623: rename d0/f1a to d0/d56/fca 0 2024-06-15T01:07:06.558 INFO:tasks.workunit.client.0.smithi137.stdout:3/624: creat d0/d42/fcb x:0 0 0 2024-06-15T01:07:06.558 INFO:tasks.workunit.client.1.smithi142.stdout:5/587: rmdir d2/d3e/d31/d7e 39 2024-06-15T01:07:06.559 INFO:tasks.workunit.client.0.smithi137.stdout:3/625: truncate d0/d56/d8f/d9c/d9f/fa5 923364 0 2024-06-15T01:07:06.561 INFO:tasks.workunit.client.0.smithi137.stdout:3/626: mkdir d0/d13/d37/dcc 0 2024-06-15T01:07:06.562 INFO:tasks.workunit.client.1.smithi142.stdout:8/670: creat d0/d17/d40/d6e/fcf x:0 0 0 2024-06-15T01:07:06.562 INFO:tasks.workunit.client.1.smithi142.stdout:6/627: write d1a/d39/d42/d4c/d58/f7f [394486,78539] 0 2024-06-15T01:07:06.563 INFO:tasks.workunit.client.0.smithi137.stdout:3/627: write d0/d2f/fb8 [605318,129083] 0 2024-06-15T01:07:06.563 INFO:tasks.workunit.client.1.smithi142.stdout:3/722: dread d0/d3/da/d71/d55/f7a [0,4194304] 0 2024-06-15T01:07:06.563 INFO:tasks.workunit.client.1.smithi142.stdout:1/666: rmdir d6/d12/d80/dae 39 2024-06-15T01:07:06.564 INFO:tasks.workunit.client.1.smithi142.stdout:3/723: chown d0/d1b/d6b/dbd 7840535 1 2024-06-15T01:07:06.564 INFO:tasks.workunit.client.1.smithi142.stdout:1/667: fsync d6/d25/fe2 0 2024-06-15T01:07:06.565 INFO:tasks.workunit.client.1.smithi142.stdout:0/639: creat d3/d39/d9a/d9d/dcd/fe4 x:0 0 0 2024-06-15T01:07:06.565 INFO:tasks.workunit.client.1.smithi142.stdout:4/710: link d5/d1c/d6f/dc7/c4d d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/df5/cf8 0 2024-06-15T01:07:06.567 INFO:tasks.workunit.client.1.smithi142.stdout:9/586: rmdir d6/dc/dd/dca 0 2024-06-15T01:07:06.568 INFO:tasks.workunit.client.0.smithi137.stdout:3/628: rename d0/d42/c4d to d0/d2/daf/ccd 0 2024-06-15T01:07:06.568 INFO:tasks.workunit.client.1.smithi142.stdout:5/588: chown d2/d18/f63 171 1 2024-06-15T01:07:06.569 INFO:tasks.workunit.client.1.smithi142.stdout:8/671: mkdir d0/d25/d5a/d84/dd0 0 2024-06-15T01:07:06.572 INFO:tasks.workunit.client.1.smithi142.stdout:1/668: dread d6/d46/dc3/f4b [0,4194304] 0 2024-06-15T01:07:06.578 INFO:tasks.workunit.client.0.smithi137.stdout:3/629: dwrite d0/d2/d22/d24/d28/fad [0,4194304] 0 2024-06-15T01:07:06.578 INFO:tasks.workunit.client.0.smithi137.stdout:3/630: write d0/d2f/f69 [650003,55010] 0 2024-06-15T01:07:06.579 INFO:tasks.workunit.client.1.smithi142.stdout:8/672: dwrite d0/d25/d53/d9c/fbd [0,4194304] 0 2024-06-15T01:07:06.580 INFO:tasks.workunit.client.1.smithi142.stdout:0/640: symlink d3/d39/d3e/d81/d1e/d98/le5 0 2024-06-15T01:07:06.582 INFO:tasks.workunit.client.0.smithi137.stdout:3/631: creat d0/d2/d22/d63/d76/fce x:0 0 0 2024-06-15T01:07:06.585 INFO:tasks.workunit.client.0.smithi137.stdout:3/632: creat d0/d56/d8f/fcf x:0 0 0 2024-06-15T01:07:06.585 INFO:tasks.workunit.client.1.smithi142.stdout:4/711: rmdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6 39 2024-06-15T01:07:06.587 INFO:tasks.workunit.client.0.smithi137.stdout:3/633: mknod d0/d2f/cd0 0 2024-06-15T01:07:06.589 INFO:tasks.workunit.client.1.smithi142.stdout:6/628: truncate d1a/d39/f3f 1380504 0 2024-06-15T01:07:06.590 INFO:tasks.workunit.client.1.smithi142.stdout:3/724: mknod d0/d1f/d20/d5d/d60/dd0/cd8 0 2024-06-15T01:07:06.592 INFO:tasks.workunit.client.1.smithi142.stdout:1/669: mkdir d6/d12/d1c/d2c/d93/db9/df2 0 2024-06-15T01:07:06.594 INFO:tasks.workunit.client.1.smithi142.stdout:0/641: dwrite d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [0,4194304] 0 2024-06-15T01:07:06.594 INFO:tasks.workunit.client.1.smithi142.stdout:7/666: dwrite d2/d8/dd/d14/d2c/f62 [0,4194304] 0 2024-06-15T01:07:06.595 INFO:tasks.workunit.client.1.smithi142.stdout:1/670: read - d6/d25/fe2 zero size 2024-06-15T01:07:06.595 INFO:tasks.workunit.client.0.smithi137.stdout:3/634: dwrite d0/d56/d8f/fcf [0,4194304] 0 2024-06-15T01:07:06.596 INFO:tasks.workunit.client.1.smithi142.stdout:3/725: write d0/d1f/d20/d5d/d60/fce [1019338,105973] 0 2024-06-15T01:07:06.597 INFO:tasks.workunit.client.1.smithi142.stdout:4/712: creat d5/d1c/d6f/ff9 x:0 0 0 2024-06-15T01:07:06.597 INFO:tasks.workunit.client.1.smithi142.stdout:5/589: mknod d2/de/d19/d5f/d68/d76/cb8 0 2024-06-15T01:07:06.598 INFO:tasks.workunit.client.1.smithi142.stdout:1/671: truncate d6/f90 1381972 0 2024-06-15T01:07:06.599 INFO:tasks.workunit.client.1.smithi142.stdout:7/667: chown d2/d8/dd/d14/d2c/d37/d6b/d80/f36 2648 1 2024-06-15T01:07:06.601 INFO:tasks.workunit.client.1.smithi142.stdout:5/590: chown d2/de/l4b 2403 1 2024-06-15T01:07:06.603 INFO:tasks.workunit.client.1.smithi142.stdout:0/642: mkdir d3/d39/d9a/d9d/de6 0 2024-06-15T01:07:06.604 INFO:tasks.workunit.client.1.smithi142.stdout:9/587: dread d6/dc/d1b/f76 [0,4194304] 0 2024-06-15T01:07:06.608 INFO:tasks.workunit.client.1.smithi142.stdout:3/726: chown d0/d1b/d9e 58 1 2024-06-15T01:07:06.608 INFO:tasks.workunit.client.1.smithi142.stdout:9/588: dread - d6/dc/f61 zero size 2024-06-15T01:07:06.609 INFO:tasks.workunit.client.1.smithi142.stdout:7/668: dwrite d2/f91 [0,4194304] 0 2024-06-15T01:07:06.613 INFO:tasks.workunit.client.1.smithi142.stdout:6/629: getdents d1a/d3c/db0 0 2024-06-15T01:07:06.614 INFO:tasks.workunit.client.1.smithi142.stdout:2/612: truncate d8/f16 1994966 0 2024-06-15T01:07:06.614 INFO:tasks.workunit.client.1.smithi142.stdout:7/669: chown d2/d6/d22/d3c/l6f 55744 1 2024-06-15T01:07:06.616 INFO:tasks.workunit.client.1.smithi142.stdout:9/589: truncate d6/dc/d4e/f94 5239671 0 2024-06-15T01:07:06.617 INFO:tasks.workunit.client.1.smithi142.stdout:6/630: symlink d1a/d39/d3a/dd2/ldd 0 2024-06-15T01:07:06.618 INFO:tasks.workunit.client.1.smithi142.stdout:0/643: dwrite d3/d39/d3e/d81/d4c/d61/f77 [0,4194304] 0 2024-06-15T01:07:06.620 INFO:tasks.workunit.client.1.smithi142.stdout:3/727: dread d0/d1f/f38 [0,4194304] 0 2024-06-15T01:07:06.620 INFO:tasks.workunit.client.1.smithi142.stdout:0/644: fsync d3/d39/d3e/d81/d44/fa7 0 2024-06-15T01:07:06.621 INFO:tasks.workunit.client.1.smithi142.stdout:6/631: dread - d1a/d2d/d3b/dab/fcb zero size 2024-06-15T01:07:06.623 INFO:tasks.workunit.client.0.smithi137.stdout:3/635: truncate d0/d2/d23/f87 1202958 0 2024-06-15T01:07:06.625 INFO:tasks.workunit.client.1.smithi142.stdout:8/673: write d0/f5f [600787,121217] 0 2024-06-15T01:07:06.626 INFO:tasks.workunit.client.1.smithi142.stdout:2/613: fsync d8/df/d1f/f3e 0 2024-06-15T01:07:06.628 INFO:tasks.workunit.client.1.smithi142.stdout:5/591: dread d2/f21 [0,4194304] 0 2024-06-15T01:07:06.628 INFO:tasks.workunit.client.0.smithi137.stdout:3/636: mknod d0/d13/d37/cd1 0 2024-06-15T01:07:06.628 INFO:tasks.workunit.client.1.smithi142.stdout:3/728: fsync d0/d3/da/d71/d55/f7a 0 2024-06-15T01:07:06.630 INFO:tasks.workunit.client.1.smithi142.stdout:6/632: write d1a/d83/daa/fb6 [824705,45397] 0 2024-06-15T01:07:06.630 INFO:tasks.workunit.client.1.smithi142.stdout:4/713: write d5/d1c/d6f/dc7/d30/d46/d50/d51/f75 [173937,78697] 0 2024-06-15T01:07:06.631 INFO:tasks.workunit.client.1.smithi142.stdout:2/614: dread - d8/d19/f75 zero size 2024-06-15T01:07:06.631 INFO:tasks.workunit.client.1.smithi142.stdout:0/645: truncate d3/d39/d9a/fc9 181017 0 2024-06-15T01:07:06.633 INFO:tasks.workunit.client.0.smithi137.stdout:3/637: getdents d0/d2/d23 0 2024-06-15T01:07:06.633 INFO:tasks.workunit.client.1.smithi142.stdout:8/674: creat d0/d25/d5a/d84/fd1 x:0 0 0 2024-06-15T01:07:06.633 INFO:tasks.workunit.client.1.smithi142.stdout:5/592: chown d2/de/d19/f2a 271 1 2024-06-15T01:07:06.634 INFO:tasks.workunit.client.1.smithi142.stdout:9/590: dwrite d6/dc/f8e [0,4194304] 0 2024-06-15T01:07:06.634 INFO:tasks.workunit.client.0.smithi137.stdout:3/638: write d0/d2/d23/d68/f3c [3443327,130145] 0 2024-06-15T01:07:06.634 INFO:tasks.workunit.client.1.smithi142.stdout:3/729: mkdir d0/d3/da/d18/dd9 0 2024-06-15T01:07:06.635 INFO:tasks.workunit.client.0.smithi137.stdout:3/639: write d0/d77/fc2 [309533,49806] 0 2024-06-15T01:07:06.637 INFO:tasks.workunit.client.1.smithi142.stdout:7/670: dwrite d2/d8/da/d33/d25/d79/fcc [0,4194304] 0 2024-06-15T01:07:06.638 INFO:tasks.workunit.client.0.smithi137.stdout:3/640: mknod d0/d56/d8f/cd2 0 2024-06-15T01:07:06.639 INFO:tasks.workunit.client.0.smithi137.stdout:3/641: truncate d0/d13/fba 572369 0 2024-06-15T01:07:06.640 INFO:tasks.workunit.client.1.smithi142.stdout:9/591: write d6/dc/d1b/fc6 [357560,72159] 0 2024-06-15T01:07:06.640 INFO:tasks.workunit.client.1.smithi142.stdout:8/675: write d0/d25/d5a/f85 [1654130,65097] 0 2024-06-15T01:07:06.643 INFO:tasks.workunit.client.1.smithi142.stdout:7/671: stat d2/d6/d22/d3c 0 2024-06-15T01:07:06.644 INFO:tasks.workunit.client.0.smithi137.stdout:3/642: getdents d0/d2/d22/d63/d76 0 2024-06-15T01:07:06.645 INFO:tasks.workunit.client.0.smithi137.stdout:3/643: chown d0/d2/d8/d67/d7e 22 1 2024-06-15T01:07:06.646 INFO:tasks.workunit.client.1.smithi142.stdout:3/730: creat d0/d1f/d20/d5d/d82/dc9/fda x:0 0 0 2024-06-15T01:07:06.648 INFO:tasks.workunit.client.1.smithi142.stdout:1/672: write d6/d3a/f88 [4172866,31373] 0 2024-06-15T01:07:06.648 INFO:tasks.workunit.client.1.smithi142.stdout:9/592: truncate d6/dc/fd9 604236 0 2024-06-15T01:07:06.648 INFO:tasks.workunit.client.0.smithi137.stdout:3/644: creat d0/d56/d8f/d7b/fd3 x:0 0 0 2024-06-15T01:07:06.650 INFO:tasks.workunit.client.1.smithi142.stdout:8/676: mknod d0/d25/d5a/d84/cd2 0 2024-06-15T01:07:06.651 INFO:tasks.workunit.client.1.smithi142.stdout:7/672: rmdir d2/d8/dd/d14/d2c/d37/d6b 39 2024-06-15T01:07:06.652 INFO:tasks.workunit.client.1.smithi142.stdout:9/593: fdatasync d6/f6a 0 2024-06-15T01:07:06.653 INFO:tasks.workunit.client.1.smithi142.stdout:2/615: dwrite d8/d19/d33/fa2 [0,4194304] 0 2024-06-15T01:07:06.653 INFO:tasks.workunit.client.0.smithi137.stdout:3/645: rename d0/d13/l31 to d0/d13/d37/d83/ld4 0 2024-06-15T01:07:06.659 INFO:tasks.workunit.client.1.smithi142.stdout:1/673: fsync d6/f2a 0 2024-06-15T01:07:06.680 INFO:tasks.workunit.client.0.smithi137.stdout:3/646: write d0/d2/d23/f3f [2832766,102183] 0 2024-06-15T01:07:06.683 INFO:tasks.workunit.client.0.smithi137.stdout:3/647: creat d0/d2/d22/d63/fd5 x:0 0 0 2024-06-15T01:07:06.684 INFO:tasks.workunit.client.0.smithi137.stdout:3/648: chown d0/d2/d22/d24/d5e/f85 500033 1 2024-06-15T01:07:06.689 INFO:tasks.workunit.client.1.smithi142.stdout:7/673: rename d2/d6/d22/d47/l5e to d2/d6/daf/dce/ld5 0 2024-06-15T01:07:06.690 INFO:tasks.workunit.client.1.smithi142.stdout:3/731: symlink d0/d3/da/d71/d55/dae/ldb 0 2024-06-15T01:07:06.692 INFO:tasks.workunit.client.0.smithi137.stdout:3/649: dwrite d0/d2/d23/f73 [0,4194304] 0 2024-06-15T01:07:06.693 INFO:tasks.workunit.client.1.smithi142.stdout:4/714: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f37 996850 0 2024-06-15T01:07:06.693 INFO:tasks.workunit.client.1.smithi142.stdout:0/646: dread d3/d39/d3e/d81/d44/f57 [0,4194304] 0 2024-06-15T01:07:06.693 INFO:tasks.workunit.client.0.smithi137.stdout:3/650: chown d0/d2/d22/d24/l25 2 1 2024-06-15T01:07:06.694 INFO:tasks.workunit.client.1.smithi142.stdout:3/732: readlink d0/d3/da/d71/d55/l56 0 2024-06-15T01:07:06.694 INFO:tasks.workunit.client.0.smithi137.stdout:3/651: write d0/d2/d22/d24/f54 [4732042,103561] 0 2024-06-15T01:07:06.695 INFO:tasks.workunit.client.0.smithi137.stdout:3/652: stat d0/d42/d79 0 2024-06-15T01:07:06.696 INFO:tasks.workunit.client.1.smithi142.stdout:9/594: write d6/dc/d1b/d25/d58/d74/f92 [1002508,93644] 0 2024-06-15T01:07:06.696 INFO:tasks.workunit.client.0.smithi137.stdout:3/653: chown d0/d56/d8f/fa8 566488 1 2024-06-15T01:07:06.697 INFO:tasks.workunit.client.0.smithi137.stdout:3/654: chown d0/d2/d22/d24/d28/fb4 4840059 1 2024-06-15T01:07:06.699 INFO:tasks.workunit.client.1.smithi142.stdout:1/674: truncate d6/d12/d3c/d63/d8d/fb8 860023 0 2024-06-15T01:07:06.699 INFO:tasks.workunit.client.0.smithi137.stdout:3/655: unlink d0/d13/d37/f5f 0 2024-06-15T01:07:06.700 INFO:tasks.workunit.client.0.smithi137.stdout:3/656: read - d0/d2/d22/d63/fd5 zero size 2024-06-15T01:07:06.701 INFO:tasks.workunit.client.1.smithi142.stdout:6/633: dwrite d1a/f3e [0,4194304] 0 2024-06-15T01:07:06.701 INFO:tasks.workunit.client.1.smithi142.stdout:2/616: link d8/df/d10/d94/d25/d22/d43/d52/fb3 d8/df/d10/d94/d25/d22/d43/d52/d7a/fbe 0 2024-06-15T01:07:06.702 INFO:tasks.workunit.client.1.smithi142.stdout:3/733: mknod d0/d3/d15/cdc 0 2024-06-15T01:07:06.703 INFO:tasks.workunit.client.1.smithi142.stdout:5/593: dwrite d2/de/d19/d5f/d68/d76/f81 [0,4194304] 0 2024-06-15T01:07:06.706 INFO:tasks.workunit.client.1.smithi142.stdout:1/675: stat d6/d12/d3c/d63/d8d/d97/dd3/caf 0 2024-06-15T01:07:06.706 INFO:tasks.workunit.client.1.smithi142.stdout:6/634: write d1a/d3c/db0/fc0 [118086,96099] 0 2024-06-15T01:07:06.707 INFO:tasks.workunit.client.1.smithi142.stdout:1/676: stat f5 0 2024-06-15T01:07:06.707 INFO:tasks.workunit.client.1.smithi142.stdout:3/734: rmdir d0/d1f 39 2024-06-15T01:07:06.709 INFO:tasks.workunit.client.1.smithi142.stdout:6/635: chown d1a/d39/d3a/d51/f63 2877610 1 2024-06-15T01:07:06.709 INFO:tasks.workunit.client.1.smithi142.stdout:9/595: mkdir d6/dc/d1b/d6f/de1 0 2024-06-15T01:07:06.710 INFO:tasks.workunit.client.1.smithi142.stdout:2/617: dread d8/fe [0,4194304] 0 2024-06-15T01:07:06.710 INFO:tasks.workunit.client.1.smithi142.stdout:3/735: write d0/d3/da/f7b [1207132,41805] 0 2024-06-15T01:07:06.713 INFO:tasks.workunit.client.1.smithi142.stdout:0/647: rename d3/d39/d3e/d81/c1a to d3/d39/d3e/d81/d4c/d61/ce7 0 2024-06-15T01:07:06.713 INFO:tasks.workunit.client.1.smithi142.stdout:1/677: mknod d6/d12/d1c/d2c/d47/cf3 0 2024-06-15T01:07:06.719 INFO:tasks.workunit.client.0.smithi137.stdout:3/657: dread d0/d2/d22/d63/d76/f7d [0,4194304] 0 2024-06-15T01:07:06.721 INFO:tasks.workunit.client.1.smithi142.stdout:3/736: creat d0/d95/fdd x:0 0 0 2024-06-15T01:07:06.722 INFO:tasks.workunit.client.0.smithi137.stdout:3/658: symlink d0/d2/d23/ld6 0 2024-06-15T01:07:06.723 INFO:tasks.workunit.client.1.smithi142.stdout:9/596: mkdir d6/dc/d1b/d25/d58/de2 0 2024-06-15T01:07:06.726 INFO:tasks.workunit.client.1.smithi142.stdout:0/648: truncate d3/d39/d3e/d81/f8d 572401 0 2024-06-15T01:07:06.726 INFO:tasks.workunit.client.1.smithi142.stdout:7/674: truncate d2/d8/dd/d14/fb8 330983 0 2024-06-15T01:07:06.731 INFO:tasks.workunit.client.1.smithi142.stdout:1/678: symlink d6/d12/d3c/d63/d8d/d97/dc6/lf4 0 2024-06-15T01:07:06.731 INFO:tasks.workunit.client.1.smithi142.stdout:4/715: dread d5/d1c/f34 [0,4194304] 0 2024-06-15T01:07:06.732 INFO:tasks.workunit.client.1.smithi142.stdout:0/649: symlink d3/d39/d3e/d81/d1b/dae/le8 0 2024-06-15T01:07:06.733 INFO:tasks.workunit.client.1.smithi142.stdout:0/650: chown d3/d39/d3e/d81/f7e 30 1 2024-06-15T01:07:06.733 INFO:tasks.workunit.client.1.smithi142.stdout:7/675: creat d2/d8/da8/fd6 x:0 0 0 2024-06-15T01:07:06.734 INFO:tasks.workunit.client.1.smithi142.stdout:3/737: creat d0/d1f/d20/d59/d61/d96/fde x:0 0 0 2024-06-15T01:07:06.734 INFO:tasks.workunit.client.1.smithi142.stdout:7/676: stat d2/d8/dd/d14/d2c/l78 0 2024-06-15T01:07:06.738 INFO:tasks.workunit.client.1.smithi142.stdout:6/636: dread d1a/d39/d42/d4c/d58/f6a [0,4194304] 0 2024-06-15T01:07:06.738 INFO:tasks.workunit.client.1.smithi142.stdout:4/716: creat d5/d9/d23/d32/ffa x:0 0 0 2024-06-15T01:07:06.739 INFO:tasks.workunit.client.1.smithi142.stdout:0/651: mknod d3/d39/d3e/d81/ce9 0 2024-06-15T01:07:06.741 INFO:tasks.workunit.client.1.smithi142.stdout:3/738: symlink d0/d1b/d6b/ldf 0 2024-06-15T01:07:06.742 INFO:tasks.workunit.client.1.smithi142.stdout:5/594: dwrite d2/fb7 [0,4194304] 0 2024-06-15T01:07:06.743 INFO:tasks.workunit.client.1.smithi142.stdout:2/618: link d8/df/d10/d94/d25/d22/d43/d52/l9e d8/df/d10/d94/d25/lbf 0 2024-06-15T01:07:06.743 INFO:tasks.workunit.client.1.smithi142.stdout:4/717: fsync d5/d1c/d6f/dc7/de0/ff7 0 2024-06-15T01:07:06.744 INFO:tasks.workunit.client.1.smithi142.stdout:0/652: chown d3/d39/d3e/d81/d44/d7c/faa 206069 1 2024-06-15T01:07:06.744 INFO:tasks.workunit.client.1.smithi142.stdout:3/739: chown d0/d3/d15/c78 419 1 2024-06-15T01:07:06.746 INFO:tasks.workunit.client.1.smithi142.stdout:3/740: write d0/d1f/f4b [7149996,94993] 0 2024-06-15T01:07:06.752 INFO:tasks.workunit.client.1.smithi142.stdout:3/741: dwrite d0/d1f/d20/d59/d61/d96/fcc [0,4194304] 0 2024-06-15T01:07:06.753 INFO:tasks.workunit.client.1.smithi142.stdout:1/679: truncate d6/d12/f2f 917487 0 2024-06-15T01:07:06.754 INFO:tasks.workunit.client.1.smithi142.stdout:2/619: mknod d8/d19/cc0 0 2024-06-15T01:07:06.754 INFO:tasks.workunit.client.1.smithi142.stdout:9/597: write d6/d34/fc0 [2503374,33597] 0 2024-06-15T01:07:06.757 INFO:tasks.workunit.client.1.smithi142.stdout:2/620: write d8/df/d10/f87 [651840,67769] 0 2024-06-15T01:07:06.757 INFO:tasks.workunit.client.1.smithi142.stdout:0/653: mkdir d3/d39/d3e/d81/d4c/d61/dea 0 2024-06-15T01:07:06.759 INFO:tasks.workunit.client.1.smithi142.stdout:9/598: write d6/d34/fc0 [2918833,5208] 0 2024-06-15T01:07:06.760 INFO:tasks.workunit.client.1.smithi142.stdout:5/595: rename d2/d3e/d31/db4 to d2/de/d27/d86/db9 0 2024-06-15T01:07:06.761 INFO:tasks.workunit.client.1.smithi142.stdout:2/621: truncate d8/df/d10/d94/d25/f53 2029693 0 2024-06-15T01:07:06.761 INFO:tasks.workunit.client.1.smithi142.stdout:3/742: rmdir d0 39 2024-06-15T01:07:06.762 INFO:tasks.workunit.client.1.smithi142.stdout:9/599: truncate d6/dc/d1b/d25/d35/fd1 414850 0 2024-06-15T01:07:06.763 INFO:tasks.workunit.client.1.smithi142.stdout:1/680: symlink d6/d12/d1c/d2c/d93/lf5 0 2024-06-15T01:07:06.767 INFO:tasks.workunit.client.1.smithi142.stdout:4/718: dread d5/d9/d18/f69 [0,4194304] 0 2024-06-15T01:07:06.767 INFO:tasks.workunit.client.1.smithi142.stdout:0/654: unlink d3/d39/d3e/d81/d1e/fdd 0 2024-06-15T01:07:06.768 INFO:tasks.workunit.client.1.smithi142.stdout:0/655: stat d3/d39/d3e/d81/d4c/d61/cb0 0 2024-06-15T01:07:06.768 INFO:tasks.workunit.client.1.smithi142.stdout:0/656: fsync d3/d39/d9a/fa8 0 2024-06-15T01:07:06.770 INFO:tasks.workunit.client.1.smithi142.stdout:3/743: write d0/d1f/f6f [2528159,104351] 0 2024-06-15T01:07:06.770 INFO:tasks.workunit.client.1.smithi142.stdout:2/622: mkdir d8/df/d10/d94/d25/d22/d43/d52/dc1 0 2024-06-15T01:07:06.770 INFO:tasks.workunit.client.1.smithi142.stdout:0/657: read d3/d39/d3e/fa5 [3220516,18856] 0 2024-06-15T01:07:06.771 INFO:tasks.workunit.client.1.smithi142.stdout:9/600: symlink d6/dc/dd/le3 0 2024-06-15T01:07:06.772 INFO:tasks.workunit.client.1.smithi142.stdout:2/623: write d8/df/d10/d94/d25/d1a/d57/f69 [887803,109137] 0 2024-06-15T01:07:06.773 INFO:tasks.workunit.client.1.smithi142.stdout:1/681: mknod d6/d3a/cf6 0 2024-06-15T01:07:06.774 INFO:tasks.workunit.client.1.smithi142.stdout:6/637: truncate d1a/d39/d42/d4c/d58/d87/f93 742955 0 2024-06-15T01:07:06.774 INFO:tasks.workunit.client.1.smithi142.stdout:6/638: read - d1a/d39/d42/f74 zero size 2024-06-15T01:07:06.775 INFO:tasks.workunit.client.1.smithi142.stdout:6/639: readlink d1a/l29 0 2024-06-15T01:07:06.776 INFO:tasks.workunit.client.1.smithi142.stdout:4/719: rename d5/d9/da/f7f to d5/d9/d23/d68/ffb 0 2024-06-15T01:07:06.782 INFO:tasks.workunit.client.1.smithi142.stdout:3/744: fsync d0/d1f/d20/f26 0 2024-06-15T01:07:06.783 INFO:tasks.workunit.client.1.smithi142.stdout:9/601: mknod d6/dc/d1b/d25/d35/ce4 0 2024-06-15T01:07:06.784 INFO:tasks.workunit.client.1.smithi142.stdout:4/720: dread d5/d9/d23/d32/f8f [0,4194304] 0 2024-06-15T01:07:06.785 INFO:tasks.workunit.client.1.smithi142.stdout:2/624: unlink d8/df/d10/d94/f26 0 2024-06-15T01:07:06.785 INFO:tasks.workunit.client.1.smithi142.stdout:4/721: chown d5/d9/d90/daf/ce4 14872933 1 2024-06-15T01:07:06.786 INFO:tasks.workunit.client.1.smithi142.stdout:1/682: mknod d6/d4d/da2/cf7 0 2024-06-15T01:07:06.787 INFO:tasks.workunit.client.1.smithi142.stdout:1/683: readlink d6/d12/d1c/d2c/la7 0 2024-06-15T01:07:06.788 INFO:tasks.workunit.client.1.smithi142.stdout:5/596: creat d2/d18/fba x:0 0 0 2024-06-15T01:07:06.790 INFO:tasks.workunit.client.1.smithi142.stdout:5/597: read d2/d18/f37 [2524314,33409] 0 2024-06-15T01:07:06.792 INFO:tasks.workunit.client.1.smithi142.stdout:3/745: dread d0/d1f/f27 [0,4194304] 0 2024-06-15T01:07:06.793 INFO:tasks.workunit.client.1.smithi142.stdout:3/746: write d0/d3/f90 [1276957,70566] 0 2024-06-15T01:07:06.794 INFO:tasks.workunit.client.1.smithi142.stdout:0/658: symlink d3/d39/d3e/d81/d1e/leb 0 2024-06-15T01:07:06.794 INFO:tasks.workunit.client.1.smithi142.stdout:3/747: write d0/d1f/d20/d5d/fbb [1569242,20401] 0 2024-06-15T01:07:06.797 INFO:tasks.workunit.client.1.smithi142.stdout:3/748: chown d0/d1f/d20/d5d/c7d 61 1 2024-06-15T01:07:06.800 INFO:tasks.workunit.client.1.smithi142.stdout:1/684: fsync d6/d12/da0/db0/f73 0 2024-06-15T01:07:06.801 INFO:tasks.workunit.client.1.smithi142.stdout:8/677: sync 2024-06-15T01:07:06.803 INFO:tasks.workunit.client.1.smithi142.stdout:6/640: rename d1a/d39/d42/d4c/c5e to d1a/d2d/d3b/dab/dce/cde 0 2024-06-15T01:07:06.804 INFO:tasks.workunit.client.1.smithi142.stdout:6/641: write d1a/d2d/d3b/faf [210975,121560] 0 2024-06-15T01:07:06.808 INFO:tasks.workunit.client.1.smithi142.stdout:6/642: dread d1a/d3c/fc8 [0,4194304] 0 2024-06-15T01:07:06.809 INFO:tasks.workunit.client.1.smithi142.stdout:6/643: read d1a/d2d/d68/da0/fa5 [721916,112590] 0 2024-06-15T01:07:06.810 INFO:tasks.workunit.client.1.smithi142.stdout:9/602: mkdir d6/dc/d1b/d6f/de5 0 2024-06-15T01:07:06.811 INFO:tasks.workunit.client.1.smithi142.stdout:4/722: mknod d5/d1c/d6f/dc7/d30/cfc 0 2024-06-15T01:07:06.812 INFO:tasks.workunit.client.1.smithi142.stdout:6/644: dread d1a/d2d/fb8 [0,4194304] 0 2024-06-15T01:07:06.813 INFO:tasks.workunit.client.1.smithi142.stdout:0/659: dread d3/d39/d3e/d81/d44/d7c/d22/d59/f9c [0,4194304] 0 2024-06-15T01:07:06.814 INFO:tasks.workunit.client.1.smithi142.stdout:0/660: write d3/d39/d3e/d81/d1e/d98/fd9 [473402,95873] 0 2024-06-15T01:07:06.815 INFO:tasks.workunit.client.1.smithi142.stdout:1/685: unlink d6/l19 0 2024-06-15T01:07:06.818 INFO:tasks.workunit.client.1.smithi142.stdout:2/625: rename d8/d19/d33/l47 to d8/df/d10/d94/d25/d1a/d63/lc2 0 2024-06-15T01:07:06.819 INFO:tasks.workunit.client.1.smithi142.stdout:6/645: dwrite d1a/d39/d3a/d51/fa3 [0,4194304] 0 2024-06-15T01:07:06.820 INFO:tasks.workunit.client.1.smithi142.stdout:5/598: creat d2/de/d27/db2/fbb x:0 0 0 2024-06-15T01:07:06.841 INFO:tasks.workunit.client.0.smithi137.stdout:7/449: sync 2024-06-15T01:07:06.842 INFO:tasks.workunit.client.0.smithi137.stdout:7/450: chown df/d13/d2e/d1a/f80 1 1 2024-06-15T01:07:06.844 INFO:tasks.workunit.client.1.smithi142.stdout:9/603: symlink d6/dc/d1b/le6 0 2024-06-15T01:07:06.845 INFO:tasks.workunit.client.1.smithi142.stdout:4/723: fdatasync d5/f10 0 2024-06-15T01:07:06.846 INFO:tasks.workunit.client.1.smithi142.stdout:8/678: symlink d0/d25/d53/da7/dcb/ld3 0 2024-06-15T01:07:06.846 INFO:tasks.workunit.client.1.smithi142.stdout:1/686: dwrite d6/d12/d80/d9b/fcb [0,4194304] 0 2024-06-15T01:07:06.848 INFO:tasks.workunit.client.1.smithi142.stdout:6/646: symlink d1a/d3c/ldf 0 2024-06-15T01:07:06.848 INFO:tasks.workunit.client.1.smithi142.stdout:1/687: chown d6/d25/lea 0 1 2024-06-15T01:07:06.850 INFO:tasks.workunit.client.1.smithi142.stdout:5/599: mkdir d2/d18/d2e/dae/dbc 0 2024-06-15T01:07:06.852 INFO:tasks.workunit.client.1.smithi142.stdout:6/647: fdatasync d1a/d83/f7c 0 2024-06-15T01:07:06.852 INFO:tasks.workunit.client.1.smithi142.stdout:9/604: unlink d6/dc/dd/c11 0 2024-06-15T01:07:06.853 INFO:tasks.workunit.client.1.smithi142.stdout:5/600: readlink d2/de/d19/d5f/lb5 0 2024-06-15T01:07:06.853 INFO:tasks.workunit.client.1.smithi142.stdout:8/679: creat d0/fd4 x:0 0 0 2024-06-15T01:07:06.853 INFO:tasks.workunit.client.1.smithi142.stdout:6/648: chown d1a/f1f 26 1 2024-06-15T01:07:06.853 INFO:tasks.workunit.client.1.smithi142.stdout:3/749: getdents d0/d1f/d20/d5d/d82 0 2024-06-15T01:07:06.854 INFO:tasks.workunit.client.1.smithi142.stdout:4/724: rename d5/d9/d1a/c87 to d5/d1c/d6f/dc7/d30/d46/cfd 0 2024-06-15T01:07:06.857 INFO:tasks.workunit.client.1.smithi142.stdout:5/601: readlink d2/d18/l42 0 2024-06-15T01:07:06.858 INFO:tasks.workunit.client.1.smithi142.stdout:4/725: dread - d5/d1c/d6f/dc7/d30/d46/fc6 zero size 2024-06-15T01:07:06.858 INFO:tasks.workunit.client.1.smithi142.stdout:1/688: creat d6/d12/da0/db0/de1/ff8 x:0 0 0 2024-06-15T01:07:06.860 INFO:tasks.workunit.client.1.smithi142.stdout:6/649: write d1a/d3c/fc8 [822126,57486] 0 2024-06-15T01:07:06.860 INFO:tasks.workunit.client.1.smithi142.stdout:8/680: truncate d0/d17/f56 1454194 0 2024-06-15T01:07:06.861 INFO:tasks.workunit.client.1.smithi142.stdout:2/626: truncate d8/df/d10/d94/d25/da9/f6f 632445 0 2024-06-15T01:07:06.864 INFO:tasks.workunit.client.1.smithi142.stdout:4/726: dread - d5/d9/da/d92/d98/fac zero size 2024-06-15T01:07:06.865 INFO:tasks.workunit.client.1.smithi142.stdout:1/689: write d6/d12/d1c/d2c/f59 [1869570,61908] 0 2024-06-15T01:07:06.867 INFO:tasks.workunit.client.1.smithi142.stdout:5/602: dread d2/d3e/d31/f3c [0,4194304] 0 2024-06-15T01:07:06.868 INFO:tasks.workunit.client.1.smithi142.stdout:3/750: dwrite d0/d1f/d20/d59/fa6 [0,4194304] 0 2024-06-15T01:07:06.868 INFO:tasks.workunit.client.1.smithi142.stdout:2/627: creat d8/df/d10/d94/db5/fc3 x:0 0 0 2024-06-15T01:07:06.869 INFO:tasks.workunit.client.1.smithi142.stdout:1/690: stat d6/d12/d1c/fe0 0 2024-06-15T01:07:06.870 INFO:tasks.workunit.client.1.smithi142.stdout:9/605: dwrite d6/dc/d4e/f94 [0,4194304] 0 2024-06-15T01:07:06.878 INFO:tasks.workunit.client.1.smithi142.stdout:6/650: rename d1a/d39/d3a/f5f to d1a/d39/d42/d4c/d58/d87/fe0 0 2024-06-15T01:07:06.879 INFO:tasks.workunit.client.1.smithi142.stdout:8/681: getdents d0/d25/d5a/d8f/d98 0 2024-06-15T01:07:06.882 INFO:tasks.workunit.client.1.smithi142.stdout:8/682: chown d0/d25/d5a/f78 0 1 2024-06-15T01:07:06.882 INFO:tasks.workunit.client.1.smithi142.stdout:7/677: sync 2024-06-15T01:07:06.884 INFO:tasks.workunit.client.1.smithi142.stdout:4/727: creat d5/d1c/d6f/dc7/de1/ffe x:0 0 0 2024-06-15T01:07:06.886 INFO:tasks.workunit.client.1.smithi142.stdout:5/603: dwrite d2/f14 [0,4194304] 0 2024-06-15T01:07:06.886 INFO:tasks.workunit.client.1.smithi142.stdout:1/691: write d6/d46/f78 [838489,55299] 0 2024-06-15T01:07:06.886 INFO:tasks.workunit.client.1.smithi142.stdout:8/683: rmdir d0/d25/d5a 39 2024-06-15T01:07:06.888 INFO:tasks.workunit.client.1.smithi142.stdout:7/678: unlink d2/d8/dd/dcf/lb3 0 2024-06-15T01:07:06.888 INFO:tasks.workunit.client.1.smithi142.stdout:6/651: symlink d1a/d3c/db0/dd8/le1 0 2024-06-15T01:07:06.890 INFO:tasks.workunit.client.1.smithi142.stdout:3/751: getdents d0 0 2024-06-15T01:07:06.891 INFO:tasks.workunit.client.1.smithi142.stdout:4/728: unlink d5/d1c/d6f/dc7/d30/d46/lf1 0 2024-06-15T01:07:06.892 INFO:tasks.workunit.client.1.smithi142.stdout:9/606: getdents d6/dc/d1b/d25/d35 0 2024-06-15T01:07:06.893 INFO:tasks.workunit.client.1.smithi142.stdout:1/692: symlink d6/d12/d80/d9b/db2/lf9 0 2024-06-15T01:07:06.895 INFO:tasks.workunit.client.1.smithi142.stdout:6/652: truncate d1a/d2d/d80/d8e/f9d 199168 0 2024-06-15T01:07:06.899 INFO:tasks.workunit.client.1.smithi142.stdout:4/729: write d5/d9/d90/daf/dcb/fdf [958498,18541] 0 2024-06-15T01:07:06.899 INFO:tasks.workunit.client.1.smithi142.stdout:9/607: write d6/dc/f8e [3013949,111971] 0 2024-06-15T01:07:06.900 INFO:tasks.workunit.client.1.smithi142.stdout:2/628: link d8/df/d10/d94/d25/da9/l61 d8/df/d10/d94/d25/d22/d43/lc4 0 2024-06-15T01:07:06.900 INFO:tasks.workunit.client.1.smithi142.stdout:3/752: symlink d0/d1b/le0 0 2024-06-15T01:07:06.901 INFO:tasks.workunit.client.1.smithi142.stdout:8/684: getdents d0/d25/d66 0 2024-06-15T01:07:06.902 INFO:tasks.workunit.client.1.smithi142.stdout:5/604: dwrite d2/d18/f99 [0,4194304] 0 2024-06-15T01:07:06.904 INFO:tasks.workunit.client.1.smithi142.stdout:2/629: write d8/df/d10/f3b [5104610,49441] 0 2024-06-15T01:07:06.905 INFO:tasks.workunit.client.1.smithi142.stdout:4/730: read d5/d9/f1f [50070,91715] 0 2024-06-15T01:07:06.905 INFO:tasks.workunit.client.1.smithi142.stdout:1/693: fdatasync d6/d12/d1c/d2c/d93/dba/feb 0 2024-06-15T01:07:06.905 INFO:tasks.workunit.client.1.smithi142.stdout:3/753: chown d0/d1f/d20/d5d/d60/fce 1630808917 1 2024-06-15T01:07:06.907 INFO:tasks.workunit.client.1.smithi142.stdout:6/653: link f7 d1a/d39/d3a/dd2/fe2 0 2024-06-15T01:07:06.909 INFO:tasks.workunit.client.1.smithi142.stdout:7/679: write d2/d8/dd/d14/f4b [941306,24183] 0 2024-06-15T01:07:06.913 INFO:tasks.workunit.client.1.smithi142.stdout:2/630: readlink d8/df/d10/d94/d25/d22/d43/d52/l9e 0 2024-06-15T01:07:06.916 INFO:tasks.workunit.client.1.smithi142.stdout:1/694: creat d6/d12/da0/db0/de1/ffa x:0 0 0 2024-06-15T01:07:06.917 INFO:tasks.workunit.client.1.smithi142.stdout:8/685: dwrite d0/f43 [0,4194304] 0 2024-06-15T01:07:06.919 INFO:tasks.workunit.client.1.smithi142.stdout:2/631: mknod d8/df/d1f/cc5 0 2024-06-15T01:07:06.922 INFO:tasks.workunit.client.1.smithi142.stdout:2/632: write d8/df/d10/d1c/fad [582809,1430] 0 2024-06-15T01:07:06.925 INFO:tasks.workunit.client.1.smithi142.stdout:4/731: write d5/d1c/d6f/dc7/d30/d46/d50/f5e [4044473,49207] 0 2024-06-15T01:07:06.925 INFO:tasks.workunit.client.1.smithi142.stdout:7/680: creat d2/d8/da/d33/fd7 x:0 0 0 2024-06-15T01:07:06.926 INFO:tasks.workunit.client.1.smithi142.stdout:9/608: truncate d6/dc/d1b/d25/d58/f64 4153727 0 2024-06-15T01:07:06.928 INFO:tasks.workunit.client.1.smithi142.stdout:6/654: rename d1a/d39/d42/d4c/ld3 to d1a/d39/d42/d4c/le3 0 2024-06-15T01:07:06.928 INFO:tasks.workunit.client.1.smithi142.stdout:2/633: creat d8/df/d10/d94/d25/d76/fc6 x:0 0 0 2024-06-15T01:07:06.930 INFO:tasks.workunit.client.1.smithi142.stdout:1/695: getdents d6/d12/d3c/d63/d8d/d97/dc6 0 2024-06-15T01:07:06.932 INFO:tasks.workunit.client.1.smithi142.stdout:7/681: write d2/d8/da/d33/f52 [1056696,80163] 0 2024-06-15T01:07:06.932 INFO:tasks.workunit.client.1.smithi142.stdout:3/754: dwrite d0/d3/da/d18/f28 [0,4194304] 0 2024-06-15T01:07:06.933 INFO:tasks.workunit.client.1.smithi142.stdout:4/732: rename d5/d9/d23/fd9 to d5/d9/d90/fff 0 2024-06-15T01:07:06.934 INFO:tasks.workunit.client.1.smithi142.stdout:1/696: fsync d6/d46/dc3/fca 0 2024-06-15T01:07:06.937 INFO:tasks.workunit.client.1.smithi142.stdout:7/682: dread - d2/d8/dd/dcf/fbb zero size 2024-06-15T01:07:06.938 INFO:tasks.workunit.client.1.smithi142.stdout:9/609: dread d6/dc/dd/f4f [0,4194304] 0 2024-06-15T01:07:06.938 INFO:tasks.workunit.client.1.smithi142.stdout:6/655: dread d1a/d2d/d2e/f31 [0,4194304] 0 2024-06-15T01:07:06.940 INFO:tasks.workunit.client.1.smithi142.stdout:9/610: stat d6/dc/dd/c85 0 2024-06-15T01:07:06.942 INFO:tasks.workunit.client.1.smithi142.stdout:4/733: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/dbc 1754 1 2024-06-15T01:07:06.942 INFO:tasks.workunit.client.1.smithi142.stdout:6/656: fdatasync d1a/d2d/f33 0 2024-06-15T01:07:06.945 INFO:tasks.workunit.client.1.smithi142.stdout:2/634: link d8/df/d10/d94/d25/c3c d8/df/daf/cc7 0 2024-06-15T01:07:06.945 INFO:tasks.workunit.client.1.smithi142.stdout:7/683: mkdir d2/d8/dd8 0 2024-06-15T01:07:06.945 INFO:tasks.workunit.client.1.smithi142.stdout:6/657: mknod d1a/d2d/d68/da0/ce4 0 2024-06-15T01:07:06.947 INFO:tasks.workunit.client.1.smithi142.stdout:2/635: creat d8/d19/fc8 x:0 0 0 2024-06-15T01:07:06.948 INFO:tasks.workunit.client.1.smithi142.stdout:7/684: read - d2/d8/da/d33/d25/d81/fa4 zero size 2024-06-15T01:07:06.949 INFO:tasks.workunit.client.1.smithi142.stdout:9/611: mkdir d6/dc/d1b/d25/d58/de2/de7 0 2024-06-15T01:07:06.949 INFO:tasks.workunit.client.1.smithi142.stdout:8/686: dread d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:06.950 INFO:tasks.workunit.client.1.smithi142.stdout:4/734: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/l100 0 2024-06-15T01:07:06.952 INFO:tasks.workunit.client.1.smithi142.stdout:6/658: symlink d1a/d39/d42/le5 0 2024-06-15T01:07:06.954 INFO:tasks.workunit.client.1.smithi142.stdout:7/685: write d2/d8/da/d33/d25/d79/fb7 [91863,109033] 0 2024-06-15T01:07:06.956 INFO:tasks.workunit.client.1.smithi142.stdout:0/661: sync 2024-06-15T01:07:06.956 INFO:tasks.workunit.client.1.smithi142.stdout:6/659: truncate d1a/d2d/f33 1976369 0 2024-06-15T01:07:06.958 INFO:tasks.workunit.client.1.smithi142.stdout:9/612: unlink d6/d31/l67 0 2024-06-15T01:07:06.959 INFO:tasks.workunit.client.1.smithi142.stdout:3/755: dread d0/d3/da/d71/d55/f57 [0,4194304] 0 2024-06-15T01:07:06.961 INFO:tasks.workunit.client.1.smithi142.stdout:2/636: dwrite d8/d19/d33/f83 [0,4194304] 0 2024-06-15T01:07:06.963 INFO:tasks.workunit.client.1.smithi142.stdout:8/687: mknod d0/d25/d53/d9c/dbf/cd5 0 2024-06-15T01:07:06.963 INFO:tasks.workunit.client.1.smithi142.stdout:1/697: dwrite d6/d12/d1c/d2c/d47/f5a [0,4194304] 0 2024-06-15T01:07:06.966 INFO:tasks.workunit.client.1.smithi142.stdout:0/662: truncate d3/f48 343988 0 2024-06-15T01:07:06.967 INFO:tasks.workunit.client.1.smithi142.stdout:1/698: write d6/d3a/f99 [2318884,40635] 0 2024-06-15T01:07:06.969 INFO:tasks.workunit.client.1.smithi142.stdout:4/735: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/lf3 to d5/d9/d23/d32/l101 0 2024-06-15T01:07:06.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/637: truncate d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa4 4291324 0 2024-06-15T01:07:06.972 INFO:tasks.workunit.client.1.smithi142.stdout:0/663: write d3/d39/d3e/d81/d4c/d61/f66 [883297,34367] 0 2024-06-15T01:07:06.972 INFO:tasks.workunit.client.1.smithi142.stdout:6/660: truncate d1a/f38 2311868 0 2024-06-15T01:07:06.973 INFO:tasks.workunit.client.1.smithi142.stdout:6/661: dread - d1a/d2d/d80/d8e/fd7 zero size 2024-06-15T01:07:06.976 INFO:tasks.workunit.client.1.smithi142.stdout:9/613: creat d6/dc/d1b/d25/d58/de2/fe8 x:0 0 0 2024-06-15T01:07:06.976 INFO:tasks.workunit.client.0.smithi137.stdout:7/451: write df/d13/d2e/d1a/f68 [2336153,48958] 0 2024-06-15T01:07:06.976 INFO:tasks.workunit.client.1.smithi142.stdout:4/736: dread d5/d9/da/d92/faa [0,4194304] 0 2024-06-15T01:07:06.978 INFO:tasks.workunit.client.1.smithi142.stdout:4/737: truncate d5/d9/d90/daf/dcb/fdf 1366604 0 2024-06-15T01:07:06.979 INFO:tasks.workunit.client.1.smithi142.stdout:1/699: unlink d6/d12/d28/l71 0 2024-06-15T01:07:06.979 INFO:tasks.workunit.client.0.smithi137.stdout:7/452: mkdir df/d13/d2e/d1a/d8c 0 2024-06-15T01:07:06.981 INFO:tasks.workunit.client.1.smithi142.stdout:3/756: rename d0/d1f/l40 to d0/d1b/d6b/dbd/le1 0 2024-06-15T01:07:06.982 INFO:tasks.workunit.client.1.smithi142.stdout:6/662: symlink d1a/d2d/d80/le6 0 2024-06-15T01:07:06.983 INFO:tasks.workunit.client.0.smithi137.stdout:7/453: mknod df/d7d/d1c/d1d/d27/d33/d43/c8d 0 2024-06-15T01:07:06.983 INFO:tasks.workunit.client.1.smithi142.stdout:7/686: creat d2/d8/dd/fd9 x:0 0 0 2024-06-15T01:07:06.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/663: write d1a/d39/d3a/d51/fa3 [3272712,35814] 0 2024-06-15T01:07:06.984 INFO:tasks.workunit.client.0.smithi137.stdout:7/454: fdatasync df/d13/d2e/f4e 0 2024-06-15T01:07:06.984 INFO:tasks.workunit.client.1.smithi142.stdout:9/614: truncate d6/dc/d82/f72 201004 0 2024-06-15T01:07:06.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/664: dread - d1a/d3c/f48 zero size 2024-06-15T01:07:06.985 INFO:tasks.workunit.client.0.smithi137.stdout:7/455: truncate df/d13/d3c/f5e 1143135 0 2024-06-15T01:07:06.986 INFO:tasks.workunit.client.1.smithi142.stdout:9/615: read d6/d34/fa0 [203386,98499] 0 2024-06-15T01:07:06.986 INFO:tasks.workunit.client.1.smithi142.stdout:4/738: unlink d5/d9/da/l31 0 2024-06-15T01:07:06.989 INFO:tasks.workunit.client.1.smithi142.stdout:1/700: mkdir d6/d12/d1c/d2c/dfb 0 2024-06-15T01:07:06.990 INFO:tasks.workunit.client.1.smithi142.stdout:0/664: rename d3/d39/d3e/d81/d44/d7c/f14 to d3/d39/d9a/da2/fec 0 2024-06-15T01:07:06.992 INFO:tasks.workunit.client.1.smithi142.stdout:9/616: creat d6/dc/d1b/d25/d58/de2/fe9 x:0 0 0 2024-06-15T01:07:06.993 INFO:tasks.workunit.client.1.smithi142.stdout:4/739: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102 0 2024-06-15T01:07:06.993 INFO:tasks.workunit.client.0.smithi137.stdout:7/456: dread df/d13/d2e/d1a/f2c [0,4194304] 0 2024-06-15T01:07:06.995 INFO:tasks.workunit.client.1.smithi142.stdout:2/638: creat d8/df/d10/d94/fc9 x:0 0 0 2024-06-15T01:07:06.997 INFO:tasks.workunit.client.0.smithi137.stdout:7/457: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e 0 2024-06-15T01:07:06.998 INFO:tasks.workunit.client.0.smithi137.stdout:7/458: chown df/d13/d2e/d1a/c25 785607 1 2024-06-15T01:07:06.999 INFO:tasks.workunit.client.1.smithi142.stdout:7/687: dwrite d2/d6/d22/f4d [0,4194304] 0 2024-06-15T01:07:07.000 INFO:tasks.workunit.client.1.smithi142.stdout:7/688: fsync d2/f2e 0 2024-06-15T01:07:07.000 INFO:tasks.workunit.client.1.smithi142.stdout:0/665: fsync d3/d39/d3e/d81/d44/d7c/d22/f65 0 2024-06-15T01:07:07.001 INFO:tasks.workunit.client.1.smithi142.stdout:4/740: fsync d5/d9/da/f3c 0 2024-06-15T01:07:07.001 INFO:tasks.workunit.client.1.smithi142.stdout:9/617: truncate d6/dc/d1b/f40 1492129 0 2024-06-15T01:07:07.001 INFO:tasks.workunit.client.1.smithi142.stdout:7/689: chown d2/d6/d22/d47/f7a 8 1 2024-06-15T01:07:07.002 INFO:tasks.workunit.client.0.smithi137.stdout:7/459: rename df/d7d/d1c/d1d/c29 to df/d7d/d3b/d5b/c8f 0 2024-06-15T01:07:07.003 INFO:tasks.workunit.client.0.smithi137.stdout:7/460: stat df/d13/d2e/d1a/f77 0 2024-06-15T01:07:07.005 INFO:tasks.workunit.client.1.smithi142.stdout:2/639: mknod d8/df/d10/d94/d25/cca 0 2024-06-15T01:07:07.009 INFO:tasks.workunit.client.1.smithi142.stdout:3/757: rename d0/d3/da/d71 to d0/d3/de2 0 2024-06-15T01:07:07.013 INFO:tasks.workunit.client.1.smithi142.stdout:4/741: mknod d5/d9/d23/d32/c103 0 2024-06-15T01:07:07.014 INFO:tasks.workunit.client.1.smithi142.stdout:4/742: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/fd7 zero size 2024-06-15T01:07:07.017 INFO:tasks.workunit.client.1.smithi142.stdout:9/618: unlink d6/dc/d82/f81 0 2024-06-15T01:07:07.021 INFO:tasks.workunit.client.0.smithi137.stdout:7/461: dread df/d13/d3c/f67 [0,4194304] 0 2024-06-15T01:07:07.022 INFO:tasks.workunit.client.1.smithi142.stdout:2/640: creat d8/df/d10/d94/d25/d22/d43/d52/dc1/fcb x:0 0 0 2024-06-15T01:07:07.023 INFO:tasks.workunit.client.1.smithi142.stdout:2/641: dread - d8/df/d10/d94/d25/d1a/d63/f95 zero size 2024-06-15T01:07:07.026 INFO:tasks.workunit.client.1.smithi142.stdout:6/665: rename d1a/d39/d42/d4c/l4d to d1a/d2d/d3b/le7 0 2024-06-15T01:07:07.027 INFO:tasks.workunit.client.1.smithi142.stdout:2/642: chown d8/df/d10/d94/d25/d1a/c8c 4 1 2024-06-15T01:07:07.028 INFO:tasks.workunit.client.0.smithi137.stdout:7/462: dwrite df/d13/d3c/f67 [0,4194304] 0 2024-06-15T01:07:07.028 INFO:tasks.workunit.client.1.smithi142.stdout:0/666: dread d3/d39/d3e/d81/d44/d7c/d22/f65 [0,4194304] 0 2024-06-15T01:07:07.029 INFO:tasks.workunit.client.1.smithi142.stdout:3/758: dread d0/d1f/d20/f26 [4194304,4194304] 0 2024-06-15T01:07:07.031 INFO:tasks.workunit.client.1.smithi142.stdout:3/759: readlink d0/d1b/d6b/ldf 0 2024-06-15T01:07:07.031 INFO:tasks.workunit.client.1.smithi142.stdout:4/743: dread d5/d1c/d6f/dc7/d30/d46/d50/f91 [0,4194304] 0 2024-06-15T01:07:07.031 INFO:tasks.workunit.client.0.smithi137.stdout:7/463: mkdir df/d13/d2e/d1a/d6a/d90 0 2024-06-15T01:07:07.031 INFO:tasks.workunit.client.1.smithi142.stdout:1/701: rename d6/d12/d28 to d6/d25/dfc 0 2024-06-15T01:07:07.033 INFO:tasks.workunit.client.1.smithi142.stdout:2/643: dread d8/d19/d33/f66 [0,4194304] 0 2024-06-15T01:07:07.033 INFO:tasks.workunit.client.1.smithi142.stdout:6/666: creat d1a/d39/d3a/fe8 x:0 0 0 2024-06-15T01:07:07.034 INFO:tasks.workunit.client.1.smithi142.stdout:0/667: rmdir d3/d39/d3e/d81/d1b/d38 39 2024-06-15T01:07:07.035 INFO:tasks.workunit.client.1.smithi142.stdout:3/760: mknod d0/d1f/d20/ce3 0 2024-06-15T01:07:07.036 INFO:tasks.workunit.client.0.smithi137.stdout:7/464: rmdir df/d7d/d1c/d7e 0 2024-06-15T01:07:07.036 INFO:tasks.workunit.client.1.smithi142.stdout:4/744: creat d5/d9/d1a/f104 x:0 0 0 2024-06-15T01:07:07.037 INFO:tasks.workunit.client.0.smithi137.stdout:7/465: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/c91 0 2024-06-15T01:07:07.039 INFO:tasks.workunit.client.0.smithi137.stdout:7/466: creat df/d13/d3c/f92 x:0 0 0 2024-06-15T01:07:07.041 INFO:tasks.workunit.client.1.smithi142.stdout:5/605: sync 2024-06-15T01:07:07.041 INFO:tasks.workunit.client.1.smithi142.stdout:0/668: dread d3/d39/d3e/d81/f7e [0,4194304] 0 2024-06-15T01:07:07.044 INFO:tasks.workunit.client.1.smithi142.stdout:4/745: rmdir d5/d1c/d6f/dc7/d30/d46 39 2024-06-15T01:07:07.049 INFO:tasks.workunit.client.1.smithi142.stdout:0/669: dread d3/d39/d3e/d81/d44/f5a [0,4194304] 0 2024-06-15T01:07:07.049 INFO:tasks.workunit.client.1.smithi142.stdout:5/606: fsync d2/de/d27/f46 0 2024-06-15T01:07:07.053 INFO:tasks.workunit.client.1.smithi142.stdout:3/761: creat d0/d1b/fe4 x:0 0 0 2024-06-15T01:07:07.054 INFO:tasks.workunit.client.1.smithi142.stdout:7/690: dwrite d2/d8/da/d33/d25/f16 [0,4194304] 0 2024-06-15T01:07:07.055 INFO:tasks.workunit.client.1.smithi142.stdout:3/762: fsync d0/d1f/d20/d5d/fc3 0 2024-06-15T01:07:07.058 INFO:tasks.workunit.client.1.smithi142.stdout:5/607: unlink d2/de/d19/f51 0 2024-06-15T01:07:07.059 INFO:tasks.workunit.client.1.smithi142.stdout:8/688: sync 2024-06-15T01:07:07.060 INFO:tasks.workunit.client.1.smithi142.stdout:6/667: link d1a/d39/d3a/dd2/cda d1a/d3c/db0/ce9 0 2024-06-15T01:07:07.060 INFO:tasks.workunit.client.1.smithi142.stdout:9/619: dwrite d6/dc/d1b/d25/fd3 [0,4194304] 0 2024-06-15T01:07:07.063 INFO:tasks.workunit.client.0.smithi137.stdout:7/467: dread df/d13/d2e/f51 [0,4194304] 0 2024-06-15T01:07:07.064 INFO:tasks.workunit.client.1.smithi142.stdout:1/702: truncate d6/d4d/f6c 3748990 0 2024-06-15T01:07:07.065 INFO:tasks.workunit.client.1.smithi142.stdout:7/691: mknod d2/d6/daf/dce/cda 0 2024-06-15T01:07:07.071 INFO:tasks.workunit.client.1.smithi142.stdout:3/763: rename d0/d1f/d20/l3a to d0/d1f/d20/d5d/d60/dc1/le5 0 2024-06-15T01:07:07.072 INFO:tasks.workunit.client.1.smithi142.stdout:5/608: mkdir d2/de/d27/d86/dbd 0 2024-06-15T01:07:07.073 INFO:tasks.workunit.client.1.smithi142.stdout:8/689: dwrite d0/d25/d53/d9c/fbd [4194304,4194304] 0 2024-06-15T01:07:07.075 INFO:tasks.workunit.client.0.smithi137.stdout:7/468: dwrite df/d13/d2e/d1a/f26 [0,4194304] 0 2024-06-15T01:07:07.075 INFO:tasks.workunit.client.1.smithi142.stdout:9/620: unlink d6/d34/f3a 0 2024-06-15T01:07:07.075 INFO:tasks.workunit.client.1.smithi142.stdout:2/644: dwrite d8/df/d10/d94/d25/f6a [0,4194304] 0 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:5/585: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:9/340: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:8/410: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:0/529: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:6/419: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:1/536: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:3/659: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:4/516: sync 2024-06-15T01:07:07.081 INFO:tasks.workunit.client.0.smithi137.stdout:2/400: sync 2024-06-15T01:07:07.084 INFO:tasks.workunit.client.1.smithi142.stdout:2/645: truncate d8/df/d10/d94/d25/d1a/d63/f95 1033544 0 2024-06-15T01:07:07.084 INFO:tasks.workunit.client.0.smithi137.stdout:8/411: chown d9/de 1794510 1 2024-06-15T01:07:07.086 INFO:tasks.workunit.client.1.smithi142.stdout:4/746: getdents d5/d1c/d6f/dc7/d30/d46/d50/d51 0 2024-06-15T01:07:07.089 INFO:tasks.workunit.client.1.smithi142.stdout:8/690: dwrite d0/d17/d40/fa8 [4194304,4194304] 0 2024-06-15T01:07:07.092 INFO:tasks.workunit.client.0.smithi137.stdout:0/530: write d1/d2/d55/faf [2755652,64379] 0 2024-06-15T01:07:07.094 INFO:tasks.workunit.client.1.smithi142.stdout:7/692: creat d2/d8/dd/d14/d2c/d37/fdb x:0 0 0 2024-06-15T01:07:07.095 INFO:tasks.workunit.client.0.smithi137.stdout:9/341: creat df/d12/d15/f5f x:0 0 0 2024-06-15T01:07:07.096 INFO:tasks.workunit.client.1.smithi142.stdout:7/693: stat d2/d8/dd/d14/l68 0 2024-06-15T01:07:07.096 INFO:tasks.workunit.client.1.smithi142.stdout:0/670: dwrite d3/d39/d9a/da2/fec [0,4194304] 0 2024-06-15T01:07:07.098 INFO:tasks.workunit.client.0.smithi137.stdout:8/412: write d9/de/f36 [2447764,78765] 0 2024-06-15T01:07:07.098 INFO:tasks.workunit.client.0.smithi137.stdout:4/517: rename dd/d15/d36/l84 to dd/d15/d36/d45/d8f/lc4 0 2024-06-15T01:07:07.099 INFO:tasks.workunit.client.1.smithi142.stdout:5/609: rmdir d2/d18/d2e/dae/dbc 0 2024-06-15T01:07:07.099 INFO:tasks.workunit.client.1.smithi142.stdout:8/691: unlink d0/d25/d5a/c71 0 2024-06-15T01:07:07.101 INFO:tasks.workunit.client.0.smithi137.stdout:8/413: write d9/d2a/d2f/f72 [322723,105979] 0 2024-06-15T01:07:07.101 INFO:tasks.workunit.client.0.smithi137.stdout:3/660: mknod d0/d56/cd7 0 2024-06-15T01:07:07.102 INFO:tasks.workunit.client.0.smithi137.stdout:3/661: fsync d0/d2/d8/f14 0 2024-06-15T01:07:07.102 INFO:tasks.workunit.client.1.smithi142.stdout:6/668: write d1a/d39/d3a/f59 [557887,65390] 0 2024-06-15T01:07:07.103 INFO:tasks.workunit.client.0.smithi137.stdout:3/662: stat d0/d13/d37/cd1 0 2024-06-15T01:07:07.105 INFO:tasks.workunit.client.0.smithi137.stdout:0/531: symlink d1/d2/dc/d10/d53/lb8 0 2024-06-15T01:07:07.107 INFO:tasks.workunit.client.0.smithi137.stdout:5/586: getdents df/d44/d54 0 2024-06-15T01:07:07.107 INFO:tasks.workunit.client.1.smithi142.stdout:8/692: creat d0/d25/d53/da7/fd6 x:0 0 0 2024-06-15T01:07:07.108 INFO:tasks.workunit.client.1.smithi142.stdout:8/693: write d0/f14 [1142697,115324] 0 2024-06-15T01:07:07.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/420: mkdir d0/d1a/d3e/d57/d1d/d2e/d62/d84 0 2024-06-15T01:07:07.110 INFO:tasks.workunit.client.0.smithi137.stdout:9/342: truncate df/d12/f1f 5079032 0 2024-06-15T01:07:07.111 INFO:tasks.workunit.client.1.smithi142.stdout:4/747: link d5/d1c/d6f/dc7/d30/d46/f52 d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f105 0 2024-06-15T01:07:07.112 INFO:tasks.workunit.client.0.smithi137.stdout:3/663: creat d0/d2/d8/dbd/fd8 x:0 0 0 2024-06-15T01:07:07.113 INFO:tasks.workunit.client.1.smithi142.stdout:7/694: symlink d2/d8/dd/ldc 0 2024-06-15T01:07:07.113 INFO:tasks.workunit.client.1.smithi142.stdout:4/748: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/f75 88774 1 2024-06-15T01:07:07.114 INFO:tasks.workunit.client.0.smithi137.stdout:3/664: chown d0/d2/d8/f7f 57 1 2024-06-15T01:07:07.114 INFO:tasks.workunit.client.1.smithi142.stdout:2/646: creat d8/df/d10/d1c/d7c/fcc x:0 0 0 2024-06-15T01:07:07.114 INFO:tasks.workunit.client.1.smithi142.stdout:4/749: stat d5/d1c/d6f/dc7/d30/c5c 0 2024-06-15T01:07:07.116 INFO:tasks.workunit.client.0.smithi137.stdout:8/414: symlink d9/de/d1e/l88 0 2024-06-15T01:07:07.116 INFO:tasks.workunit.client.0.smithi137.stdout:9/343: fsync df/d12/d15/f58 0 2024-06-15T01:07:07.116 INFO:tasks.workunit.client.1.smithi142.stdout:2/647: readlink d8/df/d10/d94/d25/d22/lb8 0 2024-06-15T01:07:07.116 INFO:tasks.workunit.client.1.smithi142.stdout:4/750: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/le7 0 2024-06-15T01:07:07.117 INFO:tasks.workunit.client.0.smithi137.stdout:3/665: creat d0/d2/d4e/fd9 x:0 0 0 2024-06-15T01:07:07.118 INFO:tasks.workunit.client.0.smithi137.stdout:8/415: readlink d9/l24 0 2024-06-15T01:07:07.118 INFO:tasks.workunit.client.1.smithi142.stdout:2/648: fdatasync d8/df/d10/d94/d25/d22/d43/f46 0 2024-06-15T01:07:07.122 INFO:tasks.workunit.client.1.smithi142.stdout:7/695: rename d2/d8/dd/d14/d2c/d37/d6b/d80 to d2/d6/d22/ddd 0 2024-06-15T01:07:07.122 INFO:tasks.workunit.client.1.smithi142.stdout:8/694: link d0/d25/d5a/c38 d0/d17/d40/db8/cd7 0 2024-06-15T01:07:07.123 INFO:tasks.workunit.client.1.smithi142.stdout:2/649: creat d8/df/daf/fcd x:0 0 0 2024-06-15T01:07:07.123 INFO:tasks.workunit.client.1.smithi142.stdout:3/764: dread d0/d3/de2/d55/faf [0,4194304] 0 2024-06-15T01:07:07.124 INFO:tasks.workunit.client.1.smithi142.stdout:4/751: dwrite d5/d9/d1a/fe5 [0,4194304] 0 2024-06-15T01:07:07.124 INFO:tasks.workunit.client.1.smithi142.stdout:6/669: link d1a/f28 d1a/fea 0 2024-06-15T01:07:07.124 INFO:tasks.workunit.client.1.smithi142.stdout:8/695: fdatasync d0/d25/d53/fbb 0 2024-06-15T01:07:07.126 INFO:tasks.workunit.client.0.smithi137.stdout:8/416: link d9/d2a/d2f/c65 d9/d2a/d2f/c89 0 2024-06-15T01:07:07.127 INFO:tasks.workunit.client.1.smithi142.stdout:7/696: mknod d2/d6/d22/d3c/cde 0 2024-06-15T01:07:07.130 INFO:tasks.workunit.client.1.smithi142.stdout:2/650: truncate d8/d19/d33/f83 4264312 0 2024-06-15T01:07:07.131 INFO:tasks.workunit.client.1.smithi142.stdout:4/752: mknod d5/d9/da/d92/d98/dc1/c106 0 2024-06-15T01:07:07.133 INFO:tasks.workunit.client.1.smithi142.stdout:6/670: truncate d1a/d2d/d68/da0/fad 44261 0 2024-06-15T01:07:07.137 INFO:tasks.workunit.client.1.smithi142.stdout:4/753: write d5/d9/d1a/fe5 [1003193,3232] 0 2024-06-15T01:07:07.137 INFO:tasks.workunit.client.1.smithi142.stdout:3/765: creat d0/d3/de2/d55/fe6 x:0 0 0 2024-06-15T01:07:07.139 INFO:tasks.workunit.client.0.smithi137.stdout:4/518: dread dd/f28 [0,4194304] 0 2024-06-15T01:07:07.139 INFO:tasks.workunit.client.0.smithi137.stdout:8/417: creat d9/d2a/f8a x:0 0 0 2024-06-15T01:07:07.141 INFO:tasks.workunit.client.0.smithi137.stdout:3/666: dwrite d0/d2/d22/d24/d28/fc7 [0,4194304] 0 2024-06-15T01:07:07.143 INFO:tasks.workunit.client.0.smithi137.stdout:9/344: truncate fd 2525734 0 2024-06-15T01:07:07.144 INFO:tasks.workunit.client.1.smithi142.stdout:3/766: write d0/d95/fdd [417317,65263] 0 2024-06-15T01:07:07.145 INFO:tasks.workunit.client.1.smithi142.stdout:8/696: dwrite d0/d25/d5a/f4d [0,4194304] 0 2024-06-15T01:07:07.145 INFO:tasks.workunit.client.0.smithi137.stdout:6/421: write d0/d1a/d3e/f77 [273858,32749] 0 2024-06-15T01:07:07.147 INFO:tasks.workunit.client.0.smithi137.stdout:1/537: dwrite d0/d3/d47/f88 [0,4194304] 0 2024-06-15T01:07:07.148 INFO:tasks.workunit.client.1.smithi142.stdout:9/621: dwrite d6/d34/ddd/f97 [0,4194304] 0 2024-06-15T01:07:07.149 INFO:tasks.workunit.client.0.smithi137.stdout:7/469: dwrite df/d7d/d3b/d5b/f6d [0,4194304] 0 2024-06-15T01:07:07.151 INFO:tasks.workunit.client.1.smithi142.stdout:5/610: dwrite d2/de/f66 [0,4194304] 0 2024-06-15T01:07:07.153 INFO:tasks.workunit.client.0.smithi137.stdout:1/538: write d0/d3/d47/d73/fb9 [43666,103752] 0 2024-06-15T01:07:07.156 INFO:tasks.workunit.client.0.smithi137.stdout:2/401: dwrite d0/dc/f3d [0,4194304] 0 2024-06-15T01:07:07.157 INFO:tasks.workunit.client.1.smithi142.stdout:4/754: getdents d5/d9/d1a/dee 0 2024-06-15T01:07:07.161 INFO:tasks.workunit.client.0.smithi137.stdout:9/345: write df/d12/d15/f5f [135059,125658] 0 2024-06-15T01:07:07.161 INFO:tasks.workunit.client.0.smithi137.stdout:7/470: chown df/d13/d2e/f58 958299617 1 2024-06-15T01:07:07.162 INFO:tasks.workunit.client.0.smithi137.stdout:8/418: symlink d9/d2a/d2f/d59/l8b 0 2024-06-15T01:07:07.164 INFO:tasks.workunit.client.1.smithi142.stdout:9/622: dwrite d6/dc/d1b/d25/d58/fd6 [4194304,4194304] 0 2024-06-15T01:07:07.166 INFO:tasks.workunit.client.1.smithi142.stdout:9/623: readlink d6/dc/d1b/d25/d35/l98 0 2024-06-15T01:07:07.171 INFO:tasks.workunit.client.1.smithi142.stdout:8/697: mknod d0/d25/d5a/d8f/d98/cd8 0 2024-06-15T01:07:07.174 INFO:tasks.workunit.client.0.smithi137.stdout:2/402: dwrite d0/de/f52 [0,4194304] 0 2024-06-15T01:07:07.174 INFO:tasks.workunit.client.0.smithi137.stdout:3/667: creat d0/d2/d23/d68/fda x:0 0 0 2024-06-15T01:07:07.174 INFO:tasks.workunit.client.1.smithi142.stdout:6/671: getdents d1a/d2d/d68 0 2024-06-15T01:07:07.177 INFO:tasks.workunit.client.1.smithi142.stdout:2/651: dwrite d8/df/d10/d94/d25/d22/d43/f96 [0,4194304] 0 2024-06-15T01:07:07.177 INFO:tasks.workunit.client.0.smithi137.stdout:6/422: rename d0/d5/d9/d29/c55 to d0/d5/d46/c85 0 2024-06-15T01:07:07.180 INFO:tasks.workunit.client.1.smithi142.stdout:9/624: fdatasync d6/d31/f86 0 2024-06-15T01:07:07.181 INFO:tasks.workunit.client.0.smithi137.stdout:9/346: dwrite df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:07.181 INFO:tasks.workunit.client.1.smithi142.stdout:8/698: dread - d0/d25/d5a/d8f/d98/da4/fc1 zero size 2024-06-15T01:07:07.183 INFO:tasks.workunit.client.0.smithi137.stdout:0/532: dread d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:07.184 INFO:tasks.workunit.client.1.smithi142.stdout:6/672: creat d1a/d3c/db0/feb x:0 0 0 2024-06-15T01:07:07.185 INFO:tasks.workunit.client.0.smithi137.stdout:0/533: chown d1/d2/dc/d10/d53/c62 9347 1 2024-06-15T01:07:07.186 INFO:tasks.workunit.client.0.smithi137.stdout:9/347: write df/d2a/f4c [1323973,24830] 0 2024-06-15T01:07:07.187 INFO:tasks.workunit.client.0.smithi137.stdout:3/668: read d0/d2/d23/f6f [3205610,115331] 0 2024-06-15T01:07:07.188 INFO:tasks.workunit.client.0.smithi137.stdout:3/669: chown d0/d2/d22/d24/d5e 3446181 1 2024-06-15T01:07:07.189 INFO:tasks.workunit.client.1.smithi142.stdout:4/755: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/fc2 [0,4194304] 0 2024-06-15T01:07:07.191 INFO:tasks.workunit.client.1.smithi142.stdout:3/767: dread d0/d3/da/d18/f28 [0,4194304] 0 2024-06-15T01:07:07.191 INFO:tasks.workunit.client.1.smithi142.stdout:5/611: truncate d2/de/d19/d5f/d68/d76/d8c/fa7 675966 0 2024-06-15T01:07:07.192 INFO:tasks.workunit.client.0.smithi137.stdout:3/670: dread d0/d13/fba [0,4194304] 0 2024-06-15T01:07:07.194 INFO:tasks.workunit.client.0.smithi137.stdout:8/419: truncate d9/de/d1e/f44 832550 0 2024-06-15T01:07:07.197 INFO:tasks.workunit.client.1.smithi142.stdout:2/652: fsync d8/df/d10/f3b 0 2024-06-15T01:07:07.198 INFO:tasks.workunit.client.1.smithi142.stdout:9/625: dread d6/d31/f32 [0,4194304] 0 2024-06-15T01:07:07.199 INFO:tasks.workunit.client.1.smithi142.stdout:6/673: write d1a/d3c/fa7 [4725904,22212] 0 2024-06-15T01:07:07.200 INFO:tasks.workunit.client.0.smithi137.stdout:0/534: rename d1/d2/dc/d6c/d88/l89 to d1/d2/dc/d10/d14/d15/d17/d54/lb9 0 2024-06-15T01:07:07.201 INFO:tasks.workunit.client.1.smithi142.stdout:7/697: dwrite d2/d6/d22/d47/f50 [0,4194304] 0 2024-06-15T01:07:07.202 INFO:tasks.workunit.client.0.smithi137.stdout:9/348: mknod df/d2a/c60 0 2024-06-15T01:07:07.202 INFO:tasks.workunit.client.1.smithi142.stdout:6/674: readlink d1a/l40 0 2024-06-15T01:07:07.204 INFO:tasks.workunit.client.0.smithi137.stdout:6/423: creat d0/d5/d9/d29/d64/f86 x:0 0 0 2024-06-15T01:07:07.207 INFO:tasks.workunit.client.0.smithi137.stdout:3/671: mkdir d0/d2/d23/ddb 0 2024-06-15T01:07:07.209 INFO:tasks.workunit.client.0.smithi137.stdout:2/403: link d0/d1c/l34 d0/dc/d22/d23/d5c/l7a 0 2024-06-15T01:07:07.210 INFO:tasks.workunit.client.0.smithi137.stdout:4/519: write dd/d15/d21/f29 [1199313,83804] 0 2024-06-15T01:07:07.211 INFO:tasks.workunit.client.0.smithi137.stdout:0/535: mknod d1/d2/dc/d10/d14/d56/cba 0 2024-06-15T01:07:07.212 INFO:tasks.workunit.client.0.smithi137.stdout:9/349: creat df/d12/d29/f61 x:0 0 0 2024-06-15T01:07:07.212 INFO:tasks.workunit.client.1.smithi142.stdout:5/612: getdents d2/d3e/d31 0 2024-06-15T01:07:07.212 INFO:tasks.workunit.client.0.smithi137.stdout:9/350: stat df/d13/l2b 0 2024-06-15T01:07:07.213 INFO:tasks.workunit.client.1.smithi142.stdout:8/699: dread d0/d25/d5a/f4d [0,4194304] 0 2024-06-15T01:07:07.213 INFO:tasks.workunit.client.0.smithi137.stdout:0/536: chown d1/d2/f47 376091203 1 2024-06-15T01:07:07.214 INFO:tasks.workunit.client.1.smithi142.stdout:6/675: truncate d1a/d3c/fc8 4810825 0 2024-06-15T01:07:07.214 INFO:tasks.workunit.client.1.smithi142.stdout:8/700: readlink d0/d17/d40/l42 0 2024-06-15T01:07:07.215 INFO:tasks.workunit.client.0.smithi137.stdout:6/424: rmdir d0/d1a/d3e/d57/d1d/d2e 39 2024-06-15T01:07:07.215 INFO:tasks.workunit.client.0.smithi137.stdout:3/672: creat d0/d13/d37/d66/fdc x:0 0 0 2024-06-15T01:07:07.216 INFO:tasks.workunit.client.1.smithi142.stdout:9/626: dwrite d6/f6a [0,4194304] 0 2024-06-15T01:07:07.217 INFO:tasks.workunit.client.0.smithi137.stdout:1/539: dread d0/d3/d1b/d4c/f8e [0,4194304] 0 2024-06-15T01:07:07.217 INFO:tasks.workunit.client.0.smithi137.stdout:3/673: chown d0/d56/d8f/d7b/f96 15 1 2024-06-15T01:07:07.219 INFO:tasks.workunit.client.0.smithi137.stdout:4/520: unlink dd/f13 0 2024-06-15T01:07:07.220 INFO:tasks.workunit.client.1.smithi142.stdout:1/703: sync 2024-06-15T01:07:07.221 INFO:tasks.workunit.client.0.smithi137.stdout:0/537: rmdir d1/d2/dc/d10/d14/d15/d17 39 2024-06-15T01:07:07.226 INFO:tasks.workunit.client.0.smithi137.stdout:6/425: dread d0/fc [0,4194304] 0 2024-06-15T01:07:07.229 INFO:tasks.workunit.client.0.smithi137.stdout:2/404: link d0/de/d2b/l36 d0/de/l7b 0 2024-06-15T01:07:07.230 INFO:tasks.workunit.client.0.smithi137.stdout:9/351: dwrite df/d12/d1b/f2f [0,4194304] 0 2024-06-15T01:07:07.231 INFO:tasks.workunit.client.1.smithi142.stdout:7/698: creat d2/d6/d22/fdf x:0 0 0 2024-06-15T01:07:07.232 INFO:tasks.workunit.client.1.smithi142.stdout:2/653: mknod d8/df/d10/d94/d25/d22/d43/d52/cce 0 2024-06-15T01:07:07.232 INFO:tasks.workunit.client.0.smithi137.stdout:6/426: write d0/d1a/f2f [116651,25685] 0 2024-06-15T01:07:07.234 INFO:tasks.workunit.client.1.smithi142.stdout:6/676: fsync d1a/d2d/f65 0 2024-06-15T01:07:07.234 INFO:tasks.workunit.client.0.smithi137.stdout:6/427: chown d0/d5/d9/d3f 7 1 2024-06-15T01:07:07.235 INFO:tasks.workunit.client.1.smithi142.stdout:8/701: rmdir d0/d17/d40/db8 39 2024-06-15T01:07:07.236 INFO:tasks.workunit.client.0.smithi137.stdout:0/538: dread d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [4194304,4194304] 0 2024-06-15T01:07:07.238 INFO:tasks.workunit.client.1.smithi142.stdout:9/627: creat d6/d34/ddd/fea x:0 0 0 2024-06-15T01:07:07.238 INFO:tasks.workunit.client.0.smithi137.stdout:2/405: truncate d0/de/d2b/f32 432015 0 2024-06-15T01:07:07.240 INFO:tasks.workunit.client.1.smithi142.stdout:7/699: write d2/d8/dd/d14/d2c/d37/d6b/f96 [2360472,41533] 0 2024-06-15T01:07:07.240 INFO:tasks.workunit.client.1.smithi142.stdout:4/756: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f107 x:0 0 0 2024-06-15T01:07:07.241 INFO:tasks.workunit.client.1.smithi142.stdout:5/613: mkdir d2/d3e/d74/da5/dbe 0 2024-06-15T01:07:07.244 INFO:tasks.workunit.client.1.smithi142.stdout:8/702: write d0/f23 [262112,123598] 0 2024-06-15T01:07:07.244 INFO:tasks.workunit.client.1.smithi142.stdout:8/703: read - d0/d25/d53/d9c/fac zero size 2024-06-15T01:07:07.245 INFO:tasks.workunit.client.1.smithi142.stdout:5/614: read d2/de/d27/f56 [908007,22089] 0 2024-06-15T01:07:07.245 INFO:tasks.workunit.client.0.smithi137.stdout:6/428: dwrite d0/d5/d9/d29/f4b [0,4194304] 0 2024-06-15T01:07:07.245 INFO:tasks.workunit.client.0.smithi137.stdout:4/521: link dd/d15/d30/f40 dd/d15/d21/d3a/d51/d83/db2/d72/d81/fc5 0 2024-06-15T01:07:07.245 INFO:tasks.workunit.client.1.smithi142.stdout:8/704: fdatasync d0/d25/d53/fbb 0 2024-06-15T01:07:07.246 INFO:tasks.workunit.client.0.smithi137.stdout:6/429: dread - d0/d5/f7c zero size 2024-06-15T01:07:07.248 INFO:tasks.workunit.client.0.smithi137.stdout:4/522: write dd/d15/d36/d45/d55/f95 [5144381,7798] 0 2024-06-15T01:07:07.249 INFO:tasks.workunit.client.0.smithi137.stdout:2/406: fsync d0/dc/d22/f4b 0 2024-06-15T01:07:07.250 INFO:tasks.workunit.client.1.smithi142.stdout:2/654: unlink d8/fe 0 2024-06-15T01:07:07.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/539: symlink d1/lbb 0 2024-06-15T01:07:07.253 INFO:tasks.workunit.client.1.smithi142.stdout:7/700: mkdir d2/d8/da/d33/d25/d79/de0 0 2024-06-15T01:07:07.254 INFO:tasks.workunit.client.1.smithi142.stdout:7/701: truncate d2/d8/f42 4956203 0 2024-06-15T01:07:07.254 INFO:tasks.workunit.client.1.smithi142.stdout:4/757: creat d5/d9/da/d92/d98/f108 x:0 0 0 2024-06-15T01:07:07.256 INFO:tasks.workunit.client.0.smithi137.stdout:6/430: chown d0/d1a/d3e/d57/d1d/d2e/d62 1750166 1 2024-06-15T01:07:07.256 INFO:tasks.workunit.client.1.smithi142.stdout:4/758: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f95 [1044363,35606] 0 2024-06-15T01:07:07.257 INFO:tasks.workunit.client.0.smithi137.stdout:6/431: readlink d0/d1a/d3e/d57/d1d/d2e/d62/l72 0 2024-06-15T01:07:07.260 INFO:tasks.workunit.client.0.smithi137.stdout:4/523: rename dd/d15/d21/d3a/d51/f7f to dd/d15/d21/d3a/d51/d5b/fc6 0 2024-06-15T01:07:07.260 INFO:tasks.workunit.client.0.smithi137.stdout:1/540: dread d0/d3/d1b/f69 [0,4194304] 0 2024-06-15T01:07:07.263 INFO:tasks.workunit.client.0.smithi137.stdout:0/540: mknod d1/d2/d13/cbc 0 2024-06-15T01:07:07.265 INFO:tasks.workunit.client.0.smithi137.stdout:6/432: mkdir d0/d1a/d3e/d87 0 2024-06-15T01:07:07.266 INFO:tasks.workunit.client.1.smithi142.stdout:2/655: fdatasync d8/df/d10/d94/d25/da9/f8b 0 2024-06-15T01:07:07.266 INFO:tasks.workunit.client.1.smithi142.stdout:2/656: chown d8/d19/c9f 462 1 2024-06-15T01:07:07.268 INFO:tasks.workunit.client.1.smithi142.stdout:6/677: link d1a/d2d/d2e/f8b d1a/d39/d3a/fec 0 2024-06-15T01:07:07.269 INFO:tasks.workunit.client.1.smithi142.stdout:6/678: fsync d1a/d3c/f48 0 2024-06-15T01:07:07.269 INFO:tasks.workunit.client.0.smithi137.stdout:1/541: mknod d0/d3/d1b/d4c/d59/cc0 0 2024-06-15T01:07:07.270 INFO:tasks.workunit.client.1.smithi142.stdout:4/759: symlink d5/d9/d18/l109 0 2024-06-15T01:07:07.271 INFO:tasks.workunit.client.0.smithi137.stdout:8/420: dwrite d9/d22/f37 [0,4194304] 0 2024-06-15T01:07:07.272 INFO:tasks.workunit.client.1.smithi142.stdout:9/628: link d6/d31/c7e d6/d34/d8a/ceb 0 2024-06-15T01:07:07.272 INFO:tasks.workunit.client.0.smithi137.stdout:4/524: dread dd/d15/d36/d45/d55/d8d/f5e [0,4194304] 0 2024-06-15T01:07:07.273 INFO:tasks.workunit.client.1.smithi142.stdout:2/657: dwrite d8/df/f59 [0,4194304] 0 2024-06-15T01:07:07.273 INFO:tasks.workunit.client.0.smithi137.stdout:9/352: dread f5 [0,4194304] 0 2024-06-15T01:07:07.274 INFO:tasks.workunit.client.1.smithi142.stdout:8/705: link d0/d17/d40/d86/d9d/fa5 d0/d25/d53/da7/dcb/fd9 0 2024-06-15T01:07:07.275 INFO:tasks.workunit.client.1.smithi142.stdout:1/704: write d6/d25/f3d [1592246,53369] 0 2024-06-15T01:07:07.276 INFO:tasks.workunit.client.0.smithi137.stdout:6/433: chown d0/d5/c53 7852 1 2024-06-15T01:07:07.276 INFO:tasks.workunit.client.1.smithi142.stdout:8/706: dread - d0/d25/d5a/d84/fa0 zero size 2024-06-15T01:07:07.277 INFO:tasks.workunit.client.0.smithi137.stdout:3/674: truncate d0/d2/d1e/f81 514494 0 2024-06-15T01:07:07.279 INFO:tasks.workunit.client.0.smithi137.stdout:0/541: dwrite d1/d2/dc/d10/d14/d15/d1c/f46 [0,4194304] 0 2024-06-15T01:07:07.279 INFO:tasks.workunit.client.0.smithi137.stdout:1/542: readlink d0/d1f/d76/l7f 0 2024-06-15T01:07:07.280 INFO:tasks.workunit.client.0.smithi137.stdout:6/434: stat d0/d5/d9/d3f 0 2024-06-15T01:07:07.283 INFO:tasks.workunit.client.0.smithi137.stdout:3/675: write d0/d42/fc9 [1048126,59457] 0 2024-06-15T01:07:07.286 INFO:tasks.workunit.client.0.smithi137.stdout:6/435: truncate d0/d5/d9/d3f/f4a 88511 0 2024-06-15T01:07:07.287 INFO:tasks.workunit.client.0.smithi137.stdout:8/421: write d9/de/f7b [865621,76144] 0 2024-06-15T01:07:07.289 INFO:tasks.workunit.client.1.smithi142.stdout:9/629: chown d6/d31/cce 10670117 1 2024-06-15T01:07:07.292 INFO:tasks.workunit.client.1.smithi142.stdout:4/760: truncate d5/d9/d18/f6e 651178 0 2024-06-15T01:07:07.296 INFO:tasks.workunit.client.1.smithi142.stdout:5/615: dwrite d2/de/f7a [0,4194304] 0 2024-06-15T01:07:07.297 INFO:tasks.workunit.client.1.smithi142.stdout:9/630: read d6/dc/d1b/d25/f3f [2440253,31034] 0 2024-06-15T01:07:07.297 INFO:tasks.workunit.client.1.smithi142.stdout:6/679: dread d1a/d39/d3a/dd2/fe2 [0,4194304] 0 2024-06-15T01:07:07.298 INFO:tasks.workunit.client.0.smithi137.stdout:3/676: unlink d0/d42/d79/ca7 0 2024-06-15T01:07:07.298 INFO:tasks.workunit.client.1.smithi142.stdout:2/658: mkdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf 0 2024-06-15T01:07:07.300 INFO:tasks.workunit.client.0.smithi137.stdout:8/422: stat d9/de/d1e/f75 0 2024-06-15T01:07:07.301 INFO:tasks.workunit.client.0.smithi137.stdout:9/353: dwrite df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:07.302 INFO:tasks.workunit.client.1.smithi142.stdout:0/671: sync 2024-06-15T01:07:07.303 INFO:tasks.workunit.client.1.smithi142.stdout:9/631: write d6/dc/d1b/d25/d58/f63 [647894,12806] 0 2024-06-15T01:07:07.304 INFO:tasks.workunit.client.0.smithi137.stdout:3/677: mknod d0/d2/d22/d63/cdd 0 2024-06-15T01:07:07.306 INFO:tasks.workunit.client.1.smithi142.stdout:7/702: dwrite d2/d8/dd/d8a/fb5 [0,4194304] 0 2024-06-15T01:07:07.306 INFO:tasks.workunit.client.0.smithi137.stdout:2/407: dread d0/de/f16 [0,4194304] 0 2024-06-15T01:07:07.307 INFO:tasks.workunit.client.0.smithi137.stdout:0/542: write d1/d2/d13/d66/f4a [1462679,103221] 0 2024-06-15T01:07:07.307 INFO:tasks.workunit.client.0.smithi137.stdout:9/354: mknod df/d49/c62 0 2024-06-15T01:07:07.308 INFO:tasks.workunit.client.0.smithi137.stdout:4/525: dwrite dd/d15/d21/d9c/d75/fc2 [0,4194304] 0 2024-06-15T01:07:07.310 INFO:tasks.workunit.client.0.smithi137.stdout:4/526: chown dd/d15 10 1 2024-06-15T01:07:07.316 INFO:tasks.workunit.client.1.smithi142.stdout:0/672: dwrite d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [0,4194304] 0 2024-06-15T01:07:07.317 INFO:tasks.workunit.client.1.smithi142.stdout:0/673: readlink d3/l3a 0 2024-06-15T01:07:07.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:07 smithi142 ceph-mon[37379]: pgmap v19: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 84 MiB/s rd, 341 MiB/s wr, 597 op/s 2024-06-15T01:07:07.318 INFO:tasks.workunit.client.1.smithi142.stdout:7/703: dwrite d2/d6/fc5 [0,4194304] 0 2024-06-15T01:07:07.321 INFO:tasks.workunit.client.1.smithi142.stdout:7/704: write d2/d8/da/d33/d25/d79/f95 [137914,61252] 0 2024-06-15T01:07:07.351 INFO:tasks.workunit.client.1.smithi142.stdout:3/768: sync 2024-06-15T01:07:07.357 INFO:tasks.workunit.client.1.smithi142.stdout:3/769: dwrite d0/d1f/d20/d5d/d82/dc9/fda [0,4194304] 0 2024-06-15T01:07:07.360 INFO:tasks.workunit.client.0.smithi137.stdout:6/436: getdents d0/d1a/d3e/d57/d1d/d2e/d62 0 2024-06-15T01:07:07.361 INFO:tasks.workunit.client.0.smithi137.stdout:8/423: creat d9/d2a/d79/d7e/f8c x:0 0 0 2024-06-15T01:07:07.362 INFO:tasks.workunit.client.1.smithi142.stdout:0/674: dread d3/d39/d3e/d81/d44/d7c/f95 [0,4194304] 0 2024-06-15T01:07:07.363 INFO:tasks.workunit.client.0.smithi137.stdout:3/678: mkdir d0/d2/d23/ddb/dde 0 2024-06-15T01:07:07.364 INFO:tasks.workunit.client.1.smithi142.stdout:6/680: rmdir d1a 39 2024-06-15T01:07:07.368 INFO:tasks.workunit.client.1.smithi142.stdout:4/761: truncate f2 810188 0 2024-06-15T01:07:07.370 INFO:tasks.workunit.client.1.smithi142.stdout:9/632: mkdir d6/d34/d8a/dec 0 2024-06-15T01:07:07.371 INFO:tasks.workunit.client.1.smithi142.stdout:4/762: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f95 [123309,114726] 0 2024-06-15T01:07:07.373 INFO:tasks.workunit.client.1.smithi142.stdout:7/705: unlink d2/d8/da/d33/d25/l82 0 2024-06-15T01:07:07.374 INFO:tasks.workunit.client.1.smithi142.stdout:3/770: write d0/d1f/f38 [969600,1355] 0 2024-06-15T01:07:07.376 INFO:tasks.workunit.client.1.smithi142.stdout:8/707: link d0/f14 d0/d25/d5a/fda 0 2024-06-15T01:07:07.377 INFO:tasks.workunit.client.1.smithi142.stdout:8/708: fdatasync d0/d25/d5a/f69 0 2024-06-15T01:07:07.377 INFO:tasks.workunit.client.1.smithi142.stdout:9/633: mknod d6/dc/d51/ced 0 2024-06-15T01:07:07.378 INFO:tasks.workunit.client.1.smithi142.stdout:4/763: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/d99/f9a 0 2024-06-15T01:07:07.379 INFO:tasks.workunit.client.1.smithi142.stdout:7/706: fdatasync d2/d8/da/d33/d25/d79/fb1 0 2024-06-15T01:07:07.379 INFO:tasks.workunit.client.1.smithi142.stdout:9/634: fsync d6/dc/d1b/d25/d58/fbb 0 2024-06-15T01:07:07.379 INFO:tasks.workunit.client.1.smithi142.stdout:3/771: dread d0/d3/de2/d75/fa7 [0,4194304] 0 2024-06-15T01:07:07.379 INFO:tasks.workunit.client.1.smithi142.stdout:4/764: fdatasync d5/d1c/d6f/dc7/de0/ff7 0 2024-06-15T01:07:07.380 INFO:tasks.workunit.client.1.smithi142.stdout:4/765: stat d5/d9/da/da3/cb6 0 2024-06-15T01:07:07.385 INFO:tasks.workunit.client.1.smithi142.stdout:0/675: rename d3/f23 to d3/d39/d3e/d81/d1b/d38/da3/fed 0 2024-06-15T01:07:07.385 INFO:tasks.workunit.client.1.smithi142.stdout:0/676: stat d3/d39/d3e/d81/d1e/c3b 0 2024-06-15T01:07:07.386 INFO:tasks.workunit.client.1.smithi142.stdout:1/705: dread d6/d12/d3c/d63/fdd [0,4194304] 0 2024-06-15T01:07:07.388 INFO:tasks.workunit.client.0.smithi137.stdout:0/543: mkdir d1/d2/dc/d10/d22/db7/dbd 0 2024-06-15T01:07:07.389 INFO:tasks.workunit.client.0.smithi137.stdout:2/408: dwrite d0/f6 [0,4194304] 0 2024-06-15T01:07:07.389 INFO:tasks.workunit.client.1.smithi142.stdout:3/772: rmdir d0/d3/de2 39 2024-06-15T01:07:07.389 INFO:tasks.workunit.client.1.smithi142.stdout:3/773: readlink d0/d1f/d20/d5d/l8d 0 2024-06-15T01:07:07.390 INFO:tasks.workunit.client.1.smithi142.stdout:9/635: mknod d6/dc/d51/d53/d8d/cee 0 2024-06-15T01:07:07.391 INFO:tasks.workunit.client.0.smithi137.stdout:9/355: dwrite df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:07.391 INFO:tasks.workunit.client.1.smithi142.stdout:4/766: rmdir d5/d1c/d6f/dc7/d30/d46/d50 39 2024-06-15T01:07:07.393 INFO:tasks.workunit.client.0.smithi137.stdout:8/424: mknod d9/d2a/d2f/c8d 0 2024-06-15T01:07:07.394 INFO:tasks.workunit.client.0.smithi137.stdout:0/544: chown d1/d7e/d87/da0/ca4 1 1 2024-06-15T01:07:07.396 INFO:tasks.workunit.client.0.smithi137.stdout:1/543: dread d0/d3/d1b/d4c/d59/f63 [0,4194304] 0 2024-06-15T01:07:07.397 INFO:tasks.workunit.client.0.smithi137.stdout:4/527: dwrite dd/d15/d21/d9c/d75/d99/fbd [0,4194304] 0 2024-06-15T01:07:07.398 INFO:tasks.workunit.client.1.smithi142.stdout:1/706: unlink d6/d25/dfc/f39 0 2024-06-15T01:07:07.399 INFO:tasks.workunit.client.1.smithi142.stdout:1/707: dread - d6/d12/da0/db0/fe8 zero size 2024-06-15T01:07:07.401 INFO:tasks.workunit.client.0.smithi137.stdout:6/437: getdents d0/d5 0 2024-06-15T01:07:07.402 INFO:tasks.workunit.client.0.smithi137.stdout:1/544: chown d0/d10/fb5 1004432851 1 2024-06-15T01:07:07.402 INFO:tasks.workunit.client.1.smithi142.stdout:8/709: symlink d0/d17/d40/ldb 0 2024-06-15T01:07:07.403 INFO:tasks.workunit.client.1.smithi142.stdout:8/710: stat d0/d25/d5a/l31 0 2024-06-15T01:07:07.403 INFO:tasks.workunit.client.1.smithi142.stdout:9/636: mknod d6/dc/d51/cef 0 2024-06-15T01:07:07.404 INFO:tasks.workunit.client.0.smithi137.stdout:9/356: creat df/d12/d15/f63 x:0 0 0 2024-06-15T01:07:07.405 INFO:tasks.workunit.client.0.smithi137.stdout:6/438: read d0/d5/f73 [4979367,12783] 0 2024-06-15T01:07:07.408 INFO:tasks.workunit.client.1.smithi142.stdout:2/659: write d8/df/d10/f11 [1340347,37064] 0 2024-06-15T01:07:07.410 INFO:tasks.workunit.client.0.smithi137.stdout:6/439: stat d0/d1a/f35 0 2024-06-15T01:07:07.411 INFO:tasks.workunit.client.1.smithi142.stdout:6/681: getdents d1a/d39/d3a/d51/d62/da8 0 2024-06-15T01:07:07.412 INFO:tasks.workunit.client.0.smithi137.stdout:9/357: truncate df/d12/f59 344466 0 2024-06-15T01:07:07.412 INFO:tasks.workunit.client.0.smithi137.stdout:1/545: creat d0/d10/fc1 x:0 0 0 2024-06-15T01:07:07.412 INFO:tasks.workunit.client.0.smithi137.stdout:3/679: dwrite d0/d4c/f82 [0,4194304] 0 2024-06-15T01:07:07.413 INFO:tasks.workunit.client.1.smithi142.stdout:4/767: unlink d5/d1c/d6f/dc7/d30/d46/d50/d51/c8c 0 2024-06-15T01:07:07.413 INFO:tasks.workunit.client.0.smithi137.stdout:4/528: symlink dd/d15/d21/d3a/d51/d83/lc7 0 2024-06-15T01:07:07.415 INFO:tasks.workunit.client.0.smithi137.stdout:2/409: link d0/d42/f5a d0/da/f7c 0 2024-06-15T01:07:07.415 INFO:tasks.workunit.client.1.smithi142.stdout:0/677: creat d3/d39/d3e/d81/d44/d7c/fee x:0 0 0 2024-06-15T01:07:07.415 INFO:tasks.workunit.client.0.smithi137.stdout:0/545: dread d1/fb6 [0,4194304] 0 2024-06-15T01:07:07.417 INFO:tasks.workunit.client.0.smithi137.stdout:1/546: write d0/d10/fc1 [194841,37846] 0 2024-06-15T01:07:07.418 INFO:tasks.workunit.client.0.smithi137.stdout:9/358: write df/d12/d15/f63 [575560,2416] 0 2024-06-15T01:07:07.420 INFO:tasks.workunit.client.1.smithi142.stdout:4/768: dwrite d5/d9/d90/daf/dcb/f8a [0,4194304] 0 2024-06-15T01:07:07.422 INFO:tasks.workunit.client.0.smithi137.stdout:1/547: read d0/d1f/d60/f91 [2928170,63590] 0 2024-06-15T01:07:07.423 INFO:tasks.workunit.client.1.smithi142.stdout:4/769: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d 0 2024-06-15T01:07:07.423 INFO:tasks.workunit.client.0.smithi137.stdout:6/440: creat d0/d1a/d3e/d57/d32/d74/f88 x:0 0 0 2024-06-15T01:07:07.424 INFO:tasks.workunit.client.0.smithi137.stdout:0/546: readlink d1/d2/dc/d10/d14/d15/d17/l26 0 2024-06-15T01:07:07.424 INFO:tasks.workunit.client.1.smithi142.stdout:5/616: truncate d2/de/f5a 1933597 0 2024-06-15T01:07:07.425 INFO:tasks.workunit.client.0.smithi137.stdout:1/548: chown d0/d3/d1b/c46 1640459 1 2024-06-15T01:07:07.427 INFO:tasks.workunit.client.0.smithi137.stdout:2/410: symlink d0/dc/d2e/d72/d56/l7d 0 2024-06-15T01:07:07.427 INFO:tasks.workunit.client.0.smithi137.stdout:1/549: truncate d0/d2d/f79 4659906 0 2024-06-15T01:07:07.428 INFO:tasks.workunit.client.1.smithi142.stdout:9/637: write d6/dc/d1b/d25/d58/f87 [1407427,55039] 0 2024-06-15T01:07:07.429 INFO:tasks.workunit.client.0.smithi137.stdout:4/529: chown dd/d15/d36/d45/c50 1952522 1 2024-06-15T01:07:07.432 INFO:tasks.workunit.client.0.smithi137.stdout:8/425: link d9/de/d1e/c32 d9/d55/c8e 0 2024-06-15T01:07:07.432 INFO:tasks.workunit.client.0.smithi137.stdout:4/530: write dd/d15/d21/f29 [2340985,114202] 0 2024-06-15T01:07:07.433 INFO:tasks.workunit.client.1.smithi142.stdout:2/660: rename d8/d19/d33/fa0 to d8/df/d10/d94/d25/da9/fd0 0 2024-06-15T01:07:07.433 INFO:tasks.workunit.client.0.smithi137.stdout:8/426: write d9/de/f83 [736021,126525] 0 2024-06-15T01:07:07.435 INFO:tasks.workunit.client.1.smithi142.stdout:6/682: mkdir d1a/d39/ded 0 2024-06-15T01:07:07.435 INFO:tasks.workunit.client.0.smithi137.stdout:8/427: write d9/d2a/f8a [445040,91605] 0 2024-06-15T01:07:07.436 INFO:tasks.workunit.client.1.smithi142.stdout:6/683: read - d1a/d39/d3a/d51/d62/fd1 zero size 2024-06-15T01:07:07.436 INFO:tasks.workunit.client.0.smithi137.stdout:9/359: mknod df/d12/d4e/c64 0 2024-06-15T01:07:07.441 INFO:tasks.workunit.client.0.smithi137.stdout:0/547: mknod d1/d2/cbe 0 2024-06-15T01:07:07.442 INFO:tasks.workunit.client.0.smithi137.stdout:9/360: dread df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:07.442 INFO:tasks.workunit.client.1.smithi142.stdout:5/617: stat d2/d3e/caa 0 2024-06-15T01:07:07.443 INFO:tasks.workunit.client.1.smithi142.stdout:9/638: write d6/f71 [4355211,44228] 0 2024-06-15T01:07:07.444 INFO:tasks.workunit.client.0.smithi137.stdout:3/680: rename d0/d2/d1e/d7c to d0/ddf 0 2024-06-15T01:07:07.445 INFO:tasks.workunit.client.0.smithi137.stdout:2/411: creat d0/d42/d49/d57/f7e x:0 0 0 2024-06-15T01:07:07.446 INFO:tasks.workunit.client.1.smithi142.stdout:2/661: mkdir d8/df/d10/d94/d25/d22/d43/d52/dd1 0 2024-06-15T01:07:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:2/412: truncate d0/dc/d22/f39 2173124 0 2024-06-15T01:07:07.448 INFO:tasks.workunit.client.1.smithi142.stdout:3/774: dwrite d0/d1f/d20/d59/f92 [0,4194304] 0 2024-06-15T01:07:07.449 INFO:tasks.workunit.client.0.smithi137.stdout:9/361: mknod df/d2a/c65 0 2024-06-15T01:07:07.450 INFO:tasks.workunit.client.0.smithi137.stdout:9/362: truncate df/f28 1377630 0 2024-06-15T01:07:07.451 INFO:tasks.workunit.client.1.smithi142.stdout:8/711: write d0/d17/d40/d86/d9d/fa5 [71454,66798] 0 2024-06-15T01:07:07.451 INFO:tasks.workunit.client.1.smithi142.stdout:1/708: link d6/d46/dc3/lbe d6/d25/lfd 0 2024-06-15T01:07:07.454 INFO:tasks.workunit.client.1.smithi142.stdout:5/618: creat d2/d3e/d98/fbf x:0 0 0 2024-06-15T01:07:07.455 INFO:tasks.workunit.client.0.smithi137.stdout:8/428: rmdir d9/de/d1e/d35 39 2024-06-15T01:07:07.455 INFO:tasks.workunit.client.1.smithi142.stdout:0/678: dread d3/d39/d3e/d81/d1b/d38/f96 [0,4194304] 0 2024-06-15T01:07:07.457 INFO:tasks.workunit.client.1.smithi142.stdout:0/679: write d3/d39/d3e/d81/d44/d7c/d22/d59/f94 [3326170,124548] 0 2024-06-15T01:07:07.459 INFO:tasks.workunit.client.0.smithi137.stdout:1/550: link d0/d1f/d60/f55 d0/d10/d41/d7b/fc2 0 2024-06-15T01:07:07.459 INFO:tasks.workunit.client.0.smithi137.stdout:9/363: dwrite df/d12/d15/f1d [0,4194304] 0 2024-06-15T01:07:07.460 INFO:tasks.workunit.client.1.smithi142.stdout:2/662: creat d8/df/d10/d94/d25/d22/d43/d51/fd2 x:0 0 0 2024-06-15T01:07:07.460 INFO:tasks.workunit.client.1.smithi142.stdout:2/663: write d8/df/d10/d94/db5/fc3 [35247,18143] 0 2024-06-15T01:07:07.463 INFO:tasks.workunit.client.1.smithi142.stdout:3/775: rmdir d0/d1f/d20/d5d/d82/dc9 39 2024-06-15T01:07:07.463 INFO:tasks.workunit.client.0.smithi137.stdout:3/681: creat d0/d2f/dbe/fe0 x:0 0 0 2024-06-15T01:07:07.464 INFO:tasks.workunit.client.0.smithi137.stdout:8/429: dwrite f8 [0,4194304] 0 2024-06-15T01:07:07.466 INFO:tasks.workunit.client.0.smithi137.stdout:8/430: write f8 [5018013,93863] 0 2024-06-15T01:07:07.467 INFO:tasks.workunit.client.1.smithi142.stdout:1/709: mkdir d6/d12/d3c/d63/d8d/d97/dc6/dfe 0 2024-06-15T01:07:07.468 INFO:tasks.workunit.client.0.smithi137.stdout:8/431: chown d9/d2a 44401849 1 2024-06-15T01:07:07.469 INFO:tasks.workunit.client.1.smithi142.stdout:9/639: symlink d6/dc/lf0 0 2024-06-15T01:07:07.469 INFO:tasks.workunit.client.1.smithi142.stdout:2/664: dwrite d8/df/d10/d94/db5/fc3 [0,4194304] 0 2024-06-15T01:07:07.469 INFO:tasks.workunit.client.0.smithi137.stdout:2/413: symlink d0/dc/d2e/d72/d56/d6e/l7f 0 2024-06-15T01:07:07.474 INFO:tasks.workunit.client.0.smithi137.stdout:3/682: creat d0/d13/d37/d83/fe1 x:0 0 0 2024-06-15T01:07:07.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:07 smithi137 ceph-mon[29354]: pgmap v19: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 84 MiB/s rd, 341 MiB/s wr, 597 op/s 2024-06-15T01:07:07.476 INFO:tasks.workunit.client.1.smithi142.stdout:5/619: chown d2/de/c83 2947 1 2024-06-15T01:07:07.477 INFO:tasks.workunit.client.1.smithi142.stdout:0/680: symlink d3/d39/d3e/d81/d1b/d38/da3/lef 0 2024-06-15T01:07:07.477 INFO:tasks.workunit.client.1.smithi142.stdout:9/640: chown d6/d34/ddd/f97 2513 1 2024-06-15T01:07:07.477 INFO:tasks.workunit.client.0.smithi137.stdout:3/683: write d0/d2/d22/d24/d28/fa6 [4718291,99443] 0 2024-06-15T01:07:07.480 INFO:tasks.workunit.client.0.smithi137.stdout:8/432: read d9/d22/f3e [1642472,95672] 0 2024-06-15T01:07:07.480 INFO:tasks.workunit.client.0.smithi137.stdout:2/414: dwrite d0/dc/d2e/d72/d56/f4f [4194304,4194304] 0 2024-06-15T01:07:07.482 INFO:tasks.workunit.client.1.smithi142.stdout:2/665: write d8/df/d10/d94/d25/da9/fd0 [2039089,32426] 0 2024-06-15T01:07:07.483 INFO:tasks.workunit.client.0.smithi137.stdout:2/415: dread - d0/d42/d49/d57/f7e zero size 2024-06-15T01:07:07.484 INFO:tasks.workunit.client.0.smithi137.stdout:0/548: dread d1/d2/dc/d10/d14/d15/d17/f18 [0,4194304] 0 2024-06-15T01:07:07.484 INFO:tasks.workunit.client.0.smithi137.stdout:3/684: fdatasync d0/d13/d37/f43 0 2024-06-15T01:07:07.486 INFO:tasks.workunit.client.1.smithi142.stdout:6/684: dwrite d1a/d2d/d68/da0/fa5 [0,4194304] 0 2024-06-15T01:07:07.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/433: mknod d9/c8f 0 2024-06-15T01:07:07.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/434: readlink d9/l10 0 2024-06-15T01:07:07.487 INFO:tasks.workunit.client.0.smithi137.stdout:3/685: write d0/d2/d23/d68/fda [927492,18620] 0 2024-06-15T01:07:07.489 INFO:tasks.workunit.client.1.smithi142.stdout:4/770: read f2 [475515,116632] 0 2024-06-15T01:07:07.489 INFO:tasks.workunit.client.1.smithi142.stdout:6/685: chown d1a/f61 935703301 1 2024-06-15T01:07:07.490 INFO:tasks.workunit.client.0.smithi137.stdout:0/549: creat d1/d2/dc/fbf x:0 0 0 2024-06-15T01:07:07.491 INFO:tasks.workunit.client.0.smithi137.stdout:8/435: creat d9/d2a/d2f/f90 x:0 0 0 2024-06-15T01:07:07.491 INFO:tasks.workunit.client.1.smithi142.stdout:5/620: mknod d2/de/d19/cc0 0 2024-06-15T01:07:07.491 INFO:tasks.workunit.client.1.smithi142.stdout:1/710: dwrite d6/d12/d3c/f9c [0,4194304] 0 2024-06-15T01:07:07.491 INFO:tasks.workunit.client.1.smithi142.stdout:2/666: write d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [3072638,56426] 0 2024-06-15T01:07:07.492 INFO:tasks.workunit.client.0.smithi137.stdout:8/436: fdatasync d9/d2a/d2f/f90 0 2024-06-15T01:07:07.494 INFO:tasks.workunit.client.0.smithi137.stdout:3/686: rename d0/d42/d79 to d0/d2/d23/de2 0 2024-06-15T01:07:07.497 INFO:tasks.workunit.client.1.smithi142.stdout:3/776: rename d0/d1f/l4a to d0/d1f/d20/d5d/d82/le7 0 2024-06-15T01:07:07.497 INFO:tasks.workunit.client.0.smithi137.stdout:3/687: write d0/d2f/f69 [4976474,46929] 0 2024-06-15T01:07:07.498 INFO:tasks.workunit.client.1.smithi142.stdout:9/641: creat d6/dc/d1b/d25/d35/ff1 x:0 0 0 2024-06-15T01:07:07.499 INFO:tasks.workunit.client.1.smithi142.stdout:0/681: unlink d3/d39/d3e/d81/d4c/la9 0 2024-06-15T01:07:07.500 INFO:tasks.workunit.client.0.smithi137.stdout:0/550: creat d1/d2/d13/fc0 x:0 0 0 2024-06-15T01:07:07.501 INFO:tasks.workunit.client.1.smithi142.stdout:6/686: rmdir d1a/d2d/d3b/dab 39 2024-06-15T01:07:07.503 INFO:tasks.workunit.client.1.smithi142.stdout:1/711: creat d6/d3a/fff x:0 0 0 2024-06-15T01:07:07.503 INFO:tasks.workunit.client.0.smithi137.stdout:8/437: mknod d9/de/d1e/d67/c91 0 2024-06-15T01:07:07.505 INFO:tasks.workunit.client.0.smithi137.stdout:1/551: write d0/d10/f23 [399371,25949] 0 2024-06-15T01:07:07.506 INFO:tasks.workunit.client.1.smithi142.stdout:0/682: mkdir d3/d39/d3e/d8f/df0 0 2024-06-15T01:07:07.506 INFO:tasks.workunit.client.1.smithi142.stdout:5/621: dwrite d2/de/d19/d5f/d68/f73 [4194304,4194304] 0 2024-06-15T01:07:07.507 INFO:tasks.workunit.client.0.smithi137.stdout:3/688: rmdir d0/d13 39 2024-06-15T01:07:07.508 INFO:tasks.workunit.client.1.smithi142.stdout:6/687: creat d1a/d2d/d80/fee x:0 0 0 2024-06-15T01:07:07.510 INFO:tasks.workunit.client.1.smithi142.stdout:0/683: mkdir d3/d39/d3e/d81/d44/df1 0 2024-06-15T01:07:07.511 INFO:tasks.workunit.client.1.smithi142.stdout:0/684: fsync d3/f6e 0 2024-06-15T01:07:07.511 INFO:tasks.workunit.client.1.smithi142.stdout:6/688: mknod d1a/d3c/cef 0 2024-06-15T01:07:07.512 INFO:tasks.workunit.client.1.smithi142.stdout:3/777: getdents d0/d1f/d20/d5d/d60/dc1 0 2024-06-15T01:07:07.513 INFO:tasks.workunit.client.1.smithi142.stdout:1/712: creat d6/d12/d80/f100 x:0 0 0 2024-06-15T01:07:07.514 INFO:tasks.workunit.client.0.smithi137.stdout:3/689: mkdir d0/d2/d8/d67/d7e/de3 0 2024-06-15T01:07:07.514 INFO:tasks.workunit.client.0.smithi137.stdout:8/438: dread d9/de/d1e/d35/f63 [0,4194304] 0 2024-06-15T01:07:07.515 INFO:tasks.workunit.client.0.smithi137.stdout:3/690: stat d0/d2/d8/f41 0 2024-06-15T01:07:07.515 INFO:tasks.workunit.client.1.smithi142.stdout:6/689: chown d1a/d39/f57 1651 1 2024-06-15T01:07:07.515 INFO:tasks.workunit.client.0.smithi137.stdout:3/691: stat d0/d2f/l97 0 2024-06-15T01:07:07.516 INFO:tasks.workunit.client.0.smithi137.stdout:3/692: chown d0/d77 499248 1 2024-06-15T01:07:07.519 INFO:tasks.workunit.client.0.smithi137.stdout:3/693: symlink d0/d42/le4 0 2024-06-15T01:07:07.519 INFO:tasks.workunit.client.1.smithi142.stdout:1/713: creat d6/d12/d3c/d63/d8d/d97/dc6/f101 x:0 0 0 2024-06-15T01:07:07.519 INFO:tasks.workunit.client.1.smithi142.stdout:7/707: sync 2024-06-15T01:07:07.519 INFO:tasks.workunit.client.1.smithi142.stdout:6/690: mknod d1a/d3c/db0/dd8/cf0 0 2024-06-15T01:07:07.520 INFO:tasks.workunit.client.0.smithi137.stdout:2/416: dread d0/de/d10/f33 [0,4194304] 0 2024-06-15T01:07:07.521 INFO:tasks.workunit.client.0.smithi137.stdout:3/694: symlink d0/d77/le5 0 2024-06-15T01:07:07.523 INFO:tasks.workunit.client.0.smithi137.stdout:2/417: readlink d0/da/l48 0 2024-06-15T01:07:07.523 INFO:tasks.workunit.client.1.smithi142.stdout:6/691: rename d1a/d2d/d80/d8e/fd7 to d1a/d39/ff1 0 2024-06-15T01:07:07.524 INFO:tasks.workunit.client.1.smithi142.stdout:7/708: mknod d2/d8/ce1 0 2024-06-15T01:07:07.524 INFO:tasks.workunit.client.1.smithi142.stdout:6/692: chown d1a/d39/d42/d4c/d58/d87/fa4 106 1 2024-06-15T01:07:07.524 INFO:tasks.workunit.client.0.smithi137.stdout:3/695: fdatasync d0/f1f 0 2024-06-15T01:07:07.526 INFO:tasks.workunit.client.1.smithi142.stdout:1/714: getdents d6/d12/d3c 0 2024-06-15T01:07:07.527 INFO:tasks.workunit.client.0.smithi137.stdout:3/696: write d0/d13/d37/d66/f88 [158216,52674] 0 2024-06-15T01:07:07.528 INFO:tasks.workunit.client.1.smithi142.stdout:8/712: sync 2024-06-15T01:07:07.528 INFO:tasks.workunit.client.0.smithi137.stdout:2/418: getdents d0/dc/d2e/d72/d56/d6e 0 2024-06-15T01:07:07.529 INFO:tasks.workunit.client.0.smithi137.stdout:2/419: fdatasync d0/f69 0 2024-06-15T01:07:07.529 INFO:tasks.workunit.client.0.smithi137.stdout:3/697: creat d0/d2/daf/fe6 x:0 0 0 2024-06-15T01:07:07.530 INFO:tasks.workunit.client.1.smithi142.stdout:6/693: creat d1a/d2d/d3b/dab/dce/ff2 x:0 0 0 2024-06-15T01:07:07.532 INFO:tasks.workunit.client.1.smithi142.stdout:1/715: fdatasync d6/d12/d80/dae/fb3 0 2024-06-15T01:07:07.532 INFO:tasks.workunit.client.0.smithi137.stdout:3/698: mknod d0/d2/d23/ce7 0 2024-06-15T01:07:07.534 INFO:tasks.workunit.client.0.smithi137.stdout:2/420: mknod d0/dc/d2e/d72/d56/d6e/c80 0 2024-06-15T01:07:07.534 INFO:tasks.workunit.client.0.smithi137.stdout:3/699: dread - d0/d56/d8f/fa8 zero size 2024-06-15T01:07:07.535 INFO:tasks.workunit.client.0.smithi137.stdout:3/700: dread - d0/d13/d37/d83/fe1 zero size 2024-06-15T01:07:07.536 INFO:tasks.workunit.client.0.smithi137.stdout:2/421: write d0/dc/d22/d23/f76 [259942,26958] 0 2024-06-15T01:07:07.537 INFO:tasks.workunit.client.1.smithi142.stdout:1/716: symlink d6/l102 0 2024-06-15T01:07:07.538 INFO:tasks.workunit.client.0.smithi137.stdout:0/551: truncate d1/d7e/fae 2453174 0 2024-06-15T01:07:07.538 INFO:tasks.workunit.client.1.smithi142.stdout:6/694: dwrite d1a/d3c/db0/feb [0,4194304] 0 2024-06-15T01:07:07.539 INFO:tasks.workunit.client.1.smithi142.stdout:8/713: creat d0/d25/d5a/fdc x:0 0 0 2024-06-15T01:07:07.539 INFO:tasks.workunit.client.0.smithi137.stdout:2/422: mknod d0/dc/d2e/c81 0 2024-06-15T01:07:07.539 INFO:tasks.workunit.client.0.smithi137.stdout:2/423: stat d0/de 0 2024-06-15T01:07:07.542 INFO:tasks.workunit.client.1.smithi142.stdout:8/714: chown d0/d25/d66/lb3 7748027 1 2024-06-15T01:07:07.542 INFO:tasks.workunit.client.1.smithi142.stdout:4/771: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f21 [0,4194304] 0 2024-06-15T01:07:07.543 INFO:tasks.workunit.client.0.smithi137.stdout:0/552: symlink d1/d2/dc/d10/lc1 0 2024-06-15T01:07:07.544 INFO:tasks.workunit.client.1.smithi142.stdout:9/642: write d6/d34/f8b [451886,107488] 0 2024-06-15T01:07:07.544 INFO:tasks.workunit.client.1.smithi142.stdout:4/772: dread - d5/d9/d90/fff zero size 2024-06-15T01:07:07.544 INFO:tasks.workunit.client.1.smithi142.stdout:8/715: dread - d0/d17/d40/d5e/f94 zero size 2024-06-15T01:07:07.544 INFO:tasks.workunit.client.0.smithi137.stdout:8/439: truncate d9/d2a/d79/f7d 1424684 0 2024-06-15T01:07:07.547 INFO:tasks.workunit.client.0.smithi137.stdout:2/424: creat d0/dc/d22/d23/d5c/f82 x:0 0 0 2024-06-15T01:07:07.547 INFO:tasks.workunit.client.1.smithi142.stdout:4/773: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/fce zero size 2024-06-15T01:07:07.547 INFO:tasks.workunit.client.1.smithi142.stdout:1/717: truncate d6/d46/f78 631915 0 2024-06-15T01:07:07.548 INFO:tasks.workunit.client.1.smithi142.stdout:1/718: write d6/d12/f2d [8060126,7257] 0 2024-06-15T01:07:07.550 INFO:tasks.workunit.client.1.smithi142.stdout:2/667: dread d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [0,4194304] 0 2024-06-15T01:07:07.551 INFO:tasks.workunit.client.1.smithi142.stdout:9/643: truncate d6/f9 1860772 0 2024-06-15T01:07:07.552 INFO:tasks.workunit.client.1.smithi142.stdout:8/716: truncate d0/f6f 489564 0 2024-06-15T01:07:07.552 INFO:tasks.workunit.client.1.smithi142.stdout:9/644: write d6/d34/fc0 [1537997,109121] 0 2024-06-15T01:07:07.553 INFO:tasks.workunit.client.1.smithi142.stdout:5/622: truncate d2/d18/f99 2408318 0 2024-06-15T01:07:07.554 INFO:tasks.workunit.client.0.smithi137.stdout:8/440: rename d9/de/l80 to d9/de/d1e/l92 0 2024-06-15T01:07:07.554 INFO:tasks.workunit.client.1.smithi142.stdout:4/774: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d10a 0 2024-06-15T01:07:07.555 INFO:tasks.workunit.client.0.smithi137.stdout:8/441: fsync d9/d2a/f6f 0 2024-06-15T01:07:07.557 INFO:tasks.workunit.client.1.smithi142.stdout:5/623: dread d2/de/f66 [0,4194304] 0 2024-06-15T01:07:07.558 INFO:tasks.workunit.client.1.smithi142.stdout:0/685: write d3/d39/d3e/d81/f17 [3025122,19906] 0 2024-06-15T01:07:07.561 INFO:tasks.workunit.client.0.smithi137.stdout:2/425: getdents d0/dc/d2e 0 2024-06-15T01:07:07.563 INFO:tasks.workunit.client.1.smithi142.stdout:2/668: rmdir d8/df/daf 39 2024-06-15T01:07:07.563 INFO:tasks.workunit.client.1.smithi142.stdout:3/778: truncate d0/d1f/d20/d5d/d60/f85 2277727 0 2024-06-15T01:07:07.564 INFO:tasks.workunit.client.0.smithi137.stdout:3/701: dwrite d0/d2/d22/d24/d5e/f85 [0,4194304] 0 2024-06-15T01:07:07.565 INFO:tasks.workunit.client.1.smithi142.stdout:3/779: truncate d0/d3/da/d18/fd4 42046 0 2024-06-15T01:07:07.567 INFO:tasks.workunit.client.0.smithi137.stdout:2/426: dread d0/da/f11 [0,4194304] 0 2024-06-15T01:07:07.573 INFO:tasks.workunit.client.1.smithi142.stdout:4/775: truncate d5/d9/d90/fc4 948264 0 2024-06-15T01:07:07.574 INFO:tasks.workunit.client.1.smithi142.stdout:7/709: truncate d2/d8/dd/dcf/f2f 1294710 0 2024-06-15T01:07:07.576 INFO:tasks.workunit.client.0.smithi137.stdout:0/553: dread d1/d2/d13/d66/f4a [0,4194304] 0 2024-06-15T01:07:07.576 INFO:tasks.workunit.client.1.smithi142.stdout:5/624: creat d2/de/d27/d86/fc1 x:0 0 0 2024-06-15T01:07:07.579 INFO:tasks.workunit.client.0.smithi137.stdout:8/442: link d9/c58 d9/d2a/d2f/d59/c93 0 2024-06-15T01:07:07.581 INFO:tasks.workunit.client.0.smithi137.stdout:0/554: rmdir d1/d2/dc/d10/d63/d77 39 2024-06-15T01:07:07.584 INFO:tasks.workunit.client.0.smithi137.stdout:3/702: dwrite d0/d56/f84 [0,4194304] 0 2024-06-15T01:07:07.585 INFO:tasks.workunit.client.0.smithi137.stdout:3/703: chown d0/d13/fba 2 1 2024-06-15T01:07:07.585 INFO:tasks.workunit.client.0.smithi137.stdout:0/555: rmdir d1/d7e/d87 39 2024-06-15T01:07:07.587 INFO:tasks.workunit.client.0.smithi137.stdout:3/704: chown d0/d2/d22/d24/f54 112772 1 2024-06-15T01:07:07.587 INFO:tasks.workunit.client.1.smithi142.stdout:4/776: symlink d5/d9/d90/l10b 0 2024-06-15T01:07:07.589 INFO:tasks.workunit.client.0.smithi137.stdout:3/705: chown d0/d2/d23 2530105 1 2024-06-15T01:07:07.589 INFO:tasks.workunit.client.1.smithi142.stdout:7/710: stat d2/d8/dd/d8a/d94/cb0 0 2024-06-15T01:07:07.591 INFO:tasks.workunit.client.0.smithi137.stdout:8/443: getdents d9/d2a/d2f/d59/d82 0 2024-06-15T01:07:07.593 INFO:tasks.workunit.client.0.smithi137.stdout:0/556: rename d1/d2/d13/d66/l74 to d1/d2/dc/d6c/d90/lc2 0 2024-06-15T01:07:07.595 INFO:tasks.workunit.client.0.smithi137.stdout:3/706: dread d0/d2/d22/d24/d5e/f85 [0,4194304] 0 2024-06-15T01:07:07.595 INFO:tasks.workunit.client.0.smithi137.stdout:2/427: dread d0/f69 [0,4194304] 0 2024-06-15T01:07:07.596 INFO:tasks.workunit.client.0.smithi137.stdout:2/428: readlink d0/dc/d2e/d72/d56/d6e/l7f 0 2024-06-15T01:07:07.605 INFO:tasks.workunit.client.0.smithi137.stdout:2/429: readlink d0/l30 0 2024-06-15T01:07:07.607 INFO:tasks.workunit.client.0.smithi137.stdout:2/430: fdatasync d0/dc/d2e/d72/f67 0 2024-06-15T01:07:07.608 INFO:tasks.workunit.client.0.smithi137.stdout:0/557: mkdir d1/d2/d13/d66/dc3 0 2024-06-15T01:07:07.610 INFO:tasks.workunit.client.0.smithi137.stdout:0/558: dread d1/d2/d13/d66/f60 [0,4194304] 0 2024-06-15T01:07:07.611 INFO:tasks.workunit.client.1.smithi142.stdout:4/777: creat d5/d9/da/d92/d98/dc1/f10c x:0 0 0 2024-06-15T01:07:07.612 INFO:tasks.workunit.client.1.smithi142.stdout:1/719: dwrite f5 [4194304,4194304] 0 2024-06-15T01:07:07.612 INFO:tasks.workunit.client.1.smithi142.stdout:6/695: truncate d1a/d39/d3a/dd2/fe2 2767121 0 2024-06-15T01:07:07.613 INFO:tasks.workunit.client.0.smithi137.stdout:8/444: creat d9/de/f94 x:0 0 0 2024-06-15T01:07:07.614 INFO:tasks.workunit.client.1.smithi142.stdout:5/625: creat d2/de/d19/d5f/d68/d76/d96/dac/fc2 x:0 0 0 2024-06-15T01:07:07.616 INFO:tasks.workunit.client.0.smithi137.stdout:3/707: dwrite d0/d2/f15 [0,4194304] 0 2024-06-15T01:07:07.616 INFO:tasks.workunit.client.0.smithi137.stdout:8/445: truncate d9/de/d1e/f7f 449817 0 2024-06-15T01:07:07.617 INFO:tasks.workunit.client.1.smithi142.stdout:5/626: read d2/de/d27/f46 [2493258,69360] 0 2024-06-15T01:07:07.617 INFO:tasks.workunit.client.1.smithi142.stdout:7/711: mkdir d2/d8/dd/d8a/de2 0 2024-06-15T01:07:07.619 INFO:tasks.workunit.client.1.smithi142.stdout:4/778: dread d5/d1c/d6f/dc7/fab [0,4194304] 0 2024-06-15T01:07:07.624 INFO:tasks.workunit.client.1.smithi142.stdout:4/779: write d5/d9/d90/daf/fed [451326,128452] 0 2024-06-15T01:07:07.626 INFO:tasks.workunit.client.0.smithi137.stdout:8/446: creat d9/de/d1e/d67/f95 x:0 0 0 2024-06-15T01:07:07.626 INFO:tasks.workunit.client.1.smithi142.stdout:8/717: write d0/d17/d40/d6e/f73 [1294648,101205] 0 2024-06-15T01:07:07.627 INFO:tasks.workunit.client.1.smithi142.stdout:6/696: creat d1a/d2d/d68/da0/ff3 x:0 0 0 2024-06-15T01:07:07.629 INFO:tasks.workunit.client.1.smithi142.stdout:2/669: dwrite d8/df/d10/d94/d25/d1a/d63/f77 [4194304,4194304] 0 2024-06-15T01:07:07.629 INFO:tasks.workunit.client.1.smithi142.stdout:5/627: creat d2/d3e/d31/fc3 x:0 0 0 2024-06-15T01:07:07.629 INFO:tasks.workunit.client.1.smithi142.stdout:9/645: dwrite d6/dc/d82/fd4 [0,4194304] 0 2024-06-15T01:07:07.632 INFO:tasks.workunit.client.1.smithi142.stdout:7/712: mknod d2/d8/dd/d14/d2c/d70/ce3 0 2024-06-15T01:07:07.632 INFO:tasks.workunit.client.1.smithi142.stdout:6/697: write d1a/d39/d42/d4c/d58/f7a [1157322,49570] 0 2024-06-15T01:07:07.632 INFO:tasks.workunit.client.1.smithi142.stdout:8/718: write d0/d25/d53/da7/dcb/fd9 [1137163,116170] 0 2024-06-15T01:07:07.632 INFO:tasks.workunit.client.0.smithi137.stdout:3/708: dwrite d0/d56/fca [0,4194304] 0 2024-06-15T01:07:07.635 INFO:tasks.workunit.client.0.smithi137.stdout:8/447: dwrite d9/f1d [0,4194304] 0 2024-06-15T01:07:07.636 INFO:tasks.workunit.client.0.smithi137.stdout:2/431: write d0/da/f7c [162314,50914] 0 2024-06-15T01:07:07.638 INFO:tasks.workunit.client.1.smithi142.stdout:6/698: write d1a/d2d/d68/da0/fa5 [3065703,118658] 0 2024-06-15T01:07:07.640 INFO:tasks.workunit.client.0.smithi137.stdout:3/709: mkdir d0/d13/d37/de8 0 2024-06-15T01:07:07.640 INFO:tasks.workunit.client.1.smithi142.stdout:8/719: fdatasync d0/d25/d5a/d84/fa0 0 2024-06-15T01:07:07.641 INFO:tasks.workunit.client.0.smithi137.stdout:2/432: write d0/dc/d22/d23/d5c/f82 [7806,17980] 0 2024-06-15T01:07:07.643 INFO:tasks.workunit.client.1.smithi142.stdout:6/699: write d1a/fdc [140554,110380] 0 2024-06-15T01:07:07.645 INFO:tasks.workunit.client.0.smithi137.stdout:3/710: readlink d0/d2/d23/l3a 0 2024-06-15T01:07:07.645 INFO:tasks.workunit.client.1.smithi142.stdout:0/686: sync 2024-06-15T01:07:07.645 INFO:tasks.workunit.client.1.smithi142.stdout:5/628: dread d2/de/d27/f38 [0,4194304] 0 2024-06-15T01:07:07.646 INFO:tasks.workunit.client.0.smithi137.stdout:2/433: write d0/f6 [114565,42511] 0 2024-06-15T01:07:07.648 INFO:tasks.workunit.client.0.smithi137.stdout:0/559: dwrite d1/d2/dc/f73 [0,4194304] 0 2024-06-15T01:07:07.650 INFO:tasks.workunit.client.1.smithi142.stdout:2/670: truncate d8/df/d10/d94/d25/d22/d43/d52/dc1/fcb 524409 0 2024-06-15T01:07:07.651 INFO:tasks.workunit.client.1.smithi142.stdout:2/671: stat d8/d19/d33/c6c 0 2024-06-15T01:07:07.654 INFO:tasks.workunit.client.1.smithi142.stdout:3/780: dwrite d0/d3/f1a [0,4194304] 0 2024-06-15T01:07:07.656 INFO:tasks.workunit.client.1.smithi142.stdout:2/672: stat d8/df/d10/d94/d25/d1a/l9b 0 2024-06-15T01:07:07.658 INFO:tasks.workunit.client.1.smithi142.stdout:9/646: mkdir d6/dc/d51/d53/df2 0 2024-06-15T01:07:07.659 INFO:tasks.workunit.client.0.smithi137.stdout:0/560: read d1/d2/f21 [93393,127122] 0 2024-06-15T01:07:07.659 INFO:tasks.workunit.client.1.smithi142.stdout:3/781: write d0/d1f/f38 [1189856,5367] 0 2024-06-15T01:07:07.660 INFO:tasks.workunit.client.1.smithi142.stdout:1/720: dread d6/d12/d80/dc8/fd1 [0,4194304] 0 2024-06-15T01:07:07.661 INFO:tasks.workunit.client.1.smithi142.stdout:7/713: creat d2/d8/da/d33/d25/d79/fe4 x:0 0 0 2024-06-15T01:07:07.663 INFO:tasks.workunit.client.0.smithi137.stdout:3/711: dwrite d0/d2/d22/d63/d76/fce [0,4194304] 0 2024-06-15T01:07:07.665 INFO:tasks.workunit.client.1.smithi142.stdout:6/700: rename d1a/d2d/d3b/l67 to d1a/d2d/d3b/dab/dce/lf4 0 2024-06-15T01:07:07.665 INFO:tasks.workunit.client.1.smithi142.stdout:8/720: dread d0/f3f [4194304,4194304] 0 2024-06-15T01:07:07.666 INFO:tasks.workunit.client.0.smithi137.stdout:0/561: mkdir d1/d7e/d87/da0/dc4 0 2024-06-15T01:07:07.666 INFO:tasks.workunit.client.1.smithi142.stdout:5/629: chown d2/de/f5a 1048 1 2024-06-15T01:07:07.669 INFO:tasks.workunit.client.0.smithi137.stdout:3/712: unlink d0/d2f/l93 0 2024-06-15T01:07:07.671 INFO:tasks.workunit.client.0.smithi137.stdout:0/562: creat d1/d2/dc/d10/d14/d15/d1c/fc5 x:0 0 0 2024-06-15T01:07:07.673 INFO:tasks.workunit.client.1.smithi142.stdout:1/721: mknod d6/d12/d1c/d2c/d93/db9/c103 0 2024-06-15T01:07:07.674 INFO:tasks.workunit.client.1.smithi142.stdout:7/714: mkdir d2/d6/d22/d47/de5 0 2024-06-15T01:07:07.675 INFO:tasks.workunit.client.1.smithi142.stdout:7/715: chown d2/d6/l64 1 1 2024-06-15T01:07:07.677 INFO:tasks.workunit.client.1.smithi142.stdout:6/701: symlink d1a/d39/d3a/d51/lf5 0 2024-06-15T01:07:07.678 INFO:tasks.workunit.client.0.smithi137.stdout:3/713: dwrite d0/d13/d37/d66/f88 [0,4194304] 0 2024-06-15T01:07:07.678 INFO:tasks.workunit.client.1.smithi142.stdout:8/721: write d0/d25/d5a/f4d [2823664,118271] 0 2024-06-15T01:07:07.678 INFO:tasks.workunit.client.0.smithi137.stdout:3/714: chown d0/d13 1731653 1 2024-06-15T01:07:07.679 INFO:tasks.workunit.client.1.smithi142.stdout:9/647: mkdir d6/d34/df3 0 2024-06-15T01:07:07.680 INFO:tasks.workunit.client.1.smithi142.stdout:2/673: symlink d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf/ld3 0 2024-06-15T01:07:07.680 INFO:tasks.workunit.client.0.smithi137.stdout:0/563: read d1/d2/dc/d10/f75 [3636012,108371] 0 2024-06-15T01:07:07.680 INFO:tasks.workunit.client.0.smithi137.stdout:3/715: write d0/d56/d8f/fcf [4602935,47449] 0 2024-06-15T01:07:07.681 INFO:tasks.workunit.client.0.smithi137.stdout:3/716: fdatasync d0/d2/d23/d68/f3c 0 2024-06-15T01:07:07.682 INFO:tasks.workunit.client.0.smithi137.stdout:0/564: write d1/d2/d55/faf [1900333,51534] 0 2024-06-15T01:07:07.684 INFO:tasks.workunit.client.1.smithi142.stdout:6/702: unlink d1a/d39/d3a/fe8 0 2024-06-15T01:07:07.686 INFO:tasks.workunit.client.1.smithi142.stdout:5/630: dread d2/d3e/d31/f3c [0,4194304] 0 2024-06-15T01:07:07.686 INFO:tasks.workunit.client.1.smithi142.stdout:8/722: symlink d0/d25/d5a/d8f/d98/da4/ldd 0 2024-06-15T01:07:07.687 INFO:tasks.workunit.client.1.smithi142.stdout:5/631: chown d2/ca 423539452 1 2024-06-15T01:07:07.687 INFO:tasks.workunit.client.0.smithi137.stdout:3/717: creat d0/d2f/dbe/fe9 x:0 0 0 2024-06-15T01:07:07.688 INFO:tasks.workunit.client.1.smithi142.stdout:9/648: symlink d6/dc/d1b/d25/d35/lf4 0 2024-06-15T01:07:07.688 INFO:tasks.workunit.client.1.smithi142.stdout:8/723: chown d0/d17/l89 7131 1 2024-06-15T01:07:07.688 INFO:tasks.workunit.client.1.smithi142.stdout:2/674: readlink l5 0 2024-06-15T01:07:07.691 INFO:tasks.workunit.client.1.smithi142.stdout:3/782: getdents d0/d3/da/d18 0 2024-06-15T01:07:07.692 INFO:tasks.workunit.client.0.smithi137.stdout:3/718: truncate d0/d2/d23/f3d 1782525 0 2024-06-15T01:07:07.692 INFO:tasks.workunit.client.1.smithi142.stdout:3/783: fsync d0/d1f/d20/d59/d61/d96/fcc 0 2024-06-15T01:07:07.694 INFO:tasks.workunit.client.1.smithi142.stdout:6/703: dread d1a/d3c/db0/fc0 [0,4194304] 0 2024-06-15T01:07:07.695 INFO:tasks.workunit.client.1.smithi142.stdout:2/675: creat d8/fd4 x:0 0 0 2024-06-15T01:07:07.696 INFO:tasks.workunit.client.1.smithi142.stdout:5/632: mkdir d2/de/d19/d5f/d68/d76/d8c/dc4 0 2024-06-15T01:07:07.697 INFO:tasks.workunit.client.1.smithi142.stdout:2/676: chown d8/df/d10/d94/d25/d22/d43/d52/cce 31927 1 2024-06-15T01:07:07.697 INFO:tasks.workunit.client.0.smithi137.stdout:3/719: read d0/d2/d23/f48 [2323742,27184] 0 2024-06-15T01:07:07.698 INFO:tasks.workunit.client.1.smithi142.stdout:8/724: rename d0/d25/d5a/c4e to d0/d17/d40/d5e/cde 0 2024-06-15T01:07:07.698 INFO:tasks.workunit.client.1.smithi142.stdout:4/780: dwrite d5/d9/d1a/f82 [0,4194304] 0 2024-06-15T01:07:07.699 INFO:tasks.workunit.client.1.smithi142.stdout:2/677: write d8/d19/d33/f83 [1476867,27133] 0 2024-06-15T01:07:07.701 INFO:tasks.workunit.client.0.smithi137.stdout:3/720: unlink d0/d2/d22/d24/d28/fb4 0 2024-06-15T01:07:07.701 INFO:tasks.workunit.client.1.smithi142.stdout:1/722: getdents d6/d12/d80/d9b 0 2024-06-15T01:07:07.701 INFO:tasks.workunit.client.1.smithi142.stdout:3/784: write d0/d1b/f88 [4185136,125101] 0 2024-06-15T01:07:07.703 INFO:tasks.workunit.client.1.smithi142.stdout:8/725: write d0/d25/d5a/d8f/d98/da4/fb5 [940841,22874] 0 2024-06-15T01:07:07.704 INFO:tasks.workunit.client.1.smithi142.stdout:1/723: dread - d6/d25/fe2 zero size 2024-06-15T01:07:07.705 INFO:tasks.workunit.client.1.smithi142.stdout:5/633: mkdir d2/de/d19/d5f/d68/d76/da6/dc5 0 2024-06-15T01:07:07.705 INFO:tasks.workunit.client.1.smithi142.stdout:7/716: dread d2/d6/d22/ddd/f9c [0,4194304] 0 2024-06-15T01:07:07.708 INFO:tasks.workunit.client.0.smithi137.stdout:3/721: link d0/d2/d23/f6f d0/d2/d22/d24/d28/fea 0 2024-06-15T01:07:07.708 INFO:tasks.workunit.client.1.smithi142.stdout:3/785: unlink d0/d3/la8 0 2024-06-15T01:07:07.708 INFO:tasks.workunit.client.1.smithi142.stdout:7/717: write d2/d6/f8f [2238899,31844] 0 2024-06-15T01:07:07.710 INFO:tasks.workunit.client.1.smithi142.stdout:4/781: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/l10d 0 2024-06-15T01:07:07.711 INFO:tasks.workunit.client.0.smithi137.stdout:3/722: creat d0/d2/d1e/feb x:0 0 0 2024-06-15T01:07:07.711 INFO:tasks.workunit.client.1.smithi142.stdout:8/726: chown d0/d25/c2b 126308 1 2024-06-15T01:07:07.711 INFO:tasks.workunit.client.0.smithi137.stdout:8/448: write d9/de/d1e/f28 [2865108,83201] 0 2024-06-15T01:07:07.712 INFO:tasks.workunit.client.0.smithi137.stdout:3/723: stat d0/d2/d23/de2/l9e 0 2024-06-15T01:07:07.713 INFO:tasks.workunit.client.0.smithi137.stdout:0/565: dread d1/d2/f47 [4194304,4194304] 0 2024-06-15T01:07:07.713 INFO:tasks.workunit.client.0.smithi137.stdout:2/434: truncate d0/d42/d49/f62 934799 0 2024-06-15T01:07:07.713 INFO:tasks.workunit.client.1.smithi142.stdout:4/782: stat d5/d9/da/da3/de9 0 2024-06-15T01:07:07.718 INFO:tasks.workunit.client.1.smithi142.stdout:4/783: chown d5/d1c/d6f/dc7/d30/lf2 8760 1 2024-06-15T01:07:07.720 INFO:tasks.workunit.client.1.smithi142.stdout:3/786: dwrite d0/d3/f43 [0,4194304] 0 2024-06-15T01:07:07.722 INFO:tasks.workunit.client.0.smithi137.stdout:0/566: rmdir d1/d2/dc/d10/d14/d15/d17/d54 39 2024-06-15T01:07:07.724 INFO:tasks.workunit.client.1.smithi142.stdout:8/727: dwrite d0/d25/d53/da7/fd6 [0,4194304] 0 2024-06-15T01:07:07.724 INFO:tasks.workunit.client.1.smithi142.stdout:9/649: dread d6/d31/f86 [0,4194304] 0 2024-06-15T01:07:07.724 INFO:tasks.workunit.client.1.smithi142.stdout:4/784: write d5/d9/d23/d32/ffa [1042130,45164] 0 2024-06-15T01:07:07.724 INFO:tasks.workunit.client.0.smithi137.stdout:3/724: dwrite d0/d56/d8f/fcf [0,4194304] 0 2024-06-15T01:07:07.726 INFO:tasks.workunit.client.1.smithi142.stdout:2/678: rmdir d8/df/d10/d94/d25/d22/d43/d52/dd1 0 2024-06-15T01:07:07.729 INFO:tasks.workunit.client.1.smithi142.stdout:2/679: truncate d8/df/d10/d94/fc9 473628 0 2024-06-15T01:07:07.730 INFO:tasks.workunit.client.0.smithi137.stdout:0/567: creat d1/d2/dc/d6c/d90/fc6 x:0 0 0 2024-06-15T01:07:07.731 INFO:tasks.workunit.client.1.smithi142.stdout:2/680: write d8/df/d10/d94/d25/f6a [1196517,6554] 0 2024-06-15T01:07:07.735 INFO:tasks.workunit.client.1.smithi142.stdout:6/704: dread d1a/f46 [0,4194304] 0 2024-06-15T01:07:07.735 INFO:tasks.workunit.client.1.smithi142.stdout:1/724: link d6/d12/d1c/d2c/d47/f5f d6/d12/d80/d9b/db2/dbf/f104 0 2024-06-15T01:07:07.736 INFO:tasks.workunit.client.1.smithi142.stdout:6/705: fdatasync d1a/f1f 0 2024-06-15T01:07:07.737 INFO:tasks.workunit.client.1.smithi142.stdout:6/706: stat d1a/d39/f3f 0 2024-06-15T01:07:07.737 INFO:tasks.workunit.client.1.smithi142.stdout:1/725: write d6/d12/da0/db0/f73 [2606614,49689] 0 2024-06-15T01:07:07.737 INFO:tasks.workunit.client.1.smithi142.stdout:3/787: creat d0/fe8 x:0 0 0 2024-06-15T01:07:07.738 INFO:tasks.workunit.client.0.smithi137.stdout:3/725: mknod d0/d13/d37/cec 0 2024-06-15T01:07:07.740 INFO:tasks.workunit.client.1.smithi142.stdout:2/681: dread d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [0,4194304] 0 2024-06-15T01:07:07.742 INFO:tasks.workunit.client.0.smithi137.stdout:8/449: write d9/f1b [230382,436] 0 2024-06-15T01:07:07.743 INFO:tasks.workunit.client.0.smithi137.stdout:2/435: dread d0/de/d10/f19 [4194304,4194304] 0 2024-06-15T01:07:07.744 INFO:tasks.workunit.client.0.smithi137.stdout:2/436: chown d0/dc/c14 2 1 2024-06-15T01:07:07.744 INFO:tasks.workunit.client.1.smithi142.stdout:4/785: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/fd7 to d5/d1c/d6f/f10e 0 2024-06-15T01:07:07.745 INFO:tasks.workunit.client.1.smithi142.stdout:6/707: dwrite d1a/d39/d42/d4c/d58/d87/fa4 [0,4194304] 0 2024-06-15T01:07:07.746 INFO:tasks.workunit.client.1.smithi142.stdout:5/634: getdents d2/de/d27/db2 0 2024-06-15T01:07:07.752 INFO:tasks.workunit.client.0.smithi137.stdout:8/450: getdents d9/d2a/d79/d7e 0 2024-06-15T01:07:07.753 INFO:tasks.workunit.client.1.smithi142.stdout:1/726: fsync d6/f10 0 2024-06-15T01:07:07.753 INFO:tasks.workunit.client.0.smithi137.stdout:8/451: dread - d9/d2a/d2f/d59/f87 zero size 2024-06-15T01:07:07.754 INFO:tasks.workunit.client.0.smithi137.stdout:0/568: write d1/d2/dc/d10/f4f [3025863,52008] 0 2024-06-15T01:07:07.754 INFO:tasks.workunit.client.1.smithi142.stdout:1/727: write d6/d12/d1c/f34 [5703304,117207] 0 2024-06-15T01:07:07.755 INFO:tasks.workunit.client.1.smithi142.stdout:9/650: creat d6/d34/d8a/dec/ff5 x:0 0 0 2024-06-15T01:07:07.756 INFO:tasks.workunit.client.0.smithi137.stdout:0/569: fdatasync d1/d2/dc/d10/d14/d56/f85 0 2024-06-15T01:07:07.757 INFO:tasks.workunit.client.1.smithi142.stdout:8/728: getdents d0/d17/d40/d86/d9d 0 2024-06-15T01:07:07.757 INFO:tasks.workunit.client.0.smithi137.stdout:3/726: truncate d0/d77/fc2 324294 0 2024-06-15T01:07:07.757 INFO:tasks.workunit.client.1.smithi142.stdout:8/729: write d0/d17/d40/fa8 [4132512,67914] 0 2024-06-15T01:07:07.758 INFO:tasks.workunit.client.1.smithi142.stdout:8/730: readlink d0/d25/d53/da7/dcb/ld3 0 2024-06-15T01:07:07.759 INFO:tasks.workunit.client.0.smithi137.stdout:2/437: write d0/dc/d22/d23/f53 [194938,55211] 0 2024-06-15T01:07:07.760 INFO:tasks.workunit.client.1.smithi142.stdout:4/786: write d5/d1c/d6f/dc7/d30/d46/d50/f91 [3500194,85159] 0 2024-06-15T01:07:07.760 INFO:tasks.workunit.client.1.smithi142.stdout:0/687: sync 2024-06-15T01:07:07.762 INFO:tasks.workunit.client.1.smithi142.stdout:7/718: write d2/d8/f2d [40895,102310] 0 2024-06-15T01:07:07.763 INFO:tasks.workunit.client.0.smithi137.stdout:3/727: mknod d0/d56/d8f/d7b/ced 0 2024-06-15T01:07:07.763 INFO:tasks.workunit.client.0.smithi137.stdout:8/452: dwrite d9/de/f7b [0,4194304] 0 2024-06-15T01:07:07.767 INFO:tasks.workunit.client.0.smithi137.stdout:0/570: dread d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:07.768 INFO:tasks.workunit.client.1.smithi142.stdout:1/728: unlink d6/c14 0 2024-06-15T01:07:07.768 INFO:tasks.workunit.client.0.smithi137.stdout:2/438: getdents d0/dc/d2e/d72 0 2024-06-15T01:07:07.768 INFO:tasks.workunit.client.1.smithi142.stdout:1/729: stat d6/d12/d1c/cce 0 2024-06-15T01:07:07.769 INFO:tasks.workunit.client.0.smithi137.stdout:3/728: symlink d0/d2/d8/lee 0 2024-06-15T01:07:07.770 INFO:tasks.workunit.client.1.smithi142.stdout:2/682: mknod d8/df/cd5 0 2024-06-15T01:07:07.772 INFO:tasks.workunit.client.0.smithi137.stdout:0/571: rmdir d1/d2/dc/d10/d14/d15/d1c 39 2024-06-15T01:07:07.775 INFO:tasks.workunit.client.1.smithi142.stdout:5/635: creat d2/d3e/d31/d7e/fc6 x:0 0 0 2024-06-15T01:07:07.775 INFO:tasks.workunit.client.1.smithi142.stdout:1/730: dwrite d6/d25/f3d [0,4194304] 0 2024-06-15T01:07:07.776 INFO:tasks.workunit.client.0.smithi137.stdout:2/439: dwrite d0/f1 [4194304,4194304] 0 2024-06-15T01:07:07.776 INFO:tasks.workunit.client.1.smithi142.stdout:7/719: mkdir d2/db6/de6 0 2024-06-15T01:07:07.776 INFO:tasks.workunit.client.1.smithi142.stdout:5/636: dread - d2/d3e/d98/fbf zero size 2024-06-15T01:07:07.780 INFO:tasks.workunit.client.1.smithi142.stdout:9/651: unlink d6/d34/c36 0 2024-06-15T01:07:07.783 INFO:tasks.workunit.client.1.smithi142.stdout:2/683: write d8/df/f3f [4657123,65437] 0 2024-06-15T01:07:07.786 INFO:tasks.workunit.client.1.smithi142.stdout:5/637: truncate d2/de/d19/d5f/d68/d76/f7c 639383 0 2024-06-15T01:07:07.786 INFO:tasks.workunit.client.1.smithi142.stdout:6/708: link d1a/d39/d42/d4c/c78 d1a/d2d/d68/cf6 0 2024-06-15T01:07:07.787 INFO:tasks.workunit.client.1.smithi142.stdout:1/731: creat d6/d12/db5/f105 x:0 0 0 2024-06-15T01:07:07.789 INFO:tasks.workunit.client.1.smithi142.stdout:5/638: creat d2/d3e/d65/fc7 x:0 0 0 2024-06-15T01:07:07.790 INFO:tasks.workunit.client.1.smithi142.stdout:6/709: mkdir d1a/d39/d42/df7 0 2024-06-15T01:07:07.791 INFO:tasks.workunit.client.1.smithi142.stdout:9/652: dwrite d6/dc/d51/d53/fc9 [0,4194304] 0 2024-06-15T01:07:07.791 INFO:tasks.workunit.client.1.smithi142.stdout:6/710: stat d1a/f76 0 2024-06-15T01:07:07.794 INFO:tasks.workunit.client.0.smithi137.stdout:5/587: sync 2024-06-15T01:07:07.797 INFO:tasks.workunit.client.1.smithi142.stdout:2/684: dwrite d8/df/d10/d94/d25/f53 [0,4194304] 0 2024-06-15T01:07:07.797 INFO:tasks.workunit.client.0.smithi137.stdout:5/588: symlink df/d11/d30/lc6 0 2024-06-15T01:07:07.799 INFO:tasks.workunit.client.1.smithi142.stdout:9/653: mkdir d6/dc/d1b/d6f/da1/df6 0 2024-06-15T01:07:07.802 INFO:tasks.workunit.client.1.smithi142.stdout:2/685: fdatasync d8/df/d10/d94/d25/f6a 0 2024-06-15T01:07:07.803 INFO:tasks.workunit.client.0.smithi137.stdout:5/589: getdents df/d76/d7c 0 2024-06-15T01:07:07.803 INFO:tasks.workunit.client.1.smithi142.stdout:6/711: dwrite f0 [4194304,4194304] 0 2024-06-15T01:07:07.804 INFO:tasks.workunit.client.1.smithi142.stdout:8/731: read d0/d25/d66/f95 [19058,107907] 0 2024-06-15T01:07:07.805 INFO:tasks.workunit.client.1.smithi142.stdout:5/639: creat d2/de/d19/d5f/d68/d76/fc8 x:0 0 0 2024-06-15T01:07:07.812 INFO:tasks.workunit.client.1.smithi142.stdout:2/686: mknod d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/cd6 0 2024-06-15T01:07:07.812 INFO:tasks.workunit.client.1.smithi142.stdout:6/712: creat d1a/d39/d3a/dd2/ff8 x:0 0 0 2024-06-15T01:07:07.814 INFO:tasks.workunit.client.1.smithi142.stdout:8/732: mkdir d0/d25/d5a/d84/dd0/ddf 0 2024-06-15T01:07:07.817 INFO:tasks.workunit.client.0.smithi137.stdout:0/572: dread d1/d2/dc/d10/d14/d15/f59 [0,4194304] 0 2024-06-15T01:07:07.819 INFO:tasks.workunit.client.1.smithi142.stdout:1/732: dread d6/f26 [0,4194304] 0 2024-06-15T01:07:07.821 INFO:tasks.workunit.client.1.smithi142.stdout:8/733: write d0/d17/f6b [4753609,36774] 0 2024-06-15T01:07:07.823 INFO:tasks.workunit.client.1.smithi142.stdout:8/734: truncate d0/d25/d53/f58 4762107 0 2024-06-15T01:07:07.827 INFO:tasks.workunit.client.0.smithi137.stdout:0/573: creat d1/d2/dc/d10/d22/fc7 x:0 0 0 2024-06-15T01:07:07.828 INFO:tasks.workunit.client.1.smithi142.stdout:7/720: dread d2/d8/da/d33/d25/f40 [0,4194304] 0 2024-06-15T01:07:07.828 INFO:tasks.workunit.client.0.smithi137.stdout:0/574: chown d1/d2/dc/d10/d14/d15/l24 1 1 2024-06-15T01:07:07.829 INFO:tasks.workunit.client.0.smithi137.stdout:8/453: write d9/d2a/f69 [489484,62307] 0 2024-06-15T01:07:07.831 INFO:tasks.workunit.client.1.smithi142.stdout:6/713: dread d1a/fea [0,4194304] 0 2024-06-15T01:07:07.831 INFO:tasks.workunit.client.0.smithi137.stdout:3/729: write d0/f1f [1658683,93096] 0 2024-06-15T01:07:07.833 INFO:tasks.workunit.client.1.smithi142.stdout:5/640: link d2/d18/f55 d2/d18/fc9 0 2024-06-15T01:07:07.833 INFO:tasks.workunit.client.1.smithi142.stdout:0/688: write d3/d39/d3e/fa5 [3812283,56359] 0 2024-06-15T01:07:07.834 INFO:tasks.workunit.client.1.smithi142.stdout:3/788: mknod d0/d3/de2/d55/dae/ce9 0 2024-06-15T01:07:07.836 INFO:tasks.workunit.client.0.smithi137.stdout:3/730: truncate d0/d2/d23/f3d 1166341 0 2024-06-15T01:07:07.837 INFO:tasks.workunit.client.0.smithi137.stdout:3/731: write d0/d2/d23/d68/f3c [4102537,56287] 0 2024-06-15T01:07:07.840 INFO:tasks.workunit.client.1.smithi142.stdout:5/641: unlink d2/de/d19/d5f/d68/d76/c7b 0 2024-06-15T01:07:07.840 INFO:tasks.workunit.client.1.smithi142.stdout:0/689: dwrite d3/d39/d9a/fa8 [0,4194304] 0 2024-06-15T01:07:07.841 INFO:tasks.workunit.client.0.smithi137.stdout:5/590: rmdir df/d11 39 2024-06-15T01:07:07.841 INFO:tasks.workunit.client.1.smithi142.stdout:4/787: dread d5/d9/d23/d68/ffb [0,4194304] 0 2024-06-15T01:07:07.845 INFO:tasks.workunit.client.1.smithi142.stdout:6/714: symlink d1a/d2d/d3b/lf9 0 2024-06-15T01:07:07.845 INFO:tasks.workunit.client.1.smithi142.stdout:9/654: dread d6/dc/d1b/f76 [0,4194304] 0 2024-06-15T01:07:07.847 INFO:tasks.workunit.client.0.smithi137.stdout:3/732: link d0/ddf/l9a d0/d2/d23/d68/lef 0 2024-06-15T01:07:07.848 INFO:tasks.workunit.client.1.smithi142.stdout:1/733: getdents d6/d12/da0/db0/de1 0 2024-06-15T01:07:07.849 INFO:tasks.workunit.client.0.smithi137.stdout:3/733: write d0/d2f/dbe/fe9 [502671,100666] 0 2024-06-15T01:07:07.850 INFO:tasks.workunit.client.0.smithi137.stdout:3/734: chown d0/d13/d37/d83 32254634 1 2024-06-15T01:07:07.853 INFO:tasks.workunit.client.0.smithi137.stdout:0/575: truncate d1/d2/dc/d10/f4f 2096104 0 2024-06-15T01:07:07.854 INFO:tasks.workunit.client.0.smithi137.stdout:3/735: truncate d0/d2/d23/f48 1287647 0 2024-06-15T01:07:07.855 INFO:tasks.workunit.client.1.smithi142.stdout:0/690: mknod d3/d39/d3e/d81/d1b/d38/da3/cf2 0 2024-06-15T01:07:07.856 INFO:tasks.workunit.client.0.smithi137.stdout:0/576: unlink d1/d2/f5e 0 2024-06-15T01:07:07.857 INFO:tasks.workunit.client.1.smithi142.stdout:4/788: mknod d5/d1c/d6f/dc7/de0/c10f 0 2024-06-15T01:07:07.859 INFO:tasks.workunit.client.0.smithi137.stdout:3/736: mkdir d0/d2/d22/d24/d28/d2e/df0 0 2024-06-15T01:07:07.859 INFO:tasks.workunit.client.1.smithi142.stdout:3/789: symlink d0/d1f/d20/d5d/d60/dd0/lea 0 2024-06-15T01:07:07.861 INFO:tasks.workunit.client.0.smithi137.stdout:3/737: mknod d0/d2/d1e/cf1 0 2024-06-15T01:07:07.863 INFO:tasks.workunit.client.1.smithi142.stdout:1/734: mkdir d6/d12/d3c/d63/d8d/d97/dd3/d106 0 2024-06-15T01:07:07.863 INFO:tasks.workunit.client.0.smithi137.stdout:3/738: rmdir d0/d2f/dbe 39 2024-06-15T01:07:07.867 INFO:tasks.workunit.client.1.smithi142.stdout:5/642: getdents d2/de/d27/d86/db9 0 2024-06-15T01:07:07.868 INFO:tasks.workunit.client.1.smithi142.stdout:2/687: truncate d8/df/d10/d94/d25/d22/d43/d52/dc1/fcb 349477 0 2024-06-15T01:07:07.870 INFO:tasks.workunit.client.0.smithi137.stdout:3/739: dwrite d0/d2f/fb8 [0,4194304] 0 2024-06-15T01:07:07.871 INFO:tasks.workunit.client.0.smithi137.stdout:0/577: fsync d1/d2/dc/d10/f4f 0 2024-06-15T01:07:07.871 INFO:tasks.workunit.client.1.smithi142.stdout:0/691: rename d3/d39/d3e/d81/d44/d7c/d22/d59/c6c to d3/d39/d3e/d81/d1b/dae/cf3 0 2024-06-15T01:07:07.872 INFO:tasks.workunit.client.0.smithi137.stdout:3/740: chown d0/d2/d8/f7f 708267 1 2024-06-15T01:07:07.872 INFO:tasks.workunit.client.1.smithi142.stdout:4/789: rmdir d5/d9/d90/daf/dcb 39 2024-06-15T01:07:07.874 INFO:tasks.workunit.client.1.smithi142.stdout:4/790: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f107 [18690,81917] 0 2024-06-15T01:07:07.876 INFO:tasks.workunit.client.1.smithi142.stdout:6/715: truncate d1a/d39/d42/d4c/d58/d87/f93 710790 0 2024-06-15T01:07:07.877 INFO:tasks.workunit.client.0.smithi137.stdout:0/578: symlink d1/d2/d55/lc8 0 2024-06-15T01:07:07.881 INFO:tasks.workunit.client.1.smithi142.stdout:4/791: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f60 [0,4194304] 0 2024-06-15T01:07:07.881 INFO:tasks.workunit.client.1.smithi142.stdout:8/735: truncate d0/d25/d5a/f8d 2801226 0 2024-06-15T01:07:07.882 INFO:tasks.workunit.client.1.smithi142.stdout:0/692: mkdir d3/d39/d9a/d9d/dbf/df4 0 2024-06-15T01:07:07.883 INFO:tasks.workunit.client.0.smithi137.stdout:3/741: rename d0/d2/d22/d24/f47 to d0/ddf/ff2 0 2024-06-15T01:07:07.884 INFO:tasks.workunit.client.1.smithi142.stdout:3/790: mknod d0/d3/da/db8/ceb 0 2024-06-15T01:07:07.885 INFO:tasks.workunit.client.1.smithi142.stdout:7/721: truncate d2/d8/dd/d14/f4b 774010 0 2024-06-15T01:07:07.885 INFO:tasks.workunit.client.0.smithi137.stdout:0/579: unlink d1/d2/dc/d10/d14/d15/d17/l65 0 2024-06-15T01:07:07.886 INFO:tasks.workunit.client.1.smithi142.stdout:6/716: creat d1a/d39/d3a/dd2/ffa x:0 0 0 2024-06-15T01:07:07.886 INFO:tasks.workunit.client.0.smithi137.stdout:0/580: write d1/d2/dc/d10/d22/fc7 [322171,112952] 0 2024-06-15T01:07:07.887 INFO:tasks.workunit.client.0.smithi137.stdout:3/742: mknod d0/d56/d8f/cf3 0 2024-06-15T01:07:07.887 INFO:tasks.workunit.client.1.smithi142.stdout:1/735: creat d6/d12/d80/d9b/db2/dc2/f107 x:0 0 0 2024-06-15T01:07:07.888 INFO:tasks.workunit.client.0.smithi137.stdout:0/581: readlink d1/d2/dc/d6c/d88/d91/lb3 0 2024-06-15T01:07:07.888 INFO:tasks.workunit.client.1.smithi142.stdout:1/736: readlink d6/d25/l72 0 2024-06-15T01:07:07.890 INFO:tasks.workunit.client.1.smithi142.stdout:3/791: dwrite d0/d1f/d20/d59/f8e [0,4194304] 0 2024-06-15T01:07:07.891 INFO:tasks.workunit.client.0.smithi137.stdout:3/743: creat d0/d2/d8/d67/ff4 x:0 0 0 2024-06-15T01:07:07.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/582: dread d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:07.893 INFO:tasks.workunit.client.1.smithi142.stdout:8/736: symlink d0/d25/d53/da7/dcb/le0 0 2024-06-15T01:07:07.893 INFO:tasks.workunit.client.1.smithi142.stdout:3/792: chown d0/d1f/f46 129 1 2024-06-15T01:07:07.893 INFO:tasks.workunit.client.0.smithi137.stdout:3/744: creat d0/d2/d23/ff5 x:0 0 0 2024-06-15T01:07:07.894 INFO:tasks.workunit.client.0.smithi137.stdout:0/583: mknod d1/d2/d13/d66/dc3/cc9 0 2024-06-15T01:07:07.896 INFO:tasks.workunit.client.1.smithi142.stdout:9/655: getdents d6/dc/d1b/d6f 0 2024-06-15T01:07:07.897 INFO:tasks.workunit.client.1.smithi142.stdout:1/737: dwrite d6/d12/d1c/fe0 [0,4194304] 0 2024-06-15T01:07:07.899 INFO:tasks.workunit.client.1.smithi142.stdout:7/722: symlink d2/d8/dd/dc0/le7 0 2024-06-15T01:07:07.900 INFO:tasks.workunit.client.0.smithi137.stdout:0/584: rename d1/d2/dc/d10/d63 to d1/d2/dc/d10/d53/dca 0 2024-06-15T01:07:07.902 INFO:tasks.workunit.client.1.smithi142.stdout:6/717: symlink d1a/d83/lfb 0 2024-06-15T01:07:07.902 INFO:tasks.workunit.client.0.smithi137.stdout:3/745: dwrite d0/d42/fc9 [0,4194304] 0 2024-06-15T01:07:07.903 INFO:tasks.workunit.client.0.smithi137.stdout:0/585: mknod d1/d2/d55/ccb 0 2024-06-15T01:07:07.904 INFO:tasks.workunit.client.1.smithi142.stdout:2/688: dread d8/d19/d8d/f93 [0,4194304] 0 2024-06-15T01:07:07.905 INFO:tasks.workunit.client.1.smithi142.stdout:6/718: dread - d1a/d2d/f65 zero size 2024-06-15T01:07:07.907 INFO:tasks.workunit.client.0.smithi137.stdout:0/586: creat d1/d2/dc/d10/d14/d52/fcc x:0 0 0 2024-06-15T01:07:07.909 INFO:tasks.workunit.client.1.smithi142.stdout:0/693: mknod d3/d39/d9a/d9d/de6/cf5 0 2024-06-15T01:07:07.911 INFO:tasks.workunit.client.0.smithi137.stdout:3/746: rename d0/d2/d22/d24/d28/fc7 to d0/d2/d8/dbd/ff6 0 2024-06-15T01:07:07.911 INFO:tasks.workunit.client.1.smithi142.stdout:7/723: dwrite d2/d6/f1c [0,4194304] 0 2024-06-15T01:07:07.911 INFO:tasks.workunit.client.1.smithi142.stdout:4/792: rename d5/d9/d90/daf/dcb/fdf to d5/d1c/f110 0 2024-06-15T01:07:07.912 INFO:tasks.workunit.client.1.smithi142.stdout:0/694: write d3/d39/d3e/d81/d1e/f2b [1948674,124767] 0 2024-06-15T01:07:07.913 INFO:tasks.workunit.client.0.smithi137.stdout:0/587: rmdir d1/d2/dc/d10/d53 39 2024-06-15T01:07:07.916 INFO:tasks.workunit.client.1.smithi142.stdout:5/643: write d2/f8b [642595,3483] 0 2024-06-15T01:07:07.916 INFO:tasks.workunit.client.1.smithi142.stdout:3/793: write d0/fba [3203298,90134] 0 2024-06-15T01:07:07.916 INFO:tasks.workunit.client.0.smithi137.stdout:3/747: creat d0/d42/ff7 x:0 0 0 2024-06-15T01:07:07.919 INFO:tasks.workunit.client.1.smithi142.stdout:9/656: unlink d6/dc/d1b/ca9 0 2024-06-15T01:07:07.921 INFO:tasks.workunit.client.1.smithi142.stdout:9/657: write d6/dc/d1b/d25/d58/f63 [1692819,111769] 0 2024-06-15T01:07:07.921 INFO:tasks.workunit.client.1.smithi142.stdout:2/689: mkdir d8/df/d10/d94/d25/d22/d43/d51/dd7 0 2024-06-15T01:07:07.922 INFO:tasks.workunit.client.0.smithi137.stdout:0/588: dread d1/d2/dc/d10/d14/d15/d1c/f46 [0,4194304] 0 2024-06-15T01:07:07.922 INFO:tasks.workunit.client.0.smithi137.stdout:3/748: read d0/d56/f6a [1018788,96180] 0 2024-06-15T01:07:07.923 INFO:tasks.workunit.client.1.smithi142.stdout:5/644: dwrite d2/d18/fba [0,4194304] 0 2024-06-15T01:07:07.927 INFO:tasks.workunit.client.0.smithi137.stdout:0/589: mknod d1/d2/ccd 0 2024-06-15T01:07:07.927 INFO:tasks.workunit.client.1.smithi142.stdout:7/724: write d2/d8/da/d33/f9e [684674,39660] 0 2024-06-15T01:07:07.928 INFO:tasks.workunit.client.1.smithi142.stdout:7/725: chown d2/d6/f1d 12734176 1 2024-06-15T01:07:07.928 INFO:tasks.workunit.client.1.smithi142.stdout:7/726: stat d2/d8/dd/ccd 0 2024-06-15T01:07:07.930 INFO:tasks.workunit.client.0.smithi137.stdout:0/590: read d1/d2/dc/d10/d14/d15/f4b [1074569,41571] 0 2024-06-15T01:07:07.931 INFO:tasks.workunit.client.1.smithi142.stdout:4/793: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d111 0 2024-06-15T01:07:07.932 INFO:tasks.workunit.client.1.smithi142.stdout:4/794: dread - d5/d9/d41/f65 zero size 2024-06-15T01:07:07.934 INFO:tasks.workunit.client.0.smithi137.stdout:0/591: symlink d1/d2/dc/d10/d22/lce 0 2024-06-15T01:07:07.935 INFO:tasks.workunit.client.1.smithi142.stdout:3/794: mknod d0/d1f/d20/d5d/d82/d6e/cec 0 2024-06-15T01:07:07.936 INFO:tasks.workunit.client.0.smithi137.stdout:0/592: write d1/d2/d55/faf [1601112,114869] 0 2024-06-15T01:07:07.937 INFO:tasks.workunit.client.0.smithi137.stdout:0/593: chown d1/d2/dc/d10/d14/d15/d1c/f46 84 1 2024-06-15T01:07:07.940 INFO:tasks.workunit.client.1.smithi142.stdout:6/719: mkdir d1a/d39/d42/df7/dfc 0 2024-06-15T01:07:07.941 INFO:tasks.workunit.client.0.smithi137.stdout:0/594: creat d1/d7e/d87/da0/fcf x:0 0 0 2024-06-15T01:07:07.941 INFO:tasks.workunit.client.0.smithi137.stdout:0/595: readlink d1/d2/d13/d66/l38 0 2024-06-15T01:07:07.944 INFO:tasks.workunit.client.1.smithi142.stdout:5/645: symlink d2/de/d19/d5f/d68/d76/d96/dac/lca 0 2024-06-15T01:07:07.945 INFO:tasks.workunit.client.0.smithi137.stdout:0/596: write d1/d2/dc/d10/d53/da2/fa5 [834742,83970] 0 2024-06-15T01:07:07.948 INFO:tasks.workunit.client.1.smithi142.stdout:0/695: mkdir d3/d39/d3e/d81/d1b/df6 0 2024-06-15T01:07:07.949 INFO:tasks.workunit.client.1.smithi142.stdout:2/690: dread d8/df/f48 [0,4194304] 0 2024-06-15T01:07:07.951 INFO:tasks.workunit.client.1.smithi142.stdout:2/691: fdatasync d8/df/d10/d94/d25/d22/d43/fac 0 2024-06-15T01:07:07.953 INFO:tasks.workunit.client.0.smithi137.stdout:0/597: dread d1/d2/dc/d10/d53/da2/fa5 [0,4194304] 0 2024-06-15T01:07:07.953 INFO:tasks.workunit.client.0.smithi137.stdout:3/749: dread d0/d2/d23/d68/f3c [0,4194304] 0 2024-06-15T01:07:07.954 INFO:tasks.workunit.client.1.smithi142.stdout:6/720: dread d1a/d39/d3a/d51/d62/f79 [0,4194304] 0 2024-06-15T01:07:07.957 INFO:tasks.workunit.client.1.smithi142.stdout:0/696: write d3/d39/d3e/d81/d1b/d38/f96 [1584529,119748] 0 2024-06-15T01:07:07.958 INFO:tasks.workunit.client.0.smithi137.stdout:0/598: mkdir d1/d2/dc/d10/d14/d15/d17/d54/d7a/dd0 0 2024-06-15T01:07:07.958 INFO:tasks.workunit.client.0.smithi137.stdout:3/750: dread d0/d2/d8/dbd/ff6 [0,4194304] 0 2024-06-15T01:07:07.965 INFO:tasks.workunit.client.1.smithi142.stdout:7/727: creat d2/d6/fe8 x:0 0 0 2024-06-15T01:07:07.965 INFO:tasks.workunit.client.1.smithi142.stdout:3/795: dwrite d0/d3/de2/d55/f57 [4194304,4194304] 0 2024-06-15T01:07:07.965 INFO:tasks.workunit.client.1.smithi142.stdout:1/738: dwrite d6/d3a/fe3 [0,4194304] 0 2024-06-15T01:07:07.966 INFO:tasks.workunit.client.1.smithi142.stdout:2/692: dwrite d8/df/d10/d94/d25/f6a [0,4194304] 0 2024-06-15T01:07:07.967 INFO:tasks.workunit.client.1.smithi142.stdout:8/737: truncate d0/f4c 911943 0 2024-06-15T01:07:07.968 INFO:tasks.workunit.client.0.smithi137.stdout:3/751: dwrite d0/d2/d4e/fd9 [0,4194304] 0 2024-06-15T01:07:07.969 INFO:tasks.workunit.client.1.smithi142.stdout:9/658: getdents d6/dc/d4e/d78 0 2024-06-15T01:07:07.973 INFO:tasks.workunit.client.1.smithi142.stdout:7/728: truncate d2/d6/d22/d3c/fc1 822291 0 2024-06-15T01:07:07.974 INFO:tasks.workunit.client.1.smithi142.stdout:6/721: dwrite d1a/d39/d42/d4c/d58/f5d [4194304,4194304] 0 2024-06-15T01:07:07.977 INFO:tasks.workunit.client.0.smithi137.stdout:3/752: rename d0/d42/le4 to d0/d2/d22/d63/d76/lf8 0 2024-06-15T01:07:07.978 INFO:tasks.workunit.client.1.smithi142.stdout:0/697: rmdir d3/d39/d3e/d81/d44/d7c 39 2024-06-15T01:07:07.979 INFO:tasks.workunit.client.0.smithi137.stdout:0/599: dread d1/d2/dc/d10/d14/d15/f4b [0,4194304] 0 2024-06-15T01:07:07.981 INFO:tasks.workunit.client.0.smithi137.stdout:3/753: mknod d0/d2/d22/d24/d28/cf9 0 2024-06-15T01:07:07.982 INFO:tasks.workunit.client.1.smithi142.stdout:3/796: mkdir d0/d1f/d20/ded 0 2024-06-15T01:07:07.982 INFO:tasks.workunit.client.0.smithi137.stdout:0/600: symlink d1/d7e/ld1 0 2024-06-15T01:07:07.983 INFO:tasks.workunit.client.1.smithi142.stdout:5/646: getdents d2/de/d19 0 2024-06-15T01:07:07.985 INFO:tasks.workunit.client.1.smithi142.stdout:5/647: chown d2/f9 143078798 1 2024-06-15T01:07:07.985 INFO:tasks.workunit.client.1.smithi142.stdout:3/797: dread d0/d3/de2/d55/f94 [0,4194304] 0 2024-06-15T01:07:07.986 INFO:tasks.workunit.client.1.smithi142.stdout:2/693: dread d8/df/d10/f87 [0,4194304] 0 2024-06-15T01:07:07.987 INFO:tasks.workunit.client.1.smithi142.stdout:1/739: rename d6/d25/dfc/f33 to d6/d12/d80/dae/f108 0 2024-06-15T01:07:07.987 INFO:tasks.workunit.client.1.smithi142.stdout:2/694: fsync d8/df/d10/d94/d25/d22/fbd 0 2024-06-15T01:07:07.989 INFO:tasks.workunit.client.1.smithi142.stdout:4/795: write d5/d9/d90/fc4 [1437639,36875] 0 2024-06-15T01:07:07.989 INFO:tasks.workunit.client.0.smithi137.stdout:0/601: dread d1/d2/d13/d66/f4a [0,4194304] 0 2024-06-15T01:07:07.989 INFO:tasks.workunit.client.0.smithi137.stdout:3/754: dwrite d0/d56/d8f/d7b/f96 [0,4194304] 0 2024-06-15T01:07:07.993 INFO:tasks.workunit.client.1.smithi142.stdout:1/740: dread d6/d12/d3c/d63/f6f [0,4194304] 0 2024-06-15T01:07:07.993 INFO:tasks.workunit.client.1.smithi142.stdout:0/698: read d3/f28 [447050,86914] 0 2024-06-15T01:07:07.995 INFO:tasks.workunit.client.1.smithi142.stdout:0/699: write d3/d39/d3e/d81/d1b/d52/f85 [2127237,127407] 0 2024-06-15T01:07:08.000 INFO:tasks.workunit.client.1.smithi142.stdout:5/648: symlink d2/de/d27/lcb 0 2024-06-15T01:07:08.001 INFO:tasks.workunit.client.1.smithi142.stdout:9/659: rename d6/dc/f15 to d6/dc/d1b/d6f/ff7 0 2024-06-15T01:07:08.003 INFO:tasks.workunit.client.1.smithi142.stdout:9/660: dread - d6/dc/d4e/d78/fd2 zero size 2024-06-15T01:07:08.003 INFO:tasks.workunit.client.1.smithi142.stdout:7/729: mknod d2/d8/dd/d14/d2c/d37/ce9 0 2024-06-15T01:07:08.004 INFO:tasks.workunit.client.1.smithi142.stdout:9/661: write d6/dc/d1b/d25/f3f [2119186,127662] 0 2024-06-15T01:07:08.008 INFO:tasks.workunit.client.1.smithi142.stdout:6/722: rename d1a/d39/d3a/d51/d62/fb4 to d1a/d39/d42/d4c/d58/d87/ffd 0 2024-06-15T01:07:08.009 INFO:tasks.workunit.client.1.smithi142.stdout:2/695: unlink d8/df/d10/d94/d25/d1a/d63/lc2 0 2024-06-15T01:07:08.010 INFO:tasks.workunit.client.1.smithi142.stdout:4/796: link d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/fce d5/d1c/d6f/db9/f112 0 2024-06-15T01:07:08.012 INFO:tasks.workunit.client.1.smithi142.stdout:9/662: mkdir d6/dc/d1b/d25/d58/df8 0 2024-06-15T01:07:08.018 INFO:tasks.workunit.client.1.smithi142.stdout:8/738: getdents d0/d25/d5a 0 2024-06-15T01:07:08.018 INFO:tasks.workunit.client.1.smithi142.stdout:1/741: creat d6/d12/d3c/d63/f109 x:0 0 0 2024-06-15T01:07:08.018 INFO:tasks.workunit.client.1.smithi142.stdout:7/730: symlink d2/d8/da/d33/d25/d79/dbc/lea 0 2024-06-15T01:07:08.019 INFO:tasks.workunit.client.1.smithi142.stdout:3/798: link d0/f3c d0/d3/de2/d75/fee 0 2024-06-15T01:07:08.019 INFO:tasks.workunit.client.1.smithi142.stdout:1/742: read - d6/d3a/fff zero size 2024-06-15T01:07:08.020 INFO:tasks.workunit.client.1.smithi142.stdout:4/797: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d111/f113 x:0 0 0 2024-06-15T01:07:08.023 INFO:tasks.workunit.client.0.smithi137.stdout:0/602: dread d1/d2/f47 [0,4194304] 0 2024-06-15T01:07:08.024 INFO:tasks.workunit.client.1.smithi142.stdout:0/700: dwrite d3/d39/d3e/d81/d44/d7c/d22/f70 [0,4194304] 0 2024-06-15T01:07:08.024 INFO:tasks.workunit.client.1.smithi142.stdout:2/696: rename d8/df/d1f/c4c to d8/df/d10/d94/d25/d22/d43/d51/cd8 0 2024-06-15T01:07:08.025 INFO:tasks.workunit.client.1.smithi142.stdout:7/731: mknod d2/db6/de6/ceb 0 2024-06-15T01:07:08.026 INFO:tasks.workunit.client.1.smithi142.stdout:3/799: mknod d0/d3/da/db8/cef 0 2024-06-15T01:07:08.028 INFO:tasks.workunit.client.1.smithi142.stdout:3/800: readlink d0/d3/de2/d55/l56 0 2024-06-15T01:07:08.029 INFO:tasks.workunit.client.1.smithi142.stdout:6/723: link d1a/d2d/d3b/dab/dce/lf4 d1a/d3c/lfe 0 2024-06-15T01:07:08.029 INFO:tasks.workunit.client.1.smithi142.stdout:8/739: dwrite d0/d25/d5a/d8f/d98/da4/fc1 [0,4194304] 0 2024-06-15T01:07:08.031 INFO:tasks.workunit.client.1.smithi142.stdout:7/732: truncate d2/d8/da/d33/f9e 1230968 0 2024-06-15T01:07:08.033 INFO:tasks.workunit.client.1.smithi142.stdout:2/697: truncate d8/df/d10/d94/d25/d22/fbd 622763 0 2024-06-15T01:07:08.033 INFO:tasks.workunit.client.1.smithi142.stdout:5/649: write d2/de/d27/f38 [327793,51944] 0 2024-06-15T01:07:08.037 INFO:tasks.workunit.client.1.smithi142.stdout:0/701: fdatasync d3/d39/d9a/fb8 0 2024-06-15T01:07:08.042 INFO:tasks.workunit.client.1.smithi142.stdout:3/801: dwrite d0/fba [0,4194304] 0 2024-06-15T01:07:08.043 INFO:tasks.workunit.client.1.smithi142.stdout:0/702: fdatasync d3/d39/d3e/d81/d44/d7c/d22/f2d 0 2024-06-15T01:07:08.044 INFO:tasks.workunit.client.1.smithi142.stdout:5/650: dwrite d2/f28 [0,4194304] 0 2024-06-15T01:07:08.046 INFO:tasks.workunit.client.1.smithi142.stdout:7/733: rename d2/d8/da/d33/d25/d81/fa4 to d2/db6/fec 0 2024-06-15T01:07:08.048 INFO:tasks.workunit.client.1.smithi142.stdout:9/663: write d6/dc/d1b/d25/d58/f64 [5044522,65211] 0 2024-06-15T01:07:08.049 INFO:tasks.workunit.client.1.smithi142.stdout:4/798: dread d5/d9/d90/daf/dcb/f7a [0,4194304] 0 2024-06-15T01:07:08.051 INFO:tasks.workunit.client.1.smithi142.stdout:8/740: link d0/f3f d0/d25/d66/fe1 0 2024-06-15T01:07:08.052 INFO:tasks.workunit.client.1.smithi142.stdout:8/741: dread - d0/fd4 zero size 2024-06-15T01:07:08.053 INFO:tasks.workunit.client.1.smithi142.stdout:8/742: chown d0/l39 25478 1 2024-06-15T01:07:08.054 INFO:tasks.workunit.client.1.smithi142.stdout:7/734: dwrite d2/f49 [0,4194304] 0 2024-06-15T01:07:08.055 INFO:tasks.workunit.client.1.smithi142.stdout:0/703: chown d3/d39/f7d 5 1 2024-06-15T01:07:08.060 INFO:tasks.workunit.client.1.smithi142.stdout:0/704: write d3/d39/d3e/d81/d44/fd6 [327556,27343] 0 2024-06-15T01:07:08.060 INFO:tasks.workunit.client.1.smithi142.stdout:8/743: chown d0/d25/d53/d9c/fbd 11 1 2024-06-15T01:07:08.062 INFO:tasks.workunit.client.1.smithi142.stdout:2/698: creat d8/df/d10/d94/fd9 x:0 0 0 2024-06-15T01:07:08.063 INFO:tasks.workunit.client.1.smithi142.stdout:1/743: write d6/d12/d1c/d2c/f91 [1136099,54427] 0 2024-06-15T01:07:08.067 INFO:tasks.workunit.client.1.smithi142.stdout:6/724: getdents d1a/d2d/d3b/dab 0 2024-06-15T01:07:08.068 INFO:tasks.workunit.client.1.smithi142.stdout:1/744: chown d6/d12/d3c/d63/d8d/fb8 6140 1 2024-06-15T01:07:08.070 INFO:tasks.workunit.client.1.smithi142.stdout:6/725: readlink d1a/d39/d3a/d51/lf5 0 2024-06-15T01:07:08.071 INFO:tasks.workunit.client.1.smithi142.stdout:9/664: rename d6/dc/d51/d53/d8d/cee to d6/dc/d1b/d6f/da1/df6/cf9 0 2024-06-15T01:07:08.072 INFO:tasks.workunit.client.1.smithi142.stdout:4/799: dwrite d5/d1c/f110 [0,4194304] 0 2024-06-15T01:07:08.073 INFO:tasks.workunit.client.1.smithi142.stdout:2/699: creat d8/df/d10/d94/d25/fda x:0 0 0 2024-06-15T01:07:08.076 INFO:tasks.workunit.client.1.smithi142.stdout:9/665: creat d6/dc/d51/d53/ffa x:0 0 0 2024-06-15T01:07:08.077 INFO:tasks.workunit.client.1.smithi142.stdout:9/666: chown d6/dc/d1b/d25/d58/f99 1042558870 1 2024-06-15T01:07:08.078 INFO:tasks.workunit.client.1.smithi142.stdout:1/745: link d6/d12/d80/f100 d6/d12/d1c/f10a 0 2024-06-15T01:07:08.078 INFO:tasks.workunit.client.1.smithi142.stdout:2/700: mknod d8/df/d10/d94/d25/d76/cdb 0 2024-06-15T01:07:08.080 INFO:tasks.workunit.client.1.smithi142.stdout:4/800: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/f114 x:0 0 0 2024-06-15T01:07:08.083 INFO:tasks.workunit.client.1.smithi142.stdout:9/667: mkdir d6/d34/df3/dfb 0 2024-06-15T01:07:08.083 INFO:tasks.workunit.client.1.smithi142.stdout:2/701: stat d8/df/d10/d94/d25/d22/d43/d51/fd2 0 2024-06-15T01:07:08.084 INFO:tasks.workunit.client.1.smithi142.stdout:1/746: symlink d6/d3a/dd4/l10b 0 2024-06-15T01:07:08.085 INFO:tasks.workunit.client.1.smithi142.stdout:4/801: unlink d5/d1c/d6f/dc7/de1/ffe 0 2024-06-15T01:07:08.087 INFO:tasks.workunit.client.1.smithi142.stdout:1/747: write d6/d12/d80/dc8/fda [255697,39398] 0 2024-06-15T01:07:08.087 INFO:tasks.workunit.client.1.smithi142.stdout:9/668: dread d6/dc/d1b/d25/d35/fd1 [0,4194304] 0 2024-06-15T01:07:08.088 INFO:tasks.workunit.client.1.smithi142.stdout:5/651: dread d2/d18/f99 [0,4194304] 0 2024-06-15T01:07:08.088 INFO:tasks.workunit.client.1.smithi142.stdout:3/802: dread d0/d3/da/f9b [0,4194304] 0 2024-06-15T01:07:08.093 INFO:tasks.workunit.client.1.smithi142.stdout:1/748: mkdir d6/d12/db5/d10c 0 2024-06-15T01:07:08.094 INFO:tasks.workunit.client.1.smithi142.stdout:2/702: creat d8/df/d10/d94/d25/d22/fdc x:0 0 0 2024-06-15T01:07:08.095 INFO:tasks.workunit.client.1.smithi142.stdout:9/669: creat d6/d34/d8a/dec/ffc x:0 0 0 2024-06-15T01:07:08.099 INFO:tasks.workunit.client.1.smithi142.stdout:1/749: unlink d6/d46/f56 0 2024-06-15T01:07:08.102 INFO:tasks.workunit.client.1.smithi142.stdout:4/802: dread d5/d9/d23/d68/ffb [0,4194304] 0 2024-06-15T01:07:08.102 INFO:tasks.workunit.client.1.smithi142.stdout:8/744: dread d0/d25/d5a/fda [0,4194304] 0 2024-06-15T01:07:08.103 INFO:tasks.workunit.client.1.smithi142.stdout:7/735: dread d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:08.104 INFO:tasks.workunit.client.1.smithi142.stdout:6/726: dwrite d1a/d2d/f2f [0,4194304] 0 2024-06-15T01:07:08.104 INFO:tasks.workunit.client.1.smithi142.stdout:3/803: getdents d0/d1f/d20/d5d/d82 0 2024-06-15T01:07:08.104 INFO:tasks.workunit.client.1.smithi142.stdout:0/705: dwrite d3/d39/d3e/d81/d44/d7c/d22/f36 [0,4194304] 0 2024-06-15T01:07:08.105 INFO:tasks.workunit.client.1.smithi142.stdout:9/670: mknod d6/dc/dd/cfd 0 2024-06-15T01:07:08.107 INFO:tasks.workunit.client.1.smithi142.stdout:8/745: chown d0/d25/d5a/f2c 114 1 2024-06-15T01:07:08.108 INFO:tasks.workunit.client.1.smithi142.stdout:0/706: read - d3/d39/d3e/db4/fda zero size 2024-06-15T01:07:08.111 INFO:tasks.workunit.client.1.smithi142.stdout:7/736: write d2/d8/dd/d14/d2c/d37/fdb [471642,6589] 0 2024-06-15T01:07:08.114 INFO:tasks.workunit.client.1.smithi142.stdout:5/652: link d2/d3e/c48 d2/d18/d79/da2/ccc 0 2024-06-15T01:07:08.115 INFO:tasks.workunit.client.1.smithi142.stdout:1/750: write d6/d12/d3c/d63/f85 [1974170,413] 0 2024-06-15T01:07:08.117 INFO:tasks.workunit.client.1.smithi142.stdout:9/671: mknod d6/dc/d1b/d25/cfe 0 2024-06-15T01:07:08.118 INFO:tasks.workunit.client.1.smithi142.stdout:8/746: mkdir d0/d25/d53/de2 0 2024-06-15T01:07:08.119 INFO:tasks.workunit.client.1.smithi142.stdout:0/707: creat d3/d39/d3e/d81/d44/d7c/d22/d59/ff7 x:0 0 0 2024-06-15T01:07:08.120 INFO:tasks.workunit.client.1.smithi142.stdout:6/727: link d1a/d39/d42/f74 d1a/d2d/d3b/dab/fff 0 2024-06-15T01:07:08.122 INFO:tasks.workunit.client.1.smithi142.stdout:5/653: creat d2/de/d27/d86/dbd/fcd x:0 0 0 2024-06-15T01:07:08.123 INFO:tasks.workunit.client.1.smithi142.stdout:4/803: dwrite d5/d9/da/d92/d98/dc1/f10c [0,4194304] 0 2024-06-15T01:07:08.123 INFO:tasks.workunit.client.1.smithi142.stdout:1/751: chown d6/d12/d3c/d63/d8d/d97/dd3/f76 420 1 2024-06-15T01:07:08.123 INFO:tasks.workunit.client.1.smithi142.stdout:8/747: fsync d0/d25/d5a/d84/fd1 0 2024-06-15T01:07:08.125 INFO:tasks.workunit.client.1.smithi142.stdout:5/654: write d2/f25 [3954968,58757] 0 2024-06-15T01:07:08.126 INFO:tasks.workunit.client.1.smithi142.stdout:1/752: chown d6/d12/f22 0 1 2024-06-15T01:07:08.128 INFO:tasks.workunit.client.1.smithi142.stdout:4/804: chown d5/d1c/d6f/dc7/d30/dcc 214321450 1 2024-06-15T01:07:08.129 INFO:tasks.workunit.client.1.smithi142.stdout:0/708: write d3/d39/d3e/d81/d44/f49 [296007,122676] 0 2024-06-15T01:07:08.132 INFO:tasks.workunit.client.1.smithi142.stdout:2/703: dwrite d8/d19/f70 [0,4194304] 0 2024-06-15T01:07:08.133 INFO:tasks.workunit.client.1.smithi142.stdout:4/805: write d5/d9/d90/daf/dcb/f8a [2280721,39785] 0 2024-06-15T01:07:08.134 INFO:tasks.workunit.client.1.smithi142.stdout:7/737: dwrite d2/d6/f69 [0,4194304] 0 2024-06-15T01:07:08.135 INFO:tasks.workunit.client.1.smithi142.stdout:0/709: write d3/d39/d3e/fe0 [755983,36588] 0 2024-06-15T01:07:08.136 INFO:tasks.workunit.client.1.smithi142.stdout:7/738: write d2/f91 [2201867,38645] 0 2024-06-15T01:07:08.136 INFO:tasks.workunit.client.1.smithi142.stdout:8/748: read d0/d17/d40/d5e/f81 [3675002,108806] 0 2024-06-15T01:07:08.137 INFO:tasks.workunit.client.1.smithi142.stdout:0/710: stat d3/d39/d3e/d81/ld 0 2024-06-15T01:07:08.138 INFO:tasks.workunit.client.1.smithi142.stdout:1/753: symlink d6/d12/d1c/l10d 0 2024-06-15T01:07:08.138 INFO:tasks.workunit.client.1.smithi142.stdout:5/655: link d2/f58 d2/d18/d79/fce 0 2024-06-15T01:07:08.138 INFO:tasks.workunit.client.1.smithi142.stdout:2/704: truncate d8/df/d10/f9d 160318 0 2024-06-15T01:07:08.140 INFO:tasks.workunit.client.1.smithi142.stdout:5/656: write d2/d18/fba [2856521,26917] 0 2024-06-15T01:07:08.143 INFO:tasks.workunit.client.1.smithi142.stdout:5/657: chown d2/d18/f61 78859 1 2024-06-15T01:07:08.148 INFO:tasks.workunit.client.1.smithi142.stdout:7/739: creat d2/d8/dd8/fed x:0 0 0 2024-06-15T01:07:08.149 INFO:tasks.workunit.client.1.smithi142.stdout:2/705: read - d8/df/d10/d94/d25/d1a/faa zero size 2024-06-15T01:07:08.150 INFO:tasks.workunit.client.1.smithi142.stdout:1/754: write d6/d12/d3c/f9c [4201958,44886] 0 2024-06-15T01:07:08.153 INFO:tasks.workunit.client.1.smithi142.stdout:3/804: dread d0/d1b/f88 [0,4194304] 0 2024-06-15T01:07:08.154 INFO:tasks.workunit.client.1.smithi142.stdout:0/711: mkdir d3/d39/d9a/d9d/df8 0 2024-06-15T01:07:08.155 INFO:tasks.workunit.client.1.smithi142.stdout:8/749: read d0/d17/d40/f76 [2602434,113714] 0 2024-06-15T01:07:08.160 INFO:tasks.workunit.client.1.smithi142.stdout:4/806: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d111/f115 x:0 0 0 2024-06-15T01:07:08.161 INFO:tasks.workunit.client.1.smithi142.stdout:7/740: mknod d2/d8/dd8/cee 0 2024-06-15T01:07:08.162 INFO:tasks.workunit.client.1.smithi142.stdout:2/706: symlink d8/df/d10/d94/d25/da9/ldd 0 2024-06-15T01:07:08.162 INFO:tasks.workunit.client.1.smithi142.stdout:0/712: rmdir d3/d39/d9a/d9d/dcd 39 2024-06-15T01:07:08.163 INFO:tasks.workunit.client.1.smithi142.stdout:3/805: creat d0/d3/de2/d75/ff0 x:0 0 0 2024-06-15T01:07:08.164 INFO:tasks.workunit.client.1.smithi142.stdout:8/750: dread d0/d17/d40/d86/f9b [0,4194304] 0 2024-06-15T01:07:08.164 INFO:tasks.workunit.client.1.smithi142.stdout:7/741: readlink d2/d8/da/d33/d25/d79/l90 0 2024-06-15T01:07:08.166 INFO:tasks.workunit.client.0.smithi137.stdout:0/603: fsync d1/d2/dc/d10/f25 0 2024-06-15T01:07:08.167 INFO:tasks.workunit.client.1.smithi142.stdout:3/806: chown d0/d1f/d20/d5d/la4 98535 1 2024-06-15T01:07:08.169 INFO:tasks.workunit.client.0.smithi137.stdout:3/755: getdents d0/d4c 0 2024-06-15T01:07:08.170 INFO:tasks.workunit.client.1.smithi142.stdout:7/742: truncate d2/d8/dd/dcf/fa6 113859 0 2024-06-15T01:07:08.170 INFO:tasks.workunit.client.1.smithi142.stdout:0/713: truncate d3/f9 4518589 0 2024-06-15T01:07:08.171 INFO:tasks.workunit.client.0.smithi137.stdout:3/756: write d0/d2/d8/f7a [586398,79175] 0 2024-06-15T01:07:08.172 INFO:tasks.workunit.client.0.smithi137.stdout:3/757: chown d0/d2/daf 102221 1 2024-06-15T01:07:08.173 INFO:tasks.workunit.client.0.smithi137.stdout:3/758: truncate d0/d2/d23/d68/fda 1196206 0 2024-06-15T01:07:08.174 INFO:tasks.workunit.client.1.smithi142.stdout:8/751: write d0/d17/f6b [5737162,41792] 0 2024-06-15T01:07:08.174 INFO:tasks.workunit.client.0.smithi137.stdout:3/759: readlink d0/d2/d22/d63/l80 0 2024-06-15T01:07:08.176 INFO:tasks.workunit.client.0.smithi137.stdout:3/760: chown d0/d2/d22/d24/cb0 58 1 2024-06-15T01:07:08.176 INFO:tasks.workunit.client.1.smithi142.stdout:8/752: dread - d0/d25/d5a/d8f/d98/fc8 zero size 2024-06-15T01:07:08.176 INFO:tasks.workunit.client.1.smithi142.stdout:4/807: dwrite d5/d9/da/d92/d98/fac [0,4194304] 0 2024-06-15T01:07:08.179 INFO:tasks.workunit.client.1.smithi142.stdout:7/743: rmdir d2/d8/da/d33/d25 39 2024-06-15T01:07:08.182 INFO:tasks.workunit.client.1.smithi142.stdout:8/753: creat d0/d17/d40/d6e/fe3 x:0 0 0 2024-06-15T01:07:08.184 INFO:tasks.workunit.client.0.smithi137.stdout:0/604: symlink d1/d2/dc/d10/d14/ld2 0 2024-06-15T01:07:08.186 INFO:tasks.workunit.client.0.smithi137.stdout:0/605: write d1/d2/d55/f6e [2312863,129106] 0 2024-06-15T01:07:08.186 INFO:tasks.workunit.client.1.smithi142.stdout:8/754: write d0/d25/d5a/f88 [4751935,42378] 0 2024-06-15T01:07:08.188 INFO:tasks.workunit.client.0.smithi137.stdout:0/606: dread - d1/d7e/d87/da0/fcf zero size 2024-06-15T01:07:08.189 INFO:tasks.workunit.client.1.smithi142.stdout:2/707: getdents d8/df/d10/d64 0 2024-06-15T01:07:08.189 INFO:tasks.workunit.client.1.smithi142.stdout:0/714: symlink d3/d39/d3e/d81/d4c/d61/dea/lf9 0 2024-06-15T01:07:08.190 INFO:tasks.workunit.client.0.smithi137.stdout:3/761: link d0/d13/l72 d0/d2/d23/de2/lfa 0 2024-06-15T01:07:08.190 INFO:tasks.workunit.client.0.smithi137.stdout:3/762: stat d0/d2/d22/d24/l78 0 2024-06-15T01:07:08.192 INFO:tasks.workunit.client.1.smithi142.stdout:2/708: stat d8/df/d10/d94/d25/d1a 0 2024-06-15T01:07:08.194 INFO:tasks.workunit.client.1.smithi142.stdout:4/808: dwrite d5/d9/da/d92/d98/fac [0,4194304] 0 2024-06-15T01:07:08.196 INFO:tasks.workunit.client.1.smithi142.stdout:7/744: dwrite d2/d6/d22/d47/f7c [0,4194304] 0 2024-06-15T01:07:08.199 INFO:tasks.workunit.client.1.smithi142.stdout:0/715: read d3/d39/d3e/d81/d44/f57 [3388861,35996] 0 2024-06-15T01:07:08.199 INFO:tasks.workunit.client.1.smithi142.stdout:2/709: dread d8/d19/d33/f66 [0,4194304] 0 2024-06-15T01:07:08.205 INFO:tasks.workunit.client.1.smithi142.stdout:7/745: mknod d2/db6/de6/cef 0 2024-06-15T01:07:08.207 INFO:tasks.workunit.client.1.smithi142.stdout:4/809: getdents d5/d9/da/da3/de9 0 2024-06-15T01:07:08.221 INFO:tasks.workunit.client.1.smithi142.stdout:3/807: dwrite d0/d1b/f41 [0,4194304] 0 2024-06-15T01:07:08.223 INFO:tasks.workunit.client.1.smithi142.stdout:2/710: fdatasync d8/df/daf/fcd 0 2024-06-15T01:07:08.225 INFO:tasks.workunit.client.1.smithi142.stdout:1/755: dwrite d6/d12/d1c/f34 [4194304,4194304] 0 2024-06-15T01:07:08.225 INFO:tasks.workunit.client.1.smithi142.stdout:5/658: dwrite d2/d3e/d31/f6d [0,4194304] 0 2024-06-15T01:07:08.228 INFO:tasks.workunit.client.1.smithi142.stdout:5/659: chown d2/d3e/d31/d7e 1846 1 2024-06-15T01:07:08.230 INFO:tasks.workunit.client.0.smithi137.stdout:3/763: truncate d0/d2f/fb8 1034851 0 2024-06-15T01:07:08.230 INFO:tasks.workunit.client.1.smithi142.stdout:2/711: fsync d8/df/d10/d94/d25/d22/d43/d51/fd2 0 2024-06-15T01:07:08.231 INFO:tasks.workunit.client.1.smithi142.stdout:8/755: dwrite d0/d25/d5a/d84/fcc [0,4194304] 0 2024-06-15T01:07:08.233 INFO:tasks.workunit.client.0.smithi137.stdout:3/764: stat d0/d2/d22/d63/d76/f7d 0 2024-06-15T01:07:08.236 INFO:tasks.workunit.client.0.smithi137.stdout:0/607: dwrite d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:08.237 INFO:tasks.workunit.client.0.smithi137.stdout:3/765: dread d0/d2/d8/f7f [0,4194304] 0 2024-06-15T01:07:08.239 INFO:tasks.workunit.client.0.smithi137.stdout:0/608: fsync d1/d2/dc/d10/d14/d15/f57 0 2024-06-15T01:07:08.240 INFO:tasks.workunit.client.1.smithi142.stdout:0/716: symlink d3/d39/d3e/d81/d44/df1/lfa 0 2024-06-15T01:07:08.240 INFO:tasks.workunit.client.1.smithi142.stdout:8/756: dwrite d0/d25/d53/da7/fd6 [0,4194304] 0 2024-06-15T01:07:08.240 INFO:tasks.workunit.client.1.smithi142.stdout:7/746: fdatasync d2/d8/da/d33/d25/f67 0 2024-06-15T01:07:08.241 INFO:tasks.workunit.client.1.smithi142.stdout:3/808: rename d0/d1f/d20/fad to d0/d3/de2/d55/ff1 0 2024-06-15T01:07:08.241 INFO:tasks.workunit.client.0.smithi137.stdout:0/609: write d1/d2/dc/d10/d14/d15/f35 [5120956,117732] 0 2024-06-15T01:07:08.244 INFO:tasks.workunit.client.0.smithi137.stdout:0/610: truncate d1/d2/dc/d10/f96 4948475 0 2024-06-15T01:07:08.244 INFO:tasks.workunit.client.1.smithi142.stdout:5/660: creat d2/d3e/d74/fcf x:0 0 0 2024-06-15T01:07:08.245 INFO:tasks.workunit.client.0.smithi137.stdout:3/766: creat d0/d2/d8/d67/d7e/de3/ffb x:0 0 0 2024-06-15T01:07:08.246 INFO:tasks.workunit.client.1.smithi142.stdout:4/810: dread d5/d1c/d6f/dc7/d30/f64 [0,4194304] 0 2024-06-15T01:07:08.247 INFO:tasks.workunit.client.0.smithi137.stdout:3/767: dread - d0/d2/d8/d67/d7e/de3/ffb zero size 2024-06-15T01:07:08.247 INFO:tasks.workunit.client.0.smithi137.stdout:0/611: fsync d1/f32 0 2024-06-15T01:07:08.250 INFO:tasks.workunit.client.0.smithi137.stdout:3/768: truncate d0/d2/d8/dbd/ff6 5206689 0 2024-06-15T01:07:08.251 INFO:tasks.workunit.client.0.smithi137.stdout:0/612: rmdir d1/d2/dc/d10/d14/d15/d17 39 2024-06-15T01:07:08.252 INFO:tasks.workunit.client.1.smithi142.stdout:1/756: rename d6/d12/d3c/l92 to d6/d12/d3c/d63/d8d/l10e 0 2024-06-15T01:07:08.253 INFO:tasks.workunit.client.1.smithi142.stdout:4/811: rmdir d5/d1c/d6f/dc7/d30/dcc 39 2024-06-15T01:07:08.255 INFO:tasks.workunit.client.0.smithi137.stdout:3/769: read d0/d2/d23/f3f [2976335,124500] 0 2024-06-15T01:07:08.255 INFO:tasks.workunit.client.1.smithi142.stdout:5/661: creat d2/d3e/d98/fd0 x:0 0 0 2024-06-15T01:07:08.257 INFO:tasks.workunit.client.0.smithi137.stdout:0/613: rename d1/d2/d13/d66/l38 to d1/d2/dc/d10/d14/d15/d17/d54/ld3 0 2024-06-15T01:07:08.259 INFO:tasks.workunit.client.1.smithi142.stdout:5/662: read d2/d18/f63 [1309277,114979] 0 2024-06-15T01:07:08.260 INFO:tasks.workunit.client.1.smithi142.stdout:0/717: link d3/d39/d3e/d81/d44/d7c/d22/d59/fba d3/d39/d3e/d81/d44/d7c/d22/ffb 0 2024-06-15T01:07:08.262 INFO:tasks.workunit.client.1.smithi142.stdout:0/718: write d3/d39/d3e/d81/d1b/d52/f85 [4983227,58780] 0 2024-06-15T01:07:08.262 INFO:tasks.workunit.client.1.smithi142.stdout:4/812: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102/d116 0 2024-06-15T01:07:08.263 INFO:tasks.workunit.client.1.smithi142.stdout:1/757: dread d6/f90 [0,4194304] 0 2024-06-15T01:07:08.265 INFO:tasks.workunit.client.1.smithi142.stdout:2/712: rename d8/df/daf/cc7 to d8/df/d10/d94/d25/d22/cde 0 2024-06-15T01:07:08.266 INFO:tasks.workunit.client.0.smithi137.stdout:0/614: dwrite d1/d2/dc/d10/d14/d15/d1c/fc5 [0,4194304] 0 2024-06-15T01:07:08.267 INFO:tasks.workunit.client.1.smithi142.stdout:5/663: symlink d2/d3e/d31/da0/ld1 0 2024-06-15T01:07:08.267 INFO:tasks.workunit.client.1.smithi142.stdout:0/719: fsync d3/d39/d3e/d81/d4c/faf 0 2024-06-15T01:07:08.268 INFO:tasks.workunit.client.1.smithi142.stdout:4/813: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117 0 2024-06-15T01:07:08.268 INFO:tasks.workunit.client.0.smithi137.stdout:0/615: mkdir d1/d2/dc/d10/dd4 0 2024-06-15T01:07:08.270 INFO:tasks.workunit.client.0.smithi137.stdout:0/616: readlink d1/d2/dc/d10/d14/d56/l94 0 2024-06-15T01:07:08.272 INFO:tasks.workunit.client.0.smithi137.stdout:0/617: write d1/d2/dc/d6c/d88/d91/f9a [1723220,93982] 0 2024-06-15T01:07:08.274 INFO:tasks.workunit.client.1.smithi142.stdout:2/713: dwrite d8/df/d10/d94/d25/fda [0,4194304] 0 2024-06-15T01:07:08.275 INFO:tasks.workunit.client.0.smithi137.stdout:0/618: symlink d1/d2/dc/d10/d14/d52/ld5 0 2024-06-15T01:07:08.278 INFO:tasks.workunit.client.0.smithi137.stdout:0/619: unlink d1/la6 0 2024-06-15T01:07:08.280 INFO:tasks.workunit.client.0.smithi137.stdout:0/620: write d1/d7e/d87/da0/fcf [479770,41965] 0 2024-06-15T01:07:08.280 INFO:tasks.workunit.client.0.smithi137.stdout:0/621: stat d1/f32 0 2024-06-15T01:07:08.288 INFO:tasks.workunit.client.0.smithi137.stdout:3/770: dwrite d0/d2/d23/f6f [0,4194304] 0 2024-06-15T01:07:08.290 INFO:tasks.workunit.client.0.smithi137.stdout:0/622: dwrite d1/d2/dc/d10/d14/d15/d1c/f46 [0,4194304] 0 2024-06-15T01:07:08.292 INFO:tasks.workunit.client.0.smithi137.stdout:0/623: readlink d1/d2/dc/d10/lc1 0 2024-06-15T01:07:08.293 INFO:tasks.workunit.client.0.smithi137.stdout:3/771: read d0/d2/d22/d24/d5e/f85 [3416573,37729] 0 2024-06-15T01:07:08.294 INFO:tasks.workunit.client.0.smithi137.stdout:0/624: rename d1/d2 to d1/d2/dc/d10/dd4/dd6 22 2024-06-15T01:07:08.294 INFO:tasks.workunit.client.0.smithi137.stdout:3/772: fdatasync d0/d2/d23/f73 0 2024-06-15T01:07:08.296 INFO:tasks.workunit.client.0.smithi137.stdout:0/625: write d1/d2/d13/d66/f86 [855600,66757] 0 2024-06-15T01:07:08.298 INFO:tasks.workunit.client.0.smithi137.stdout:3/773: readlink d0/d56/d8f/d9c/lc0 0 2024-06-15T01:07:08.298 INFO:tasks.workunit.client.0.smithi137.stdout:0/626: rmdir d1/d2/dc/d10/d53/dca/d8d 39 2024-06-15T01:07:08.301 INFO:tasks.workunit.client.0.smithi137.stdout:0/627: mkdir d1/d7e/d87/da0/dc4/dd7 0 2024-06-15T01:07:08.303 INFO:tasks.workunit.client.0.smithi137.stdout:0/628: write d1/d2/dc/d10/d14/d15/d1c/f46 [1602852,99072] 0 2024-06-15T01:07:08.303 INFO:tasks.workunit.client.0.smithi137.stdout:3/774: creat d0/d56/ffc x:0 0 0 2024-06-15T01:07:08.307 INFO:tasks.workunit.client.0.smithi137.stdout:3/775: read d0/d77/fc2 [54326,8283] 0 2024-06-15T01:07:08.308 INFO:tasks.workunit.client.0.smithi137.stdout:0/629: symlink d1/d2/dc/d10/d14/d15/d17/d54/d7a/dd0/ld8 0 2024-06-15T01:07:08.309 INFO:tasks.workunit.client.0.smithi137.stdout:0/630: chown d1/d2/dc/d6c/d88/d91/f9a 661683672 1 2024-06-15T01:07:08.310 INFO:tasks.workunit.client.0.smithi137.stdout:3/776: creat d0/ddf/ffd x:0 0 0 2024-06-15T01:07:08.311 INFO:tasks.workunit.client.0.smithi137.stdout:0/631: rmdir d1/d2/dc/d10/d14/d15/d17 39 2024-06-15T01:07:08.317 INFO:tasks.workunit.client.0.smithi137.stdout:0/632: dwrite d1/d2/dc/d10/f96 [0,4194304] 0 2024-06-15T01:07:08.321 INFO:tasks.workunit.client.0.smithi137.stdout:0/633: dread d1/f32 [0,4194304] 0 2024-06-15T01:07:08.325 INFO:tasks.workunit.client.0.smithi137.stdout:0/634: mkdir d1/d2/dc/d10/d53/dca/d8d/dd9 0 2024-06-15T01:07:08.326 INFO:tasks.workunit.client.0.smithi137.stdout:0/635: chown d1/d7e/d87/la8 2 1 2024-06-15T01:07:08.329 INFO:tasks.workunit.client.0.smithi137.stdout:7/471: sync 2024-06-15T01:07:08.329 INFO:tasks.workunit.client.0.smithi137.stdout:0/636: mknod d1/d2/dc/d10/d14/d15/d17/d54/d7a/dd0/cda 0 2024-06-15T01:07:08.331 INFO:tasks.workunit.client.0.smithi137.stdout:7/472: creat df/d7d/d3b/f93 x:0 0 0 2024-06-15T01:07:08.332 INFO:tasks.workunit.client.0.smithi137.stdout:0/637: mknod d1/d2/dc/d10/d14/d56/d99/cdb 0 2024-06-15T01:07:08.332 INFO:tasks.workunit.client.0.smithi137.stdout:0/638: stat d1/d2/dc/fbf 0 2024-06-15T01:07:08.335 INFO:tasks.workunit.client.0.smithi137.stdout:0/639: getdents d1/d2/d55 0 2024-06-15T01:07:08.337 INFO:tasks.workunit.client.0.smithi137.stdout:3/777: dread d0/d13/d37/f43 [4194304,4194304] 0 2024-06-15T01:07:08.338 INFO:tasks.workunit.client.0.smithi137.stdout:3/778: readlink d0/d2/d22/d63/d76/lf8 0 2024-06-15T01:07:08.340 INFO:tasks.workunit.client.0.smithi137.stdout:3/779: mknod d0/d56/d8f/d9c/d9f/cfe 0 2024-06-15T01:07:08.342 INFO:tasks.workunit.client.0.smithi137.stdout:3/780: mkdir d0/d2/d22/d24/d28/d2e/df0/dff 0 2024-06-15T01:07:08.343 INFO:tasks.workunit.client.0.smithi137.stdout:3/781: creat d0/d56/d8f/d9c/d9f/f100 x:0 0 0 2024-06-15T01:07:08.346 INFO:tasks.workunit.client.0.smithi137.stdout:3/782: link d0/d13/d37/f43 d0/d2/d23/de2/f101 0 2024-06-15T01:07:08.348 INFO:tasks.workunit.client.0.smithi137.stdout:3/783: getdents d0/d2f 0 2024-06-15T01:07:08.348 INFO:tasks.workunit.client.0.smithi137.stdout:3/784: fsync d0/d2f/dbe/fe0 0 2024-06-15T01:07:08.354 INFO:tasks.workunit.client.0.smithi137.stdout:3/785: dwrite d0/d2/d1e/f6e [0,4194304] 0 2024-06-15T01:07:08.355 INFO:tasks.workunit.client.0.smithi137.stdout:3/786: fsync d0/d13/d37/d83/fe1 0 2024-06-15T01:07:08.362 INFO:tasks.workunit.client.0.smithi137.stdout:3/787: dwrite d0/d2/d22/d63/d76/fce [0,4194304] 0 2024-06-15T01:07:08.365 INFO:tasks.workunit.client.0.smithi137.stdout:3/788: symlink d0/d2/d1e/l102 0 2024-06-15T01:07:08.367 INFO:tasks.workunit.client.1.smithi142.stdout:5/664: creat d2/de/d19/d5f/fd2 x:0 0 0 2024-06-15T01:07:08.371 INFO:tasks.workunit.client.1.smithi142.stdout:0/720: unlink d3/c8 0 2024-06-15T01:07:08.373 INFO:tasks.workunit.client.1.smithi142.stdout:4/814: creat d5/d9/d41/f118 x:0 0 0 2024-06-15T01:07:08.377 INFO:tasks.workunit.client.1.smithi142.stdout:1/758: link d6/d12/d3c/d63/d8d/d97/dc6/lf4 d6/d4d/l10f 0 2024-06-15T01:07:08.378 INFO:tasks.workunit.client.1.smithi142.stdout:0/721: truncate d3/d39/d3e/d81/d44/d7c/f95 3801113 0 2024-06-15T01:07:08.379 INFO:tasks.workunit.client.1.smithi142.stdout:2/714: mknod d8/df/d10/d1c/cdf 0 2024-06-15T01:07:08.380 INFO:tasks.workunit.client.1.smithi142.stdout:2/715: fdatasync d8/fd4 0 2024-06-15T01:07:08.380 INFO:tasks.workunit.client.1.smithi142.stdout:7/747: truncate d2/d8/dd/d14/d2c/d37/f75 421120 0 2024-06-15T01:07:08.381 INFO:tasks.workunit.client.1.smithi142.stdout:2/716: chown d8/df/d10/d94/d25/f53 1835575 1 2024-06-15T01:07:08.382 INFO:tasks.workunit.client.1.smithi142.stdout:1/759: creat d6/d12/d80/dc8/f110 x:0 0 0 2024-06-15T01:07:08.383 INFO:tasks.workunit.client.1.smithi142.stdout:1/760: stat d6/d12/d3c/d63/d8d/d97/dd3/fac 0 2024-06-15T01:07:08.383 INFO:tasks.workunit.client.1.smithi142.stdout:5/665: creat d2/de/d19/d5f/d68/d76/fd3 x:0 0 0 2024-06-15T01:07:08.387 INFO:tasks.workunit.client.1.smithi142.stdout:2/717: dread d8/df/d10/d94/d25/d22/d43/d52/dc1/fcb [0,4194304] 0 2024-06-15T01:07:08.387 INFO:tasks.workunit.client.1.smithi142.stdout:1/761: write d6/f26 [1108916,16081] 0 2024-06-15T01:07:08.389 INFO:tasks.workunit.client.1.smithi142.stdout:5/666: link d2/de/d19/d5f/d68/f73 d2/de/d27/d86/db9/fd4 0 2024-06-15T01:07:08.390 INFO:tasks.workunit.client.1.smithi142.stdout:2/718: write d8/d19/f70 [522068,110207] 0 2024-06-15T01:07:08.392 INFO:tasks.workunit.client.1.smithi142.stdout:5/667: chown d2/de/d19/d5f/l6f 167 1 2024-06-15T01:07:08.392 INFO:tasks.workunit.client.1.smithi142.stdout:1/762: mknod d6/d3a/c111 0 2024-06-15T01:07:08.393 INFO:tasks.workunit.client.1.smithi142.stdout:7/748: dwrite d2/db6/fec [0,4194304] 0 2024-06-15T01:07:08.396 INFO:tasks.workunit.client.1.smithi142.stdout:2/719: dread - d8/df/d10/d1c/d7c/f90 zero size 2024-06-15T01:07:08.397 INFO:tasks.workunit.client.1.smithi142.stdout:1/763: creat d6/d12/d80/d9b/db2/dc2/f112 x:0 0 0 2024-06-15T01:07:08.397 INFO:tasks.workunit.client.0.smithi137.stdout:3/789: dread d0/d13/f20 [0,4194304] 0 2024-06-15T01:07:08.398 INFO:tasks.workunit.client.0.smithi137.stdout:3/790: fsync d0/d2/d23/f6f 0 2024-06-15T01:07:08.399 INFO:tasks.workunit.client.1.smithi142.stdout:5/668: truncate d2/de/d27/f46 90296 0 2024-06-15T01:07:08.399 INFO:tasks.workunit.client.0.smithi137.stdout:3/791: chown d0/d2/daf 0 1 2024-06-15T01:07:08.401 INFO:tasks.workunit.client.1.smithi142.stdout:7/749: rename d2/d8/da/d33/f9e to d2/d8/dd/dcf/ff0 0 2024-06-15T01:07:08.402 INFO:tasks.workunit.client.1.smithi142.stdout:7/750: write d2/d8/dd/fd9 [616829,120084] 0 2024-06-15T01:07:08.402 INFO:tasks.workunit.client.1.smithi142.stdout:2/720: symlink d8/d19/d8d/le0 0 2024-06-15T01:07:08.403 INFO:tasks.workunit.client.1.smithi142.stdout:7/751: write d2/d8/da/d33/f52 [878496,57283] 0 2024-06-15T01:07:08.404 INFO:tasks.workunit.client.1.smithi142.stdout:9/672: sync 2024-06-15T01:07:08.406 INFO:tasks.workunit.client.1.smithi142.stdout:3/809: write d0/f6a [861348,43237] 0 2024-06-15T01:07:08.408 INFO:tasks.workunit.client.1.smithi142.stdout:8/757: dwrite d0/f23 [0,4194304] 0 2024-06-15T01:07:08.408 INFO:tasks.workunit.client.1.smithi142.stdout:2/721: creat d8/df/d10/d94/d25/d1a/d63/fe1 x:0 0 0 2024-06-15T01:07:08.417 INFO:tasks.workunit.client.1.smithi142.stdout:8/758: mknod d0/d25/d53/da7/dcb/ce4 0 2024-06-15T01:07:08.418 INFO:tasks.workunit.client.1.smithi142.stdout:7/752: truncate d2/d8/dd/dcf/f2f 742284 0 2024-06-15T01:07:08.421 INFO:tasks.workunit.client.1.smithi142.stdout:3/810: rename d0/fe8 to d0/d95/ff2 0 2024-06-15T01:07:08.421 INFO:tasks.workunit.client.1.smithi142.stdout:2/722: write d8/df/d10/d94/d25/fda [5237377,129500] 0 2024-06-15T01:07:08.423 INFO:tasks.workunit.client.1.smithi142.stdout:9/673: getdents d6/dc/d1b/d6f/de1 0 2024-06-15T01:07:08.425 INFO:tasks.workunit.client.1.smithi142.stdout:4/815: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fda [4789746,91565] 0 2024-06-15T01:07:08.427 INFO:tasks.workunit.client.1.smithi142.stdout:0/722: write d3/d39/d9a/d9d/dcd/fcf [507817,71527] 0 2024-06-15T01:07:08.430 INFO:tasks.workunit.client.1.smithi142.stdout:4/816: unlink d5/d9/d23/d32/fb4 0 2024-06-15T01:07:08.430 INFO:tasks.workunit.client.1.smithi142.stdout:8/759: getdents d0/d25/d5a/d84 0 2024-06-15T01:07:08.432 INFO:tasks.workunit.client.1.smithi142.stdout:2/723: dwrite d8/df/d10/d94/d25/f74 [0,4194304] 0 2024-06-15T01:07:08.434 INFO:tasks.workunit.client.1.smithi142.stdout:9/674: getdents d6/dc/d1b/d6f/da1 0 2024-06-15T01:07:08.436 INFO:tasks.workunit.client.1.smithi142.stdout:8/760: creat d0/d25/d53/da7/dcb/fe5 x:0 0 0 2024-06-15T01:07:08.437 INFO:tasks.workunit.client.1.smithi142.stdout:2/724: read d8/d19/f56 [3091442,14287] 0 2024-06-15T01:07:08.438 INFO:tasks.workunit.client.1.smithi142.stdout:5/669: write d2/f21 [232246,88904] 0 2024-06-15T01:07:08.441 INFO:tasks.workunit.client.1.smithi142.stdout:9/675: mknod d6/dc/d1b/cff 0 2024-06-15T01:07:08.443 INFO:tasks.workunit.client.1.smithi142.stdout:2/725: symlink d8/df/d10/d94/d25/d22/d43/d52/dc1/le2 0 2024-06-15T01:07:08.447 INFO:tasks.workunit.client.1.smithi142.stdout:5/670: dwrite d2/d3e/d31/d7e/fa8 [0,4194304] 0 2024-06-15T01:07:08.447 INFO:tasks.workunit.client.1.smithi142.stdout:9/676: fsync d6/d31/f84 0 2024-06-15T01:07:08.452 INFO:tasks.workunit.client.1.smithi142.stdout:8/761: rename d0/d25/d5a/d84/fd1 to d0/d17/d40/fe6 0 2024-06-15T01:07:08.453 INFO:tasks.workunit.client.1.smithi142.stdout:2/726: mkdir d8/df/d10/d94/d25/d22/d43/de3 0 2024-06-15T01:07:08.454 INFO:tasks.workunit.client.1.smithi142.stdout:7/753: dwrite d2/d6/f6d [0,4194304] 0 2024-06-15T01:07:08.455 INFO:tasks.workunit.client.1.smithi142.stdout:0/723: dread f1 [0,4194304] 0 2024-06-15T01:07:08.456 INFO:tasks.workunit.client.1.smithi142.stdout:0/724: readlink d3/d39/d3e/d81/ld 0 2024-06-15T01:07:08.459 INFO:tasks.workunit.client.1.smithi142.stdout:7/754: truncate d2/d8/da/d33/d25/d79/fe4 71114 0 2024-06-15T01:07:08.461 INFO:tasks.workunit.client.1.smithi142.stdout:3/811: truncate d0/d3/fcb 568053 0 2024-06-15T01:07:08.462 INFO:tasks.workunit.client.1.smithi142.stdout:8/762: mknod d0/d25/d53/d9c/ce7 0 2024-06-15T01:07:08.466 INFO:tasks.workunit.client.1.smithi142.stdout:9/677: rmdir d6/dc/d1b/d25/d58/df8 0 2024-06-15T01:07:08.469 INFO:tasks.workunit.client.1.smithi142.stdout:4/817: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f105 [0,4194304] 0 2024-06-15T01:07:08.471 INFO:tasks.workunit.client.1.smithi142.stdout:5/671: write d2/d18/f55 [1964525,20898] 0 2024-06-15T01:07:08.471 INFO:tasks.workunit.client.1.smithi142.stdout:0/725: dwrite d3/d39/d3e/d8f/fd2 [0,4194304] 0 2024-06-15T01:07:08.475 INFO:tasks.workunit.client.1.smithi142.stdout:9/678: creat d6/d34/df3/f100 x:0 0 0 2024-06-15T01:07:08.475 INFO:tasks.workunit.client.1.smithi142.stdout:5/672: chown d2/de/f66 227217944 1 2024-06-15T01:07:08.476 INFO:tasks.workunit.client.1.smithi142.stdout:0/726: chown d3/d39/d3e/d81/d1b/d38/ccc 13219148 1 2024-06-15T01:07:08.477 INFO:tasks.workunit.client.1.smithi142.stdout:3/812: dwrite d0/d3/da/d18/f98 [0,4194304] 0 2024-06-15T01:07:08.478 INFO:tasks.workunit.client.1.smithi142.stdout:4/818: creat d5/d9/da/d92/f119 x:0 0 0 2024-06-15T01:07:08.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/763: getdents d0/d17/d40 0 2024-06-15T01:07:08.485 INFO:tasks.workunit.client.1.smithi142.stdout:3/813: fdatasync d0/d3/da/f9b 0 2024-06-15T01:07:08.486 INFO:tasks.workunit.client.1.smithi142.stdout:4/819: unlink d5/d1c/d6f/dc7/d30/f64 0 2024-06-15T01:07:08.489 INFO:tasks.workunit.client.1.smithi142.stdout:8/764: chown d0/d25/fc0 381 1 2024-06-15T01:07:08.491 INFO:tasks.workunit.client.1.smithi142.stdout:5/673: getdents d2/d3e/d74/da5 0 2024-06-15T01:07:08.492 INFO:tasks.workunit.client.1.smithi142.stdout:3/814: chown d0/f39 7572 1 2024-06-15T01:07:08.493 INFO:tasks.workunit.client.1.smithi142.stdout:9/679: dwrite d6/dc/d1b/d25/fd3 [0,4194304] 0 2024-06-15T01:07:08.495 INFO:tasks.workunit.client.1.smithi142.stdout:5/674: fsync d2/d18/fc9 0 2024-06-15T01:07:08.496 INFO:tasks.workunit.client.1.smithi142.stdout:7/755: dwrite d2/d8/da/fae [4194304,4194304] 0 2024-06-15T01:07:08.496 INFO:tasks.workunit.client.1.smithi142.stdout:4/820: mkdir d5/d1c/d6f/dc7/d30/dcc/d11a 0 2024-06-15T01:07:08.499 INFO:tasks.workunit.client.1.smithi142.stdout:8/765: creat d0/d25/d53/da7/fe8 x:0 0 0 2024-06-15T01:07:08.500 INFO:tasks.workunit.client.1.smithi142.stdout:0/727: dwrite d3/d39/d3e/d81/d44/fd6 [0,4194304] 0 2024-06-15T01:07:08.505 INFO:tasks.workunit.client.1.smithi142.stdout:8/766: rename d0/d17/c5c to d0/d25/d53/da7/dcb/ce9 0 2024-06-15T01:07:08.506 INFO:tasks.workunit.client.1.smithi142.stdout:8/767: chown d0/d17/d40/d6e/fe3 3280 1 2024-06-15T01:07:08.507 INFO:tasks.workunit.client.1.smithi142.stdout:0/728: creat d3/d39/d3e/d81/d1e/ffc x:0 0 0 2024-06-15T01:07:08.509 INFO:tasks.workunit.client.1.smithi142.stdout:5/675: creat d2/de/d19/d5f/d68/d76/da6/dc5/fd5 x:0 0 0 2024-06-15T01:07:08.510 INFO:tasks.workunit.client.1.smithi142.stdout:4/821: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117/f11b x:0 0 0 2024-06-15T01:07:08.511 INFO:tasks.workunit.client.1.smithi142.stdout:7/756: mknod d2/d8/da/d33/d25/d79/dd4/cf1 0 2024-06-15T01:07:08.512 INFO:tasks.workunit.client.1.smithi142.stdout:8/768: mknod d0/d17/d40/d5e/cea 0 2024-06-15T01:07:08.519 INFO:tasks.workunit.client.1.smithi142.stdout:7/757: link d2/d6/l21 d2/lf2 0 2024-06-15T01:07:08.521 INFO:tasks.workunit.client.1.smithi142.stdout:5/676: dwrite d2/d3e/d98/fbf [0,4194304] 0 2024-06-15T01:07:08.523 INFO:tasks.workunit.client.1.smithi142.stdout:4/822: dwrite d5/d9/d1a/f104 [0,4194304] 0 2024-06-15T01:07:08.525 INFO:tasks.workunit.client.1.smithi142.stdout:6/728: sync 2024-06-15T01:07:08.527 INFO:tasks.workunit.client.1.smithi142.stdout:8/769: dwrite d0/d25/d5a/f4d [0,4194304] 0 2024-06-15T01:07:08.529 INFO:tasks.workunit.client.1.smithi142.stdout:4/823: symlink d5/d9/l11c 0 2024-06-15T01:07:08.532 INFO:tasks.workunit.client.1.smithi142.stdout:9/680: dwrite d6/f45 [0,4194304] 0 2024-06-15T01:07:08.534 INFO:tasks.workunit.client.1.smithi142.stdout:6/729: rename d1a/d39/d42/d4c to d1a/d3c/d100 0 2024-06-15T01:07:08.537 INFO:tasks.workunit.client.1.smithi142.stdout:4/824: symlink d5/d9/da/d92/l11d 0 2024-06-15T01:07:08.537 INFO:tasks.workunit.client.1.smithi142.stdout:7/758: dwrite d2/db6/fec [0,4194304] 0 2024-06-15T01:07:08.538 INFO:tasks.workunit.client.1.smithi142.stdout:5/677: dread d2/f28 [0,4194304] 0 2024-06-15T01:07:08.541 INFO:tasks.workunit.client.1.smithi142.stdout:4/825: fsync d5/d9/da/d92/d98/dc1/f10c 0 2024-06-15T01:07:08.542 INFO:tasks.workunit.client.1.smithi142.stdout:6/730: creat d1a/d39/ded/f101 x:0 0 0 2024-06-15T01:07:08.542 INFO:tasks.workunit.client.1.smithi142.stdout:4/826: stat d5/d1c/d6f/lc9 0 2024-06-15T01:07:08.543 INFO:tasks.workunit.client.1.smithi142.stdout:5/678: write d2/de/d27/db2/fbb [718884,12230] 0 2024-06-15T01:07:08.543 INFO:tasks.workunit.client.1.smithi142.stdout:7/759: unlink d2/d8/cba 0 2024-06-15T01:07:08.544 INFO:tasks.workunit.client.1.smithi142.stdout:4/827: chown d5/d9/d18/f80 2 1 2024-06-15T01:07:08.546 INFO:tasks.workunit.client.1.smithi142.stdout:5/679: write d2/d3e/d98/fd0 [196222,49351] 0 2024-06-15T01:07:08.547 INFO:tasks.workunit.client.1.smithi142.stdout:7/760: mkdir d2/d8/da/d33/d25/d81/df3 0 2024-06-15T01:07:08.552 INFO:tasks.workunit.client.1.smithi142.stdout:4/828: rename d5/d9/d90/daf/dcb to d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117/d11e 0 2024-06-15T01:07:08.552 INFO:tasks.workunit.client.1.smithi142.stdout:5/680: truncate d2/de/d19/d5f/d68/d76/f7c 835947 0 2024-06-15T01:07:08.554 INFO:tasks.workunit.client.1.smithi142.stdout:5/681: mkdir d2/d3e/d65/dd6 0 2024-06-15T01:07:08.555 INFO:tasks.workunit.client.1.smithi142.stdout:1/764: sync 2024-06-15T01:07:08.558 INFO:tasks.workunit.client.1.smithi142.stdout:0/729: dread d3/d39/d3e/d81/d44/d7c/f95 [0,4194304] 0 2024-06-15T01:07:08.561 INFO:tasks.workunit.client.1.smithi142.stdout:4/829: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f107 [0,4194304] 0 2024-06-15T01:07:08.563 INFO:tasks.workunit.client.1.smithi142.stdout:4/830: stat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6 0 2024-06-15T01:07:08.564 INFO:tasks.workunit.client.1.smithi142.stdout:6/731: dread d1a/d83/fbb [0,4194304] 0 2024-06-15T01:07:08.564 INFO:tasks.workunit.client.1.smithi142.stdout:7/761: dread d2/d8/dd/d14/f4b [0,4194304] 0 2024-06-15T01:07:08.572 INFO:tasks.workunit.client.0.smithi137.stdout:6/441: sync 2024-06-15T01:07:08.572 INFO:tasks.workunit.client.0.smithi137.stdout:4/531: sync 2024-06-15T01:07:08.580 INFO:tasks.workunit.client.0.smithi137.stdout:4/532: symlink dd/d15/d21/d3a/d6e/d7b/d96/lc8 0 2024-06-15T01:07:08.584 INFO:tasks.workunit.client.0.smithi137.stdout:4/533: creat dd/d15/d21/d3a/d51/fc9 x:0 0 0 2024-06-15T01:07:08.589 INFO:tasks.workunit.client.0.smithi137.stdout:4/534: rmdir dd/d15/d21/d3a/d51/d83/db2/db8 0 2024-06-15T01:07:08.595 INFO:tasks.workunit.client.0.smithi137.stdout:6/442: dread d0/d5/d9/f4f [0,4194304] 0 2024-06-15T01:07:08.596 INFO:tasks.workunit.client.1.smithi142.stdout:3/815: dwrite d0/d3/da/f62 [0,4194304] 0 2024-06-15T01:07:08.596 INFO:tasks.workunit.client.0.smithi137.stdout:4/535: link dd/d15/d21/d3a/d51/d83/db2/d67/l77 dd/d15/d21/d3a/d51/d83/db2/d67/lca 0 2024-06-15T01:07:08.603 INFO:tasks.workunit.client.1.smithi142.stdout:1/765: dread d6/d12/d80/dae/fb7 [0,4194304] 0 2024-06-15T01:07:08.614 INFO:tasks.workunit.client.0.smithi137.stdout:4/536: write dd/fe [2594645,31498] 0 2024-06-15T01:07:08.616 INFO:tasks.workunit.client.0.smithi137.stdout:6/443: truncate d0/d1a/d3e/d57/d1d/d2e/f38 1135550 0 2024-06-15T01:07:08.617 INFO:tasks.workunit.client.1.smithi142.stdout:8/770: dread d0/d25/d5a/f88 [0,4194304] 0 2024-06-15T01:07:08.617 INFO:tasks.workunit.client.1.smithi142.stdout:4/831: mkdir d5/d9/d23/d32/d11f 0 2024-06-15T01:07:08.618 INFO:tasks.workunit.client.0.smithi137.stdout:6/444: write d0/d1a/d3e/d57/d1d/f45 [896890,24262] 0 2024-06-15T01:07:08.620 INFO:tasks.workunit.client.0.smithi137.stdout:6/445: read d0/d5/d9/d29/f4b [989112,120729] 0 2024-06-15T01:07:08.622 INFO:tasks.workunit.client.0.smithi137.stdout:4/537: rename dd/d15/d21/d3a/d51 to dd/d15/d36/d45/dcb 0 2024-06-15T01:07:08.628 INFO:tasks.workunit.client.0.smithi137.stdout:6/446: symlink d0/d1a/d3e/d57/d1d/d2e/d62/d84/l89 0 2024-06-15T01:07:08.629 INFO:tasks.workunit.client.0.smithi137.stdout:6/447: readlink d0/l37 0 2024-06-15T01:07:08.630 INFO:tasks.workunit.client.1.smithi142.stdout:4/832: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d [0,4194304] 0 2024-06-15T01:07:08.633 INFO:tasks.workunit.client.0.smithi137.stdout:4/538: rmdir dd/d15/d36/d45/dcb/d83/db2/da1 0 2024-06-15T01:07:08.634 INFO:tasks.workunit.client.0.smithi137.stdout:4/539: truncate dd/d15/d36/d45/d55/f82 163953 0 2024-06-15T01:07:08.634 INFO:tasks.workunit.client.0.smithi137.stdout:6/448: mknod d0/d1a/d3e/d57/d32/d74/c8a 0 2024-06-15T01:07:08.637 INFO:tasks.workunit.client.0.smithi137.stdout:4/540: mknod dd/d15/d30/ccc 0 2024-06-15T01:07:08.638 INFO:tasks.workunit.client.1.smithi142.stdout:7/762: dread d2/d8/da/d33/f35 [0,4194304] 0 2024-06-15T01:07:08.639 INFO:tasks.workunit.client.1.smithi142.stdout:2/727: sync 2024-06-15T01:07:08.641 INFO:tasks.workunit.client.0.smithi137.stdout:6/449: rename d0/d1a/d3e/d57/d1d/d2e/f51 to d0/d1a/d3e/d57/d32/d74/d5a/f8b 0 2024-06-15T01:07:08.642 INFO:tasks.workunit.client.0.smithi137.stdout:6/450: dread - d0/d1a/d3e/f71 zero size 2024-06-15T01:07:08.642 INFO:tasks.workunit.client.1.smithi142.stdout:3/816: dread d0/d1f/d20/d59/d61/d96/faa [0,4194304] 0 2024-06-15T01:07:08.643 INFO:tasks.workunit.client.0.smithi137.stdout:6/451: chown d0 2127001677 1 2024-06-15T01:07:08.643 INFO:tasks.workunit.client.1.smithi142.stdout:7/763: fsync d2/d8/da/d33/d25/d79/f92 0 2024-06-15T01:07:08.644 INFO:tasks.workunit.client.1.smithi142.stdout:3/817: dread - d0/d1f/fd3 zero size 2024-06-15T01:07:08.644 INFO:tasks.workunit.client.0.smithi137.stdout:6/452: fsync d0/d1a/d3e/d57/d1d/f40 0 2024-06-15T01:07:08.644 INFO:tasks.workunit.client.1.smithi142.stdout:2/728: unlink d8/df/d1f/c29 0 2024-06-15T01:07:08.645 INFO:tasks.workunit.client.0.smithi137.stdout:4/541: rename dd/d15/d36/d45/dcb/d83/db2/d72/ca0 to dd/d15/d36/ccd 0 2024-06-15T01:07:08.646 INFO:tasks.workunit.client.1.smithi142.stdout:9/681: dwrite d6/dc/dd/f46 [0,4194304] 0 2024-06-15T01:07:08.648 INFO:tasks.workunit.client.0.smithi137.stdout:4/542: dread dd/d15/d36/d45/d55/f82 [0,4194304] 0 2024-06-15T01:07:08.649 INFO:tasks.workunit.client.1.smithi142.stdout:7/764: dread d2/db6/fec [0,4194304] 0 2024-06-15T01:07:08.650 INFO:tasks.workunit.client.1.smithi142.stdout:7/765: stat d2/d8/dd/dc0/fc3 0 2024-06-15T01:07:08.661 INFO:tasks.workunit.client.1.smithi142.stdout:3/818: rename d0/fba to d0/d3/de2/d75/ff3 0 2024-06-15T01:07:08.661 INFO:tasks.workunit.client.1.smithi142.stdout:5/682: write d2/d18/f99 [1495229,118379] 0 2024-06-15T01:07:08.661 INFO:tasks.workunit.client.1.smithi142.stdout:7/766: mknod d2/d8/dd/dc0/cf4 0 2024-06-15T01:07:08.663 INFO:tasks.workunit.client.1.smithi142.stdout:0/730: sync 2024-06-15T01:07:08.665 INFO:tasks.workunit.client.1.smithi142.stdout:2/729: creat d8/df/d10/d94/d25/d22/d43/d51/dd7/fe4 x:0 0 0 2024-06-15T01:07:08.665 INFO:tasks.workunit.client.0.smithi137.stdout:6/453: dread d0/d5/f12 [0,4194304] 0 2024-06-15T01:07:08.668 INFO:tasks.workunit.client.1.smithi142.stdout:7/767: chown d2/d6/lb4 0 1 2024-06-15T01:07:08.669 INFO:tasks.workunit.client.0.smithi137.stdout:6/454: mknod d0/d1a/d3e/d57/d32/d74/d5a/c8c 0 2024-06-15T01:07:08.669 INFO:tasks.workunit.client.0.smithi137.stdout:6/455: readlink d0/d5/l50 0 2024-06-15T01:07:08.672 INFO:tasks.workunit.client.1.smithi142.stdout:0/731: rmdir d3/d39/d3e/d81/d44/d7c/d22 39 2024-06-15T01:07:08.673 INFO:tasks.workunit.client.1.smithi142.stdout:2/730: creat d8/df/d10/d94/d25/d1a/d57/fe5 x:0 0 0 2024-06-15T01:07:08.674 INFO:tasks.workunit.client.1.smithi142.stdout:2/731: chown d8/df/d1f 6984 1 2024-06-15T01:07:08.676 INFO:tasks.workunit.client.1.smithi142.stdout:0/732: mknod d3/d39/d3e/d8f/cfd 0 2024-06-15T01:07:08.676 INFO:tasks.workunit.client.0.smithi137.stdout:6/456: write d0/d5/d9/d3f/f4a [1077419,129510] 0 2024-06-15T01:07:08.678 INFO:tasks.workunit.client.1.smithi142.stdout:2/732: fdatasync d8/df/d10/d94/d25/d22/d43/d52/d7a/fbe 0 2024-06-15T01:07:08.678 INFO:tasks.workunit.client.1.smithi142.stdout:9/682: dread d6/d34/fa0 [0,4194304] 0 2024-06-15T01:07:08.680 INFO:tasks.workunit.client.1.smithi142.stdout:9/683: write d6/d34/d8a/dec/ffc [1025249,75161] 0 2024-06-15T01:07:08.682 INFO:tasks.workunit.client.1.smithi142.stdout:0/733: mkdir d3/d39/d3e/d81/d1b/d86/dfe 0 2024-06-15T01:07:08.684 INFO:tasks.workunit.client.1.smithi142.stdout:0/734: rmdir d3/d39/d9a/d9d/dbf 39 2024-06-15T01:07:08.687 INFO:tasks.workunit.client.1.smithi142.stdout:6/732: mknod d1a/d83/c102 0 2024-06-15T01:07:08.690 INFO:tasks.workunit.client.1.smithi142.stdout:6/733: mknod d1a/d2d/c103 0 2024-06-15T01:07:08.692 INFO:tasks.workunit.client.0.smithi137.stdout:6/457: dread d0/f39 [0,4194304] 0 2024-06-15T01:07:08.694 INFO:tasks.workunit.client.0.smithi137.stdout:6/458: fdatasync d0/d1a/d3e/d57/d32/d74/f61 0 2024-06-15T01:07:08.694 INFO:tasks.workunit.client.1.smithi142.stdout:6/734: truncate d1a/d39/d3a/fec 330178 0 2024-06-15T01:07:08.695 INFO:tasks.workunit.client.0.smithi137.stdout:6/459: chown d0/d5/f83 0 1 2024-06-15T01:07:08.696 INFO:tasks.workunit.client.0.smithi137.stdout:6/460: write d0/f39 [3608859,77863] 0 2024-06-15T01:07:08.700 INFO:tasks.workunit.client.0.smithi137.stdout:6/461: creat d0/d1a/d3e/d57/d1d/d2e/f8d x:0 0 0 2024-06-15T01:07:08.701 INFO:tasks.workunit.client.0.smithi137.stdout:6/462: fdatasync d0/d5/f6d 0 2024-06-15T01:07:08.705 INFO:tasks.workunit.client.1.smithi142.stdout:8/771: creat d0/d17/d40/feb x:0 0 0 2024-06-15T01:07:08.706 INFO:tasks.workunit.client.0.smithi137.stdout:6/463: write d0/f10 [1818352,63366] 0 2024-06-15T01:07:08.710 INFO:tasks.workunit.client.1.smithi142.stdout:4/833: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117/d11e/f8a 930628 0 2024-06-15T01:07:08.712 INFO:tasks.workunit.client.1.smithi142.stdout:1/766: dwrite d6/d3a/f44 [0,4194304] 0 2024-06-15T01:07:08.714 INFO:tasks.workunit.client.1.smithi142.stdout:4/834: unlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/ccf 0 2024-06-15T01:07:08.716 INFO:tasks.workunit.client.1.smithi142.stdout:1/767: chown d6/d12/d1c/d2c/d47/c49 6885 1 2024-06-15T01:07:08.717 INFO:tasks.workunit.client.1.smithi142.stdout:1/768: truncate d6/f13 4769027 0 2024-06-15T01:07:08.717 INFO:tasks.workunit.client.1.smithi142.stdout:4/835: symlink d5/d1c/d6f/db9/l120 0 2024-06-15T01:07:08.718 INFO:tasks.workunit.client.1.smithi142.stdout:4/836: chown d5/d9/d1a/f104 26 1 2024-06-15T01:07:08.718 INFO:tasks.workunit.client.1.smithi142.stdout:1/769: truncate d6/d12/d80/f100 405856 0 2024-06-15T01:07:08.719 INFO:tasks.workunit.client.1.smithi142.stdout:5/683: creat d2/de/d19/d5f/d68/fd7 x:0 0 0 2024-06-15T01:07:08.720 INFO:tasks.workunit.client.1.smithi142.stdout:5/684: chown d2/d3e/d65/dd6 4893480 1 2024-06-15T01:07:08.721 INFO:tasks.workunit.client.1.smithi142.stdout:1/770: truncate d6/d12/d80/dc8/fe7 1002256 0 2024-06-15T01:07:08.723 INFO:tasks.workunit.client.1.smithi142.stdout:4/837: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/c121 0 2024-06-15T01:07:08.723 INFO:tasks.workunit.client.0.smithi137.stdout:6/464: dread d0/d5/d9/f63 [4194304,4194304] 0 2024-06-15T01:07:08.724 INFO:tasks.workunit.client.0.smithi137.stdout:6/465: fdatasync d0/d1a/d3e/d57/d1d/f45 0 2024-06-15T01:07:08.725 INFO:tasks.workunit.client.1.smithi142.stdout:3/819: rmdir d0/d1f/d20/d5d/d82/d6e 39 2024-06-15T01:07:08.726 INFO:tasks.workunit.client.0.smithi137.stdout:6/466: write d0/d1a/d3e/d57/d1d/d2e/d62/f76 [914046,86592] 0 2024-06-15T01:07:08.729 INFO:tasks.workunit.client.1.smithi142.stdout:1/771: dwrite d6/d12/d1c/fe0 [0,4194304] 0 2024-06-15T01:07:08.730 INFO:tasks.workunit.client.0.smithi137.stdout:6/467: creat d0/d1a/d3e/d57/d1d/d2e/f8e x:0 0 0 2024-06-15T01:07:08.730 INFO:tasks.workunit.client.1.smithi142.stdout:4/838: creat d5/d1c/d6f/dc7/f122 x:0 0 0 2024-06-15T01:07:08.731 INFO:tasks.workunit.client.1.smithi142.stdout:1/772: write d6/d12/d3c/d63/f109 [308890,39452] 0 2024-06-15T01:07:08.734 INFO:tasks.workunit.client.1.smithi142.stdout:4/839: creat d5/d9/da/da3/f123 x:0 0 0 2024-06-15T01:07:08.735 INFO:tasks.workunit.client.1.smithi142.stdout:1/773: symlink d6/d12/d1c/d2c/d93/l113 0 2024-06-15T01:07:08.738 INFO:tasks.workunit.client.1.smithi142.stdout:1/774: creat d6/d12/d80/d9b/db2/dc2/f114 x:0 0 0 2024-06-15T01:07:08.738 INFO:tasks.workunit.client.0.smithi137.stdout:6/468: write d0/d5/d9/f41 [2482299,114276] 0 2024-06-15T01:07:08.740 INFO:tasks.workunit.client.1.smithi142.stdout:1/775: rmdir d6/d12/d52 39 2024-06-15T01:07:08.740 INFO:tasks.workunit.client.1.smithi142.stdout:1/776: fdatasync d6/f13 0 2024-06-15T01:07:08.741 INFO:tasks.workunit.client.0.smithi137.stdout:6/469: mkdir d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f 0 2024-06-15T01:07:08.743 INFO:tasks.workunit.client.0.smithi137.stdout:6/470: creat d0/d5/f90 x:0 0 0 2024-06-15T01:07:08.744 INFO:tasks.workunit.client.0.smithi137.stdout:6/471: chown d0/d1a/c60 2427 1 2024-06-15T01:07:08.747 INFO:tasks.workunit.client.1.smithi142.stdout:2/733: rename d8/df/d10/d94/d25/d22/d43/d52/l9e to d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf/le6 0 2024-06-15T01:07:08.748 INFO:tasks.workunit.client.1.smithi142.stdout:5/685: dread d2/de/d27/f56 [0,4194304] 0 2024-06-15T01:07:08.749 INFO:tasks.workunit.client.1.smithi142.stdout:9/684: rename d6/dc/d1b/f4b to d6/dc/d4e/d78/f101 0 2024-06-15T01:07:08.751 INFO:tasks.workunit.client.0.smithi137.stdout:6/472: creat d0/d5/d9/f91 x:0 0 0 2024-06-15T01:07:08.754 INFO:tasks.workunit.client.0.smithi137.stdout:6/473: mkdir d0/d1a/d3e/d57/d32/d92 0 2024-06-15T01:07:08.755 INFO:tasks.workunit.client.1.smithi142.stdout:7/768: symlink d2/lf5 0 2024-06-15T01:07:08.755 INFO:tasks.workunit.client.1.smithi142.stdout:5/686: stat d2/d18/d79/da2/ccc 0 2024-06-15T01:07:08.756 INFO:tasks.workunit.client.1.smithi142.stdout:9/685: dread d6/dc/dd/d22/f54 [0,4194304] 0 2024-06-15T01:07:08.757 INFO:tasks.workunit.client.1.smithi142.stdout:0/735: rename d3/d39/d3e/d81/f1d to d3/d39/d3e/d81/d4c/da0/fff 0 2024-06-15T01:07:08.759 INFO:tasks.workunit.client.1.smithi142.stdout:8/772: creat d0/d17/d40/d86/fec x:0 0 0 2024-06-15T01:07:08.763 INFO:tasks.workunit.client.1.smithi142.stdout:4/840: rename d5/d9/da/d92/d98/fac to d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f124 0 2024-06-15T01:07:08.764 INFO:tasks.workunit.client.1.smithi142.stdout:3/820: truncate d0/d1f/f4b 6693318 0 2024-06-15T01:07:08.765 INFO:tasks.workunit.client.1.smithi142.stdout:8/773: mkdir d0/d25/d5a/d8f/d97/ded 0 2024-06-15T01:07:08.769 INFO:tasks.workunit.client.1.smithi142.stdout:9/686: rename d6/dc/dd/f77 to d6/dc/d1b/d25/d58/d74/f102 0 2024-06-15T01:07:08.769 INFO:tasks.workunit.client.1.smithi142.stdout:6/735: write d1a/d39/d3a/fec [277149,25439] 0 2024-06-15T01:07:08.769 INFO:tasks.workunit.client.1.smithi142.stdout:0/736: fsync d3/d39/d3e/d81/d44/d7c/d22/d59/f9c 0 2024-06-15T01:07:08.770 INFO:tasks.workunit.client.1.smithi142.stdout:9/687: chown d6/d31/f86 176636095 1 2024-06-15T01:07:08.770 INFO:tasks.workunit.client.1.smithi142.stdout:1/777: truncate d6/d12/d3c/d63/f6f 773768 0 2024-06-15T01:07:08.773 INFO:tasks.workunit.client.1.smithi142.stdout:8/774: unlink d0/d25/fc0 0 2024-06-15T01:07:08.774 INFO:tasks.workunit.client.1.smithi142.stdout:8/775: truncate d0/d17/d40/d86/f9f 294896 0 2024-06-15T01:07:08.774 INFO:tasks.workunit.client.1.smithi142.stdout:8/776: readlink d0/l4f 0 2024-06-15T01:07:08.775 INFO:tasks.workunit.client.1.smithi142.stdout:8/777: write d0/d17/d40/d86/fec [960732,60341] 0 2024-06-15T01:07:08.775 INFO:tasks.workunit.client.1.smithi142.stdout:2/734: write d8/df/d10/d1c/d7c/f90 [225743,59496] 0 2024-06-15T01:07:08.776 INFO:tasks.workunit.client.1.smithi142.stdout:0/737: mknod d3/c100 0 2024-06-15T01:07:08.777 INFO:tasks.workunit.client.1.smithi142.stdout:0/738: stat d3/l46 0 2024-06-15T01:07:08.777 INFO:tasks.workunit.client.1.smithi142.stdout:2/735: dread - d8/df/daf/fb9 zero size 2024-06-15T01:07:08.777 INFO:tasks.workunit.client.1.smithi142.stdout:6/736: creat d1a/d3c/db0/f104 x:0 0 0 2024-06-15T01:07:08.779 INFO:tasks.workunit.client.1.smithi142.stdout:4/841: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102/d116/f125 x:0 0 0 2024-06-15T01:07:08.782 INFO:tasks.workunit.client.1.smithi142.stdout:5/687: link d2/de/d85/l9a d2/ld8 0 2024-06-15T01:07:08.783 INFO:tasks.workunit.client.1.smithi142.stdout:5/688: chown d2/de/d19/d93/c94 21868354 1 2024-06-15T01:07:08.784 INFO:tasks.workunit.client.1.smithi142.stdout:6/737: fdatasync d1a/f1e 0 2024-06-15T01:07:08.785 INFO:tasks.workunit.client.1.smithi142.stdout:7/769: dwrite d2/d8/dd/d14/f58 [0,4194304] 0 2024-06-15T01:07:08.785 INFO:tasks.workunit.client.1.smithi142.stdout:0/739: unlink d3/d39/d3e/d81/d1b/c74 0 2024-06-15T01:07:08.786 INFO:tasks.workunit.client.1.smithi142.stdout:4/842: creat d5/d1c/d6f/db9/f126 x:0 0 0 2024-06-15T01:07:08.791 INFO:tasks.workunit.client.1.smithi142.stdout:3/821: getdents d0/d1f/d20/d5d 0 2024-06-15T01:07:08.792 INFO:tasks.workunit.client.1.smithi142.stdout:8/778: symlink d0/d17/d40/lee 0 2024-06-15T01:07:08.794 INFO:tasks.workunit.client.1.smithi142.stdout:6/738: mkdir d1a/d3c/d105 0 2024-06-15T01:07:08.795 INFO:tasks.workunit.client.1.smithi142.stdout:7/770: dwrite d2/d6/f8f [0,4194304] 0 2024-06-15T01:07:08.795 INFO:tasks.workunit.client.1.smithi142.stdout:6/739: chown d1a/d2d/fb8 9999090 1 2024-06-15T01:07:08.797 INFO:tasks.workunit.client.1.smithi142.stdout:1/778: creat d6/d12/da0/db0/f115 x:0 0 0 2024-06-15T01:07:08.798 INFO:tasks.workunit.client.1.smithi142.stdout:1/779: read - d6/d12/d80/fdf zero size 2024-06-15T01:07:08.802 INFO:tasks.workunit.client.1.smithi142.stdout:9/688: rename d6/dc/d51/d53 to d6/dc/d1b/d6f/d103 0 2024-06-15T01:07:08.805 INFO:tasks.workunit.client.1.smithi142.stdout:3/822: mknod d0/d1f/d20/d5d/d60/dc1/cf4 0 2024-06-15T01:07:08.806 INFO:tasks.workunit.client.1.smithi142.stdout:6/740: creat d1a/d3c/f106 x:0 0 0 2024-06-15T01:07:08.808 INFO:tasks.workunit.client.1.smithi142.stdout:1/780: rmdir d6/d3a/dc4 39 2024-06-15T01:07:08.809 INFO:tasks.workunit.client.1.smithi142.stdout:0/740: getdents d3/d39/d3e/d81/d1b/df6 0 2024-06-15T01:07:08.810 INFO:tasks.workunit.client.1.smithi142.stdout:4/843: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/l127 0 2024-06-15T01:07:08.811 INFO:tasks.workunit.client.1.smithi142.stdout:1/781: readlink d6/d25/lea 0 2024-06-15T01:07:08.812 INFO:tasks.workunit.client.1.smithi142.stdout:5/689: rename d2/ld8 to d2/de/d19/ld9 0 2024-06-15T01:07:08.813 INFO:tasks.workunit.client.1.smithi142.stdout:8/779: dwrite d0/d25/f7e [0,4194304] 0 2024-06-15T01:07:08.813 INFO:tasks.workunit.client.1.smithi142.stdout:9/689: dwrite d6/dc/d1b/d25/d58/d74/f92 [0,4194304] 0 2024-06-15T01:07:08.815 INFO:tasks.workunit.client.1.smithi142.stdout:7/771: link d2/d8/dd/dcf/ff0 d2/ff6 0 2024-06-15T01:07:08.819 INFO:tasks.workunit.client.1.smithi142.stdout:3/823: creat d0/d3/da/db8/ff5 x:0 0 0 2024-06-15T01:07:08.820 INFO:tasks.workunit.client.1.smithi142.stdout:7/772: read - d2/d8/da/d33/d25/d79/f92 zero size 2024-06-15T01:07:08.821 INFO:tasks.workunit.client.1.smithi142.stdout:6/741: dwrite d1a/f2c [0,4194304] 0 2024-06-15T01:07:08.822 INFO:tasks.workunit.client.1.smithi142.stdout:4/844: fdatasync f3 0 2024-06-15T01:07:08.823 INFO:tasks.workunit.client.1.smithi142.stdout:5/690: creat d2/d18/d2e/dae/fda x:0 0 0 2024-06-15T01:07:08.823 INFO:tasks.workunit.client.1.smithi142.stdout:1/782: mknod d6/c116 0 2024-06-15T01:07:08.828 INFO:tasks.workunit.client.1.smithi142.stdout:4/845: write d5/d1c/d6f/db9/f126 [1043219,94119] 0 2024-06-15T01:07:08.829 INFO:tasks.workunit.client.1.smithi142.stdout:7/773: read d2/d8/da/d33/d25/d79/fcc [1341605,34518] 0 2024-06-15T01:07:08.830 INFO:tasks.workunit.client.1.smithi142.stdout:8/780: dwrite d0/d25/d5a/f78 [0,4194304] 0 2024-06-15T01:07:08.831 INFO:tasks.workunit.client.1.smithi142.stdout:0/741: getdents d3/d39/d9a/d9d/dcd 0 2024-06-15T01:07:08.834 INFO:tasks.workunit.client.1.smithi142.stdout:3/824: dread d0/d1f/d20/fa3 [0,4194304] 0 2024-06-15T01:07:08.837 INFO:tasks.workunit.client.1.smithi142.stdout:9/690: dread d6/dc/d1b/d6f/d103/d8d/fb9 [0,4194304] 0 2024-06-15T01:07:08.838 INFO:tasks.workunit.client.1.smithi142.stdout:0/742: creat d3/d39/d9a/d9d/de6/f101 x:0 0 0 2024-06-15T01:07:08.838 INFO:tasks.workunit.client.1.smithi142.stdout:7/774: creat d2/d8/dd/d14/d2c/d70/ff7 x:0 0 0 2024-06-15T01:07:08.839 INFO:tasks.workunit.client.1.smithi142.stdout:3/825: mkdir d0/d1b/d6b/df6 0 2024-06-15T01:07:08.840 INFO:tasks.workunit.client.1.smithi142.stdout:9/691: dread - d6/dc/d1b/d25/d58/de2/fe8 zero size 2024-06-15T01:07:08.840 INFO:tasks.workunit.client.1.smithi142.stdout:4/846: symlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102/d116/l128 0 2024-06-15T01:07:08.841 INFO:tasks.workunit.client.1.smithi142.stdout:7/775: unlink d2/d8/da/d33/d25/d79/fe4 0 2024-06-15T01:07:08.842 INFO:tasks.workunit.client.1.smithi142.stdout:9/692: unlink c4 0 2024-06-15T01:07:08.844 INFO:tasks.workunit.client.1.smithi142.stdout:4/847: mknod d5/d9/d23/d32/c129 0 2024-06-15T01:07:08.845 INFO:tasks.workunit.client.1.smithi142.stdout:3/826: dread d0/d3/da/d18/f8a [0,4194304] 0 2024-06-15T01:07:08.845 INFO:tasks.workunit.client.1.smithi142.stdout:9/693: unlink d6/d34/c50 0 2024-06-15T01:07:08.846 INFO:tasks.workunit.client.1.smithi142.stdout:0/743: dwrite d3/d39/d9a/d9d/dcd/fe4 [0,4194304] 0 2024-06-15T01:07:08.848 INFO:tasks.workunit.client.1.smithi142.stdout:3/827: symlink d0/d3/da/db8/lf7 0 2024-06-15T01:07:08.848 INFO:tasks.workunit.client.1.smithi142.stdout:4/848: rmdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12 39 2024-06-15T01:07:08.851 INFO:tasks.workunit.client.1.smithi142.stdout:9/694: creat d6/dc/d1b/d6f/da1/f104 x:0 0 0 2024-06-15T01:07:08.852 INFO:tasks.workunit.client.1.smithi142.stdout:3/828: mknod d0/d1f/d20/d59/cf8 0 2024-06-15T01:07:08.855 INFO:tasks.workunit.client.1.smithi142.stdout:9/695: write d6/dc/d1b/d25/d58/d74/f92 [4011964,72484] 0 2024-06-15T01:07:08.857 INFO:tasks.workunit.client.1.smithi142.stdout:9/696: symlink d6/d31/l105 0 2024-06-15T01:07:08.858 INFO:tasks.workunit.client.1.smithi142.stdout:0/744: dwrite d3/d39/d3e/fa5 [0,4194304] 0 2024-06-15T01:07:08.858 INFO:tasks.workunit.client.1.smithi142.stdout:3/829: dread d0/d3/de2/d55/f94 [0,4194304] 0 2024-06-15T01:07:08.859 INFO:tasks.workunit.client.1.smithi142.stdout:4/849: dwrite d5/d9/fd [0,4194304] 0 2024-06-15T01:07:08.860 INFO:tasks.workunit.client.1.smithi142.stdout:4/850: dread - d5/d1c/d6f/f10e zero size 2024-06-15T01:07:08.861 INFO:tasks.workunit.client.1.smithi142.stdout:9/697: read d6/dc/d1b/d25/d58/f63 [1266428,53397] 0 2024-06-15T01:07:08.863 INFO:tasks.workunit.client.1.smithi142.stdout:3/830: creat d0/d1f/d20/d5d/ff9 x:0 0 0 2024-06-15T01:07:08.870 INFO:tasks.workunit.client.1.smithi142.stdout:9/698: dwrite d6/dc/d1b/d25/d58/de2/fe8 [0,4194304] 0 2024-06-15T01:07:08.879 INFO:tasks.workunit.client.1.smithi142.stdout:4/851: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/f6b 3102454 0 2024-06-15T01:07:08.880 INFO:tasks.workunit.client.1.smithi142.stdout:4/852: dread - d5/d1c/d6f/dc7/f122 zero size 2024-06-15T01:07:08.881 INFO:tasks.workunit.client.1.smithi142.stdout:4/853: readlink d5/d1c/d6f/dc7/d30/dcc/l85 0 2024-06-15T01:07:08.881 INFO:tasks.workunit.client.1.smithi142.stdout:9/699: creat d6/dc/d1b/d25/d58/de2/de7/f106 x:0 0 0 2024-06-15T01:07:08.882 INFO:tasks.workunit.client.1.smithi142.stdout:6/742: dread d1a/d39/d3a/d51/d62/f79 [0,4194304] 0 2024-06-15T01:07:08.884 INFO:tasks.workunit.client.1.smithi142.stdout:6/743: chown d1a/d2d/d80/lc5 658 1 2024-06-15T01:07:08.885 INFO:tasks.workunit.client.1.smithi142.stdout:4/854: creat d5/d1c/d6f/dc7/de0/f12a x:0 0 0 2024-06-15T01:07:08.887 INFO:tasks.workunit.client.1.smithi142.stdout:6/744: mkdir d1a/d83/d107 0 2024-06-15T01:07:08.888 INFO:tasks.workunit.client.1.smithi142.stdout:4/855: creat d5/d9/d23/dbe/f12b x:0 0 0 2024-06-15T01:07:08.889 INFO:tasks.workunit.client.1.smithi142.stdout:6/745: write d1a/d2d/d2e/f31 [1157882,12331] 0 2024-06-15T01:07:08.890 INFO:tasks.workunit.client.1.smithi142.stdout:9/700: dwrite d6/d34/d8a/dec/ffc [0,4194304] 0 2024-06-15T01:07:08.892 INFO:tasks.workunit.client.1.smithi142.stdout:9/701: chown d6/dc/d1b/d25/fd3 0 1 2024-06-15T01:07:08.892 INFO:tasks.workunit.client.1.smithi142.stdout:4/856: truncate d5/d9/d23/d68/ffb 2890576 0 2024-06-15T01:07:08.894 INFO:tasks.workunit.client.1.smithi142.stdout:6/746: getdents d1a/d2d/d3b 0 2024-06-15T01:07:08.900 INFO:tasks.workunit.client.1.smithi142.stdout:4/857: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/l1e to d5/d1c/d6f/dc7/de1/l12c 0 2024-06-15T01:07:08.902 INFO:tasks.workunit.client.1.smithi142.stdout:6/747: link d1a/d39/d3a/d51/d62/fd1 d1a/d39/ded/f108 0 2024-06-15T01:07:08.902 INFO:tasks.workunit.client.1.smithi142.stdout:9/702: dwrite d6/dc/d1b/d6f/d103/ffa [0,4194304] 0 2024-06-15T01:07:08.903 INFO:tasks.workunit.client.1.smithi142.stdout:4/858: write d5/d1c/d6f/dc7/d30/d46/d50/f5e [4085882,37875] 0 2024-06-15T01:07:08.904 INFO:tasks.workunit.client.1.smithi142.stdout:9/703: dread - d6/dc/d1b/d6f/da1/f104 zero size 2024-06-15T01:07:08.905 INFO:tasks.workunit.client.1.smithi142.stdout:9/704: chown d6/dc/d4e/f57 123 1 2024-06-15T01:07:08.906 INFO:tasks.workunit.client.1.smithi142.stdout:6/748: creat d1a/d2d/d3b/dab/f109 x:0 0 0 2024-06-15T01:07:08.908 INFO:tasks.workunit.client.1.smithi142.stdout:4/859: rename d5/d9/d18/l109 to d5/d1c/d6f/dc7/de0/l12d 0 2024-06-15T01:07:08.909 INFO:tasks.workunit.client.1.smithi142.stdout:6/749: write d1a/d39/ff1 [952941,65929] 0 2024-06-15T01:07:08.911 INFO:tasks.workunit.client.1.smithi142.stdout:9/705: mknod d6/d34/d8a/dec/c107 0 2024-06-15T01:07:08.913 INFO:tasks.workunit.client.1.smithi142.stdout:6/750: symlink d1a/d39/d3a/d51/l10a 0 2024-06-15T01:07:08.914 INFO:tasks.workunit.client.1.smithi142.stdout:4/860: unlink d5/d9/d23/l2c 0 2024-06-15T01:07:08.917 INFO:tasks.workunit.client.1.smithi142.stdout:6/751: chown d1a/d39/d3a/dd2/fe2 59105514 1 2024-06-15T01:07:08.918 INFO:tasks.workunit.client.1.smithi142.stdout:6/752: write d1a/d3c/db0/f104 [301695,11200] 0 2024-06-15T01:07:08.921 INFO:tasks.workunit.client.1.smithi142.stdout:9/706: dwrite d6/dc/dd/fcc [0,4194304] 0 2024-06-15T01:07:08.926 INFO:tasks.workunit.client.1.smithi142.stdout:6/753: rename d1a/d2d/d80/d8e/f9d to d1a/d2d/d2e/f10b 0 2024-06-15T01:07:08.927 INFO:tasks.workunit.client.1.smithi142.stdout:9/707: fdatasync d6/d31/f32 0 2024-06-15T01:07:08.927 INFO:tasks.workunit.client.1.smithi142.stdout:3/831: dread d0/d3/f43 [0,4194304] 0 2024-06-15T01:07:08.929 INFO:tasks.workunit.client.1.smithi142.stdout:4/861: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f124 [0,4194304] 0 2024-06-15T01:07:08.931 INFO:tasks.workunit.client.1.smithi142.stdout:9/708: rmdir d6/d34/df3 39 2024-06-15T01:07:08.932 INFO:tasks.workunit.client.1.smithi142.stdout:3/832: unlink d0/d3/da/d18/f98 0 2024-06-15T01:07:08.934 INFO:tasks.workunit.client.1.smithi142.stdout:4/862: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/c6c to d5/d9/d23/dbe/c12e 0 2024-06-15T01:07:08.935 INFO:tasks.workunit.client.1.smithi142.stdout:9/709: mkdir d6/dc/d108 0 2024-06-15T01:07:08.937 INFO:tasks.workunit.client.1.smithi142.stdout:9/710: write d6/d34/d8a/fbf [238839,93264] 0 2024-06-15T01:07:08.939 INFO:tasks.workunit.client.1.smithi142.stdout:9/711: fsync d6/dc/d4e/d78/f9d 0 2024-06-15T01:07:08.943 INFO:tasks.workunit.client.1.smithi142.stdout:9/712: creat d6/dc/d1b/d25/d58/de2/de7/f109 x:0 0 0 2024-06-15T01:07:08.943 INFO:tasks.workunit.client.1.smithi142.stdout:2/736: truncate d8/df/d10/d94/d25/d22/d43/f96 1210761 0 2024-06-15T01:07:08.945 INFO:tasks.workunit.client.1.smithi142.stdout:9/713: truncate d6/d31/f86 58233 0 2024-06-15T01:07:08.950 INFO:tasks.workunit.client.1.smithi142.stdout:9/714: symlink d6/dc/d1b/l10a 0 2024-06-15T01:07:08.952 INFO:tasks.workunit.client.1.smithi142.stdout:9/715: dread - d6/dc/d4e/d78/fb0 zero size 2024-06-15T01:07:08.954 INFO:tasks.workunit.client.1.smithi142.stdout:2/737: getdents d8/df/d10/d94/d25 0 2024-06-15T01:07:08.955 INFO:tasks.workunit.client.1.smithi142.stdout:9/716: rmdir d6/dc/d1b/d25/d58 39 2024-06-15T01:07:08.955 INFO:tasks.workunit.client.1.smithi142.stdout:1/783: sync 2024-06-15T01:07:08.955 INFO:tasks.workunit.client.1.smithi142.stdout:2/738: truncate d8/df/daf/fcd 453413 0 2024-06-15T01:07:08.956 INFO:tasks.workunit.client.1.smithi142.stdout:1/784: chown d6/d25/f3d 6 1 2024-06-15T01:07:08.962 INFO:tasks.workunit.client.1.smithi142.stdout:1/785: symlink d6/d12/d1c/d2c/d93/l117 0 2024-06-15T01:07:08.971 INFO:tasks.workunit.client.1.smithi142.stdout:5/691: dwrite d2/d18/d2e/f71 [0,4194304] 0 2024-06-15T01:07:08.971 INFO:tasks.workunit.client.1.smithi142.stdout:8/781: dwrite d0/d17/d40/d6e/fa9 [0,4194304] 0 2024-06-15T01:07:08.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/739: dwrite d8/df/d10/d94/fd9 [0,4194304] 0 2024-06-15T01:07:08.975 INFO:tasks.workunit.client.1.smithi142.stdout:9/717: dwrite d6/dc/d1b/d25/d58/de2/fe8 [0,4194304] 0 2024-06-15T01:07:08.977 INFO:tasks.workunit.client.1.smithi142.stdout:2/740: chown d8/df/d10/d94/db5 191 1 2024-06-15T01:07:08.981 INFO:tasks.workunit.client.1.smithi142.stdout:1/786: dread d6/d12/f3e [0,4194304] 0 2024-06-15T01:07:08.983 INFO:tasks.workunit.client.1.smithi142.stdout:1/787: stat d6/d12/l42 0 2024-06-15T01:07:08.987 INFO:tasks.workunit.client.1.smithi142.stdout:5/692: rename d2/d3e/d31/f6d to d2/d3e/d65/dd6/fdb 0 2024-06-15T01:07:08.988 INFO:tasks.workunit.client.1.smithi142.stdout:2/741: fsync d8/df/d10/d94/d25/d22/d43/d52/d7a/fbe 0 2024-06-15T01:07:08.989 INFO:tasks.workunit.client.1.smithi142.stdout:3/833: rmdir d0/d1f/d20/d5d/d60/dc1 39 2024-06-15T01:07:08.990 INFO:tasks.workunit.client.1.smithi142.stdout:8/782: mknod d0/d25/d5a/d8f/d97/ded/cef 0 2024-06-15T01:07:08.991 INFO:tasks.workunit.client.1.smithi142.stdout:1/788: dwrite d6/d25/dfc/fd9 [0,4194304] 0 2024-06-15T01:07:08.993 INFO:tasks.workunit.client.1.smithi142.stdout:1/789: truncate d6/d12/d3c/d63/d8d/d97/dc6/f101 586792 0 2024-06-15T01:07:08.993 INFO:tasks.workunit.client.1.smithi142.stdout:1/790: read - d6/d25/dfc/fd6 zero size 2024-06-15T01:07:08.996 INFO:tasks.workunit.client.1.smithi142.stdout:7/776: sync 2024-06-15T01:07:08.996 INFO:tasks.workunit.client.1.smithi142.stdout:0/745: sync 2024-06-15T01:07:08.997 INFO:tasks.workunit.client.1.smithi142.stdout:2/742: creat d8/d19/fe7 x:0 0 0 2024-06-15T01:07:08.999 INFO:tasks.workunit.client.1.smithi142.stdout:7/777: write d2/d8/da/d33/d25/d79/fb7 [19403,117378] 0 2024-06-15T01:07:09.000 INFO:tasks.workunit.client.1.smithi142.stdout:8/783: creat d0/d25/db9/ff0 x:0 0 0 2024-06-15T01:07:09.001 INFO:tasks.workunit.client.1.smithi142.stdout:1/791: dread d6/d4d/f6c [0,4194304] 0 2024-06-15T01:07:09.002 INFO:tasks.workunit.client.1.smithi142.stdout:0/746: unlink d3/d39/d9a/fa8 0 2024-06-15T01:07:09.003 INFO:tasks.workunit.client.1.smithi142.stdout:3/834: mkdir d0/d95/dfa 0 2024-06-15T01:07:09.005 INFO:tasks.workunit.client.1.smithi142.stdout:2/743: creat d8/df/d10/d94/d25/d1a/d57/fe8 x:0 0 0 2024-06-15T01:07:09.006 INFO:tasks.workunit.client.1.smithi142.stdout:8/784: stat d0/d25/d53/da7/dcb/ce9 0 2024-06-15T01:07:09.007 INFO:tasks.workunit.client.1.smithi142.stdout:8/785: truncate d0/f87 830270 0 2024-06-15T01:07:09.007 INFO:tasks.workunit.client.1.smithi142.stdout:1/792: creat d6/d46/dc3/f118 x:0 0 0 2024-06-15T01:07:09.013 INFO:tasks.workunit.client.1.smithi142.stdout:6/754: truncate f6 2154779 0 2024-06-15T01:07:09.013 INFO:tasks.workunit.client.1.smithi142.stdout:8/786: creat d0/d25/d53/da7/ff1 x:0 0 0 2024-06-15T01:07:09.014 INFO:tasks.workunit.client.1.smithi142.stdout:6/755: chown c19 258205 1 2024-06-15T01:07:09.014 INFO:tasks.workunit.client.1.smithi142.stdout:0/747: dwrite d3/d39/d3e/d81/d4c/da0/fff [0,4194304] 0 2024-06-15T01:07:09.019 INFO:tasks.workunit.client.1.smithi142.stdout:7/778: getdents d2/d8/dd/d8a 0 2024-06-15T01:07:09.019 INFO:tasks.workunit.client.1.smithi142.stdout:6/756: write d1a/d39/fbe [605083,63476] 0 2024-06-15T01:07:09.027 INFO:tasks.workunit.client.1.smithi142.stdout:1/793: dwrite d6/d12/d5b/f8a [0,4194304] 0 2024-06-15T01:07:09.027 INFO:tasks.workunit.client.1.smithi142.stdout:7/779: mkdir d2/db6/de6/df8 0 2024-06-15T01:07:09.028 INFO:tasks.workunit.client.1.smithi142.stdout:6/757: symlink d1a/d3c/db0/l10c 0 2024-06-15T01:07:09.030 INFO:tasks.workunit.client.1.smithi142.stdout:2/744: link d8/df/d10/d94/d25/lbf d8/df/d10/d94/d25/d22/d43/d51/dd7/le9 0 2024-06-15T01:07:09.031 INFO:tasks.workunit.client.1.smithi142.stdout:3/835: dread d0/d1f/d20/d59/f92 [0,4194304] 0 2024-06-15T01:07:09.034 INFO:tasks.workunit.client.1.smithi142.stdout:4/863: dwrite d5/d1c/d6f/dc7/fab [0,4194304] 0 2024-06-15T01:07:09.036 INFO:tasks.workunit.client.1.smithi142.stdout:1/794: chown d6/d12/ca5 98755 1 2024-06-15T01:07:09.036 INFO:tasks.workunit.client.1.smithi142.stdout:4/864: fsync d5/d1c/d6f/dc7/d30/d46/d50/f5e 0 2024-06-15T01:07:09.039 INFO:tasks.workunit.client.1.smithi142.stdout:4/865: chown d5/d1c/d6f/dc7/d30/d46 2295039 1 2024-06-15T01:07:09.042 INFO:tasks.workunit.client.1.smithi142.stdout:2/745: write d8/df/f85 [3330446,82029] 0 2024-06-15T01:07:09.046 INFO:tasks.workunit.client.1.smithi142.stdout:3/836: symlink d0/d1f/d20/d59/lfb 0 2024-06-15T01:07:09.046 INFO:tasks.workunit.client.1.smithi142.stdout:9/718: mkdir d6/dc/d1b/d6f/d103/d8d/d10b 0 2024-06-15T01:07:09.047 INFO:tasks.workunit.client.1.smithi142.stdout:1/795: mkdir d6/d12/d3c/d63/d65/d119 0 2024-06-15T01:07:09.053 INFO:tasks.workunit.client.1.smithi142.stdout:3/837: mkdir d0/d3/de2/d55/dfc 0 2024-06-15T01:07:09.054 INFO:tasks.workunit.client.1.smithi142.stdout:9/719: dread d6/dc/d1b/d6f/d103/d8d/fb9 [0,4194304] 0 2024-06-15T01:07:09.054 INFO:tasks.workunit.client.1.smithi142.stdout:5/693: fsync d2/d3e/d31/d7e/fa8 0 2024-06-15T01:07:09.055 INFO:tasks.workunit.client.1.smithi142.stdout:1/796: dread d6/d3a/f43 [0,4194304] 0 2024-06-15T01:07:09.055 INFO:tasks.workunit.client.1.smithi142.stdout:5/694: readlink d2/d18/lb6 0 2024-06-15T01:07:09.056 INFO:tasks.workunit.client.1.smithi142.stdout:7/780: dread d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:07:09.058 INFO:tasks.workunit.client.1.smithi142.stdout:4/866: link d5/d9/da/f3c d5/d1c/d6f/dc7/de0/f12f 0 2024-06-15T01:07:09.061 INFO:tasks.workunit.client.1.smithi142.stdout:3/838: mkdir d0/d3/de2/d75/dfd 0 2024-06-15T01:07:09.062 INFO:tasks.workunit.client.1.smithi142.stdout:5/695: dread d2/de/d27/f56 [0,4194304] 0 2024-06-15T01:07:09.066 INFO:tasks.workunit.client.1.smithi142.stdout:1/797: creat d6/d12/d1c/d2c/d93/db9/f11a x:0 0 0 2024-06-15T01:07:09.070 INFO:tasks.workunit.client.1.smithi142.stdout:6/758: dread d1a/d2d/f52 [0,4194304] 0 2024-06-15T01:07:09.070 INFO:tasks.workunit.client.1.smithi142.stdout:1/798: write d6/d12/d1c/d2c/d47/f5a [5198999,83443] 0 2024-06-15T01:07:09.071 INFO:tasks.workunit.client.1.smithi142.stdout:0/748: write d3/d39/d3e/d81/f7e [1290462,16305] 0 2024-06-15T01:07:09.072 INFO:tasks.workunit.client.1.smithi142.stdout:2/746: getdents d8/df/daf 0 2024-06-15T01:07:09.073 INFO:tasks.workunit.client.1.smithi142.stdout:9/720: dwrite d6/f45 [0,4194304] 0 2024-06-15T01:07:09.074 INFO:tasks.workunit.client.1.smithi142.stdout:3/839: mknod d0/d3/d15/cfe 0 2024-06-15T01:07:09.075 INFO:tasks.workunit.client.1.smithi142.stdout:5/696: creat d2/de/d19/d5f/d68/fdc x:0 0 0 2024-06-15T01:07:09.077 INFO:tasks.workunit.client.1.smithi142.stdout:7/781: truncate d2/d6/d22/f6c 1204490 0 2024-06-15T01:07:09.078 INFO:tasks.workunit.client.1.smithi142.stdout:2/747: chown d8/df/d10/d94/d25/d22/d43/d52/d7a/d98 6685125 1 2024-06-15T01:07:09.084 INFO:tasks.workunit.client.1.smithi142.stdout:5/697: readlink d2/d3e/d31/da0/ld1 0 2024-06-15T01:07:09.085 INFO:tasks.workunit.client.1.smithi142.stdout:4/867: dread d5/d1c/d6f/dc7/de0/f12f [0,4194304] 0 2024-06-15T01:07:09.086 INFO:tasks.workunit.client.1.smithi142.stdout:7/782: symlink d2/d6/d22/lf9 0 2024-06-15T01:07:09.087 INFO:tasks.workunit.client.1.smithi142.stdout:3/840: truncate d0/d1f/d20/d59/d61/d96/faa 1743925 0 2024-06-15T01:07:09.087 INFO:tasks.workunit.client.1.smithi142.stdout:7/783: chown d2/d6/c44 100392 1 2024-06-15T01:07:09.090 INFO:tasks.workunit.client.1.smithi142.stdout:0/749: symlink d3/d39/l102 0 2024-06-15T01:07:09.092 INFO:tasks.workunit.client.1.smithi142.stdout:5/698: rmdir d2/d3e/d65/dd6 39 2024-06-15T01:07:09.095 INFO:tasks.workunit.client.1.smithi142.stdout:3/841: dwrite d0/d1b/f41 [0,4194304] 0 2024-06-15T01:07:09.098 INFO:tasks.workunit.client.1.smithi142.stdout:4/868: rmdir d5/d1c/d6f/dc7 39 2024-06-15T01:07:09.101 INFO:tasks.workunit.client.1.smithi142.stdout:9/721: dread d6/dc/dd/d22/fbe [0,4194304] 0 2024-06-15T01:07:09.103 INFO:tasks.workunit.client.1.smithi142.stdout:5/699: symlink d2/de/d19/d5f/d68/d76/da6/dc5/ldd 0 2024-06-15T01:07:09.103 INFO:tasks.workunit.client.1.smithi142.stdout:9/722: fdatasync d6/dc/d1b/d6f/d103/ffa 0 2024-06-15T01:07:09.106 INFO:tasks.workunit.client.1.smithi142.stdout:7/784: getdents d2/d6/d22/d47/de5 0 2024-06-15T01:07:09.106 INFO:tasks.workunit.client.1.smithi142.stdout:9/723: fsync d6/dc/d4e/f94 0 2024-06-15T01:07:09.106 INFO:tasks.workunit.client.1.smithi142.stdout:0/750: dread d3/d39/d3e/d81/d1e/f63 [0,4194304] 0 2024-06-15T01:07:09.109 INFO:tasks.workunit.client.1.smithi142.stdout:2/748: rename d8/df/d10/d94/d25/d22/d43/d52/cce to d8/df/d10/d94/cea 0 2024-06-15T01:07:09.111 INFO:tasks.workunit.client.1.smithi142.stdout:0/751: truncate d3/d39/d3e/d8f/fd2 4528306 0 2024-06-15T01:07:09.112 INFO:tasks.workunit.client.1.smithi142.stdout:5/700: creat d2/de/d19/d5f/d68/d76/d96/fde x:0 0 0 2024-06-15T01:07:09.114 INFO:tasks.workunit.client.1.smithi142.stdout:5/701: chown d2/d3e 9 1 2024-06-15T01:07:09.116 INFO:tasks.workunit.client.1.smithi142.stdout:9/724: chown d6/dc/cf 1080 1 2024-06-15T01:07:09.119 INFO:tasks.workunit.client.1.smithi142.stdout:0/752: creat d3/d39/d3e/d81/f103 x:0 0 0 2024-06-15T01:07:09.120 INFO:tasks.workunit.client.1.smithi142.stdout:9/725: chown d6/dc/cf 191926 1 2024-06-15T01:07:09.120 INFO:tasks.workunit.client.1.smithi142.stdout:7/785: dread d2/d8/dd/f12 [0,4194304] 0 2024-06-15T01:07:09.120 INFO:tasks.workunit.client.1.smithi142.stdout:6/759: truncate d1a/f22 2124316 0 2024-06-15T01:07:09.122 INFO:tasks.workunit.client.1.smithi142.stdout:0/753: fdatasync d3/d39/d3e/fe0 0 2024-06-15T01:07:09.122 INFO:tasks.workunit.client.1.smithi142.stdout:5/702: creat d2/de/d27/d86/dbd/fdf x:0 0 0 2024-06-15T01:07:09.124 INFO:tasks.workunit.client.1.smithi142.stdout:2/749: dwrite d8/df/d10/d1c/fad [0,4194304] 0 2024-06-15T01:07:09.126 INFO:tasks.workunit.client.1.smithi142.stdout:3/842: dread d0/d1f/d20/d5d/d60/f85 [0,4194304] 0 2024-06-15T01:07:09.127 INFO:tasks.workunit.client.1.smithi142.stdout:7/786: chown d2/d6/c44 0 1 2024-06-15T01:07:09.130 INFO:tasks.workunit.client.1.smithi142.stdout:5/703: unlink d2/d3e/d31/f64 0 2024-06-15T01:07:09.131 INFO:tasks.workunit.client.1.smithi142.stdout:0/754: read d3/d39/d3e/d81/d1b/d38/f96 [712395,43254] 0 2024-06-15T01:07:09.132 INFO:tasks.workunit.client.1.smithi142.stdout:7/787: stat d2/d6/d22/fdf 0 2024-06-15T01:07:09.132 INFO:tasks.workunit.client.1.smithi142.stdout:6/760: link fc d1a/d39/d3a/d51/d62/f10d 0 2024-06-15T01:07:09.136 INFO:tasks.workunit.client.1.smithi142.stdout:9/726: getdents d6/dc/d51 0 2024-06-15T01:07:09.136 INFO:tasks.workunit.client.1.smithi142.stdout:3/843: mknod d0/d1f/d20/d5d/d60/dd0/cff 0 2024-06-15T01:07:09.136 INFO:tasks.workunit.client.1.smithi142.stdout:6/761: truncate d1a/d2d/f52 4512666 0 2024-06-15T01:07:09.136 INFO:tasks.workunit.client.1.smithi142.stdout:7/788: rename d2/d8/da/c41 to d2/da2/cfa 0 2024-06-15T01:07:09.138 INFO:tasks.workunit.client.1.smithi142.stdout:9/727: read d6/dc/d4e/d78/f101 [2851383,76726] 0 2024-06-15T01:07:09.139 INFO:tasks.workunit.client.1.smithi142.stdout:8/787: sync 2024-06-15T01:07:09.140 INFO:tasks.workunit.client.1.smithi142.stdout:6/762: rmdir d1a/d83/daa 39 2024-06-15T01:07:09.143 INFO:tasks.workunit.client.1.smithi142.stdout:5/704: rename d2/d3e/d31/f3c to d2/de/d27/d86/dbd/fe0 0 2024-06-15T01:07:09.143 INFO:tasks.workunit.client.1.smithi142.stdout:8/788: read d0/d25/d66/fb4 [26273,21028] 0 2024-06-15T01:07:09.143 INFO:tasks.workunit.client.1.smithi142.stdout:2/750: getdents d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf 0 2024-06-15T01:07:09.144 INFO:tasks.workunit.client.1.smithi142.stdout:3/844: creat d0/d3/f100 x:0 0 0 2024-06-15T01:07:09.145 INFO:tasks.workunit.client.1.smithi142.stdout:4/869: dread d5/d1c/d6f/dc7/d30/d46/f76 [0,4194304] 0 2024-06-15T01:07:09.146 INFO:tasks.workunit.client.1.smithi142.stdout:2/751: chown d8/df/daf/fcd 1154 1 2024-06-15T01:07:09.146 INFO:tasks.workunit.client.1.smithi142.stdout:6/763: creat d1a/d39/f10e x:0 0 0 2024-06-15T01:07:09.149 INFO:tasks.workunit.client.1.smithi142.stdout:9/728: dread d6/dc/dd/d22/f2b [0,4194304] 0 2024-06-15T01:07:09.150 INFO:tasks.workunit.client.1.smithi142.stdout:6/764: fsync d1a/d2d/d2e/fd9 0 2024-06-15T01:07:09.150 INFO:tasks.workunit.client.1.smithi142.stdout:8/789: write d0/d17/d40/d86/d9d/fa5 [2153872,109439] 0 2024-06-15T01:07:09.151 INFO:tasks.workunit.client.1.smithi142.stdout:5/705: creat d2/d18/d2e/fe1 x:0 0 0 2024-06-15T01:07:09.152 INFO:tasks.workunit.client.1.smithi142.stdout:0/755: truncate d3/d39/d3e/fe0 357016 0 2024-06-15T01:07:09.152 INFO:tasks.workunit.client.1.smithi142.stdout:8/790: write d0/f3f [2538260,100283] 0 2024-06-15T01:07:09.152 INFO:tasks.workunit.client.1.smithi142.stdout:5/706: fdatasync d2/de/d27/db2/fbb 0 2024-06-15T01:07:09.154 INFO:tasks.workunit.client.1.smithi142.stdout:2/752: mkdir d8/d19/deb 0 2024-06-15T01:07:09.155 INFO:tasks.workunit.client.1.smithi142.stdout:3/845: write d0/d1b/f88 [2475059,65648] 0 2024-06-15T01:07:09.159 INFO:tasks.workunit.client.1.smithi142.stdout:3/846: truncate d0/d3/f54 1948322 0 2024-06-15T01:07:09.160 INFO:tasks.workunit.client.1.smithi142.stdout:7/789: write d2/d6/d22/d47/f72 [612207,93366] 0 2024-06-15T01:07:09.161 INFO:tasks.workunit.client.1.smithi142.stdout:4/870: mknod d5/d1c/d6f/dc7/d30/dcc/d11a/c130 0 2024-06-15T01:07:09.163 INFO:tasks.workunit.client.1.smithi142.stdout:5/707: chown d2/d18/c7d 657642 1 2024-06-15T01:07:09.165 INFO:tasks.workunit.client.1.smithi142.stdout:0/756: creat d3/d39/d9a/d9d/dbf/f104 x:0 0 0 2024-06-15T01:07:09.165 INFO:tasks.workunit.client.1.smithi142.stdout:2/753: mknod d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf/cec 0 2024-06-15T01:07:09.167 INFO:tasks.workunit.client.1.smithi142.stdout:2/754: fdatasync d8/df/d10/d94/d25/f6a 0 2024-06-15T01:07:09.168 INFO:tasks.workunit.client.1.smithi142.stdout:0/757: fdatasync d3/d39/d3e/d81/d44/d7c/d22/d59/f9c 0 2024-06-15T01:07:09.168 INFO:tasks.workunit.client.1.smithi142.stdout:6/765: dread d1a/f76 [0,4194304] 0 2024-06-15T01:07:09.168 INFO:tasks.workunit.client.1.smithi142.stdout:4/871: write d5/d1c/d6f/f10e [720170,75049] 0 2024-06-15T01:07:09.172 INFO:tasks.workunit.client.1.smithi142.stdout:3/847: getdents d0/d1f/d20/d5d/d60/d99 0 2024-06-15T01:07:09.172 INFO:tasks.workunit.client.1.smithi142.stdout:2/755: read d8/df/d10/d1c/f2c [3053654,79571] 0 2024-06-15T01:07:09.172 INFO:tasks.workunit.client.1.smithi142.stdout:6/766: dread - d1a/d39/d42/f74 zero size 2024-06-15T01:07:09.173 INFO:tasks.workunit.client.1.smithi142.stdout:0/758: read d3/d39/d9a/da2/fec [2583010,65477] 0 2024-06-15T01:07:09.173 INFO:tasks.workunit.client.1.smithi142.stdout:9/729: dread d6/f21 [0,4194304] 0 2024-06-15T01:07:09.177 INFO:tasks.workunit.client.1.smithi142.stdout:8/791: dread d0/d25/d5a/d84/fcc [0,4194304] 0 2024-06-15T01:07:09.179 INFO:tasks.workunit.client.1.smithi142.stdout:1/799: sync 2024-06-15T01:07:09.179 INFO:tasks.workunit.client.1.smithi142.stdout:6/767: mkdir d1a/d3c/d100/d58/d10f 0 2024-06-15T01:07:09.181 INFO:tasks.workunit.client.1.smithi142.stdout:0/759: write d3/d39/d9a/da2/fbc [1634643,49707] 0 2024-06-15T01:07:09.184 INFO:tasks.workunit.client.1.smithi142.stdout:2/756: rename d8/df/d10/d94/d25/d1a/d57 to d8/df/d10/d94/db5/ded 0 2024-06-15T01:07:09.185 INFO:tasks.workunit.client.1.smithi142.stdout:7/790: getdents d2/d6/daf 0 2024-06-15T01:07:09.186 INFO:tasks.workunit.client.1.smithi142.stdout:4/872: creat d5/d9/f131 x:0 0 0 2024-06-15T01:07:09.187 INFO:tasks.workunit.client.1.smithi142.stdout:8/792: creat d0/d25/d5a/d8f/d98/ff2 x:0 0 0 2024-06-15T01:07:09.188 INFO:tasks.workunit.client.1.smithi142.stdout:9/730: truncate d6/d31/f32 591157 0 2024-06-15T01:07:09.189 INFO:tasks.workunit.client.1.smithi142.stdout:6/768: unlink d1a/d39/l8a 0 2024-06-15T01:07:09.190 INFO:tasks.workunit.client.1.smithi142.stdout:0/760: dwrite d3/d39/fc4 [0,4194304] 0 2024-06-15T01:07:09.192 INFO:tasks.workunit.client.1.smithi142.stdout:1/800: creat d6/d12/d1c/d2c/d93/db9/f11b x:0 0 0 2024-06-15T01:07:09.192 INFO:tasks.workunit.client.1.smithi142.stdout:3/848: dread d0/d3/de2/d55/f57 [4194304,4194304] 0 2024-06-15T01:07:09.194 INFO:tasks.workunit.client.1.smithi142.stdout:7/791: mknod d2/db6/de6/cfb 0 2024-06-15T01:07:09.195 INFO:tasks.workunit.client.1.smithi142.stdout:5/708: getdents d2/d3e/d31/d7e 0 2024-06-15T01:07:09.196 INFO:tasks.workunit.client.1.smithi142.stdout:0/761: dread d3/d39/d3e/d81/d4c/d61/f91 [0,4194304] 0 2024-06-15T01:07:09.196 INFO:tasks.workunit.client.1.smithi142.stdout:4/873: rmdir d5/d9/da/d92/d98 39 2024-06-15T01:07:09.197 INFO:tasks.workunit.client.1.smithi142.stdout:8/793: mkdir d0/d25/d5a/d8f/df3 0 2024-06-15T01:07:09.199 INFO:tasks.workunit.client.1.smithi142.stdout:5/709: dread - d2/d3e/d31/d7e/fc6 zero size 2024-06-15T01:07:09.199 INFO:tasks.workunit.client.1.smithi142.stdout:6/769: fsync fc 0 2024-06-15T01:07:09.200 INFO:tasks.workunit.client.1.smithi142.stdout:8/794: write d0/fd4 [261234,26455] 0 2024-06-15T01:07:09.202 INFO:tasks.workunit.client.1.smithi142.stdout:8/795: chown d0/c16 111093851 1 2024-06-15T01:07:09.203 INFO:tasks.workunit.client.1.smithi142.stdout:0/762: read d3/d39/d3e/d81/d44/d7c/d22/d59/fba [518465,61977] 0 2024-06-15T01:07:09.203 INFO:tasks.workunit.client.1.smithi142.stdout:2/757: symlink d8/df/d10/d1c/lee 0 2024-06-15T01:07:09.204 INFO:tasks.workunit.client.1.smithi142.stdout:1/801: dwrite d6/d12/f22 [0,4194304] 0 2024-06-15T01:07:09.205 INFO:tasks.workunit.client.1.smithi142.stdout:3/849: mknod d0/d3/de2/d75/c101 0 2024-06-15T01:07:09.206 INFO:tasks.workunit.client.1.smithi142.stdout:1/802: fsync d6/d46/f78 0 2024-06-15T01:07:09.207 INFO:tasks.workunit.client.1.smithi142.stdout:4/874: creat d5/d9/da/f132 x:0 0 0 2024-06-15T01:07:09.208 INFO:tasks.workunit.client.1.smithi142.stdout:1/803: readlink d6/d3a/dd4/l10b 0 2024-06-15T01:07:09.208 INFO:tasks.workunit.client.1.smithi142.stdout:7/792: truncate d2/d8/da/d33/d25/d79/f92 139387 0 2024-06-15T01:07:09.209 INFO:tasks.workunit.client.1.smithi142.stdout:5/710: dwrite d2/de/d19/f6a [0,4194304] 0 2024-06-15T01:07:09.209 INFO:tasks.workunit.client.1.smithi142.stdout:8/796: fdatasync d0/d17/d40/d5e/fad 0 2024-06-15T01:07:09.209 INFO:tasks.workunit.client.1.smithi142.stdout:0/763: symlink d3/d39/d9a/d9d/de6/l105 0 2024-06-15T01:07:09.210 INFO:tasks.workunit.client.1.smithi142.stdout:1/804: chown d6/d3a/fee 870062238 1 2024-06-15T01:07:09.212 INFO:tasks.workunit.client.1.smithi142.stdout:2/758: getdents d8/df/daf 0 2024-06-15T01:07:09.213 INFO:tasks.workunit.client.1.smithi142.stdout:0/764: dread d3/d39/d9a/fc9 [0,4194304] 0 2024-06-15T01:07:09.213 INFO:tasks.workunit.client.1.smithi142.stdout:3/850: truncate d0/d3/da/d18/f87 724616 0 2024-06-15T01:07:09.214 INFO:tasks.workunit.client.1.smithi142.stdout:3/851: write d0/d3/de2/d55/fe6 [719921,33575] 0 2024-06-15T01:07:09.214 INFO:tasks.workunit.client.1.smithi142.stdout:9/731: rename d6/d31/f86 to d6/dc/f10c 0 2024-06-15T01:07:09.215 INFO:tasks.workunit.client.1.smithi142.stdout:0/765: write d3/d39/d3e/fc5 [531325,2300] 0 2024-06-15T01:07:09.216 INFO:tasks.workunit.client.1.smithi142.stdout:3/852: fdatasync d0/d3/da/db8/ff5 0 2024-06-15T01:07:09.217 INFO:tasks.workunit.client.1.smithi142.stdout:9/732: chown d6/dc/d1b/d6f/d103/d8d 183865306 1 2024-06-15T01:07:09.225 INFO:tasks.workunit.client.1.smithi142.stdout:5/711: creat d2/d3e/d74/da5/fe2 x:0 0 0 2024-06-15T01:07:09.225 INFO:tasks.workunit.client.1.smithi142.stdout:8/797: fsync d0/d25/d5a/f8d 0 2024-06-15T01:07:09.226 INFO:tasks.workunit.client.1.smithi142.stdout:1/805: unlink d6/d12/da0/db0/de1/ff8 0 2024-06-15T01:07:09.226 INFO:tasks.workunit.client.1.smithi142.stdout:4/875: mkdir d5/d9/da/d92/d98/dc1/d133 0 2024-06-15T01:07:09.227 INFO:tasks.workunit.client.1.smithi142.stdout:6/770: rename d1a/d2d/d68 to d1a/d2d/d80/d8e/d110 0 2024-06-15T01:07:09.228 INFO:tasks.workunit.client.0.smithi137.stdout:9/364: sync 2024-06-15T01:07:09.229 INFO:tasks.workunit.client.0.smithi137.stdout:9/365: chown df/d2a/f34 1220 1 2024-06-15T01:07:09.229 INFO:tasks.workunit.client.1.smithi142.stdout:5/712: dread - d2/de/d19/d5f/d68/d76/d96/fde zero size 2024-06-15T01:07:09.229 INFO:tasks.workunit.client.0.smithi137.stdout:9/366: readlink df/d13/l2b 0 2024-06-15T01:07:09.231 INFO:tasks.workunit.client.1.smithi142.stdout:5/713: dread - d2/d3e/d31/d7e/fc6 zero size 2024-06-15T01:07:09.232 INFO:tasks.workunit.client.1.smithi142.stdout:3/853: dread d0/d1b/f41 [0,4194304] 0 2024-06-15T01:07:09.234 INFO:tasks.workunit.client.1.smithi142.stdout:5/714: write d2/de/d19/d5f/d68/d76/fc8 [722235,69463] 0 2024-06-15T01:07:09.235 INFO:tasks.workunit.client.1.smithi142.stdout:8/798: dwrite d0/d25/d53/da7/fe8 [0,4194304] 0 2024-06-15T01:07:09.237 INFO:tasks.workunit.client.0.smithi137.stdout:9/367: dwrite df/d12/d15/f52 [0,4194304] 0 2024-06-15T01:07:09.238 INFO:tasks.workunit.client.0.smithi137.stdout:9/368: chown df/d12/d15/d16 22258 1 2024-06-15T01:07:09.249 INFO:tasks.workunit.client.1.smithi142.stdout:2/759: symlink d8/df/lef 0 2024-06-15T01:07:09.251 INFO:tasks.workunit.client.1.smithi142.stdout:1/806: mknod d6/d12/d1c/d2c/c11c 0 2024-06-15T01:07:09.252 INFO:tasks.workunit.client.1.smithi142.stdout:8/799: dread d0/d25/d5a/fda [0,4194304] 0 2024-06-15T01:07:09.252 INFO:tasks.workunit.client.1.smithi142.stdout:6/771: truncate d1a/d2d/f65 461740 0 2024-06-15T01:07:09.253 INFO:tasks.workunit.client.1.smithi142.stdout:6/772: dread - d1a/d2d/d3b/dab/f109 zero size 2024-06-15T01:07:09.254 INFO:tasks.workunit.client.1.smithi142.stdout:4/876: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f134 x:0 0 0 2024-06-15T01:07:09.255 INFO:tasks.workunit.client.1.smithi142.stdout:7/793: dread d2/d8/da/d33/d25/f16 [0,4194304] 0 2024-06-15T01:07:09.257 INFO:tasks.workunit.client.1.smithi142.stdout:5/715: creat d2/de/d27/fe3 x:0 0 0 2024-06-15T01:07:09.258 INFO:tasks.workunit.client.0.smithi137.stdout:9/369: write df/d12/d15/d16/f27 [604012,54121] 0 2024-06-15T01:07:09.259 INFO:tasks.workunit.client.1.smithi142.stdout:9/733: rename d6/dc/d1b/d6f/ff7 to d6/dc/d4e/d78/f10d 0 2024-06-15T01:07:09.260 INFO:tasks.workunit.client.1.smithi142.stdout:1/807: creat d6/d12/d80/dc8/f11d x:0 0 0 2024-06-15T01:07:09.261 INFO:tasks.workunit.client.0.smithi137.stdout:9/370: write df/d12/f3e [707987,104963] 0 2024-06-15T01:07:09.261 INFO:tasks.workunit.client.1.smithi142.stdout:8/800: rmdir d0/d25/d5a/d8f 39 2024-06-15T01:07:09.262 INFO:tasks.workunit.client.1.smithi142.stdout:8/801: fdatasync d0/d17/d40/d5e/f94 0 2024-06-15T01:07:09.263 INFO:tasks.workunit.client.1.smithi142.stdout:4/877: rmdir d5/d9/d18 39 2024-06-15T01:07:09.263 INFO:tasks.workunit.client.1.smithi142.stdout:8/802: write d0/f3f [2070279,110975] 0 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:4/543: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:8/454: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:1/552: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:3/792: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:7/473: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:0/640: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:2/440: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:6/474: sync 2024-06-15T01:07:09.264 INFO:tasks.workunit.client.0.smithi137.stdout:5/591: sync 2024-06-15T01:07:09.265 INFO:tasks.workunit.client.1.smithi142.stdout:4/878: write d5/d9/f131 [838943,30818] 0 2024-06-15T01:07:09.266 INFO:tasks.workunit.client.0.smithi137.stdout:9/371: mkdir df/d12/d29/d66 0 2024-06-15T01:07:09.266 INFO:tasks.workunit.client.1.smithi142.stdout:6/773: symlink d1a/d2d/d80/d8e/d110/l111 0 2024-06-15T01:07:09.267 INFO:tasks.workunit.client.0.smithi137.stdout:8/455: readlink d9/de/d1e/d35/l3a 0 2024-06-15T01:07:09.268 INFO:tasks.workunit.client.1.smithi142.stdout:3/854: unlink d0/d1f/d20/d5d/fbb 0 2024-06-15T01:07:09.269 INFO:tasks.workunit.client.0.smithi137.stdout:6/475: write d0/d1a/d68/f6a [832455,80945] 0 2024-06-15T01:07:09.270 INFO:tasks.workunit.client.0.smithi137.stdout:4/544: chown dd/d15/d21/d9c/d91 0 1 2024-06-15T01:07:09.271 INFO:tasks.workunit.client.0.smithi137.stdout:1/553: chown d0/d1f/d76/f8f 341786 1 2024-06-15T01:07:09.273 INFO:tasks.workunit.client.1.smithi142.stdout:5/716: unlink d2/f25 0 2024-06-15T01:07:09.274 INFO:tasks.workunit.client.1.smithi142.stdout:5/717: dread - d2/de/d19/d5f/d68/d76/d96/dac/fc2 zero size 2024-06-15T01:07:09.275 INFO:tasks.workunit.client.1.smithi142.stdout:1/808: rmdir d6/d12/da0/db0/de1 39 2024-06-15T01:07:09.276 INFO:tasks.workunit.client.0.smithi137.stdout:2/441: fsync d0/de/d10/f33 0 2024-06-15T01:07:09.284 INFO:tasks.workunit.client.1.smithi142.stdout:8/803: rename d0/d25/d53/da7/dcb/fe5 to d0/d17/d40/d5e/ff4 0 2024-06-15T01:07:09.285 INFO:tasks.workunit.client.0.smithi137.stdout:2/442: mkdir d0/dc/d22/d83 0 2024-06-15T01:07:09.287 INFO:tasks.workunit.client.0.smithi137.stdout:8/456: rename d9/d2a/d2f/d59/d82/f57 to d9/d2a/d2f/d59/f96 0 2024-06-15T01:07:09.287 INFO:tasks.workunit.client.0.smithi137.stdout:0/641: dwrite d1/d2/d13/f40 [0,4194304] 0 2024-06-15T01:07:09.290 INFO:tasks.workunit.client.0.smithi137.stdout:3/793: truncate d0/d2/d8/f41 3334073 0 2024-06-15T01:07:09.290 INFO:tasks.workunit.client.1.smithi142.stdout:2/760: creat d8/df/ff0 x:0 0 0 2024-06-15T01:07:09.291 INFO:tasks.workunit.client.1.smithi142.stdout:8/804: dwrite d0/d25/d53/da7/ff1 [0,4194304] 0 2024-06-15T01:07:09.292 INFO:tasks.workunit.client.1.smithi142.stdout:2/761: truncate d8/df/d10/d94/d25/d22/fdc 975208 0 2024-06-15T01:07:09.293 INFO:tasks.workunit.client.0.smithi137.stdout:1/554: symlink d0/d1f/d76/da1/lc3 0 2024-06-15T01:07:09.297 INFO:tasks.workunit.client.0.smithi137.stdout:6/476: dwrite d0/d5/d9/d3f/f5f [0,4194304] 0 2024-06-15T01:07:09.303 INFO:tasks.workunit.client.0.smithi137.stdout:1/555: write d0/d3/d47/f88 [1884389,36620] 0 2024-06-15T01:07:09.303 INFO:tasks.workunit.client.0.smithi137.stdout:5/592: dwrite df/f12 [0,4194304] 0 2024-06-15T01:07:09.304 INFO:tasks.workunit.client.0.smithi137.stdout:9/372: truncate f2 2470901 0 2024-06-15T01:07:09.306 INFO:tasks.workunit.client.1.smithi142.stdout:0/766: truncate d3/d39/d3e/d81/d44/d7c/d22/d59/f94 1324212 0 2024-06-15T01:07:09.312 INFO:tasks.workunit.client.0.smithi137.stdout:8/457: dwrite d9/d2a/f7a [0,4194304] 0 2024-06-15T01:07:09.314 INFO:tasks.workunit.client.0.smithi137.stdout:7/474: dread df/d7d/d1c/d1d/d27/f48 [0,4194304] 0 2024-06-15T01:07:09.314 INFO:tasks.workunit.client.0.smithi137.stdout:0/642: mkdir d1/d2/dc/ddc 0 2024-06-15T01:07:09.314 INFO:tasks.workunit.client.0.smithi137.stdout:6/477: rmdir d0/d5/d9/d29/d64 39 2024-06-15T01:07:09.315 INFO:tasks.workunit.client.1.smithi142.stdout:7/794: creat d2/d8/da/d33/ffc x:0 0 0 2024-06-15T01:07:09.318 INFO:tasks.workunit.client.0.smithi137.stdout:0/643: readlink d1/d2/le 0 2024-06-15T01:07:09.320 INFO:tasks.workunit.client.0.smithi137.stdout:4/545: link dd/d15/d36/d45/d55/cb9 dd/d15/d36/d45/d55/d8d/d1f/cce 0 2024-06-15T01:07:09.321 INFO:tasks.workunit.client.0.smithi137.stdout:6/478: stat d0/d1a/d3e/d57/c44 0 2024-06-15T01:07:09.323 INFO:tasks.workunit.client.0.smithi137.stdout:5/593: mkdir df/d11/d14/d6c/dc7 0 2024-06-15T01:07:09.327 INFO:tasks.workunit.client.0.smithi137.stdout:0/644: readlink d1/d2/dc/d10/d14/d52/la9 0 2024-06-15T01:07:09.328 INFO:tasks.workunit.client.1.smithi142.stdout:5/718: getdents d2/de/d27/db2 0 2024-06-15T01:07:09.328 INFO:tasks.workunit.client.0.smithi137.stdout:9/373: fsync df/d12/f17 0 2024-06-15T01:07:09.329 INFO:tasks.workunit.client.1.smithi142.stdout:9/734: truncate d6/dc/d1b/d25/d58/de2/fe8 1480498 0 2024-06-15T01:07:09.330 INFO:tasks.workunit.client.0.smithi137.stdout:8/458: dwrite d9/d2a/d2f/d59/f87 [0,4194304] 0 2024-06-15T01:07:09.330 INFO:tasks.workunit.client.0.smithi137.stdout:1/556: creat d0/d1f/d76/fc4 x:0 0 0 2024-06-15T01:07:09.331 INFO:tasks.workunit.client.0.smithi137.stdout:2/443: dread d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:07:09.332 INFO:tasks.workunit.client.0.smithi137.stdout:0/645: write d1/d2/d55/f6e [2509017,55633] 0 2024-06-15T01:07:09.334 INFO:tasks.workunit.client.1.smithi142.stdout:4/879: creat d5/d9/d23/f135 x:0 0 0 2024-06-15T01:07:09.335 INFO:tasks.workunit.client.0.smithi137.stdout:4/546: rmdir dd/d15/d21/d9c/d75/d99 39 2024-06-15T01:07:09.336 INFO:tasks.workunit.client.0.smithi137.stdout:9/374: truncate df/d12/d15/f63 603343 0 2024-06-15T01:07:09.336 INFO:tasks.workunit.client.0.smithi137.stdout:3/794: write d0/d13/d37/d66/f88 [4860298,56172] 0 2024-06-15T01:07:09.337 INFO:tasks.workunit.client.1.smithi142.stdout:5/719: dwrite d2/de/d19/d5f/d68/fd7 [0,4194304] 0 2024-06-15T01:07:09.338 INFO:tasks.workunit.client.1.smithi142.stdout:3/855: creat d0/d1f/d20/d5d/d82/dc9/f102 x:0 0 0 2024-06-15T01:07:09.340 INFO:tasks.workunit.client.1.smithi142.stdout:3/856: truncate d0/d3/de2/d55/f7a 5185093 0 2024-06-15T01:07:09.341 INFO:tasks.workunit.client.0.smithi137.stdout:1/557: mkdir d0/d3/d47/d73/dc5 0 2024-06-15T01:07:09.343 INFO:tasks.workunit.client.1.smithi142.stdout:8/805: dread d0/d25/d5a/f48 [0,4194304] 0 2024-06-15T01:07:09.344 INFO:tasks.workunit.client.1.smithi142.stdout:6/774: getdents d1a/d2d/d3b/dab 0 2024-06-15T01:07:09.344 INFO:tasks.workunit.client.1.smithi142.stdout:3/857: write d0/d1f/d20/d59/d61/d96/fcc [4786389,24402] 0 2024-06-15T01:07:09.345 INFO:tasks.workunit.client.0.smithi137.stdout:1/558: write d0/d1f/f4d [1820842,54392] 0 2024-06-15T01:07:09.345 INFO:tasks.workunit.client.1.smithi142.stdout:6/775: stat d1a/d3c/fc8 0 2024-06-15T01:07:09.345 INFO:tasks.workunit.client.0.smithi137.stdout:5/594: mkdir df/d44/d54/d95/d3b/d4b/dc8 0 2024-06-15T01:07:09.346 INFO:tasks.workunit.client.0.smithi137.stdout:6/479: mknod d0/d1a/d3e/d57/d1d/d2e/d70/c93 0 2024-06-15T01:07:09.347 INFO:tasks.workunit.client.0.smithi137.stdout:8/459: chown d9/de/d1e/c4f 6768 1 2024-06-15T01:07:09.350 INFO:tasks.workunit.client.1.smithi142.stdout:1/809: creat d6/d12/d5b/f11e x:0 0 0 2024-06-15T01:07:09.353 INFO:tasks.workunit.client.0.smithi137.stdout:4/547: mknod dd/d15/d21/d9c/ccf 0 2024-06-15T01:07:09.353 INFO:tasks.workunit.client.1.smithi142.stdout:7/795: fsync d2/d6/f83 0 2024-06-15T01:07:09.353 INFO:tasks.workunit.client.1.smithi142.stdout:7/796: chown d2/d8/da8 1529081 1 2024-06-15T01:07:09.354 INFO:tasks.workunit.client.0.smithi137.stdout:9/375: dread df/f28 [0,4194304] 0 2024-06-15T01:07:09.354 INFO:tasks.workunit.client.0.smithi137.stdout:4/548: fsync dd/d15/d21/d3a/f73 0 2024-06-15T01:07:09.354 INFO:tasks.workunit.client.1.smithi142.stdout:5/720: symlink d2/de/d19/d5f/d68/d76/d8c/le4 0 2024-06-15T01:07:09.356 INFO:tasks.workunit.client.0.smithi137.stdout:0/646: mknod d1/d7e/d87/da0/dc4/cdd 0 2024-06-15T01:07:09.356 INFO:tasks.workunit.client.1.smithi142.stdout:2/762: link d8/df/d10/d94/d25/d22/d43/d52/d7a/fbe d8/df/d10/d94/d25/d22/d43/d52/dc1/ff1 0 2024-06-15T01:07:09.358 INFO:tasks.workunit.client.0.smithi137.stdout:7/475: rename df/d13/d3c/c71 to df/d7d/d1c/c94 0 2024-06-15T01:07:09.358 INFO:tasks.workunit.client.0.smithi137.stdout:7/476: readlink df/d13/l44 0 2024-06-15T01:07:09.359 INFO:tasks.workunit.client.1.smithi142.stdout:1/810: write d6/d12/da0/db0/de1/ffa [364254,92891] 0 2024-06-15T01:07:09.359 INFO:tasks.workunit.client.0.smithi137.stdout:0/647: chown d1/d2/d13/d66/dc3/cc9 9321900 1 2024-06-15T01:07:09.359 INFO:tasks.workunit.client.1.smithi142.stdout:1/811: fdatasync d6/d12/da0/db0/fe8 0 2024-06-15T01:07:09.360 INFO:tasks.workunit.client.1.smithi142.stdout:7/797: rmdir d2/d8/dd/d14/d2c/d37 39 2024-06-15T01:07:09.360 INFO:tasks.workunit.client.0.smithi137.stdout:1/559: creat d0/d2d/fc6 x:0 0 0 2024-06-15T01:07:09.361 INFO:tasks.workunit.client.1.smithi142.stdout:5/721: creat d2/de/d27/d86/dbd/fe5 x:0 0 0 2024-06-15T01:07:09.361 INFO:tasks.workunit.client.0.smithi137.stdout:0/648: chown d1/d2/dc/d10/d14/d15/d17/l26 10267396 1 2024-06-15T01:07:09.362 INFO:tasks.workunit.client.1.smithi142.stdout:9/735: mknod d6/dc/d82/c10e 0 2024-06-15T01:07:09.362 INFO:tasks.workunit.client.1.smithi142.stdout:5/722: rename d2 to d2/d3e/d74/da5/de6 22 2024-06-15T01:07:09.363 INFO:tasks.workunit.client.0.smithi137.stdout:6/480: dread d0/d1a/d3e/d57/d1d/f1f [0,4194304] 0 2024-06-15T01:07:09.364 INFO:tasks.workunit.client.1.smithi142.stdout:2/763: rmdir d8/d19 39 2024-06-15T01:07:09.366 INFO:tasks.workunit.client.0.smithi137.stdout:9/376: creat df/d12/d1b/d31/f67 x:0 0 0 2024-06-15T01:07:09.366 INFO:tasks.workunit.client.1.smithi142.stdout:6/776: mknod d1a/d3c/d105/c112 0 2024-06-15T01:07:09.366 INFO:tasks.workunit.client.1.smithi142.stdout:2/764: write d8/df/d10/d94/db5/ded/fe5 [947077,41695] 0 2024-06-15T01:07:09.368 INFO:tasks.workunit.client.1.smithi142.stdout:8/806: unlink d0/d25/d5a/d8f/d97/fbe 0 2024-06-15T01:07:09.369 INFO:tasks.workunit.client.1.smithi142.stdout:2/765: write d8/df/d10/f3b [1400274,84721] 0 2024-06-15T01:07:09.370 INFO:tasks.workunit.client.1.smithi142.stdout:9/736: dwrite d6/dc/dd/f46 [0,4194304] 0 2024-06-15T01:07:09.370 INFO:tasks.workunit.client.0.smithi137.stdout:8/460: rename d9/de/d1e/d35/d5f/f78 to d9/d2a/f97 0 2024-06-15T01:07:09.370 INFO:tasks.workunit.client.1.smithi142.stdout:7/798: mknod d2/d8/da/d33/d25/d79/dd4/cfd 0 2024-06-15T01:07:09.371 INFO:tasks.workunit.client.0.smithi137.stdout:7/477: mknod df/d7d/d1c/d1d/d27/d33/d59/c95 0 2024-06-15T01:07:09.373 INFO:tasks.workunit.client.0.smithi137.stdout:2/444: dread d0/dc/d2e/d72/f67 [0,4194304] 0 2024-06-15T01:07:09.373 INFO:tasks.workunit.client.1.smithi142.stdout:1/812: dread d6/d12/d3c/d63/d8d/fb8 [0,4194304] 0 2024-06-15T01:07:09.374 INFO:tasks.workunit.client.1.smithi142.stdout:5/723: mknod d2/de/d19/d5f/d68/d76/d96/ce7 0 2024-06-15T01:07:09.375 INFO:tasks.workunit.client.1.smithi142.stdout:6/777: creat d1a/d3c/d105/f113 x:0 0 0 2024-06-15T01:07:09.376 INFO:tasks.workunit.client.1.smithi142.stdout:0/767: write d3/d39/d3e/d81/f34 [592138,82753] 0 2024-06-15T01:07:09.376 INFO:tasks.workunit.client.0.smithi137.stdout:8/461: write d9/de/ff [3430001,113939] 0 2024-06-15T01:07:09.377 INFO:tasks.workunit.client.0.smithi137.stdout:9/377: dwrite df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:09.377 INFO:tasks.workunit.client.1.smithi142.stdout:8/807: chown d0/d17/d40/l42 421074529 1 2024-06-15T01:07:09.379 INFO:tasks.workunit.client.0.smithi137.stdout:2/445: truncate d0/f5b 165573 0 2024-06-15T01:07:09.382 INFO:tasks.workunit.client.0.smithi137.stdout:5/595: write df/d3d/f5f [2259562,114739] 0 2024-06-15T01:07:09.383 INFO:tasks.workunit.client.0.smithi137.stdout:1/560: creat d0/d3/d47/d73/fc7 x:0 0 0 2024-06-15T01:07:09.388 INFO:tasks.workunit.client.0.smithi137.stdout:6/481: symlink d0/d1a/d3e/d87/l94 0 2024-06-15T01:07:09.389 INFO:tasks.workunit.client.0.smithi137.stdout:9/378: symlink df/d12/d29/d66/l68 0 2024-06-15T01:07:09.391 INFO:tasks.workunit.client.0.smithi137.stdout:5/596: symlink df/d11/d14/d6c/lc9 0 2024-06-15T01:07:09.393 INFO:tasks.workunit.client.0.smithi137.stdout:6/482: mknod d0/d1a/d68/c95 0 2024-06-15T01:07:09.394 INFO:tasks.workunit.client.0.smithi137.stdout:0/649: dread d1/d2/d55/faf [0,4194304] 0 2024-06-15T01:07:09.394 INFO:tasks.workunit.client.1.smithi142.stdout:5/724: dread d2/de/d27/f46 [0,4194304] 0 2024-06-15T01:07:09.396 INFO:tasks.workunit.client.0.smithi137.stdout:2/446: truncate d0/dc/d22/d23/f76 739880 0 2024-06-15T01:07:09.397 INFO:tasks.workunit.client.0.smithi137.stdout:3/795: truncate d0/d13/d37/d66/f88 1025164 0 2024-06-15T01:07:09.398 INFO:tasks.workunit.client.0.smithi137.stdout:6/483: chown d0/d1a/d3e/d57/d1d 7790216 1 2024-06-15T01:07:09.401 INFO:tasks.workunit.client.1.smithi142.stdout:7/799: rename d2/d8/dd/d8a/d94/cd1 to d2/d6/cfe 0 2024-06-15T01:07:09.402 INFO:tasks.workunit.client.1.smithi142.stdout:9/737: link d6/f21 d6/dc/dd/d22/f10f 0 2024-06-15T01:07:09.402 INFO:tasks.workunit.client.0.smithi137.stdout:1/561: dwrite d0/d3/d47/d73/fb9 [0,4194304] 0 2024-06-15T01:07:09.402 INFO:tasks.workunit.client.1.smithi142.stdout:7/800: write d2/d8/dd/dcf/fa6 [456881,104279] 0 2024-06-15T01:07:09.403 INFO:tasks.workunit.client.0.smithi137.stdout:9/379: truncate df/d12/d1b/d31/d42/f56 843585 0 2024-06-15T01:07:09.405 INFO:tasks.workunit.client.0.smithi137.stdout:3/796: chown d0/d2/d22/d24/d28/fea 625081 1 2024-06-15T01:07:09.405 INFO:tasks.workunit.client.0.smithi137.stdout:0/650: link d1/f32 d1/d2/dc/d10/d53/dca/d8d/fde 0 2024-06-15T01:07:09.406 INFO:tasks.workunit.client.1.smithi142.stdout:0/768: dread d3/d39/d3e/d81/d1b/d38/f9b [0,4194304] 0 2024-06-15T01:07:09.406 INFO:tasks.workunit.client.1.smithi142.stdout:5/725: truncate d2/d3e/d31/d7e/fa8 2099926 0 2024-06-15T01:07:09.406 INFO:tasks.workunit.client.0.smithi137.stdout:8/462: dwrite d9/de/d1e/d35/f5b [0,4194304] 0 2024-06-15T01:07:09.407 INFO:tasks.workunit.client.1.smithi142.stdout:5/726: truncate d2/d3e/d98/fd0 852983 0 2024-06-15T01:07:09.410 INFO:tasks.workunit.client.0.smithi137.stdout:9/380: creat df/f69 x:0 0 0 2024-06-15T01:07:09.410 INFO:tasks.workunit.client.0.smithi137.stdout:6/484: mknod d0/d5/d9/d29/d6b/c96 0 2024-06-15T01:07:09.413 INFO:tasks.workunit.client.0.smithi137.stdout:8/463: chown d9/de/d1e/d3f 1835654 1 2024-06-15T01:07:09.414 INFO:tasks.workunit.client.1.smithi142.stdout:2/766: truncate d8/df/d10/d94/d25/fda 1977413 0 2024-06-15T01:07:09.415 INFO:tasks.workunit.client.0.smithi137.stdout:9/381: mknod df/d12/d1b/d31/c6a 0 2024-06-15T01:07:09.417 INFO:tasks.workunit.client.1.smithi142.stdout:9/738: dwrite d6/f71 [4194304,4194304] 0 2024-06-15T01:07:09.417 INFO:tasks.workunit.client.0.smithi137.stdout:3/797: write d0/d2/d22/d24/f54 [2012870,30871] 0 2024-06-15T01:07:09.417 INFO:tasks.workunit.client.0.smithi137.stdout:2/447: read d0/d42/d49/d57/f58 [2352366,121953] 0 2024-06-15T01:07:09.418 INFO:tasks.workunit.client.1.smithi142.stdout:8/808: truncate d0/d25/d53/da7/dcb/fd9 454049 0 2024-06-15T01:07:09.418 INFO:tasks.workunit.client.1.smithi142.stdout:9/739: stat d6/dc/d1b/d25/d58/cba 0 2024-06-15T01:07:09.420 INFO:tasks.workunit.client.0.smithi137.stdout:8/464: creat d9/d22/f98 x:0 0 0 2024-06-15T01:07:09.420 INFO:tasks.workunit.client.0.smithi137.stdout:9/382: creat df/f6b x:0 0 0 2024-06-15T01:07:09.423 INFO:tasks.workunit.client.0.smithi137.stdout:2/448: creat d0/da/d73/f84 x:0 0 0 2024-06-15T01:07:09.423 INFO:tasks.workunit.client.1.smithi142.stdout:1/813: rename d6/d12/d80/dae/ced to d6/d3a/c11f 0 2024-06-15T01:07:09.423 INFO:tasks.workunit.client.0.smithi137.stdout:9/383: write df/d12/d1b/d31/f33 [73695,77196] 0 2024-06-15T01:07:09.425 INFO:tasks.workunit.client.0.smithi137.stdout:8/465: mknod d9/d55/c99 0 2024-06-15T01:07:09.425 INFO:tasks.workunit.client.1.smithi142.stdout:8/809: chown d0/f47 6519101 1 2024-06-15T01:07:09.426 INFO:tasks.workunit.client.1.smithi142.stdout:0/769: link d3/d39/d3e/d81/d4c/d61/cc3 d3/d39/d3e/d81/d4c/c106 0 2024-06-15T01:07:09.426 INFO:tasks.workunit.client.1.smithi142.stdout:8/810: write d0/f5f [43547,110481] 0 2024-06-15T01:07:09.429 INFO:tasks.workunit.client.1.smithi142.stdout:6/778: rename d1a/d3c/d105/c112 to d1a/d3c/d100/d58/d10f/c114 0 2024-06-15T01:07:09.429 INFO:tasks.workunit.client.1.smithi142.stdout:8/811: read d0/f23 [1151555,4101] 0 2024-06-15T01:07:09.430 INFO:tasks.workunit.client.0.smithi137.stdout:6/485: rename d0/d1a/d3e/d57/d1d/d2e/d62/c7e to d0/c97 0 2024-06-15T01:07:09.430 INFO:tasks.workunit.client.1.smithi142.stdout:5/727: link d2/de/d19/d93/c94 d2/de/d27/db2/ce8 0 2024-06-15T01:07:09.430 INFO:tasks.workunit.client.0.smithi137.stdout:6/486: chown d0/fc 102417 1 2024-06-15T01:07:09.431 INFO:tasks.workunit.client.1.smithi142.stdout:9/740: mkdir d6/dc/d1b/d6f/de1/d110 0 2024-06-15T01:07:09.432 INFO:tasks.workunit.client.0.smithi137.stdout:8/466: symlink d9/d2a/d79/d7e/l9a 0 2024-06-15T01:07:09.432 INFO:tasks.workunit.client.0.smithi137.stdout:8/467: readlink d9/d2a/d2f/l4a 0 2024-06-15T01:07:09.434 INFO:tasks.workunit.client.0.smithi137.stdout:9/384: link df/d12/d15/f58 df/d12/d29/d4a/f6c 0 2024-06-15T01:07:09.437 INFO:tasks.workunit.client.1.smithi142.stdout:8/812: creat d0/d17/d40/d6e/ff5 x:0 0 0 2024-06-15T01:07:09.437 INFO:tasks.workunit.client.0.smithi137.stdout:2/449: link d0/de/f1a d0/da/d73/f85 0 2024-06-15T01:07:09.438 INFO:tasks.workunit.client.1.smithi142.stdout:8/813: read - d0/d25/d5a/d8f/d98/ff2 zero size 2024-06-15T01:07:09.440 INFO:tasks.workunit.client.0.smithi137.stdout:0/651: dread d1/f9 [0,4194304] 0 2024-06-15T01:07:09.440 INFO:tasks.workunit.client.0.smithi137.stdout:9/385: unlink df/d12/d4e/c64 0 2024-06-15T01:07:09.444 INFO:tasks.workunit.client.1.smithi142.stdout:7/801: rename d2/d8/da/d33/c2a to d2/d6/daf/dce/cff 0 2024-06-15T01:07:09.445 INFO:tasks.workunit.client.1.smithi142.stdout:7/802: chown d2/d8/da/d33/d25/d79/fcc 2248209 1 2024-06-15T01:07:09.447 INFO:tasks.workunit.client.0.smithi137.stdout:9/386: read f2 [1544437,40825] 0 2024-06-15T01:07:09.448 INFO:tasks.workunit.client.1.smithi142.stdout:5/728: link d2/de/d27/db2/fbb d2/de/d19/d5f/fe9 0 2024-06-15T01:07:09.449 INFO:tasks.workunit.client.1.smithi142.stdout:5/729: write d2/de/d27/d86/dbd/fcd [353588,52175] 0 2024-06-15T01:07:09.450 INFO:tasks.workunit.client.0.smithi137.stdout:6/487: link d0/d5/d9/d29/f4d d0/d1a/f98 0 2024-06-15T01:07:09.453 INFO:tasks.workunit.client.1.smithi142.stdout:5/730: read - d2/d18/d2e/dae/fda zero size 2024-06-15T01:07:09.455 INFO:tasks.workunit.client.1.smithi142.stdout:9/741: rmdir d6/dc/d1b 39 2024-06-15T01:07:09.455 INFO:tasks.workunit.client.0.smithi137.stdout:9/387: dwrite df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:09.456 INFO:tasks.workunit.client.0.smithi137.stdout:6/488: mkdir d0/d1a/d99 0 2024-06-15T01:07:09.456 INFO:tasks.workunit.client.0.smithi137.stdout:6/489: chown d0/d1a/f26 31822 1 2024-06-15T01:07:09.459 INFO:tasks.workunit.client.1.smithi142.stdout:5/731: mknod d2/d18/d2e/cea 0 2024-06-15T01:07:09.460 INFO:tasks.workunit.client.0.smithi137.stdout:9/388: truncate df/d12/d29/f44 4699395 0 2024-06-15T01:07:09.461 INFO:tasks.workunit.client.0.smithi137.stdout:3/798: truncate d0/d2/d22/d24/f54 4499469 0 2024-06-15T01:07:09.462 INFO:tasks.workunit.client.1.smithi142.stdout:5/732: write d2/de/d19/d5f/d68/fd7 [2485267,46414] 0 2024-06-15T01:07:09.463 INFO:tasks.workunit.client.1.smithi142.stdout:1/814: dwrite d6/d12/f2f [0,4194304] 0 2024-06-15T01:07:09.464 INFO:tasks.workunit.client.0.smithi137.stdout:1/562: dwrite d0/d10/d41/d7b/fc2 [0,4194304] 0 2024-06-15T01:07:09.465 INFO:tasks.workunit.client.1.smithi142.stdout:5/733: write d2/de/d19/d5f/d68/d76/d96/dac/fc2 [31799,56477] 0 2024-06-15T01:07:09.465 INFO:tasks.workunit.client.1.smithi142.stdout:0/770: rename d3/f48 to d3/d39/d3e/d81/d4c/f107 0 2024-06-15T01:07:09.472 INFO:tasks.workunit.client.0.smithi137.stdout:9/389: dwrite df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:09.473 INFO:tasks.workunit.client.0.smithi137.stdout:6/490: creat d0/d5/d9/d29/f9a x:0 0 0 2024-06-15T01:07:09.474 INFO:tasks.workunit.client.1.smithi142.stdout:7/803: creat d2/d6/d22/d47/de5/f100 x:0 0 0 2024-06-15T01:07:09.477 INFO:tasks.workunit.client.0.smithi137.stdout:9/390: readlink df/l14 0 2024-06-15T01:07:09.478 INFO:tasks.workunit.client.1.smithi142.stdout:6/779: rename d1a/d2d/d3b/c99 to d1a/d39/c115 0 2024-06-15T01:07:09.480 INFO:tasks.workunit.client.1.smithi142.stdout:0/771: fsync d3/d39/d9a/da2/fec 0 2024-06-15T01:07:09.480 INFO:tasks.workunit.client.0.smithi137.stdout:3/799: creat d0/d2/d8/d67/d7e/de3/f103 x:0 0 0 2024-06-15T01:07:09.480 INFO:tasks.workunit.client.0.smithi137.stdout:9/391: stat df/d13/l2b 0 2024-06-15T01:07:09.482 INFO:tasks.workunit.client.0.smithi137.stdout:3/800: write d0/d56/f84 [4528536,100108] 0 2024-06-15T01:07:09.482 INFO:tasks.workunit.client.0.smithi137.stdout:1/563: rename d0/d2d/l83 to d0/d10/lc8 0 2024-06-15T01:07:09.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/814: getdents d0/d25/d53/d9c 0 2024-06-15T01:07:09.483 INFO:tasks.workunit.client.0.smithi137.stdout:1/564: readlink d0/d10/l16 0 2024-06-15T01:07:09.483 INFO:tasks.workunit.client.1.smithi142.stdout:8/815: fsync d0/d17/d40/d6e/fbc 0 2024-06-15T01:07:09.484 INFO:tasks.workunit.client.0.smithi137.stdout:0/652: write d1/d2/dc/d10/d14/d15/d1c/f1d [8119660,29862] 0 2024-06-15T01:07:09.486 INFO:tasks.workunit.client.0.smithi137.stdout:8/468: dwrite d9/d2a/d2f/d59/f71 [0,4194304] 0 2024-06-15T01:07:09.486 INFO:tasks.workunit.client.0.smithi137.stdout:3/801: write d0/d2/d8/f14 [417385,89426] 0 2024-06-15T01:07:09.486 INFO:tasks.workunit.client.0.smithi137.stdout:2/450: write d0/de/f24 [973139,92574] 0 2024-06-15T01:07:09.487 INFO:tasks.workunit.client.0.smithi137.stdout:1/565: dread - d0/d2d/fc6 zero size 2024-06-15T01:07:09.488 INFO:tasks.workunit.client.0.smithi137.stdout:0/653: fsync d1/d2/dc/d10/f96 0 2024-06-15T01:07:09.489 INFO:tasks.workunit.client.1.smithi142.stdout:7/804: truncate d2/d8/da/d33/d25/d79/fb1 2707578 0 2024-06-15T01:07:09.490 INFO:tasks.workunit.client.0.smithi137.stdout:6/491: chown d0/d5/d9/c6f 127 1 2024-06-15T01:07:09.490 INFO:tasks.workunit.client.0.smithi137.stdout:8/469: write d9/d55/f5c [4641290,37997] 0 2024-06-15T01:07:09.492 INFO:tasks.workunit.client.1.smithi142.stdout:6/780: dwrite d1a/d3c/d100/d58/d87/fe0 [0,4194304] 0 2024-06-15T01:07:09.492 INFO:tasks.workunit.client.1.smithi142.stdout:1/815: creat d6/d3a/f120 x:0 0 0 2024-06-15T01:07:09.493 INFO:tasks.workunit.client.0.smithi137.stdout:6/492: readlink d0/d1a/l21 0 2024-06-15T01:07:09.493 INFO:tasks.workunit.client.1.smithi142.stdout:7/805: chown d2/d8/dd/d14/d2c/d70/l9b 1993906 1 2024-06-15T01:07:09.493 INFO:tasks.workunit.client.1.smithi142.stdout:0/772: mkdir d3/d39/d9a/d108 0 2024-06-15T01:07:09.494 INFO:tasks.workunit.client.1.smithi142.stdout:8/816: dwrite d0/f5f [0,4194304] 0 2024-06-15T01:07:09.496 INFO:tasks.workunit.client.0.smithi137.stdout:6/493: stat d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f 0 2024-06-15T01:07:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:1/566: symlink d0/d10/daf/lc9 0 2024-06-15T01:07:09.500 INFO:tasks.workunit.client.1.smithi142.stdout:7/806: fdatasync d2/d8/da/d33/f35 0 2024-06-15T01:07:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:1/567: chown d0/d3/d89/c97 2076646 1 2024-06-15T01:07:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:2/451: creat d0/dc/f86 x:0 0 0 2024-06-15T01:07:09.504 INFO:tasks.workunit.client.1.smithi142.stdout:6/781: dwrite d1a/d3c/fc8 [0,4194304] 0 2024-06-15T01:07:09.506 INFO:tasks.workunit.client.0.smithi137.stdout:6/494: stat d0/d5/l50 0 2024-06-15T01:07:09.506 INFO:tasks.workunit.client.1.smithi142.stdout:1/816: dwrite d6/d12/d1c/d2c/f91 [0,4194304] 0 2024-06-15T01:07:09.509 INFO:tasks.workunit.client.0.smithi137.stdout:9/392: link df/d12/f1f df/d12/d4e/f6d 0 2024-06-15T01:07:09.509 INFO:tasks.workunit.client.0.smithi137.stdout:6/495: write d0/d1a/d3e/d57/d32/d74/f78 [27997,128387] 0 2024-06-15T01:07:09.514 INFO:tasks.workunit.client.1.smithi142.stdout:7/807: creat d2/d6/d22/ddd/d85/f101 x:0 0 0 2024-06-15T01:07:09.515 INFO:tasks.workunit.client.1.smithi142.stdout:7/808: chown d2/d6/d22/d3c/c4c 52881364 1 2024-06-15T01:07:09.515 INFO:tasks.workunit.client.0.smithi137.stdout:1/568: mknod d0/d3/d1b/cca 0 2024-06-15T01:07:09.515 INFO:tasks.workunit.client.0.smithi137.stdout:0/654: write d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [2986955,83222] 0 2024-06-15T01:07:09.516 INFO:tasks.workunit.client.1.smithi142.stdout:9/742: write d6/f21 [2431383,85926] 0 2024-06-15T01:07:09.523 INFO:tasks.workunit.client.1.smithi142.stdout:6/782: creat d1a/d39/d42/f116 x:0 0 0 2024-06-15T01:07:09.525 INFO:tasks.workunit.client.1.smithi142.stdout:6/783: rename d1a/d2d/d80 to d1a/d2d/d80/d8e/d110/da0/d117 22 2024-06-15T01:07:09.527 INFO:tasks.workunit.client.1.smithi142.stdout:1/817: mknod d6/d12/da0/c121 0 2024-06-15T01:07:09.527 INFO:tasks.workunit.client.1.smithi142.stdout:7/809: rmdir d2/db6 39 2024-06-15T01:07:09.528 INFO:tasks.workunit.client.0.smithi137.stdout:3/802: dread d0/d2f/faa [0,4194304] 0 2024-06-15T01:07:09.528 INFO:tasks.workunit.client.1.smithi142.stdout:1/818: dread - d6/d12/d80/d9b/db2/dbf/fe5 zero size 2024-06-15T01:07:09.529 INFO:tasks.workunit.client.0.smithi137.stdout:9/393: rename df/d1a/l47 to df/d12/d15/d16/d3c/l6e 0 2024-06-15T01:07:09.530 INFO:tasks.workunit.client.1.smithi142.stdout:1/819: chown d6/d12/d1c/d2c/d47 66620890 1 2024-06-15T01:07:09.531 INFO:tasks.workunit.client.0.smithi137.stdout:6/496: write d0/d5/d9/f63 [6643043,6986] 0 2024-06-15T01:07:09.533 INFO:tasks.workunit.client.1.smithi142.stdout:8/817: getdents d0/d25/d5a/d8f/d98 0 2024-06-15T01:07:09.534 INFO:tasks.workunit.client.1.smithi142.stdout:1/820: dread d6/d12/d80/f100 [0,4194304] 0 2024-06-15T01:07:09.534 INFO:tasks.workunit.client.0.smithi137.stdout:9/394: mkdir df/d13/d6f 0 2024-06-15T01:07:09.535 INFO:tasks.workunit.client.0.smithi137.stdout:9/395: fsync df/d12/d15/f5f 0 2024-06-15T01:07:09.538 INFO:tasks.workunit.client.0.smithi137.stdout:9/396: creat df/d12/d29/d4a/f70 x:0 0 0 2024-06-15T01:07:09.538 INFO:tasks.workunit.client.0.smithi137.stdout:0/655: getdents d1/d2/d13/d66/dc3 0 2024-06-15T01:07:09.538 INFO:tasks.workunit.client.1.smithi142.stdout:6/784: mknod d1a/d83/c118 0 2024-06-15T01:07:09.539 INFO:tasks.workunit.client.1.smithi142.stdout:7/810: dwrite d2/d8/dd/dcf/fbb [0,4194304] 0 2024-06-15T01:07:09.539 INFO:tasks.workunit.client.1.smithi142.stdout:6/785: stat d1a/d39/d42 0 2024-06-15T01:07:09.539 INFO:tasks.workunit.client.1.smithi142.stdout:9/743: creat d6/dc/d1b/d6f/d103/df2/f111 x:0 0 0 2024-06-15T01:07:09.539 INFO:tasks.workunit.client.1.smithi142.stdout:8/818: creat d0/d25/d5a/d8f/d98/da4/ff6 x:0 0 0 2024-06-15T01:07:09.541 INFO:tasks.workunit.client.0.smithi137.stdout:9/397: creat df/d13/f71 x:0 0 0 2024-06-15T01:07:09.542 INFO:tasks.workunit.client.1.smithi142.stdout:7/811: dread d2/d8/dd/d14/f4b [0,4194304] 0 2024-06-15T01:07:09.543 INFO:tasks.workunit.client.1.smithi142.stdout:6/786: rmdir d1a/d3c 39 2024-06-15T01:07:09.544 INFO:tasks.workunit.client.0.smithi137.stdout:0/656: truncate d1/d2/dc/d10/d14/d15/d17/f95 2795680 0 2024-06-15T01:07:09.544 INFO:tasks.workunit.client.1.smithi142.stdout:6/787: write d1a/d2d/f2f [3097390,84422] 0 2024-06-15T01:07:09.547 INFO:tasks.workunit.client.0.smithi137.stdout:9/398: fdatasync df/d12/f1f 0 2024-06-15T01:07:09.548 INFO:tasks.workunit.client.0.smithi137.stdout:8/470: dread f6 [0,4194304] 0 2024-06-15T01:07:09.548 INFO:tasks.workunit.client.1.smithi142.stdout:9/744: creat d6/dc/d51/f112 x:0 0 0 2024-06-15T01:07:09.550 INFO:tasks.workunit.client.1.smithi142.stdout:4/880: sync 2024-06-15T01:07:09.550 INFO:tasks.workunit.client.1.smithi142.stdout:1/821: rename d6/d12/d1c/d2c/d93/l113 to d6/d12/d52/l122 0 2024-06-15T01:07:09.551 INFO:tasks.workunit.client.1.smithi142.stdout:7/812: creat d2/d8/da/d33/d25/d79/f102 x:0 0 0 2024-06-15T01:07:09.551 INFO:tasks.workunit.client.0.smithi137.stdout:9/399: write f5 [5039214,55189] 0 2024-06-15T01:07:09.553 INFO:tasks.workunit.client.0.smithi137.stdout:8/471: symlink d9/l9b 0 2024-06-15T01:07:09.553 INFO:tasks.workunit.client.1.smithi142.stdout:6/788: symlink d1a/d39/d42/df7/l119 0 2024-06-15T01:07:09.553 INFO:tasks.workunit.client.1.smithi142.stdout:7/813: write d2/d6/d22/d47/f7c [3941752,36138] 0 2024-06-15T01:07:09.555 INFO:tasks.workunit.client.0.smithi137.stdout:0/657: dwrite d1/fb6 [4194304,4194304] 0 2024-06-15T01:07:09.555 INFO:tasks.workunit.client.1.smithi142.stdout:9/745: dread d6/dc/d1b/fc6 [0,4194304] 0 2024-06-15T01:07:09.556 INFO:tasks.workunit.client.1.smithi142.stdout:6/789: fdatasync d1a/f28 0 2024-06-15T01:07:09.557 INFO:tasks.workunit.client.0.smithi137.stdout:9/400: truncate df/d12/d1b/d31/f67 468553 0 2024-06-15T01:07:09.558 INFO:tasks.workunit.client.0.smithi137.stdout:8/472: symlink d9/de/d1e/d3f/l9c 0 2024-06-15T01:07:09.558 INFO:tasks.workunit.client.1.smithi142.stdout:9/746: write d6/dc/f61 [106614,60415] 0 2024-06-15T01:07:09.559 INFO:tasks.workunit.client.1.smithi142.stdout:4/881: dread d5/d9/da/f3c [0,4194304] 0 2024-06-15T01:07:09.559 INFO:tasks.workunit.client.1.smithi142.stdout:6/790: fdatasync d1a/d39/d42/f116 0 2024-06-15T01:07:09.559 INFO:tasks.workunit.client.0.smithi137.stdout:0/658: write d1/d2/dc/d10/d14/d56/f85 [1339022,81262] 0 2024-06-15T01:07:09.561 INFO:tasks.workunit.client.1.smithi142.stdout:1/822: dwrite d6/d12/f2d [4194304,4194304] 0 2024-06-15T01:07:09.561 INFO:tasks.workunit.client.1.smithi142.stdout:7/814: write d2/d8/dd/d14/d2c/d37/d6b/f96 [4961475,1985] 0 2024-06-15T01:07:09.563 INFO:tasks.workunit.client.1.smithi142.stdout:9/747: write d6/dc/d1b/d6f/da1/fa7 [1008042,58930] 0 2024-06-15T01:07:09.565 INFO:tasks.workunit.client.0.smithi137.stdout:0/659: dread d1/d2/dc/d10/d53/dca/d8d/fde [0,4194304] 0 2024-06-15T01:07:09.566 INFO:tasks.workunit.client.1.smithi142.stdout:0/773: dread d3/d39/d3e/f5f [0,4194304] 0 2024-06-15T01:07:09.568 INFO:tasks.workunit.client.0.smithi137.stdout:8/473: rename d9/l52 to d9/de/d1e/d35/d5f/l9d 0 2024-06-15T01:07:09.569 INFO:tasks.workunit.client.0.smithi137.stdout:8/474: write d9/de/f36 [901416,2948] 0 2024-06-15T01:07:09.570 INFO:tasks.workunit.client.0.smithi137.stdout:8/475: dread - d9/d2a/d2f/d59/d82/f70 zero size 2024-06-15T01:07:09.572 INFO:tasks.workunit.client.1.smithi142.stdout:1/823: rmdir d6/d12/d1c 39 2024-06-15T01:07:09.573 INFO:tasks.workunit.client.0.smithi137.stdout:0/660: mknod d1/d2/dc/d10/d22/db7/cdf 0 2024-06-15T01:07:09.574 INFO:tasks.workunit.client.0.smithi137.stdout:1/569: write d0/d3/d1b/f50 [531000,7826] 0 2024-06-15T01:07:09.576 INFO:tasks.workunit.client.0.smithi137.stdout:2/452: truncate d0/f1 7321287 0 2024-06-15T01:07:09.577 INFO:tasks.workunit.client.1.smithi142.stdout:7/815: dwrite d2/d8/da/d33/f52 [0,4194304] 0 2024-06-15T01:07:09.579 INFO:tasks.workunit.client.0.smithi137.stdout:0/661: symlink d1/d2/dc/d10/d14/d15/d17/d54/d7a/le0 0 2024-06-15T01:07:09.583 INFO:tasks.workunit.client.0.smithi137.stdout:8/476: mknod d9/de/c9e 0 2024-06-15T01:07:09.584 INFO:tasks.workunit.client.1.smithi142.stdout:3/858: sync 2024-06-15T01:07:09.586 INFO:tasks.workunit.client.1.smithi142.stdout:8/819: write d0/f8e [1020927,106884] 0 2024-06-15T01:07:09.588 INFO:tasks.workunit.client.1.smithi142.stdout:8/820: write d0/d17/f27 [289361,72624] 0 2024-06-15T01:07:09.590 INFO:tasks.workunit.client.0.smithi137.stdout:3/803: dwrite d0/d2/d22/d63/f6d [0,4194304] 0 2024-06-15T01:07:09.593 INFO:tasks.workunit.client.1.smithi142.stdout:3/859: dwrite d0/d1b/fe4 [0,4194304] 0 2024-06-15T01:07:09.595 INFO:tasks.workunit.client.1.smithi142.stdout:1/824: getdents d6/d12/d1c/d2c/d93/db9/df2 0 2024-06-15T01:07:09.598 INFO:tasks.workunit.client.0.smithi137.stdout:1/570: dwrite d0/d1f/d76/da1/fa2 [0,4194304] 0 2024-06-15T01:07:09.601 INFO:tasks.workunit.client.1.smithi142.stdout:0/774: write d3/d39/d3e/d81/d1b/f35 [2666203,79305] 0 2024-06-15T01:07:09.602 INFO:tasks.workunit.client.0.smithi137.stdout:0/662: rename d1/d2/f6 to d1/d2/dc/d6c/d90/fe1 0 2024-06-15T01:07:09.603 INFO:tasks.workunit.client.0.smithi137.stdout:1/571: chown d0/d3/d1b/d4c/d59/fb4 5130439 1 2024-06-15T01:07:09.606 INFO:tasks.workunit.client.1.smithi142.stdout:9/748: getdents d6/dc/d1b/d6f/de1 0 2024-06-15T01:07:09.608 INFO:tasks.workunit.client.1.smithi142.stdout:8/821: mknod d0/d25/d5a/d8f/df3/cf7 0 2024-06-15T01:07:09.608 INFO:tasks.workunit.client.1.smithi142.stdout:1/825: dwrite d6/d12/d80/d9b/db2/dc2/f107 [0,4194304] 0 2024-06-15T01:07:09.611 INFO:tasks.workunit.client.1.smithi142.stdout:0/775: fsync d3/d39/d3e/d81/d1b/d52/f85 0 2024-06-15T01:07:09.612 INFO:tasks.workunit.client.0.smithi137.stdout:8/477: dwrite d9/de/d1e/d67/f7c [0,4194304] 0 2024-06-15T01:07:09.615 INFO:tasks.workunit.client.1.smithi142.stdout:4/882: dwrite d5/d9/d23/d32/f8f [0,4194304] 0 2024-06-15T01:07:09.616 INFO:tasks.workunit.client.0.smithi137.stdout:0/663: dwrite d1/d2/dc/d10/d14/d15/f4b [0,4194304] 0 2024-06-15T01:07:09.617 INFO:tasks.workunit.client.1.smithi142.stdout:3/860: creat d0/f103 x:0 0 0 2024-06-15T01:07:09.618 INFO:tasks.workunit.client.0.smithi137.stdout:0/664: readlink d1/d2/d13/d66/l9e 0 2024-06-15T01:07:09.621 INFO:tasks.workunit.client.0.smithi137.stdout:1/572: symlink d0/d3/d1b/d4c/d59/d64/lcb 0 2024-06-15T01:07:09.626 INFO:tasks.workunit.client.0.smithi137.stdout:8/478: creat d9/de/d1e/d35/f9f x:0 0 0 2024-06-15T01:07:09.632 INFO:tasks.workunit.client.0.smithi137.stdout:2/453: dwrite d0/de/d10/f33 [0,4194304] 0 2024-06-15T01:07:09.633 INFO:tasks.workunit.client.0.smithi137.stdout:9/401: dread f3 [0,4194304] 0 2024-06-15T01:07:09.635 INFO:tasks.workunit.client.0.smithi137.stdout:8/479: symlink d9/d22/la0 0 2024-06-15T01:07:09.642 INFO:tasks.workunit.client.0.smithi137.stdout:1/573: truncate d0/d1f/d60/f91 3760473 0 2024-06-15T01:07:09.643 INFO:tasks.workunit.client.0.smithi137.stdout:9/402: mkdir df/d49/d72 0 2024-06-15T01:07:09.643 INFO:tasks.workunit.client.0.smithi137.stdout:9/403: fsync df/d12/d4e/f5c 0 2024-06-15T01:07:09.644 INFO:tasks.workunit.client.0.smithi137.stdout:2/454: symlink d0/dc/d22/l87 0 2024-06-15T01:07:09.645 INFO:tasks.workunit.client.0.smithi137.stdout:8/480: stat d9/de/d1e/l92 0 2024-06-15T01:07:09.646 INFO:tasks.workunit.client.0.smithi137.stdout:1/574: fdatasync d0/d3/d1b/d4c/d59/d64/f6e 0 2024-06-15T01:07:09.648 INFO:tasks.workunit.client.0.smithi137.stdout:2/455: mknod d0/c88 0 2024-06-15T01:07:09.657 INFO:tasks.workunit.client.0.smithi137.stdout:1/575: creat d0/d1f/d76/fcc x:0 0 0 2024-06-15T01:07:09.658 INFO:tasks.workunit.client.0.smithi137.stdout:9/404: getdents df/d12/d1b 0 2024-06-15T01:07:09.661 INFO:tasks.workunit.client.0.smithi137.stdout:0/665: fsync d1/d2/dc/d6c/d88/d91/f9a 0 2024-06-15T01:07:09.662 INFO:tasks.workunit.client.0.smithi137.stdout:1/576: rename d0/d10/daf/lc9 to d0/lcd 0 2024-06-15T01:07:09.663 INFO:tasks.workunit.client.0.smithi137.stdout:0/666: write d1/d2/d13/f40 [4290810,87164] 0 2024-06-15T01:07:09.663 INFO:tasks.workunit.client.0.smithi137.stdout:1/577: chown d0/d10/d41/d7b/fc2 20385 1 2024-06-15T01:07:09.664 INFO:tasks.workunit.client.0.smithi137.stdout:1/578: stat d0/d3/d1b/d4c/f75 0 2024-06-15T01:07:09.668 INFO:tasks.workunit.client.0.smithi137.stdout:0/667: rename d1/d2/dc/d10/d53/dca to d1/d7e/d87/da0/dc4/dd7/de2 0 2024-06-15T01:07:09.670 INFO:tasks.workunit.client.0.smithi137.stdout:0/668: read d1/d2/dc/d10/f75 [4088370,5494] 0 2024-06-15T01:07:09.673 INFO:tasks.workunit.client.0.smithi137.stdout:1/579: getdents d0/d3/d47/d73 0 2024-06-15T01:07:09.673 INFO:tasks.workunit.client.0.smithi137.stdout:0/669: mkdir d1/d2/dc/d10/d22/db7/de3 0 2024-06-15T01:07:09.675 INFO:tasks.workunit.client.0.smithi137.stdout:3/804: dwrite d0/d2/d22/d24/d5e/f98 [0,4194304] 0 2024-06-15T01:07:09.676 INFO:tasks.workunit.client.0.smithi137.stdout:9/405: dread df/d12/d1b/f2f [0,4194304] 0 2024-06-15T01:07:09.681 INFO:tasks.workunit.client.0.smithi137.stdout:0/670: mkdir d1/d7e/d87/da0/dc4/de4 0 2024-06-15T01:07:09.682 INFO:tasks.workunit.client.0.smithi137.stdout:9/406: dread df/d12/d15/d16/f27 [0,4194304] 0 2024-06-15T01:07:09.683 INFO:tasks.workunit.client.0.smithi137.stdout:2/456: write d0/f8 [2341522,86499] 0 2024-06-15T01:07:09.686 INFO:tasks.workunit.client.0.smithi137.stdout:1/580: symlink d0/d3/d1b/d4c/d59/d93/lce 0 2024-06-15T01:07:09.686 INFO:tasks.workunit.client.0.smithi137.stdout:9/407: fsync df/d12/d1b/d31/f33 0 2024-06-15T01:07:09.687 INFO:tasks.workunit.client.1.smithi142.stdout:4/883: chown d5/d9/da/da3/de9 562296948 1 2024-06-15T01:07:09.687 INFO:tasks.workunit.client.0.smithi137.stdout:2/457: write d0/f69 [1967895,14549] 0 2024-06-15T01:07:09.688 INFO:tasks.workunit.client.0.smithi137.stdout:3/805: mkdir d0/d2/d23/ddb/dde/d104 0 2024-06-15T01:07:09.688 INFO:tasks.workunit.client.0.smithi137.stdout:8/481: dwrite d9/de/f3c [0,4194304] 0 2024-06-15T01:07:09.689 INFO:tasks.workunit.client.0.smithi137.stdout:9/408: read df/d12/d15/f52 [2648378,41729] 0 2024-06-15T01:07:09.691 INFO:tasks.workunit.client.1.smithi142.stdout:4/884: symlink d5/d1c/d6f/l136 0 2024-06-15T01:07:09.693 INFO:tasks.workunit.client.1.smithi142.stdout:6/791: dread d1a/d3c/db0/feb [0,4194304] 0 2024-06-15T01:07:09.696 INFO:tasks.workunit.client.1.smithi142.stdout:0/776: dwrite d3/d39/d3e/d81/d1e/f2b [0,4194304] 0 2024-06-15T01:07:09.696 INFO:tasks.workunit.client.1.smithi142.stdout:6/792: stat d1a/d3c/f106 0 2024-06-15T01:07:09.696 INFO:tasks.workunit.client.0.smithi137.stdout:1/581: unlink d0/d3/d1b/cca 0 2024-06-15T01:07:09.696 INFO:tasks.workunit.client.1.smithi142.stdout:1/826: dwrite d6/d12/da0/db0/de1/ffa [0,4194304] 0 2024-06-15T01:07:09.697 INFO:tasks.workunit.client.1.smithi142.stdout:8/822: dread d0/d25/d5a/fda [0,4194304] 0 2024-06-15T01:07:09.697 INFO:tasks.workunit.client.1.smithi142.stdout:4/885: unlink d5/d1c/c70 0 2024-06-15T01:07:09.698 INFO:tasks.workunit.client.1.smithi142.stdout:3/861: rename d0/d1f/d20/ce3 to d0/d95/dbc/c104 0 2024-06-15T01:07:09.700 INFO:tasks.workunit.client.0.smithi137.stdout:2/458: mknod d0/dc/d22/d23/c89 0 2024-06-15T01:07:09.702 INFO:tasks.workunit.client.1.smithi142.stdout:8/823: dread - d0/d17/d40/d6e/fbc zero size 2024-06-15T01:07:09.704 INFO:tasks.workunit.client.0.smithi137.stdout:8/482: creat d9/d2a/d2f/fa1 x:0 0 0 2024-06-15T01:07:09.705 INFO:tasks.workunit.client.1.smithi142.stdout:0/777: creat d3/d39/d3e/d81/d1b/d38/f109 x:0 0 0 2024-06-15T01:07:09.708 INFO:tasks.workunit.client.0.smithi137.stdout:9/409: creat df/d12/d15/f73 x:0 0 0 2024-06-15T01:07:09.710 INFO:tasks.workunit.client.0.smithi137.stdout:1/582: fsync d0/d3/d1b/f69 0 2024-06-15T01:07:09.710 INFO:tasks.workunit.client.1.smithi142.stdout:8/824: symlink d0/d25/d5a/d84/dd0/lf8 0 2024-06-15T01:07:09.711 INFO:tasks.workunit.client.1.smithi142.stdout:6/793: dwrite d1a/d3c/db0/f104 [0,4194304] 0 2024-06-15T01:07:09.711 INFO:tasks.workunit.client.1.smithi142.stdout:3/862: truncate d0/d3/de2/d55/f94 30756 0 2024-06-15T01:07:09.711 INFO:tasks.workunit.client.1.smithi142.stdout:1/827: dwrite d6/d3a/fe3 [0,4194304] 0 2024-06-15T01:07:09.713 INFO:tasks.workunit.client.0.smithi137.stdout:9/410: dread df/d12/d1b/d31/f67 [0,4194304] 0 2024-06-15T01:07:09.714 INFO:tasks.workunit.client.0.smithi137.stdout:8/483: dwrite d9/d2a/d2f/fa1 [0,4194304] 0 2024-06-15T01:07:09.714 INFO:tasks.workunit.client.0.smithi137.stdout:0/671: truncate d1/d2/dc/d10/d14/f1b 1549844 0 2024-06-15T01:07:09.716 INFO:tasks.workunit.client.0.smithi137.stdout:0/672: read d1/d2/d13/d66/f4a [2401169,74337] 0 2024-06-15T01:07:09.720 INFO:tasks.workunit.client.1.smithi142.stdout:3/863: truncate d0/d1f/d20/d59/d61/fb7 975196 0 2024-06-15T01:07:09.720 INFO:tasks.workunit.client.1.smithi142.stdout:1/828: creat d6/d12/d80/f123 x:0 0 0 2024-06-15T01:07:09.722 INFO:tasks.workunit.client.1.smithi142.stdout:0/778: dwrite d3/d39/d3e/d81/d4c/d61/f77 [0,4194304] 0 2024-06-15T01:07:09.723 INFO:tasks.workunit.client.1.smithi142.stdout:5/734: sync 2024-06-15T01:07:09.723 INFO:tasks.workunit.client.1.smithi142.stdout:2/767: sync 2024-06-15T01:07:09.724 INFO:tasks.workunit.client.1.smithi142.stdout:3/864: rmdir d0/d1f/d20 39 2024-06-15T01:07:09.725 INFO:tasks.workunit.client.0.smithi137.stdout:1/583: rename d0/d3/d1b/c42 to d0/d1f/d67/ccf 0 2024-06-15T01:07:09.725 INFO:tasks.workunit.client.1.smithi142.stdout:6/794: dread d1a/d39/d3a/fec [0,4194304] 0 2024-06-15T01:07:09.725 INFO:tasks.workunit.client.1.smithi142.stdout:0/779: symlink d3/d39/d3e/d81/d1b/d38/da3/l10a 0 2024-06-15T01:07:09.726 INFO:tasks.workunit.client.0.smithi137.stdout:0/673: symlink d1/d2/dc/d6c/d88/d91/le5 0 2024-06-15T01:07:09.726 INFO:tasks.workunit.client.1.smithi142.stdout:3/865: dread - d0/d3/f100 zero size 2024-06-15T01:07:09.727 INFO:tasks.workunit.client.0.smithi137.stdout:0/674: write d1/fb6 [7836218,66592] 0 2024-06-15T01:07:09.728 INFO:tasks.workunit.client.1.smithi142.stdout:1/829: symlink d6/d12/d3c/d63/d8d/d97/dc6/dfe/l124 0 2024-06-15T01:07:09.728 INFO:tasks.workunit.client.1.smithi142.stdout:2/768: fsync d8/df/d10/d94/d25/d22/d43/d51/fd2 0 2024-06-15T01:07:09.732 INFO:tasks.workunit.client.1.smithi142.stdout:1/830: write d6/d12/da0/ff1 [568371,62538] 0 2024-06-15T01:07:09.732 INFO:tasks.workunit.client.1.smithi142.stdout:2/769: read d8/df/d10/d94/db5/ded/fe5 [405612,69080] 0 2024-06-15T01:07:09.734 INFO:tasks.workunit.client.0.smithi137.stdout:8/484: link d9/de/d1e/f75 d9/de/d1e/d35/fa2 0 2024-06-15T01:07:09.734 INFO:tasks.workunit.client.1.smithi142.stdout:2/770: chown d8/df/d10/d94/db5/ded 3903 1 2024-06-15T01:07:09.738 INFO:tasks.workunit.client.1.smithi142.stdout:5/735: getdents d2/de/d19 0 2024-06-15T01:07:09.738 INFO:tasks.workunit.client.1.smithi142.stdout:0/780: dread d3/d39/d3e/d81/d1b/d52/f85 [4194304,4194304] 0 2024-06-15T01:07:09.739 INFO:tasks.workunit.client.1.smithi142.stdout:5/736: fsync d2/d18/d2e/dae/fda 0 2024-06-15T01:07:09.740 INFO:tasks.workunit.client.1.smithi142.stdout:0/781: read - d3/d39/d3e/d81/d1e/ffc zero size 2024-06-15T01:07:09.740 INFO:tasks.workunit.client.1.smithi142.stdout:2/771: read d8/f40 [3080256,46464] 0 2024-06-15T01:07:09.741 INFO:tasks.workunit.client.1.smithi142.stdout:2/772: chown d8/df/d10/d94/d25/cca 5 1 2024-06-15T01:07:09.741 INFO:tasks.workunit.client.1.smithi142.stdout:0/782: dread - d3/d39/d3e/d81/d44/d7c/fee zero size 2024-06-15T01:07:09.742 INFO:tasks.workunit.client.0.smithi137.stdout:8/485: rmdir d9/d2a/d2f 39 2024-06-15T01:07:09.742 INFO:tasks.workunit.client.1.smithi142.stdout:2/773: fsync d8/df/d10/d94/d25/f27 0 2024-06-15T01:07:09.743 INFO:tasks.workunit.client.1.smithi142.stdout:5/737: creat d2/d3e/feb x:0 0 0 2024-06-15T01:07:09.744 INFO:tasks.workunit.client.1.smithi142.stdout:0/783: mknod d3/c10b 0 2024-06-15T01:07:09.745 INFO:tasks.workunit.client.0.smithi137.stdout:9/411: getdents df/d13 0 2024-06-15T01:07:09.745 INFO:tasks.workunit.client.1.smithi142.stdout:0/784: fsync d3/d39/d3e/d81/d1e/ffc 0 2024-06-15T01:07:09.746 INFO:tasks.workunit.client.0.smithi137.stdout:2/459: dwrite d0/fb [4194304,4194304] 0 2024-06-15T01:07:09.748 INFO:tasks.workunit.client.0.smithi137.stdout:0/675: dread d1/d2/dc/d10/f25 [0,4194304] 0 2024-06-15T01:07:09.749 INFO:tasks.workunit.client.0.smithi137.stdout:9/412: fsync df/d12/d15/f1d 0 2024-06-15T01:07:09.751 INFO:tasks.workunit.client.1.smithi142.stdout:0/785: unlink d3/d39/d3e/d81/d1e/l42 0 2024-06-15T01:07:09.751 INFO:tasks.workunit.client.0.smithi137.stdout:2/460: read d0/dc/d2e/d72/d56/f4f [2415753,100962] 0 2024-06-15T01:07:09.753 INFO:tasks.workunit.client.1.smithi142.stdout:6/795: dread d1a/d39/fb5 [0,4194304] 0 2024-06-15T01:07:09.754 INFO:tasks.workunit.client.0.smithi137.stdout:3/806: dread d0/d2/d8/f41 [0,4194304] 0 2024-06-15T01:07:09.754 INFO:tasks.workunit.client.1.smithi142.stdout:5/738: dwrite d2/de/d19/d5f/d68/d76/d96/dac/fc2 [0,4194304] 0 2024-06-15T01:07:09.755 INFO:tasks.workunit.client.0.smithi137.stdout:3/807: chown d0/d42/la3 46 1 2024-06-15T01:07:09.757 INFO:tasks.workunit.client.1.smithi142.stdout:0/786: chown d3/d39/d3e/d81/f17 321 1 2024-06-15T01:07:09.757 INFO:tasks.workunit.client.1.smithi142.stdout:8/825: truncate d0/d25/d5a/d8f/d98/da4/fb5 649554 0 2024-06-15T01:07:09.757 INFO:tasks.workunit.client.0.smithi137.stdout:1/584: write d0/d3/d14/f19 [2096824,8764] 0 2024-06-15T01:07:09.758 INFO:tasks.workunit.client.1.smithi142.stdout:1/831: fsync d6/d12/d80/f123 0 2024-06-15T01:07:09.758 INFO:tasks.workunit.client.0.smithi137.stdout:9/413: dread f5 [4194304,4194304] 0 2024-06-15T01:07:09.759 INFO:tasks.workunit.client.1.smithi142.stdout:6/796: write d1a/d3c/d100/d58/f7f [1491128,57064] 0 2024-06-15T01:07:09.760 INFO:tasks.workunit.client.0.smithi137.stdout:9/414: chown df/f6b 0 1 2024-06-15T01:07:09.761 INFO:tasks.workunit.client.1.smithi142.stdout:1/832: dread - d6/d12/d1c/d2c/d93/db9/f11b zero size 2024-06-15T01:07:09.761 INFO:tasks.workunit.client.0.smithi137.stdout:1/585: write d0/d10/fc1 [81975,130289] 0 2024-06-15T01:07:09.763 INFO:tasks.workunit.client.1.smithi142.stdout:6/797: write d1a/d2d/d2e/f31 [828550,79729] 0 2024-06-15T01:07:09.764 INFO:tasks.workunit.client.1.smithi142.stdout:1/833: dread - d6/d12/d80/dc8/f110 zero size 2024-06-15T01:07:09.765 INFO:tasks.workunit.client.0.smithi137.stdout:0/676: unlink d1/d2/dc/d10/d14/d15/d17/d54/lb9 0 2024-06-15T01:07:09.765 INFO:tasks.workunit.client.1.smithi142.stdout:5/739: truncate d2/de/d19/f22 5112946 0 2024-06-15T01:07:09.766 INFO:tasks.workunit.client.0.smithi137.stdout:0/677: chown d1/d2/dc/d10/l44 473228904 1 2024-06-15T01:07:09.767 INFO:tasks.workunit.client.0.smithi137.stdout:2/461: dread d0/de/d2b/f32 [0,4194304] 0 2024-06-15T01:07:09.770 INFO:tasks.workunit.client.1.smithi142.stdout:8/826: dwrite d0/d25/d5a/f85 [0,4194304] 0 2024-06-15T01:07:09.771 INFO:tasks.workunit.client.1.smithi142.stdout:1/834: mkdir d6/d12/d52/d125 0 2024-06-15T01:07:09.772 INFO:tasks.workunit.client.0.smithi137.stdout:2/462: mkdir d0/dc/d2e/d72/d8a 0 2024-06-15T01:07:09.772 INFO:tasks.workunit.client.1.smithi142.stdout:3/866: dwrite d0/d1f/d20/d5d/d60/fb0 [0,4194304] 0 2024-06-15T01:07:09.772 INFO:tasks.workunit.client.1.smithi142.stdout:5/740: read d2/de/f50 [1106852,44215] 0 2024-06-15T01:07:09.774 INFO:tasks.workunit.client.1.smithi142.stdout:1/835: stat d6/d12/d3c/d63 0 2024-06-15T01:07:09.775 INFO:tasks.workunit.client.1.smithi142.stdout:2/774: dread d8/df/d1f/f3e [0,4194304] 0 2024-06-15T01:07:09.777 INFO:tasks.workunit.client.0.smithi137.stdout:3/808: write d0/d2/d23/f3d [627577,72594] 0 2024-06-15T01:07:09.778 INFO:tasks.workunit.client.0.smithi137.stdout:2/463: fdatasync d0/de/f16 0 2024-06-15T01:07:09.780 INFO:tasks.workunit.client.0.smithi137.stdout:0/678: rename d1/d7e/d87/la8 to d1/d2/dc/d10/le6 0 2024-06-15T01:07:09.782 INFO:tasks.workunit.client.1.smithi142.stdout:6/798: truncate d1a/d3c/d100/d58/d87/f93 1555874 0 2024-06-15T01:07:09.783 INFO:tasks.workunit.client.0.smithi137.stdout:9/415: truncate f5 120884 0 2024-06-15T01:07:09.784 INFO:tasks.workunit.client.1.smithi142.stdout:6/799: stat d1a/d2d/d3b/c73 0 2024-06-15T01:07:09.784 INFO:tasks.workunit.client.1.smithi142.stdout:3/867: mkdir d0/d95/dfa/d105 0 2024-06-15T01:07:09.785 INFO:tasks.workunit.client.0.smithi137.stdout:9/416: write df/d13/f22 [1085526,42202] 0 2024-06-15T01:07:09.786 INFO:tasks.workunit.client.1.smithi142.stdout:6/800: chown d1a/d3c/db0 30194 1 2024-06-15T01:07:09.786 INFO:tasks.workunit.client.0.smithi137.stdout:0/679: dread d1/d2/d13/d66/f3f [0,4194304] 0 2024-06-15T01:07:09.788 INFO:tasks.workunit.client.0.smithi137.stdout:9/417: write df/d12/d29/d4a/f70 [220869,114110] 0 2024-06-15T01:07:09.789 INFO:tasks.workunit.client.1.smithi142.stdout:8/827: dwrite d0/d17/d40/d5e/f94 [0,4194304] 0 2024-06-15T01:07:09.789 INFO:tasks.workunit.client.1.smithi142.stdout:5/741: rename d2/d3e/d98/la3 to d2/d3e/d98/lec 0 2024-06-15T01:07:09.791 INFO:tasks.workunit.client.1.smithi142.stdout:7/816: sync 2024-06-15T01:07:09.792 INFO:tasks.workunit.client.1.smithi142.stdout:1/836: unlink d6/d12/d5b/l6e 0 2024-06-15T01:07:09.793 INFO:tasks.workunit.client.0.smithi137.stdout:9/418: rename df/d12/d15/f1d to df/d12/d1b/f74 0 2024-06-15T01:07:09.794 INFO:tasks.workunit.client.1.smithi142.stdout:0/787: dwrite d3/f2a [8388608,4194304] 0 2024-06-15T01:07:09.796 INFO:tasks.workunit.client.1.smithi142.stdout:5/742: rmdir d2/de/d19/d93 39 2024-06-15T01:07:09.797 INFO:tasks.workunit.client.1.smithi142.stdout:3/868: dwrite d0/d1f/d20/d5d/ff9 [0,4194304] 0 2024-06-15T01:07:09.798 INFO:tasks.workunit.client.1.smithi142.stdout:5/743: read - d2/de/d19/d5f/d68/fdc zero size 2024-06-15T01:07:09.799 INFO:tasks.workunit.client.1.smithi142.stdout:5/744: stat d2/d3e/d31/d7e 0 2024-06-15T01:07:09.805 INFO:tasks.workunit.client.1.smithi142.stdout:6/801: mkdir d1a/d83/d107/d11a 0 2024-06-15T01:07:09.806 INFO:tasks.workunit.client.1.smithi142.stdout:6/802: stat d1a/d2d/d80/d8e 0 2024-06-15T01:07:09.809 INFO:tasks.workunit.client.1.smithi142.stdout:7/817: mknod d2/d8/da/d33/d25/d81/c103 0 2024-06-15T01:07:09.810 INFO:tasks.workunit.client.0.smithi137.stdout:3/809: truncate d0/d2/d1e/f6e 1349598 0 2024-06-15T01:07:09.810 INFO:tasks.workunit.client.0.smithi137.stdout:1/586: dread d0/d1f/d60/da9/fac [0,4194304] 0 2024-06-15T01:07:09.811 INFO:tasks.workunit.client.1.smithi142.stdout:3/869: dread d0/d1f/f38 [0,4194304] 0 2024-06-15T01:07:09.812 INFO:tasks.workunit.client.0.smithi137.stdout:1/587: dread - d0/d3/d47/f92 zero size 2024-06-15T01:07:09.812 INFO:tasks.workunit.client.1.smithi142.stdout:3/870: chown d0/d3/de2/d55/l68 40 1 2024-06-15T01:07:09.814 INFO:tasks.workunit.client.0.smithi137.stdout:3/810: mknod d0/d2/d8/d67/d7e/c105 0 2024-06-15T01:07:09.816 INFO:tasks.workunit.client.0.smithi137.stdout:9/419: dwrite df/d12/d15/f58 [4194304,4194304] 0 2024-06-15T01:07:09.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:09 smithi142 ceph-mon[37379]: pgmap v20: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 266 MiB/s wr, 487 op/s 2024-06-15T01:07:09.820 INFO:tasks.workunit.client.1.smithi142.stdout:3/871: write d0/d3/de2/d75/ff3 [3172643,78104] 0 2024-06-15T01:07:09.822 INFO:tasks.workunit.client.1.smithi142.stdout:0/788: link d3/d39/d3e/d81/d44/d7c/d22/f36 d3/d39/d3e/d81/d4c/da0/f10c 0 2024-06-15T01:07:09.823 INFO:tasks.workunit.client.0.smithi137.stdout:3/811: rename d0/d2/d22/d24/d28 to d0/d2/d8/dbd/d106 0 2024-06-15T01:07:09.824 INFO:tasks.workunit.client.1.smithi142.stdout:9/749: sync 2024-06-15T01:07:09.824 INFO:tasks.workunit.client.1.smithi142.stdout:4/886: sync 2024-06-15T01:07:09.824 INFO:tasks.workunit.client.1.smithi142.stdout:2/775: link d8/df/d10/d1c/lee d8/lf2 0 2024-06-15T01:07:09.827 INFO:tasks.workunit.client.1.smithi142.stdout:5/745: getdents d2/d3e/d31/d7e 0 2024-06-15T01:07:09.828 INFO:tasks.workunit.client.0.smithi137.stdout:0/680: dread d1/d7e/fae [0,4194304] 0 2024-06-15T01:07:09.829 INFO:tasks.workunit.client.1.smithi142.stdout:6/803: rmdir d1a/d83/daa 39 2024-06-15T01:07:09.833 INFO:tasks.workunit.client.1.smithi142.stdout:0/789: dwrite d3/d39/d3e/fa5 [0,4194304] 0 2024-06-15T01:07:09.833 INFO:tasks.workunit.client.1.smithi142.stdout:7/818: link d2/d8/dd/dcf/fa6 d2/d8/dd/dc0/f104 0 2024-06-15T01:07:09.834 INFO:tasks.workunit.client.1.smithi142.stdout:8/828: dwrite d0/d25/d66/fb4 [0,4194304] 0 2024-06-15T01:07:09.835 INFO:tasks.workunit.client.1.smithi142.stdout:3/872: stat d0/d1b/c9d 0 2024-06-15T01:07:09.837 INFO:tasks.workunit.client.1.smithi142.stdout:3/873: chown d0/d3/da 348 1 2024-06-15T01:07:09.837 INFO:tasks.workunit.client.0.smithi137.stdout:0/681: dwrite d1/fb6 [0,4194304] 0 2024-06-15T01:07:09.842 INFO:tasks.workunit.client.1.smithi142.stdout:3/874: chown d0/d1f/d20/d5d/d60/f85 1 1 2024-06-15T01:07:09.842 INFO:tasks.workunit.client.1.smithi142.stdout:5/746: dwrite d2/d18/f55 [0,4194304] 0 2024-06-15T01:07:09.848 INFO:tasks.workunit.client.1.smithi142.stdout:7/819: dread d2/d8/da/d33/d25/d79/fb7 [0,4194304] 0 2024-06-15T01:07:09.850 INFO:tasks.workunit.client.1.smithi142.stdout:1/837: getdents d6 0 2024-06-15T01:07:09.852 INFO:tasks.workunit.client.0.smithi137.stdout:9/420: dread df/f24 [0,4194304] 0 2024-06-15T01:07:09.852 INFO:tasks.workunit.client.1.smithi142.stdout:5/747: read d2/f14 [1303105,7691] 0 2024-06-15T01:07:09.852 INFO:tasks.workunit.client.1.smithi142.stdout:7/820: fsync d2/d8/dd/f76 0 2024-06-15T01:07:09.853 INFO:tasks.workunit.client.1.smithi142.stdout:0/790: dwrite d3/d39/d9a/d9d/f9e [4194304,4194304] 0 2024-06-15T01:07:09.857 INFO:tasks.workunit.client.1.smithi142.stdout:9/750: mknod d6/dc/d1b/d6f/c113 0 2024-06-15T01:07:09.858 INFO:tasks.workunit.client.1.smithi142.stdout:8/829: dread d0/f6f [0,4194304] 0 2024-06-15T01:07:09.858 INFO:tasks.workunit.client.1.smithi142.stdout:9/751: write d6/dc/d1b/d6f/d103/f9f [1364976,70638] 0 2024-06-15T01:07:09.866 INFO:tasks.workunit.client.1.smithi142.stdout:2/776: mknod d8/df/d10/cf3 0 2024-06-15T01:07:09.866 INFO:tasks.workunit.client.0.smithi137.stdout:0/682: chown d1/d2/dc/d10/d14/d15/d17/lab 30833141 1 2024-06-15T01:07:09.866 INFO:tasks.workunit.client.0.smithi137.stdout:0/683: chown d1/d2/d13/d66/dc3/cc9 183 1 2024-06-15T01:07:09.866 INFO:tasks.workunit.client.0.smithi137.stdout:9/421: mkdir df/d12/d15/d75 0 2024-06-15T01:07:09.868 INFO:tasks.workunit.client.0.smithi137.stdout:1/588: dread d0/d3/d1b/f69 [0,4194304] 0 2024-06-15T01:07:09.869 INFO:tasks.workunit.client.0.smithi137.stdout:1/589: stat d0/d3/d47/f88 0 2024-06-15T01:07:09.869 INFO:tasks.workunit.client.1.smithi142.stdout:5/748: rmdir d2/de/d19/d5f/d68/d76/d8c 39 2024-06-15T01:07:09.870 INFO:tasks.workunit.client.0.smithi137.stdout:0/684: creat d1/d2/dc/d10/d22/db7/de3/fe7 x:0 0 0 2024-06-15T01:07:09.870 INFO:tasks.workunit.client.0.smithi137.stdout:1/590: stat d0/d3/d1b/d4c/d59/d64 0 2024-06-15T01:07:09.871 INFO:tasks.workunit.client.1.smithi142.stdout:2/777: dread - d8/df/d10/d94/d25/d1a/d63/f9a zero size 2024-06-15T01:07:09.872 INFO:tasks.workunit.client.0.smithi137.stdout:9/422: dread df/d12/d1b/f74 [0,4194304] 0 2024-06-15T01:07:09.873 INFO:tasks.workunit.client.0.smithi137.stdout:9/423: stat fd 0 2024-06-15T01:07:09.874 INFO:tasks.workunit.client.1.smithi142.stdout:4/887: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f6d [4194304,4194304] 0 2024-06-15T01:07:09.874 INFO:tasks.workunit.client.1.smithi142.stdout:1/838: creat d6/d12/d1c/d2c/d93/db9/df2/f126 x:0 0 0 2024-06-15T01:07:09.877 INFO:tasks.workunit.client.1.smithi142.stdout:7/821: truncate d2/f2e 3692061 0 2024-06-15T01:07:09.878 INFO:tasks.workunit.client.0.smithi137.stdout:0/685: creat d1/d2/dc/d10/d14/d15/d17/d54/d7a/fe8 x:0 0 0 2024-06-15T01:07:09.878 INFO:tasks.workunit.client.1.smithi142.stdout:8/830: dwrite d0/f87 [0,4194304] 0 2024-06-15T01:07:09.878 INFO:tasks.workunit.client.0.smithi137.stdout:9/424: chown df/d2a/c3a 1635397289 1 2024-06-15T01:07:09.879 INFO:tasks.workunit.client.0.smithi137.stdout:1/591: dwrite d0/d3/d47/f5d [0,4194304] 0 2024-06-15T01:07:09.881 INFO:tasks.workunit.client.1.smithi142.stdout:5/749: write d2/de/d19/d5f/d68/d76/f81 [1819682,29579] 0 2024-06-15T01:07:09.882 INFO:tasks.workunit.client.1.smithi142.stdout:8/831: stat d0/d17/l1e 0 2024-06-15T01:07:09.882 INFO:tasks.workunit.client.1.smithi142.stdout:3/875: dwrite d0/d1f/d20/d5d/fc3 [0,4194304] 0 2024-06-15T01:07:09.883 INFO:tasks.workunit.client.0.smithi137.stdout:0/686: chown d1/d7e/d87/da0/dc4/dd7/de2/d77 164 1 2024-06-15T01:07:09.887 INFO:tasks.workunit.client.0.smithi137.stdout:9/425: creat df/d1a/f76 x:0 0 0 2024-06-15T01:07:09.890 INFO:tasks.workunit.client.1.smithi142.stdout:9/752: mknod d6/dc/d1b/d6f/de1/d110/c114 0 2024-06-15T01:07:09.891 INFO:tasks.workunit.client.0.smithi137.stdout:0/687: rename d1/d2/dc/d10/l7b to d1/d7e/d87/da0/le9 0 2024-06-15T01:07:09.892 INFO:tasks.workunit.client.0.smithi137.stdout:0/688: chown d1/d7e/d87/da0 58 1 2024-06-15T01:07:09.892 INFO:tasks.workunit.client.0.smithi137.stdout:1/592: dwrite d0/d2d/f62 [0,4194304] 0 2024-06-15T01:07:09.893 INFO:tasks.workunit.client.0.smithi137.stdout:9/426: symlink df/d13/l77 0 2024-06-15T01:07:09.895 INFO:tasks.workunit.client.1.smithi142.stdout:2/778: dwrite d8/df/daf/fcd [0,4194304] 0 2024-06-15T01:07:09.896 INFO:tasks.workunit.client.1.smithi142.stdout:4/888: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/d137 0 2024-06-15T01:07:09.899 INFO:tasks.workunit.client.1.smithi142.stdout:8/832: stat d0/d25/c36 0 2024-06-15T01:07:09.901 INFO:tasks.workunit.client.1.smithi142.stdout:3/876: symlink d0/d95/l106 0 2024-06-15T01:07:09.903 INFO:tasks.workunit.client.1.smithi142.stdout:7/822: dwrite d2/d8/da/d33/fd7 [0,4194304] 0 2024-06-15T01:07:09.906 INFO:tasks.workunit.client.0.smithi137.stdout:9/427: link df/l14 df/l78 0 2024-06-15T01:07:09.907 INFO:tasks.workunit.client.1.smithi142.stdout:3/877: write d0/d1b/f91 [1284055,60714] 0 2024-06-15T01:07:09.908 INFO:tasks.workunit.client.1.smithi142.stdout:5/750: creat d2/de/d19/d5f/d68/d76/d96/dac/fed x:0 0 0 2024-06-15T01:07:09.908 INFO:tasks.workunit.client.0.smithi137.stdout:1/593: getdents d0/d1f/d60/d99 0 2024-06-15T01:07:09.910 INFO:tasks.workunit.client.0.smithi137.stdout:9/428: mkdir df/d12/d29/d79 0 2024-06-15T01:07:09.910 INFO:tasks.workunit.client.0.smithi137.stdout:3/812: dwrite d0/d2/d22/d63/d76/f7d [0,4194304] 0 2024-06-15T01:07:09.911 INFO:tasks.workunit.client.1.smithi142.stdout:9/753: mkdir d6/dc/d1b/d6f/d103/df2/d115 0 2024-06-15T01:07:09.911 INFO:tasks.workunit.client.1.smithi142.stdout:8/833: rmdir d0/d17/d40/d86 39 2024-06-15T01:07:09.911 INFO:tasks.workunit.client.1.smithi142.stdout:2/779: truncate d8/df/d10/d94/d25/d22/fbd 1562032 0 2024-06-15T01:07:09.912 INFO:tasks.workunit.client.1.smithi142.stdout:1/839: creat d6/d46/f127 x:0 0 0 2024-06-15T01:07:09.913 INFO:tasks.workunit.client.1.smithi142.stdout:6/804: write f0 [1147000,62728] 0 2024-06-15T01:07:09.913 INFO:tasks.workunit.client.1.smithi142.stdout:9/754: chown d6/dc/d1b/d25 1458 1 2024-06-15T01:07:09.914 INFO:tasks.workunit.client.1.smithi142.stdout:2/780: chown d8/df/d10/d1c/dbb 374873 1 2024-06-15T01:07:09.917 INFO:tasks.workunit.client.0.smithi137.stdout:1/594: rename d0/d10/cf to d0/d1f/d60/d99/da3/cd0 0 2024-06-15T01:07:09.918 INFO:tasks.workunit.client.0.smithi137.stdout:3/813: dread d0/d13/fba [0,4194304] 0 2024-06-15T01:07:09.919 INFO:tasks.workunit.client.1.smithi142.stdout:3/878: creat d0/d3/de2/d75/f107 x:0 0 0 2024-06-15T01:07:09.919 INFO:tasks.workunit.client.0.smithi137.stdout:1/595: readlink d0/d3/d1b/d4c/d59/d93/lce 0 2024-06-15T01:07:09.920 INFO:tasks.workunit.client.0.smithi137.stdout:9/429: mknod df/d12/d15/d16/d3c/d51/c7a 0 2024-06-15T01:07:09.921 INFO:tasks.workunit.client.0.smithi137.stdout:3/814: mknod d0/d2/d23/de2/c107 0 2024-06-15T01:07:09.922 INFO:tasks.workunit.client.1.smithi142.stdout:5/751: chown d2/d3e/d31/l5b 584 1 2024-06-15T01:07:09.922 INFO:tasks.workunit.client.1.smithi142.stdout:8/834: mknod d0/d25/d5a/d8f/df3/cf9 0 2024-06-15T01:07:09.923 INFO:tasks.workunit.client.1.smithi142.stdout:1/840: rmdir d6/d12/d80/dae 39 2024-06-15T01:07:09.924 INFO:tasks.workunit.client.0.smithi137.stdout:3/815: symlink d0/d2/d8/l108 0 2024-06-15T01:07:09.925 INFO:tasks.workunit.client.0.smithi137.stdout:3/816: stat d0/d42/ff7 0 2024-06-15T01:07:09.925 INFO:tasks.workunit.client.0.smithi137.stdout:9/430: creat df/d13/d6f/f7b x:0 0 0 2024-06-15T01:07:09.926 INFO:tasks.workunit.client.1.smithi142.stdout:6/805: dwrite d1a/d39/d3a/dd2/ffa [0,4194304] 0 2024-06-15T01:07:09.926 INFO:tasks.workunit.client.0.smithi137.stdout:9/431: chown df/f28 3476888 1 2024-06-15T01:07:09.926 INFO:tasks.workunit.client.0.smithi137.stdout:3/817: chown d0/d2/d8/d67/lc1 1873638 1 2024-06-15T01:07:09.927 INFO:tasks.workunit.client.1.smithi142.stdout:2/781: truncate d8/d19/fe7 24666 0 2024-06-15T01:07:09.928 INFO:tasks.workunit.client.1.smithi142.stdout:0/791: write d3/d39/d3e/d81/f8d [1409703,52289] 0 2024-06-15T01:07:09.929 INFO:tasks.workunit.client.0.smithi137.stdout:3/818: creat d0/d13/d37/d83/f109 x:0 0 0 2024-06-15T01:07:09.931 INFO:tasks.workunit.client.0.smithi137.stdout:3/819: symlink d0/d2f/l10a 0 2024-06-15T01:07:09.932 INFO:tasks.workunit.client.1.smithi142.stdout:3/879: dwrite d0/d1b/f91 [0,4194304] 0 2024-06-15T01:07:09.937 INFO:tasks.workunit.client.1.smithi142.stdout:1/841: creat d6/d12/d3c/d63/d8d/d97/dc6/dfe/f128 x:0 0 0 2024-06-15T01:07:09.939 INFO:tasks.workunit.client.1.smithi142.stdout:8/835: fsync d0/d25/f9e 0 2024-06-15T01:07:09.939 INFO:tasks.workunit.client.1.smithi142.stdout:2/782: unlink d8/df/d10/d94/d25/f53 0 2024-06-15T01:07:09.940 INFO:tasks.workunit.client.1.smithi142.stdout:6/806: symlink d1a/d83/l11b 0 2024-06-15T01:07:09.940 INFO:tasks.workunit.client.1.smithi142.stdout:5/752: dread d2/de/d19/d5f/d68/d76/f7c [0,4194304] 0 2024-06-15T01:07:09.942 INFO:tasks.workunit.client.1.smithi142.stdout:7/823: link d2/da2/cfa d2/d8/dd/d14/d2c/d37/c105 0 2024-06-15T01:07:09.945 INFO:tasks.workunit.client.1.smithi142.stdout:1/842: creat d6/d12/d3c/d63/d8d/d97/f129 x:0 0 0 2024-06-15T01:07:09.945 INFO:tasks.workunit.client.1.smithi142.stdout:0/792: dwrite d3/d39/d3e/d81/d44/d7c/d22/f70 [0,4194304] 0 2024-06-15T01:07:09.949 INFO:tasks.workunit.client.1.smithi142.stdout:1/843: dread - d6/d25/dfc/fd6 zero size 2024-06-15T01:07:09.949 INFO:tasks.workunit.client.1.smithi142.stdout:7/824: truncate d2/d6/d22/d47/de5/f100 311536 0 2024-06-15T01:07:09.950 INFO:tasks.workunit.client.1.smithi142.stdout:5/753: symlink d2/de/d19/d5f/d68/d76/d8c/lee 0 2024-06-15T01:07:09.952 INFO:tasks.workunit.client.1.smithi142.stdout:1/844: read - d6/d12/d5b/f11e zero size 2024-06-15T01:07:09.953 INFO:tasks.workunit.client.1.smithi142.stdout:7/825: fsync d2/d8/da/d33/ffc 0 2024-06-15T01:07:09.956 INFO:tasks.workunit.client.1.smithi142.stdout:0/793: dread d3/d39/d3e/d81/f7e [0,4194304] 0 2024-06-15T01:07:09.956 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:09 smithi137 ceph-mon[29354]: pgmap v20: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 266 MiB/s wr, 487 op/s 2024-06-15T01:07:09.956 INFO:tasks.workunit.client.1.smithi142.stdout:6/807: dwrite d1a/d39/d42/f45 [0,4194304] 0 2024-06-15T01:07:09.958 INFO:tasks.workunit.client.1.smithi142.stdout:1/845: fdatasync d6/d12/d80/dc8/fda 0 2024-06-15T01:07:09.960 INFO:tasks.workunit.client.1.smithi142.stdout:8/836: getdents d0/d25/d5a/d8f/d98/da4 0 2024-06-15T01:07:09.962 INFO:tasks.workunit.client.1.smithi142.stdout:7/826: creat d2/db6/f106 x:0 0 0 2024-06-15T01:07:09.962 INFO:tasks.workunit.client.1.smithi142.stdout:4/889: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f43 [0,4194304] 0 2024-06-15T01:07:09.963 INFO:tasks.workunit.client.0.smithi137.stdout:0/689: truncate d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d 5465933 0 2024-06-15T01:07:09.964 INFO:tasks.workunit.client.0.smithi137.stdout:0/690: chown d1/d2/dc/d10/dd4 133654796 1 2024-06-15T01:07:09.965 INFO:tasks.workunit.client.1.smithi142.stdout:2/783: dread d8/df/d10/d94/d25/fda [0,4194304] 0 2024-06-15T01:07:09.969 INFO:tasks.workunit.client.0.smithi137.stdout:1/596: dwrite d0/d3/d1b/d4c/f95 [0,4194304] 0 2024-06-15T01:07:09.986 INFO:tasks.workunit.client.1.smithi142.stdout:0/794: creat d3/d39/d9a/d9d/dbf/f10d x:0 0 0 2024-06-15T01:07:09.987 INFO:tasks.workunit.client.1.smithi142.stdout:1/846: symlink d6/d12/d3c/d63/d65/l12a 0 2024-06-15T01:07:09.987 INFO:tasks.workunit.client.1.smithi142.stdout:0/795: readlink d3/d39/d3e/d81/d1e/d98/le5 0 2024-06-15T01:07:09.987 INFO:tasks.workunit.client.1.smithi142.stdout:7/827: symlink d2/d8/dd/d8a/d94/l107 0 2024-06-15T01:07:09.989 INFO:tasks.workunit.client.1.smithi142.stdout:4/890: mkdir d5/d1c/d6f/db9/d138 0 2024-06-15T01:07:09.989 INFO:tasks.workunit.client.1.smithi142.stdout:6/808: mknod d1a/d83/d107/d11a/c11c 0 2024-06-15T01:07:09.991 INFO:tasks.workunit.client.1.smithi142.stdout:9/755: dwrite d6/dc/d4e/d78/f10d [4194304,4194304] 0 2024-06-15T01:07:09.991 INFO:tasks.workunit.client.1.smithi142.stdout:2/784: unlink d8/df/d10/d94/d25/cca 0 2024-06-15T01:07:09.992 INFO:tasks.workunit.client.1.smithi142.stdout:9/756: fsync d6/dc/d1b/d25/d58/fd6 0 2024-06-15T01:07:09.992 INFO:tasks.workunit.client.1.smithi142.stdout:8/837: truncate d0/f4c 1314969 0 2024-06-15T01:07:09.994 INFO:tasks.workunit.client.1.smithi142.stdout:1/847: symlink d6/d4d/l12b 0 2024-06-15T01:07:09.995 INFO:tasks.workunit.client.1.smithi142.stdout:7/828: mkdir d2/d8/da/d33/d25/d79/dd4/d108 0 2024-06-15T01:07:09.998 INFO:tasks.workunit.client.1.smithi142.stdout:6/809: dread d1a/d2d/f65 [0,4194304] 0 2024-06-15T01:07:10.001 INFO:tasks.workunit.client.1.smithi142.stdout:3/880: dwrite d0/d1f/d20/d5d/d60/f85 [0,4194304] 0 2024-06-15T01:07:10.002 INFO:tasks.workunit.client.1.smithi142.stdout:5/754: truncate d2/d18/fba 3553362 0 2024-06-15T01:07:10.004 INFO:tasks.workunit.client.1.smithi142.stdout:4/891: mknod d5/d9/d23/d32/d11f/c139 0 2024-06-15T01:07:10.004 INFO:tasks.workunit.client.1.smithi142.stdout:7/829: mknod d2/db6/de6/c109 0 2024-06-15T01:07:10.006 INFO:tasks.workunit.client.1.smithi142.stdout:7/830: chown d2/d8/dd/fd9 1748225 1 2024-06-15T01:07:10.007 INFO:tasks.workunit.client.1.smithi142.stdout:2/785: mkdir d8/df4 0 2024-06-15T01:07:10.012 INFO:tasks.workunit.client.1.smithi142.stdout:8/838: truncate d0/d25/d53/d9c/fce 1742206 0 2024-06-15T01:07:10.014 INFO:tasks.workunit.client.1.smithi142.stdout:8/839: read d0/d25/d5a/f88 [130741,101447] 0 2024-06-15T01:07:10.015 INFO:tasks.workunit.client.1.smithi142.stdout:6/810: dread f0 [0,4194304] 0 2024-06-15T01:07:10.016 INFO:tasks.workunit.client.1.smithi142.stdout:5/755: dread d2/de/d27/f9e [0,4194304] 0 2024-06-15T01:07:10.017 INFO:tasks.workunit.client.1.smithi142.stdout:5/756: chown d2/d18/d2e/dae 30220 1 2024-06-15T01:07:10.018 INFO:tasks.workunit.client.1.smithi142.stdout:5/757: dread - d2/d3e/feb zero size 2024-06-15T01:07:10.019 INFO:tasks.workunit.client.1.smithi142.stdout:9/757: mkdir d6/d34/df3/d116 0 2024-06-15T01:07:10.019 INFO:tasks.workunit.client.1.smithi142.stdout:3/881: truncate d0/d3/de2/f97 1820150 0 2024-06-15T01:07:10.020 INFO:tasks.workunit.client.1.smithi142.stdout:8/840: rmdir d0/d17/d40 39 2024-06-15T01:07:10.023 INFO:tasks.workunit.client.1.smithi142.stdout:0/796: rename d3/d39/d3e/d81/d44/d7c/d22/d59/f94 to d3/d39/d3e/d81/d1e/f10e 0 2024-06-15T01:07:10.025 INFO:tasks.workunit.client.1.smithi142.stdout:0/797: dread d3/d39/d3e/d81/f7e [0,4194304] 0 2024-06-15T01:07:10.029 INFO:tasks.workunit.client.1.smithi142.stdout:9/758: readlink d6/dc/d1b/d6f/lc3 0 2024-06-15T01:07:10.030 INFO:tasks.workunit.client.1.smithi142.stdout:3/882: symlink d0/d1f/d20/l108 0 2024-06-15T01:07:10.031 INFO:tasks.workunit.client.1.smithi142.stdout:0/798: dread d3/d39/d3e/d81/f8d [0,4194304] 0 2024-06-15T01:07:10.033 INFO:tasks.workunit.client.1.smithi142.stdout:2/786: write d8/df/f2d [130531,18685] 0 2024-06-15T01:07:10.034 INFO:tasks.workunit.client.1.smithi142.stdout:1/848: dwrite d6/d12/d80/dae/fb3 [0,4194304] 0 2024-06-15T01:07:10.034 INFO:tasks.workunit.client.1.smithi142.stdout:2/787: fsync d8/fd4 0 2024-06-15T01:07:10.035 INFO:tasks.workunit.client.1.smithi142.stdout:2/788: chown d8/df/d10/d94/d25/da9 0 1 2024-06-15T01:07:10.036 INFO:tasks.workunit.client.1.smithi142.stdout:7/831: creat d2/d8/dd/d14/d2c/f10a x:0 0 0 2024-06-15T01:07:10.036 INFO:tasks.workunit.client.1.smithi142.stdout:6/811: truncate d1a/f3e 433118 0 2024-06-15T01:07:10.037 INFO:tasks.workunit.client.1.smithi142.stdout:7/832: write d2/d8/dd/d14/d2c/f10a [219629,20013] 0 2024-06-15T01:07:10.038 INFO:tasks.workunit.client.1.smithi142.stdout:5/758: creat d2/de/d19/d93/fef x:0 0 0 2024-06-15T01:07:10.038 INFO:tasks.workunit.client.1.smithi142.stdout:7/833: chown d2/db6/de6 56634189 1 2024-06-15T01:07:10.040 INFO:tasks.workunit.client.1.smithi142.stdout:3/883: mkdir d0/d3/de2/d109 0 2024-06-15T01:07:10.040 INFO:tasks.workunit.client.1.smithi142.stdout:5/759: write d2/f21 [71778,18753] 0 2024-06-15T01:07:10.041 INFO:tasks.workunit.client.1.smithi142.stdout:4/892: getdents d5/d1c/d6f/dc7/d30/dcc/d11a 0 2024-06-15T01:07:10.045 INFO:tasks.workunit.client.1.smithi142.stdout:1/849: rmdir d6/d12/d1c/d2c/d93/db9/df2 39 2024-06-15T01:07:10.048 INFO:tasks.workunit.client.1.smithi142.stdout:5/760: dread d2/f12 [0,4194304] 0 2024-06-15T01:07:10.051 INFO:tasks.workunit.client.1.smithi142.stdout:6/812: rename d1a/d39/d3a/la2 to d1a/d2d/d80/d8e/d110/da0/l11d 0 2024-06-15T01:07:10.051 INFO:tasks.workunit.client.1.smithi142.stdout:8/841: dread d0/f15 [0,4194304] 0 2024-06-15T01:07:10.053 INFO:tasks.workunit.client.1.smithi142.stdout:2/789: truncate d8/f16 679828 0 2024-06-15T01:07:10.054 INFO:tasks.workunit.client.1.smithi142.stdout:6/813: chown d1a/d39/d3a/dd2/ff8 26381481 1 2024-06-15T01:07:10.055 INFO:tasks.workunit.client.1.smithi142.stdout:5/761: creat d2/de/d27/db2/ff0 x:0 0 0 2024-06-15T01:07:10.055 INFO:tasks.workunit.client.1.smithi142.stdout:7/834: rename d2/d8/da to d2/d8/da/d33/d25/d10b 22 2024-06-15T01:07:10.055 INFO:tasks.workunit.client.1.smithi142.stdout:1/850: dwrite d6/d12/d80/d9b/db2/dc2/f107 [4194304,4194304] 0 2024-06-15T01:07:10.056 INFO:tasks.workunit.client.1.smithi142.stdout:8/842: write d0/d17/f27 [3186233,118805] 0 2024-06-15T01:07:10.058 INFO:tasks.workunit.client.1.smithi142.stdout:0/799: creat d3/d39/d3e/d81/d1b/f10f x:0 0 0 2024-06-15T01:07:10.060 INFO:tasks.workunit.client.1.smithi142.stdout:2/790: write d8/fc [1391619,65562] 0 2024-06-15T01:07:10.063 INFO:tasks.workunit.client.1.smithi142.stdout:9/759: dwrite d6/d31/f84 [0,4194304] 0 2024-06-15T01:07:10.063 INFO:tasks.workunit.client.1.smithi142.stdout:7/835: rmdir d2/db6 39 2024-06-15T01:07:10.065 INFO:tasks.workunit.client.1.smithi142.stdout:2/791: write d8/df/d10/d94/d25/d22/fdc [166055,29122] 0 2024-06-15T01:07:10.074 INFO:tasks.workunit.client.1.smithi142.stdout:4/893: truncate d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/f6b 3385231 0 2024-06-15T01:07:10.075 INFO:tasks.workunit.client.1.smithi142.stdout:4/894: fsync d5/f10 0 2024-06-15T01:07:10.075 INFO:tasks.workunit.client.1.smithi142.stdout:4/895: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/fce zero size 2024-06-15T01:07:10.075 INFO:tasks.workunit.client.1.smithi142.stdout:6/814: write d1a/f28 [352554,128364] 0 2024-06-15T01:07:10.080 INFO:tasks.workunit.client.1.smithi142.stdout:6/815: rmdir d1a/d2d/d2e 39 2024-06-15T01:07:10.082 INFO:tasks.workunit.client.1.smithi142.stdout:5/762: dwrite d2/de/d19/d5f/fe9 [0,4194304] 0 2024-06-15T01:07:10.084 INFO:tasks.workunit.client.1.smithi142.stdout:9/760: creat d6/dc/d82/f117 x:0 0 0 2024-06-15T01:07:10.085 INFO:tasks.workunit.client.1.smithi142.stdout:7/836: getdents d2/d8/da/d33/d25 0 2024-06-15T01:07:10.086 INFO:tasks.workunit.client.1.smithi142.stdout:6/816: fsync d1a/d39/d3a/d51/d62/fd1 0 2024-06-15T01:07:10.088 INFO:tasks.workunit.client.1.smithi142.stdout:8/843: dwrite d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:10.088 INFO:tasks.workunit.client.1.smithi142.stdout:1/851: link d6/d12/d1c/l6a d6/d12/da0/db0/l12c 0 2024-06-15T01:07:10.091 INFO:tasks.workunit.client.1.smithi142.stdout:5/763: unlink d2/de/f66 0 2024-06-15T01:07:10.092 INFO:tasks.workunit.client.1.smithi142.stdout:1/852: readlink d6/d25/l27 0 2024-06-15T01:07:10.092 INFO:tasks.workunit.client.1.smithi142.stdout:7/837: chown d2/db6/de6 26036 1 2024-06-15T01:07:10.094 INFO:tasks.workunit.client.1.smithi142.stdout:8/844: read - d0/d25/d5a/d84/fa0 zero size 2024-06-15T01:07:10.095 INFO:tasks.workunit.client.1.smithi142.stdout:9/761: creat d6/dc/d1b/d6f/da1/df6/f118 x:0 0 0 2024-06-15T01:07:10.096 INFO:tasks.workunit.client.1.smithi142.stdout:5/764: chown d2/de/d27/f72 31582 1 2024-06-15T01:07:10.097 INFO:tasks.workunit.client.1.smithi142.stdout:7/838: write d2/d6/f8f [3512305,98112] 0 2024-06-15T01:07:10.100 INFO:tasks.workunit.client.1.smithi142.stdout:1/853: fsync d6/d12/d80/d9b/db2/dbf/f104 0 2024-06-15T01:07:10.102 INFO:tasks.workunit.client.1.smithi142.stdout:0/800: dread d3/d39/d3e/d81/fb [0,4194304] 0 2024-06-15T01:07:10.102 INFO:tasks.workunit.client.1.smithi142.stdout:8/845: rename d0/f47 to d0/d25/d5a/d8f/d98/ffa 0 2024-06-15T01:07:10.103 INFO:tasks.workunit.client.1.smithi142.stdout:2/792: dread d8/df/d10/d94/d25/f18 [0,4194304] 0 2024-06-15T01:07:10.103 INFO:tasks.workunit.client.1.smithi142.stdout:5/765: creat d2/d3e/d65/dd6/ff1 x:0 0 0 2024-06-15T01:07:10.106 INFO:tasks.workunit.client.1.smithi142.stdout:6/817: dwrite d1a/d39/d42/fbf [0,4194304] 0 2024-06-15T01:07:10.106 INFO:tasks.workunit.client.1.smithi142.stdout:9/762: creat d6/dc/dd/d22/f119 x:0 0 0 2024-06-15T01:07:10.106 INFO:tasks.workunit.client.1.smithi142.stdout:7/839: creat d2/d8/da/d33/d25/d81/f10c x:0 0 0 2024-06-15T01:07:10.108 INFO:tasks.workunit.client.1.smithi142.stdout:8/846: mkdir d0/d25/d53/da7/dcb/dfb 0 2024-06-15T01:07:10.111 INFO:tasks.workunit.client.1.smithi142.stdout:6/818: chown d1a/d39/d3a/c64 276 1 2024-06-15T01:07:10.113 INFO:tasks.workunit.client.1.smithi142.stdout:2/793: rename d8/df/d10/cf3 to d8/d19/deb/cf5 0 2024-06-15T01:07:10.115 INFO:tasks.workunit.client.1.smithi142.stdout:0/801: symlink d3/d39/d9a/d9d/df8/l110 0 2024-06-15T01:07:10.116 INFO:tasks.workunit.client.1.smithi142.stdout:1/854: creat d6/d3a/f12d x:0 0 0 2024-06-15T01:07:10.118 INFO:tasks.workunit.client.1.smithi142.stdout:9/763: rename d6/d31/fc4 to d6/d34/d8a/f11a 0 2024-06-15T01:07:10.122 INFO:tasks.workunit.client.1.smithi142.stdout:6/819: dwrite d1a/d3c/fc8 [0,4194304] 0 2024-06-15T01:07:10.122 INFO:tasks.workunit.client.1.smithi142.stdout:2/794: getdents d8 0 2024-06-15T01:07:10.127 INFO:tasks.workunit.client.1.smithi142.stdout:9/764: creat d6/dc/d1b/d6f/d103/df2/d115/f11b x:0 0 0 2024-06-15T01:07:10.137 INFO:tasks.workunit.client.1.smithi142.stdout:0/802: dwrite d3/d39/d3e/d81/d4c/da0/fb2 [4194304,4194304] 0 2024-06-15T01:07:10.137 INFO:tasks.workunit.client.1.smithi142.stdout:5/766: dread d2/d18/fba [0,4194304] 0 2024-06-15T01:07:10.138 INFO:tasks.workunit.client.1.smithi142.stdout:6/820: dwrite d1a/d39/ded/f101 [0,4194304] 0 2024-06-15T01:07:10.139 INFO:tasks.workunit.client.1.smithi142.stdout:2/795: getdents d8/df/d10/d94/d25/d22 0 2024-06-15T01:07:10.140 INFO:tasks.workunit.client.1.smithi142.stdout:8/847: dread d0/d25/d66/f92 [0,4194304] 0 2024-06-15T01:07:10.142 INFO:tasks.workunit.client.1.smithi142.stdout:7/840: dread d2/d6/d22/d47/f50 [0,4194304] 0 2024-06-15T01:07:10.143 INFO:tasks.workunit.client.1.smithi142.stdout:2/796: write d8/d19/d33/f83 [4158172,107583] 0 2024-06-15T01:07:10.146 INFO:tasks.workunit.client.1.smithi142.stdout:1/855: truncate d6/d12/d80/dae/fb3 157780 0 2024-06-15T01:07:10.147 INFO:tasks.workunit.client.1.smithi142.stdout:8/848: write d0/d17/f27 [1417460,28017] 0 2024-06-15T01:07:10.150 INFO:tasks.workunit.client.1.smithi142.stdout:9/765: dread d6/dc/d4e/d78/f10d [4194304,4194304] 0 2024-06-15T01:07:10.150 INFO:tasks.workunit.client.1.smithi142.stdout:2/797: creat d8/d19/ff6 x:0 0 0 2024-06-15T01:07:10.151 INFO:tasks.workunit.client.1.smithi142.stdout:8/849: mknod d0/d25/d5a/d8f/d98/cfc 0 2024-06-15T01:07:10.152 INFO:tasks.workunit.client.1.smithi142.stdout:1/856: mkdir d6/d12/d3c/d63/d8d/d97/dc6/d12e 0 2024-06-15T01:07:10.152 INFO:tasks.workunit.client.1.smithi142.stdout:7/841: chown d2/d8/da/d33/d25/d79/fb1 37 1 2024-06-15T01:07:10.152 INFO:tasks.workunit.client.1.smithi142.stdout:6/821: getdents d1a/d3c/d105 0 2024-06-15T01:07:10.154 INFO:tasks.workunit.client.1.smithi142.stdout:9/766: read - d6/dc/d1b/d6f/d103/df2/d115/f11b zero size 2024-06-15T01:07:10.154 INFO:tasks.workunit.client.1.smithi142.stdout:1/857: stat d6/d12/d3c/d63/d65 0 2024-06-15T01:07:10.155 INFO:tasks.workunit.client.1.smithi142.stdout:6/822: dread - d1a/d2d/d80/d8e/d110/da0/ff3 zero size 2024-06-15T01:07:10.157 INFO:tasks.workunit.client.1.smithi142.stdout:8/850: dread d0/d25/d66/f92 [0,4194304] 0 2024-06-15T01:07:10.160 INFO:tasks.workunit.client.1.smithi142.stdout:1/858: dwrite d6/f13 [4194304,4194304] 0 2024-06-15T01:07:10.166 INFO:tasks.workunit.client.1.smithi142.stdout:1/859: dwrite d6/d12/d80/fdf [0,4194304] 0 2024-06-15T01:07:10.169 INFO:tasks.workunit.client.1.smithi142.stdout:5/767: truncate d2/de/d19/d5f/d68/d76/f7c 928333 0 2024-06-15T01:07:10.171 INFO:tasks.workunit.client.1.smithi142.stdout:2/798: rename d8/df/d10/d94/db5/fc3 to d8/df/d10/d94/d25/ff7 0 2024-06-15T01:07:10.173 INFO:tasks.workunit.client.1.smithi142.stdout:2/799: fsync d8/df/d10/d94/d25/d22/d43/d51/fd2 0 2024-06-15T01:07:10.174 INFO:tasks.workunit.client.1.smithi142.stdout:7/842: read d2/d8/dd/dcf/f7b [2862635,4598] 0 2024-06-15T01:07:10.179 INFO:tasks.workunit.client.1.smithi142.stdout:5/768: mkdir d2/d3e/df2 0 2024-06-15T01:07:10.180 INFO:tasks.workunit.client.1.smithi142.stdout:9/767: truncate d6/f9 801934 0 2024-06-15T01:07:10.181 INFO:tasks.workunit.client.1.smithi142.stdout:1/860: creat d6/d12/d3c/d63/d8d/def/f12f x:0 0 0 2024-06-15T01:07:10.184 INFO:tasks.workunit.client.1.smithi142.stdout:7/843: unlink d2/d8/dd/dc0/le7 0 2024-06-15T01:07:10.186 INFO:tasks.workunit.client.0.smithi137.stdout:1/597: write d0/d3/d14/d34/f52 [229509,48067] 0 2024-06-15T01:07:10.188 INFO:tasks.workunit.client.1.smithi142.stdout:9/768: fsync d6/dc/d4e/d78/fb3 0 2024-06-15T01:07:10.189 INFO:tasks.workunit.client.0.smithi137.stdout:3/820: dwrite d0/d2f/dbe/fe0 [0,4194304] 0 2024-06-15T01:07:10.194 INFO:tasks.workunit.client.0.smithi137.stdout:3/821: getdents d0/d2/d23/de2 0 2024-06-15T01:07:10.195 INFO:tasks.workunit.client.0.smithi137.stdout:0/691: dwrite d1/d2/dc/d10/d14/d15/d17/d54/d7a/f7d [4194304,4194304] 0 2024-06-15T01:07:10.197 INFO:tasks.workunit.client.1.smithi142.stdout:9/769: mkdir d6/dc/d51/d11c 0 2024-06-15T01:07:10.198 INFO:tasks.workunit.client.0.smithi137.stdout:3/822: write d0/d2/d8/dbd/ff6 [2634606,73966] 0 2024-06-15T01:07:10.198 INFO:tasks.workunit.client.0.smithi137.stdout:0/692: mknod d1/d2/dc/d10/d14/d52/cea 0 2024-06-15T01:07:10.198 INFO:tasks.workunit.client.0.smithi137.stdout:3/823: fsync d0/d56/d8f/d7b/fd3 0 2024-06-15T01:07:10.199 INFO:tasks.workunit.client.1.smithi142.stdout:9/770: dread - d6/dc/d1b/d25/d58/de2/de7/f106 zero size 2024-06-15T01:07:10.199 INFO:tasks.workunit.client.1.smithi142.stdout:9/771: stat d6/dc/d51 0 2024-06-15T01:07:10.200 INFO:tasks.workunit.client.1.smithi142.stdout:8/851: link d0/d17/l51 d0/d17/d40/d5e/lfd 0 2024-06-15T01:07:10.202 INFO:tasks.workunit.client.0.smithi137.stdout:0/693: dread d1/f9 [0,4194304] 0 2024-06-15T01:07:10.203 INFO:tasks.workunit.client.1.smithi142.stdout:9/772: symlink d6/dc/d1b/d6f/d103/l11d 0 2024-06-15T01:07:10.204 INFO:tasks.workunit.client.0.smithi137.stdout:3/824: link d0/d2f/faa d0/d2/d22/d24/f10b 0 2024-06-15T01:07:10.204 INFO:tasks.workunit.client.0.smithi137.stdout:0/694: mknod d1/d2/d55/ceb 0 2024-06-15T01:07:10.205 INFO:tasks.workunit.client.1.smithi142.stdout:5/769: getdents d2/d18 0 2024-06-15T01:07:10.206 INFO:tasks.workunit.client.0.smithi137.stdout:0/695: write d1/d2/dc/f73 [4364387,117282] 0 2024-06-15T01:07:10.206 INFO:tasks.workunit.client.1.smithi142.stdout:9/773: read - d6/dc/d4e/d78/fd2 zero size 2024-06-15T01:07:10.207 INFO:tasks.workunit.client.1.smithi142.stdout:6/823: dwrite d1a/d39/d3a/d51/f5c [0,4194304] 0 2024-06-15T01:07:10.210 INFO:tasks.workunit.client.1.smithi142.stdout:9/774: creat d6/dc/d4e/f11e x:0 0 0 2024-06-15T01:07:10.211 INFO:tasks.workunit.client.0.smithi137.stdout:0/696: dread d1/f9 [0,4194304] 0 2024-06-15T01:07:10.212 INFO:tasks.workunit.client.1.smithi142.stdout:6/824: rename d1a/d2d/d80/d8e/fd4 to d1a/d83/d107/d11a/f11e 0 2024-06-15T01:07:10.213 INFO:tasks.workunit.client.1.smithi142.stdout:6/825: stat d1a/fdc 0 2024-06-15T01:07:10.215 INFO:tasks.workunit.client.1.smithi142.stdout:9/775: mknod d6/d34/d8a/c11f 0 2024-06-15T01:07:10.216 INFO:tasks.workunit.client.1.smithi142.stdout:5/770: getdents d2/d3e/d74/da5 0 2024-06-15T01:07:10.217 INFO:tasks.workunit.client.0.smithi137.stdout:0/697: unlink d1/d2/dc/d6c/d90/lc2 0 2024-06-15T01:07:10.217 INFO:tasks.workunit.client.1.smithi142.stdout:5/771: write d2/d18/f55 [2808790,55988] 0 2024-06-15T01:07:10.218 INFO:tasks.workunit.client.0.smithi137.stdout:3/825: dread d0/d13/d37/d66/f88 [0,4194304] 0 2024-06-15T01:07:10.218 INFO:tasks.workunit.client.1.smithi142.stdout:5/772: stat d2/d3e/d31/d7e 0 2024-06-15T01:07:10.219 INFO:tasks.workunit.client.1.smithi142.stdout:6/826: link d1a/d39/d3a/d51/l71 d1a/d3c/d100/d58/d87/l11f 0 2024-06-15T01:07:10.220 INFO:tasks.workunit.client.0.smithi137.stdout:0/698: dread d1/d7e/fae [0,4194304] 0 2024-06-15T01:07:10.222 INFO:tasks.workunit.client.1.smithi142.stdout:5/773: truncate d2/de/d27/db2/ff0 843800 0 2024-06-15T01:07:10.222 INFO:tasks.workunit.client.1.smithi142.stdout:3/884: sync 2024-06-15T01:07:10.223 INFO:tasks.workunit.client.1.smithi142.stdout:6/827: mkdir d1a/d2d/d80/d8e/d110/da0/d120 0 2024-06-15T01:07:10.224 INFO:tasks.workunit.client.1.smithi142.stdout:9/776: dwrite d6/dc/d1b/d25/d58/de2/fe9 [0,4194304] 0 2024-06-15T01:07:10.224 INFO:tasks.workunit.client.0.smithi137.stdout:3/826: link d0/d2/d8/dbd/ff6 d0/d2/d8/d67/d7e/de3/f10c 0 2024-06-15T01:07:10.224 INFO:tasks.workunit.client.0.smithi137.stdout:3/827: truncate d0/d42/ff7 31891 0 2024-06-15T01:07:10.225 INFO:tasks.workunit.client.0.smithi137.stdout:0/699: symlink d1/d2/dc/d10/d14/d15/d17/d54/d7a/lec 0 2024-06-15T01:07:10.226 INFO:tasks.workunit.client.0.smithi137.stdout:0/700: stat d1/d2/dc/d10/d14/d15/f4b 0 2024-06-15T01:07:10.228 INFO:tasks.workunit.client.0.smithi137.stdout:3/828: fdatasync d0/d2/d1e/f81 0 2024-06-15T01:07:10.232 INFO:tasks.workunit.client.1.smithi142.stdout:9/777: creat d6/d31/f120 x:0 0 0 2024-06-15T01:07:10.232 INFO:tasks.workunit.client.1.smithi142.stdout:5/774: rmdir d2/de/d27 39 2024-06-15T01:07:10.233 INFO:tasks.workunit.client.0.smithi137.stdout:0/701: creat d1/d2/fed x:0 0 0 2024-06-15T01:07:10.233 INFO:tasks.workunit.client.1.smithi142.stdout:5/775: chown d2/de/d19/d5f/d68/d76/d8c/dc4 5 1 2024-06-15T01:07:10.240 INFO:tasks.workunit.client.0.smithi137.stdout:3/829: dwrite d0/d13/d37/d66/fae [0,4194304] 0 2024-06-15T01:07:10.241 INFO:tasks.workunit.client.0.smithi137.stdout:3/830: readlink d0/d2f/l10a 0 2024-06-15T01:07:10.245 INFO:tasks.workunit.client.0.smithi137.stdout:3/831: symlink d0/ddf/da1/l10d 0 2024-06-15T01:07:10.247 INFO:tasks.workunit.client.1.smithi142.stdout:6/828: symlink d1a/d2d/l121 0 2024-06-15T01:07:10.248 INFO:tasks.workunit.client.1.smithi142.stdout:6/829: dread - d1a/d39/d3a/dd2/ff8 zero size 2024-06-15T01:07:10.249 INFO:tasks.workunit.client.0.smithi137.stdout:3/832: rename d0/d56/f84 to d0/d2/d23/d68/f10e 0 2024-06-15T01:07:10.249 INFO:tasks.workunit.client.1.smithi142.stdout:5/776: mknod d2/de/d19/d5f/d68/d76/d96/cf3 0 2024-06-15T01:07:10.250 INFO:tasks.workunit.client.0.smithi137.stdout:3/833: chown d0/d13/d37/d66/fdc 0 1 2024-06-15T01:07:10.251 INFO:tasks.workunit.client.1.smithi142.stdout:5/777: write d2/d3e/d65/dd6/ff1 [794119,50504] 0 2024-06-15T01:07:10.252 INFO:tasks.workunit.client.1.smithi142.stdout:3/885: dread d0/d3/f1a [0,4194304] 0 2024-06-15T01:07:10.254 INFO:tasks.workunit.client.1.smithi142.stdout:3/886: readlink d0/d3/da/la9 0 2024-06-15T01:07:10.255 INFO:tasks.workunit.client.1.smithi142.stdout:5/778: creat d2/d18/d2e/dae/ff4 x:0 0 0 2024-06-15T01:07:10.256 INFO:tasks.workunit.client.1.smithi142.stdout:5/779: read - d2/de/d19/d5f/fd2 zero size 2024-06-15T01:07:10.258 INFO:tasks.workunit.client.1.smithi142.stdout:4/896: sync 2024-06-15T01:07:10.259 INFO:tasks.workunit.client.1.smithi142.stdout:3/887: rename d0/d3/da/f9b to d0/d1f/d20/d5d/d60/f10a 0 2024-06-15T01:07:10.260 INFO:tasks.workunit.client.1.smithi142.stdout:3/888: chown d0/d1b/f41 3061391 1 2024-06-15T01:07:10.260 INFO:tasks.workunit.client.1.smithi142.stdout:5/780: symlink d2/de/d19/d5f/d68/d76/da6/lf5 0 2024-06-15T01:07:10.261 INFO:tasks.workunit.client.1.smithi142.stdout:4/897: mknod d5/d9/d41/c13a 0 2024-06-15T01:07:10.262 INFO:tasks.workunit.client.0.smithi137.stdout:0/702: dread d1/d2/dc/d10/d14/d15/f59 [4194304,4194304] 0 2024-06-15T01:07:10.264 INFO:tasks.workunit.client.1.smithi142.stdout:5/781: fdatasync d2/f12 0 2024-06-15T01:07:10.267 INFO:tasks.workunit.client.0.smithi137.stdout:0/703: write d1/d2/dc/d6c/d90/fe1 [3228386,130982] 0 2024-06-15T01:07:10.269 INFO:tasks.workunit.client.1.smithi142.stdout:4/898: creat d5/d9/d41/f13b x:0 0 0 2024-06-15T01:07:10.269 INFO:tasks.workunit.client.0.smithi137.stdout:3/834: dwrite d0/d2/d23/f48 [0,4194304] 0 2024-06-15T01:07:10.271 INFO:tasks.workunit.client.1.smithi142.stdout:4/899: chown d5/d1c/d6f/db9/l120 6900 1 2024-06-15T01:07:10.272 INFO:tasks.workunit.client.1.smithi142.stdout:4/900: chown d5/d9 4725 1 2024-06-15T01:07:10.273 INFO:tasks.workunit.client.0.smithi137.stdout:0/704: creat d1/d2/dc/ddc/fee x:0 0 0 2024-06-15T01:07:10.274 INFO:tasks.workunit.client.0.smithi137.stdout:3/835: mknod d0/d2/d8/d67/d7e/de3/c10f 0 2024-06-15T01:07:10.277 INFO:tasks.workunit.client.1.smithi142.stdout:2/800: dwrite d8/f40 [4194304,4194304] 0 2024-06-15T01:07:10.277 INFO:tasks.workunit.client.1.smithi142.stdout:5/782: creat d2/d3e/d31/d7e/ff6 x:0 0 0 2024-06-15T01:07:10.279 INFO:tasks.workunit.client.1.smithi142.stdout:5/783: chown d2/de/d19/d5f/d68/d76/d96/l9c 20908755 1 2024-06-15T01:07:10.279 INFO:tasks.workunit.client.1.smithi142.stdout:3/889: creat d0/d1f/d20/d5d/d82/f10b x:0 0 0 2024-06-15T01:07:10.283 INFO:tasks.workunit.client.1.smithi142.stdout:2/801: readlink d8/df/d10/d1c/lee 0 2024-06-15T01:07:10.287 INFO:tasks.workunit.client.1.smithi142.stdout:1/861: dwrite d6/f10 [0,4194304] 0 2024-06-15T01:07:10.288 INFO:tasks.workunit.client.1.smithi142.stdout:1/862: write d6/d12/d3c/d63/d8d/d97/f129 [1030029,73824] 0 2024-06-15T01:07:10.290 INFO:tasks.workunit.client.0.smithi137.stdout:1/598: dread d0/d3/f2f [0,4194304] 0 2024-06-15T01:07:10.291 INFO:tasks.workunit.client.0.smithi137.stdout:3/836: dwrite d0/d56/d8f/d9c/d9f/fb7 [0,4194304] 0 2024-06-15T01:07:10.292 INFO:tasks.workunit.client.1.smithi142.stdout:6/830: getdents d1a/d3c/d100/d58/d87 0 2024-06-15T01:07:10.292 INFO:tasks.workunit.client.1.smithi142.stdout:6/831: stat d1a/d39/d3a/dd2/ld6 0 2024-06-15T01:07:10.296 INFO:tasks.workunit.client.0.smithi137.stdout:1/599: truncate d0/d3/d89/fb3 3003412 0 2024-06-15T01:07:10.296 INFO:tasks.workunit.client.1.smithi142.stdout:5/784: unlink d2/de/d27/fe3 0 2024-06-15T01:07:10.297 INFO:tasks.workunit.client.1.smithi142.stdout:1/863: rmdir d6/d12 39 2024-06-15T01:07:10.298 INFO:tasks.workunit.client.0.smithi137.stdout:1/600: rmdir d0/d3/d47/d73 39 2024-06-15T01:07:10.302 INFO:tasks.workunit.client.0.smithi137.stdout:1/601: mknod d0/cd1 0 2024-06-15T01:07:10.303 INFO:tasks.workunit.client.0.smithi137.stdout:1/602: write d0/d3/d1b/f50 [1519759,26677] 0 2024-06-15T01:07:10.305 INFO:tasks.workunit.client.1.smithi142.stdout:5/785: creat d2/de/d85/ff7 x:0 0 0 2024-06-15T01:07:10.305 INFO:tasks.workunit.client.0.smithi137.stdout:1/603: mknod d0/d3/d89/cd2 0 2024-06-15T01:07:10.307 INFO:tasks.workunit.client.1.smithi142.stdout:5/786: write d2/de/d19/d5f/d68/fd7 [910325,37727] 0 2024-06-15T01:07:10.309 INFO:tasks.workunit.client.0.smithi137.stdout:1/604: creat d0/d3/d47/fd3 x:0 0 0 2024-06-15T01:07:10.310 INFO:tasks.workunit.client.1.smithi142.stdout:5/787: fdatasync d2/de/d19/d5f/d68/d76/d96/dac/fed 0 2024-06-15T01:07:10.313 INFO:tasks.workunit.client.1.smithi142.stdout:1/864: dwrite d6/d25/fe2 [0,4194304] 0 2024-06-15T01:07:10.313 INFO:tasks.workunit.client.1.smithi142.stdout:7/844: dread d2/d6/d22/d47/f7a [0,4194304] 0 2024-06-15T01:07:10.314 INFO:tasks.workunit.client.1.smithi142.stdout:7/845: chown d2/d8/dd/d8a 33648 1 2024-06-15T01:07:10.317 INFO:tasks.workunit.client.1.smithi142.stdout:9/778: truncate d6/dc/d82/fd4 3704770 0 2024-06-15T01:07:10.319 INFO:tasks.workunit.client.1.smithi142.stdout:2/802: dread d8/df/d10/d94/d25/f74 [0,4194304] 0 2024-06-15T01:07:10.320 INFO:tasks.workunit.client.1.smithi142.stdout:5/788: rename d2/de/l4b to d2/de/d19/d5f/d68/d76/da6/lf8 0 2024-06-15T01:07:10.320 INFO:tasks.workunit.client.0.smithi137.stdout:3/837: dread d0/d2/d1e/f6e [0,4194304] 0 2024-06-15T01:07:10.321 INFO:tasks.workunit.client.1.smithi142.stdout:2/803: write d8/fd4 [689994,63790] 0 2024-06-15T01:07:10.322 INFO:tasks.workunit.client.1.smithi142.stdout:5/789: write d2/de/d19/d5f/d68/d76/da6/dc5/fd5 [5443,96734] 0 2024-06-15T01:07:10.323 INFO:tasks.workunit.client.0.smithi137.stdout:3/838: getdents d0/d4c 0 2024-06-15T01:07:10.324 INFO:tasks.workunit.client.1.smithi142.stdout:5/790: truncate d2/d3e/d31/fc3 236651 0 2024-06-15T01:07:10.325 INFO:tasks.workunit.client.1.smithi142.stdout:7/846: dread - d2/d8/dd/d14/f65 zero size 2024-06-15T01:07:10.326 INFO:tasks.workunit.client.1.smithi142.stdout:7/847: dread - d2/d8/da/d33/ffc zero size 2024-06-15T01:07:10.326 INFO:tasks.workunit.client.0.smithi137.stdout:3/839: creat d0/d2/d8/dbd/f110 x:0 0 0 2024-06-15T01:07:10.327 INFO:tasks.workunit.client.0.smithi137.stdout:3/840: fsync d0/d56/d8f/d7b/f96 0 2024-06-15T01:07:10.328 INFO:tasks.workunit.client.1.smithi142.stdout:8/852: dread d0/d25/d53/d9c/fbd [4194304,4194304] 0 2024-06-15T01:07:10.330 INFO:tasks.workunit.client.1.smithi142.stdout:5/791: stat d2/de/d19/d5f/d68/d76/f7c 0 2024-06-15T01:07:10.330 INFO:tasks.workunit.client.1.smithi142.stdout:8/853: read d0/d17/f61 [5286423,60520] 0 2024-06-15T01:07:10.331 INFO:tasks.workunit.client.0.smithi137.stdout:3/841: link d0/d2f/faa d0/d77/f111 0 2024-06-15T01:07:10.331 INFO:tasks.workunit.client.1.smithi142.stdout:7/848: mknod d2/d6/d22/c10d 0 2024-06-15T01:07:10.332 INFO:tasks.workunit.client.1.smithi142.stdout:8/854: chown d0/d25/d5a/d8f/df3/cf7 88268 1 2024-06-15T01:07:10.332 INFO:tasks.workunit.client.1.smithi142.stdout:7/849: chown d2/db6/de6 0 1 2024-06-15T01:07:10.333 INFO:tasks.workunit.client.1.smithi142.stdout:1/865: rename d6/d12/l48 to d6/d4d/da2/l130 0 2024-06-15T01:07:10.333 INFO:tasks.workunit.client.0.smithi137.stdout:3/842: dread d0/d13/fba [0,4194304] 0 2024-06-15T01:07:10.334 INFO:tasks.workunit.client.1.smithi142.stdout:5/792: dread d2/d3e/d65/dd6/ff1 [0,4194304] 0 2024-06-15T01:07:10.335 INFO:tasks.workunit.client.1.smithi142.stdout:1/866: dread d6/d12/d80/f100 [0,4194304] 0 2024-06-15T01:07:10.335 INFO:tasks.workunit.client.0.smithi137.stdout:3/843: creat d0/d56/d8f/f112 x:0 0 0 2024-06-15T01:07:10.336 INFO:tasks.workunit.client.0.smithi137.stdout:3/844: chown d0/d2/d8/d67/d7e/de3 0 1 2024-06-15T01:07:10.337 INFO:tasks.workunit.client.0.smithi137.stdout:1/605: dread d0/d1f/f4d [0,4194304] 0 2024-06-15T01:07:10.337 INFO:tasks.workunit.client.0.smithi137.stdout:3/845: fsync d0/d2/d8/d67/ff4 0 2024-06-15T01:07:10.339 INFO:tasks.workunit.client.1.smithi142.stdout:7/850: mkdir d2/d8/dd/d8a/d10e 0 2024-06-15T01:07:10.340 INFO:tasks.workunit.client.0.smithi137.stdout:1/606: truncate d0/d3/d1b/f9d 529473 0 2024-06-15T01:07:10.340 INFO:tasks.workunit.client.1.smithi142.stdout:4/901: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fd0 [0,4194304] 0 2024-06-15T01:07:10.341 INFO:tasks.workunit.client.1.smithi142.stdout:5/793: mknod d2/de/d19/d5f/d68/d76/da6/dc5/cf9 0 2024-06-15T01:07:10.341 INFO:tasks.workunit.client.0.smithi137.stdout:1/607: truncate d0/d1f/d76/da1/fa2 4381176 0 2024-06-15T01:07:10.342 INFO:tasks.workunit.client.0.smithi137.stdout:3/846: getdents d0/d2/d8/d67/d7e/de3 0 2024-06-15T01:07:10.345 INFO:tasks.workunit.client.0.smithi137.stdout:1/608: dread d0/f3b [0,4194304] 0 2024-06-15T01:07:10.347 INFO:tasks.workunit.client.1.smithi142.stdout:7/851: read d2/d8/dd/d14/d2c/f10a [192742,16265] 0 2024-06-15T01:07:10.347 INFO:tasks.workunit.client.1.smithi142.stdout:1/867: fsync d6/d12/d1c/f10a 0 2024-06-15T01:07:10.347 INFO:tasks.workunit.client.0.smithi137.stdout:3/847: mknod d0/d13/d37/dcc/c113 0 2024-06-15T01:07:10.348 INFO:tasks.workunit.client.1.smithi142.stdout:8/855: dwrite d0/d25/d66/f95 [0,4194304] 0 2024-06-15T01:07:10.349 INFO:tasks.workunit.client.0.smithi137.stdout:1/609: mkdir d0/d3/d1b/d4c/d59/dd4 0 2024-06-15T01:07:10.349 INFO:tasks.workunit.client.1.smithi142.stdout:5/794: mkdir d2/d3e/d74/dfa 0 2024-06-15T01:07:10.353 INFO:tasks.workunit.client.1.smithi142.stdout:5/795: fdatasync d2/de/d27/db2/ff0 0 2024-06-15T01:07:10.354 INFO:tasks.workunit.client.0.smithi137.stdout:1/610: creat d0/d3/d1b/d4c/d59/d93/fd5 x:0 0 0 2024-06-15T01:07:10.357 INFO:tasks.workunit.client.1.smithi142.stdout:6/832: dwrite d1a/f25 [0,4194304] 0 2024-06-15T01:07:10.357 INFO:tasks.workunit.client.1.smithi142.stdout:0/803: sync 2024-06-15T01:07:10.357 INFO:tasks.workunit.client.1.smithi142.stdout:7/852: dwrite d2/d8/da/fae [4194304,4194304] 0 2024-06-15T01:07:10.359 INFO:tasks.workunit.client.0.smithi137.stdout:3/848: link d0/d56/d8f/d9c/lc0 d0/d13/d37/d83/l114 0 2024-06-15T01:07:10.362 INFO:tasks.workunit.client.0.smithi137.stdout:1/611: creat d0/d3/fd6 x:0 0 0 2024-06-15T01:07:10.365 INFO:tasks.workunit.client.1.smithi142.stdout:2/804: truncate d8/df/d10/d1c/fad 2730595 0 2024-06-15T01:07:10.366 INFO:tasks.workunit.client.0.smithi137.stdout:1/612: truncate d0/d1f/d76/da1/fa2 4802969 0 2024-06-15T01:07:10.366 INFO:tasks.workunit.client.0.smithi137.stdout:3/849: dwrite d0/d56/d8f/fcf [0,4194304] 0 2024-06-15T01:07:10.367 INFO:tasks.workunit.client.1.smithi142.stdout:8/856: truncate d0/d25/d5a/d8f/d98/da4/fb5 1133205 0 2024-06-15T01:07:10.369 INFO:tasks.workunit.client.1.smithi142.stdout:5/796: dread - d2/f58 zero size 2024-06-15T01:07:10.369 INFO:tasks.workunit.client.1.smithi142.stdout:7/853: creat d2/d8/dd/dcf/f10f x:0 0 0 2024-06-15T01:07:10.370 INFO:tasks.workunit.client.0.smithi137.stdout:3/850: creat d0/d2/d22/d63/f115 x:0 0 0 2024-06-15T01:07:10.370 INFO:tasks.workunit.client.1.smithi142.stdout:6/833: mkdir d1a/d39/d3a/d51/d81/d122 0 2024-06-15T01:07:10.370 INFO:tasks.workunit.client.1.smithi142.stdout:9/779: dwrite d6/d34/ddd/fcb [0,4194304] 0 2024-06-15T01:07:10.371 INFO:tasks.workunit.client.0.smithi137.stdout:1/613: mkdir d0/d3/d47/dd7 0 2024-06-15T01:07:10.372 INFO:tasks.workunit.client.1.smithi142.stdout:6/834: fsync d1a/d3c/d100/fc6 0 2024-06-15T01:07:10.375 INFO:tasks.workunit.client.1.smithi142.stdout:4/902: truncate d5/d9/da/d92/d98/dc1/f10c 3756668 0 2024-06-15T01:07:10.376 INFO:tasks.workunit.client.1.smithi142.stdout:1/868: creat d6/de4/f131 x:0 0 0 2024-06-15T01:07:10.377 INFO:tasks.workunit.client.1.smithi142.stdout:1/869: fsync d6/d12/d80/dc8/fda 0 2024-06-15T01:07:10.378 INFO:tasks.workunit.client.0.smithi137.stdout:3/851: dwrite d0/d13/d37/d66/fae [0,4194304] 0 2024-06-15T01:07:10.380 INFO:tasks.workunit.client.1.smithi142.stdout:8/857: rmdir d0/d25/d5a/d8f/d97/ded 39 2024-06-15T01:07:10.381 INFO:tasks.workunit.client.0.smithi137.stdout:3/852: fdatasync d0/d2/d22/d63/fd5 0 2024-06-15T01:07:10.381 INFO:tasks.workunit.client.0.smithi137.stdout:1/614: dwrite d0/f2 [0,4194304] 0 2024-06-15T01:07:10.384 INFO:tasks.workunit.client.0.smithi137.stdout:3/853: creat d0/d2/d22/d24/d5e/f116 x:0 0 0 2024-06-15T01:07:10.384 INFO:tasks.workunit.client.1.smithi142.stdout:9/780: symlink d6/dc/d1b/d25/d35/l121 0 2024-06-15T01:07:10.385 INFO:tasks.workunit.client.1.smithi142.stdout:6/835: mknod d1a/d39/d3a/d51/c123 0 2024-06-15T01:07:10.386 INFO:tasks.workunit.client.1.smithi142.stdout:9/781: chown d6/dc/d1b/d25/c75 0 1 2024-06-15T01:07:10.387 INFO:tasks.workunit.client.0.smithi137.stdout:3/854: mknod d0/c117 0 2024-06-15T01:07:10.387 INFO:tasks.workunit.client.0.smithi137.stdout:1/615: dread d0/d3/d47/f5d [0,4194304] 0 2024-06-15T01:07:10.389 INFO:tasks.workunit.client.1.smithi142.stdout:8/858: dwrite d0/d17/d40/d6e/fcf [0,4194304] 0 2024-06-15T01:07:10.390 INFO:tasks.workunit.client.1.smithi142.stdout:1/870: rename d6/d12/d80/d9b to d6/d25/dfc/d132 0 2024-06-15T01:07:10.390 INFO:tasks.workunit.client.0.smithi137.stdout:1/616: readlink d0/d2d/l48 0 2024-06-15T01:07:10.391 INFO:tasks.workunit.client.1.smithi142.stdout:1/871: chown d6/d12/d52/ca9 4 1 2024-06-15T01:07:10.391 INFO:tasks.workunit.client.1.smithi142.stdout:3/890: sync 2024-06-15T01:07:10.392 INFO:tasks.workunit.client.0.smithi137.stdout:1/617: mknod d0/d3/d1b/d4c/cd8 0 2024-06-15T01:07:10.393 INFO:tasks.workunit.client.0.smithi137.stdout:3/855: dwrite d0/d13/f20 [0,4194304] 0 2024-06-15T01:07:10.393 INFO:tasks.workunit.client.1.smithi142.stdout:6/836: mknod d1a/d3c/d100/d58/d87/c124 0 2024-06-15T01:07:10.394 INFO:tasks.workunit.client.0.smithi137.stdout:4/549: sync 2024-06-15T01:07:10.397 INFO:tasks.workunit.client.0.smithi137.stdout:3/856: write d0/d2/d8/d67/d7e/de3/ffb [543708,61511] 0 2024-06-15T01:07:10.397 INFO:tasks.workunit.client.1.smithi142.stdout:8/859: fdatasync d0/d17/d40/fa2 0 2024-06-15T01:07:10.398 INFO:tasks.workunit.client.0.smithi137.stdout:3/857: stat d0/d2/d22/d24/d5e/f98 0 2024-06-15T01:07:10.400 INFO:tasks.workunit.client.1.smithi142.stdout:3/891: mknod d0/d3/da/d18/dd9/c10c 0 2024-06-15T01:07:10.402 INFO:tasks.workunit.client.0.smithi137.stdout:4/550: creat dd/d15/d36/d45/dcb/d83/db2/d67/dbe/fd0 x:0 0 0 2024-06-15T01:07:10.407 INFO:tasks.workunit.client.0.smithi137.stdout:3/858: write d0/d2f/f69 [199196,33413] 0 2024-06-15T01:07:10.409 INFO:tasks.workunit.client.1.smithi142.stdout:1/872: link d6/d12/d3c/d63/d8d/d97/dd3/caf d6/d12/d1c/d2c/c133 0 2024-06-15T01:07:10.409 INFO:tasks.workunit.client.0.smithi137.stdout:4/551: write dd/d15/d21/d9c/d75/d99/fbd [1831822,77715] 0 2024-06-15T01:07:10.411 INFO:tasks.workunit.client.1.smithi142.stdout:6/837: creat d1a/f125 x:0 0 0 2024-06-15T01:07:10.416 INFO:tasks.workunit.client.1.smithi142.stdout:6/838: rename d1a/d39/d3a/d51/f5c to d1a/d39/d3a/d51/f126 0 2024-06-15T01:07:10.417 INFO:tasks.workunit.client.1.smithi142.stdout:1/873: dread d6/f13 [4194304,4194304] 0 2024-06-15T01:07:10.418 INFO:tasks.workunit.client.1.smithi142.stdout:6/839: mkdir d1a/d2d/d80/d127 0 2024-06-15T01:07:10.419 INFO:tasks.workunit.client.1.smithi142.stdout:1/874: mknod d6/d12/d80/dae/c134 0 2024-06-15T01:07:10.421 INFO:tasks.workunit.client.1.smithi142.stdout:6/840: dread f0 [0,4194304] 0 2024-06-15T01:07:10.422 INFO:tasks.workunit.client.1.smithi142.stdout:1/875: symlink d6/d12/d1c/l135 0 2024-06-15T01:07:10.423 INFO:tasks.workunit.client.1.smithi142.stdout:1/876: stat d6/d12/d52/ca9 0 2024-06-15T01:07:10.426 INFO:tasks.workunit.client.1.smithi142.stdout:6/841: dwrite d1a/d3c/fa7 [0,4194304] 0 2024-06-15T01:07:10.428 INFO:tasks.workunit.client.1.smithi142.stdout:0/804: dread d3/d39/d3e/f5f [0,4194304] 0 2024-06-15T01:07:10.434 INFO:tasks.workunit.client.1.smithi142.stdout:0/805: symlink d3/d39/d3e/d81/l111 0 2024-06-15T01:07:10.434 INFO:tasks.workunit.client.1.smithi142.stdout:0/806: write d3/d39/d3e/d81/d1b/f35 [3225526,65315] 0 2024-06-15T01:07:10.434 INFO:tasks.workunit.client.1.smithi142.stdout:6/842: link d1a/d2d/d2e/l36 d1a/l128 0 2024-06-15T01:07:10.435 INFO:tasks.workunit.client.1.smithi142.stdout:6/843: chown f18 19 1 2024-06-15T01:07:10.436 INFO:tasks.workunit.client.1.smithi142.stdout:6/844: chown d1a/d2d/f33 2652662 1 2024-06-15T01:07:10.436 INFO:tasks.workunit.client.1.smithi142.stdout:0/807: creat d3/d39/d3e/d81/d1b/df6/f112 x:0 0 0 2024-06-15T01:07:10.438 INFO:tasks.workunit.client.1.smithi142.stdout:6/845: creat d1a/d2d/d80/d8e/d110/da0/d120/f129 x:0 0 0 2024-06-15T01:07:10.440 INFO:tasks.workunit.client.1.smithi142.stdout:0/808: dread d3/d39/d3e/d81/d1e/f5d [0,4194304] 0 2024-06-15T01:07:10.442 INFO:tasks.workunit.client.1.smithi142.stdout:0/809: truncate d3/d39/d3e/f5f 4555981 0 2024-06-15T01:07:10.451 INFO:tasks.workunit.client.1.smithi142.stdout:6/846: dread d1a/d2d/d80/d8e/d110/da0/fa5 [0,4194304] 0 2024-06-15T01:07:10.453 INFO:tasks.workunit.client.1.smithi142.stdout:6/847: fdatasync f6 0 2024-06-15T01:07:10.545 INFO:tasks.workunit.client.0.smithi137.stdout:3/859: creat d0/d2/f118 x:0 0 0 2024-06-15T01:07:10.545 INFO:tasks.workunit.client.0.smithi137.stdout:3/860: chown d0/d2/d8/d67/d7e 10235 1 2024-06-15T01:07:10.546 INFO:tasks.workunit.client.1.smithi142.stdout:2/805: write d8/df/d10/d94/d25/fda [206723,24166] 0 2024-06-15T01:07:10.547 INFO:tasks.workunit.client.1.smithi142.stdout:7/854: write d2/d8/dd/dcf/fa6 [131103,87215] 0 2024-06-15T01:07:10.549 INFO:tasks.workunit.client.0.smithi137.stdout:3/861: creat d0/d2/d23/f119 x:0 0 0 2024-06-15T01:07:10.549 INFO:tasks.workunit.client.1.smithi142.stdout:4/903: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/f37 [1412884,36248] 0 2024-06-15T01:07:10.549 INFO:tasks.workunit.client.1.smithi142.stdout:9/782: write d6/d31/f32 [1105204,101259] 0 2024-06-15T01:07:10.551 INFO:tasks.workunit.client.1.smithi142.stdout:7/855: chown d2/d8/da/d33/d25/d79/f95 7 1 2024-06-15T01:07:10.552 INFO:tasks.workunit.client.0.smithi137.stdout:1/618: write d0/d1f/d7d/fae [822386,35053] 0 2024-06-15T01:07:10.554 INFO:tasks.workunit.client.1.smithi142.stdout:5/797: dwrite d2/f14 [0,4194304] 0 2024-06-15T01:07:10.558 INFO:tasks.workunit.client.1.smithi142.stdout:2/806: dwrite d8/df/d10/d94/d25/d1a/d63/f77 [0,4194304] 0 2024-06-15T01:07:10.562 INFO:tasks.workunit.client.1.smithi142.stdout:4/904: dwrite d5/d9/d23/d32/f8f [0,4194304] 0 2024-06-15T01:07:10.565 INFO:tasks.workunit.client.1.smithi142.stdout:7/856: dwrite d2/d8/dd/dcf/fbb [0,4194304] 0 2024-06-15T01:07:10.565 INFO:tasks.workunit.client.0.smithi137.stdout:1/619: truncate d0/d3/d47/f5d 914086 0 2024-06-15T01:07:10.566 INFO:tasks.workunit.client.0.smithi137.stdout:1/620: dread - d0/d3/d89/fbc zero size 2024-06-15T01:07:10.567 INFO:tasks.workunit.client.1.smithi142.stdout:7/857: chown d2/d8/dd/d14/d2c/d70/ff7 569 1 2024-06-15T01:07:10.569 INFO:tasks.workunit.client.0.smithi137.stdout:3/862: dwrite d0/ddf/ff2 [0,4194304] 0 2024-06-15T01:07:10.569 INFO:tasks.workunit.client.1.smithi142.stdout:7/858: stat d2/d8/dd/fd9 0 2024-06-15T01:07:10.579 INFO:tasks.workunit.client.0.smithi137.stdout:4/552: dread dd/f16 [0,4194304] 0 2024-06-15T01:07:10.583 INFO:tasks.workunit.client.0.smithi137.stdout:4/553: rename dd/d15/d30/f40 to dd/dbf/fd1 0 2024-06-15T01:07:10.584 INFO:tasks.workunit.client.0.smithi137.stdout:4/554: creat dd/d15/d30/fd2 x:0 0 0 2024-06-15T01:07:10.588 INFO:tasks.workunit.client.1.smithi142.stdout:8/860: sync 2024-06-15T01:07:10.598 INFO:tasks.workunit.client.1.smithi142.stdout:7/859: dread d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:07:10.598 INFO:tasks.workunit.client.1.smithi142.stdout:7/860: readlink d2/d8/dd/dcf/l53 0 2024-06-15T01:07:10.609 INFO:tasks.workunit.client.0.smithi137.stdout:4/555: dwrite dd/d15/d36/d45/d55/f62 [0,4194304] 0 2024-06-15T01:07:10.610 INFO:tasks.workunit.client.0.smithi137.stdout:7/478: sync 2024-06-15T01:07:10.610 INFO:tasks.workunit.client.1.smithi142.stdout:6/848: sync 2024-06-15T01:07:10.610 INFO:tasks.workunit.client.0.smithi137.stdout:7/479: readlink df/d13/l44 0 2024-06-15T01:07:10.613 INFO:tasks.workunit.client.0.smithi137.stdout:4/556: mknod dd/d15/d36/cd3 0 2024-06-15T01:07:10.614 INFO:tasks.workunit.client.1.smithi142.stdout:6/849: dread d1a/d39/d3a/f59 [0,4194304] 0 2024-06-15T01:07:10.615 INFO:tasks.workunit.client.0.smithi137.stdout:7/480: truncate df/d13/d2e/f4f 2253378 0 2024-06-15T01:07:10.615 INFO:tasks.workunit.client.1.smithi142.stdout:3/892: getdents d0/d3/da/d18/dd9 0 2024-06-15T01:07:10.618 INFO:tasks.workunit.client.1.smithi142.stdout:5/798: rename d2/de/d19/f6a to d2/d3e/d74/da5/ffb 0 2024-06-15T01:07:10.618 INFO:tasks.workunit.client.0.smithi137.stdout:1/621: dread d0/d2d/d3e/f77 [0,4194304] 0 2024-06-15T01:07:10.619 INFO:tasks.workunit.client.0.smithi137.stdout:1/622: readlink d0/d2d/d3e/l5b 0 2024-06-15T01:07:10.619 INFO:tasks.workunit.client.0.smithi137.stdout:7/481: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/f96 x:0 0 0 2024-06-15T01:07:10.620 INFO:tasks.workunit.client.1.smithi142.stdout:2/807: mkdir d8/d19/deb/df8 0 2024-06-15T01:07:10.620 INFO:tasks.workunit.client.0.smithi137.stdout:1/623: dread - d0/d3/d89/fbc zero size 2024-06-15T01:07:10.621 INFO:tasks.workunit.client.1.smithi142.stdout:2/808: dread - d8/df/d10/d94/d25/d76/fc6 zero size 2024-06-15T01:07:10.621 INFO:tasks.workunit.client.0.smithi137.stdout:7/482: write df/d13/d2e/f4e [3199530,54156] 0 2024-06-15T01:07:10.622 INFO:tasks.workunit.client.0.smithi137.stdout:1/624: write d0/d3/d1b/d4c/d59/d93/fd5 [335120,39645] 0 2024-06-15T01:07:10.623 INFO:tasks.workunit.client.0.smithi137.stdout:1/625: write d0/d10/d41/d7b/fc2 [3491595,5330] 0 2024-06-15T01:07:10.624 INFO:tasks.workunit.client.0.smithi137.stdout:7/483: unlink df/d13/d3c/f67 0 2024-06-15T01:07:10.624 INFO:tasks.workunit.client.1.smithi142.stdout:8/861: mkdir d0/d25/d53/da7/dcb/dfe 0 2024-06-15T01:07:10.624 INFO:tasks.workunit.client.1.smithi142.stdout:9/783: dread d6/d34/ddd/fa2 [0,4194304] 0 2024-06-15T01:07:10.629 INFO:tasks.workunit.client.1.smithi142.stdout:4/905: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/db0/dbc/f13c x:0 0 0 2024-06-15T01:07:10.630 INFO:tasks.workunit.client.1.smithi142.stdout:2/809: write d8/df/d10/d94/d25/d1a/d63/f95 [2045340,67737] 0 2024-06-15T01:07:10.630 INFO:tasks.workunit.client.1.smithi142.stdout:1/877: write d6/d12/d3c/d63/d8d/fb8 [415965,90041] 0 2024-06-15T01:07:10.635 INFO:tasks.workunit.client.1.smithi142.stdout:4/906: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/fc8 zero size 2024-06-15T01:07:10.636 INFO:tasks.workunit.client.1.smithi142.stdout:8/862: creat d0/d17/d40/d86/fff x:0 0 0 2024-06-15T01:07:10.639 INFO:tasks.workunit.client.1.smithi142.stdout:1/878: dwrite d6/d25/dfc/d132/db2/dc2/f112 [0,4194304] 0 2024-06-15T01:07:10.639 INFO:tasks.workunit.client.1.smithi142.stdout:0/810: write d3/d39/d3e/d81/d44/d7c/d22/f36 [3750672,48145] 0 2024-06-15T01:07:10.641 INFO:tasks.workunit.client.1.smithi142.stdout:2/810: creat d8/df/d10/d1c/dbb/ff9 x:0 0 0 2024-06-15T01:07:10.641 INFO:tasks.workunit.client.1.smithi142.stdout:1/879: dread - d6/d12/db5/f105 zero size 2024-06-15T01:07:10.641 INFO:tasks.workunit.client.1.smithi142.stdout:9/784: link d6/dc/d1b/d25/d58/c7d d6/d34/d8a/dec/c122 0 2024-06-15T01:07:10.642 INFO:tasks.workunit.client.1.smithi142.stdout:6/850: getdents d1a/d2d/d80/d8e/d110/da0/d120 0 2024-06-15T01:07:10.645 INFO:tasks.workunit.client.1.smithi142.stdout:9/785: read d6/dc/d1b/d6f/d103/fc9 [804848,5804] 0 2024-06-15T01:07:10.646 INFO:tasks.workunit.client.1.smithi142.stdout:2/811: symlink d8/df/d10/d1c/lfa 0 2024-06-15T01:07:10.647 INFO:tasks.workunit.client.1.smithi142.stdout:1/880: symlink d6/d12/d3c/d63/d8d/d97/dd3/d106/l136 0 2024-06-15T01:07:10.650 INFO:tasks.workunit.client.1.smithi142.stdout:9/786: creat d6/dc/f123 x:0 0 0 2024-06-15T01:07:10.651 INFO:tasks.workunit.client.1.smithi142.stdout:1/881: chown d6/d12/da0/db0/f73 34 1 2024-06-15T01:07:10.653 INFO:tasks.workunit.client.1.smithi142.stdout:6/851: dwrite d1a/d2d/f2f [0,4194304] 0 2024-06-15T01:07:10.653 INFO:tasks.workunit.client.1.smithi142.stdout:2/812: creat d8/df/d10/ffb x:0 0 0 2024-06-15T01:07:10.653 INFO:tasks.workunit.client.1.smithi142.stdout:8/863: dread d0/d25/d66/fe1 [4194304,4194304] 0 2024-06-15T01:07:10.654 INFO:tasks.workunit.client.1.smithi142.stdout:0/811: dwrite d3/d39/d3e/d81/d44/d7c/d22/f70 [0,4194304] 0 2024-06-15T01:07:10.654 INFO:tasks.workunit.client.1.smithi142.stdout:9/787: mknod d6/dc/d1b/d25/dcf/c124 0 2024-06-15T01:07:10.657 INFO:tasks.workunit.client.1.smithi142.stdout:9/788: write d6/d34/ddd/f97 [3928210,25065] 0 2024-06-15T01:07:10.657 INFO:tasks.workunit.client.1.smithi142.stdout:1/882: creat d6/d12/d3c/d63/d8d/d97/dc6/dfe/f137 x:0 0 0 2024-06-15T01:07:10.658 INFO:tasks.workunit.client.1.smithi142.stdout:0/812: mknod d3/d39/d9a/c113 0 2024-06-15T01:07:10.659 INFO:tasks.workunit.client.1.smithi142.stdout:6/852: getdents d1a/d39/d3a/d51/d81/d122 0 2024-06-15T01:07:10.660 INFO:tasks.workunit.client.1.smithi142.stdout:9/789: write d6/dc/dd/d22/f119 [1031746,96120] 0 2024-06-15T01:07:10.665 INFO:tasks.workunit.client.1.smithi142.stdout:1/883: dwrite d6/d12/da0/db0/fe8 [0,4194304] 0 2024-06-15T01:07:10.668 INFO:tasks.workunit.client.0.smithi137.stdout:7/484: chown df/c12 15290 1 2024-06-15T01:07:10.673 INFO:tasks.workunit.client.1.smithi142.stdout:8/864: getdents d0/d25/d5a/d84 0 2024-06-15T01:07:10.675 INFO:tasks.workunit.client.1.smithi142.stdout:6/853: stat d1a/d3c/d100/c7e 0 2024-06-15T01:07:10.676 INFO:tasks.workunit.client.1.smithi142.stdout:1/884: fsync d6/fc 0 2024-06-15T01:07:10.677 INFO:tasks.workunit.client.1.smithi142.stdout:6/854: truncate d1a/f22 2154450 0 2024-06-15T01:07:10.678 INFO:tasks.workunit.client.1.smithi142.stdout:1/885: read - d6/d12/d80/dc8/f110 zero size 2024-06-15T01:07:10.678 INFO:tasks.workunit.client.1.smithi142.stdout:8/865: dread d0/f3f [4194304,4194304] 0 2024-06-15T01:07:10.679 INFO:tasks.workunit.client.1.smithi142.stdout:0/813: read d3/d39/d3e/d81/d1b/f40 [126646,14170] 0 2024-06-15T01:07:10.680 INFO:tasks.workunit.client.0.smithi137.stdout:1/626: symlink d0/d10/d41/ld9 0 2024-06-15T01:07:10.681 INFO:tasks.workunit.client.0.smithi137.stdout:3/863: dwrite d0/fd [0,4194304] 0 2024-06-15T01:07:10.681 INFO:tasks.workunit.client.0.smithi137.stdout:1/627: chown d0/d3/l3a 5258 1 2024-06-15T01:07:10.686 INFO:tasks.workunit.client.1.smithi142.stdout:1/886: creat d6/de4/f138 x:0 0 0 2024-06-15T01:07:10.686 INFO:tasks.workunit.client.1.smithi142.stdout:1/887: chown d6/d12/d1c/d2c/d47/cf3 453117 1 2024-06-15T01:07:10.687 INFO:tasks.workunit.client.0.smithi137.stdout:7/485: mkdir df/d7d/d1c/d1d/d27/d97 0 2024-06-15T01:07:10.688 INFO:tasks.workunit.client.1.smithi142.stdout:8/866: truncate d0/d25/d66/f92 1258746 0 2024-06-15T01:07:10.691 INFO:tasks.workunit.client.0.smithi137.stdout:3/864: dwrite d0/d4c/f82 [0,4194304] 0 2024-06-15T01:07:10.691 INFO:tasks.workunit.client.1.smithi142.stdout:1/888: chown d6/d3a/c111 21695143 1 2024-06-15T01:07:10.693 INFO:tasks.workunit.client.0.smithi137.stdout:3/865: write d0/d56/fca [2187781,17102] 0 2024-06-15T01:07:10.695 INFO:tasks.workunit.client.1.smithi142.stdout:7/861: dread d2/d8/da/d33/d25/d79/fb1 [0,4194304] 0 2024-06-15T01:07:10.695 INFO:tasks.workunit.client.1.smithi142.stdout:8/867: mkdir d0/d25/d100 0 2024-06-15T01:07:10.697 INFO:tasks.workunit.client.1.smithi142.stdout:7/862: fdatasync d2/d8/da/d33/f35 0 2024-06-15T01:07:10.698 INFO:tasks.workunit.client.1.smithi142.stdout:1/889: dwrite d6/d25/dfc/d132/db2/dc2/f107 [4194304,4194304] 0 2024-06-15T01:07:10.698 INFO:tasks.workunit.client.1.smithi142.stdout:7/863: write d2/d6/f6d [2345266,113949] 0 2024-06-15T01:07:10.703 INFO:tasks.workunit.client.0.smithi137.stdout:1/628: getdents d0/d1f/d7d 0 2024-06-15T01:07:10.704 INFO:tasks.workunit.client.0.smithi137.stdout:3/866: mknod d0/d13/d37/c11a 0 2024-06-15T01:07:10.705 INFO:tasks.workunit.client.0.smithi137.stdout:7/486: getdents df/d13/d2e/d1a/d6a 0 2024-06-15T01:07:10.706 INFO:tasks.workunit.client.0.smithi137.stdout:3/867: chown d0/d56/d8f/d7b/ced 2 1 2024-06-15T01:07:10.707 INFO:tasks.workunit.client.1.smithi142.stdout:1/890: rename d6/d12/da0/db0/de1 to d6/d4d/da2/d139 0 2024-06-15T01:07:10.707 INFO:tasks.workunit.client.0.smithi137.stdout:7/487: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/l4d 9567141 1 2024-06-15T01:07:10.708 INFO:tasks.workunit.client.1.smithi142.stdout:7/864: read d2/f2e [2472051,81209] 0 2024-06-15T01:07:10.710 INFO:tasks.workunit.client.1.smithi142.stdout:3/893: dwrite d0/d1f/d20/d59/d61/d96/faa [0,4194304] 0 2024-06-15T01:07:10.712 INFO:tasks.workunit.client.0.smithi137.stdout:7/488: symlink df/d7d/d1c/d1d/d27/d33/d59/l98 0 2024-06-15T01:07:10.718 INFO:tasks.workunit.client.1.smithi142.stdout:1/891: unlink d6/d12/d3c/d63/d8d/d97/dc6/dfe/f128 0 2024-06-15T01:07:10.720 INFO:tasks.workunit.client.1.smithi142.stdout:3/894: rmdir d0/d1f/d20/d59 39 2024-06-15T01:07:10.722 INFO:tasks.workunit.client.1.smithi142.stdout:3/895: write d0/d3/fcb [1537261,69411] 0 2024-06-15T01:07:10.723 INFO:tasks.workunit.client.1.smithi142.stdout:1/892: mknod d6/d4d/da2/d139/c13a 0 2024-06-15T01:07:10.724 INFO:tasks.workunit.client.1.smithi142.stdout:4/907: dread d5/d9/d23/d68/f8e [0,4194304] 0 2024-06-15T01:07:10.725 INFO:tasks.workunit.client.0.smithi137.stdout:3/868: truncate d0/d2/d8/d67/d7e/de3/f10c 3977749 0 2024-06-15T01:07:10.726 INFO:tasks.workunit.client.1.smithi142.stdout:4/908: stat d5/d1c/d6f/dc7/d30/d46/d50/d51 0 2024-06-15T01:07:10.728 INFO:tasks.workunit.client.1.smithi142.stdout:3/896: getdents d0/d3/de2/d109 0 2024-06-15T01:07:10.729 INFO:tasks.workunit.client.0.smithi137.stdout:3/869: symlink d0/d2/d8/dbd/d106/l11b 0 2024-06-15T01:07:10.730 INFO:tasks.workunit.client.1.smithi142.stdout:4/909: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f13d x:0 0 0 2024-06-15T01:07:10.731 INFO:tasks.workunit.client.1.smithi142.stdout:1/893: dwrite d6/f9 [0,4194304] 0 2024-06-15T01:07:10.731 INFO:tasks.workunit.client.0.smithi137.stdout:7/489: dwrite df/f78 [0,4194304] 0 2024-06-15T01:07:10.732 INFO:tasks.workunit.client.1.smithi142.stdout:4/910: chown d5/d9/d90/fff 5848 1 2024-06-15T01:07:10.732 INFO:tasks.workunit.client.1.smithi142.stdout:1/894: chown d6/d25/dfc/f51 13910 1 2024-06-15T01:07:10.733 INFO:tasks.workunit.client.1.smithi142.stdout:1/895: chown d6/d12/d80/f100 39883 1 2024-06-15T01:07:10.737 INFO:tasks.workunit.client.1.smithi142.stdout:3/897: creat d0/d1f/d20/d5d/d60/dc1/f10d x:0 0 0 2024-06-15T01:07:10.737 INFO:tasks.workunit.client.1.smithi142.stdout:6/855: write d1a/d39/d42/f74 [796221,27991] 0 2024-06-15T01:07:10.738 INFO:tasks.workunit.client.0.smithi137.stdout:3/870: dwrite d0/d2/d4e/fd9 [0,4194304] 0 2024-06-15T01:07:10.742 INFO:tasks.workunit.client.0.smithi137.stdout:1/629: dread d0/d3/d47/f88 [0,4194304] 0 2024-06-15T01:07:10.745 INFO:tasks.workunit.client.1.smithi142.stdout:7/865: dread d2/d8/da/d33/d25/f67 [0,4194304] 0 2024-06-15T01:07:10.745 INFO:tasks.workunit.client.0.smithi137.stdout:3/871: creat d0/d56/d8f/d9c/d9f/f11c x:0 0 0 2024-06-15T01:07:10.746 INFO:tasks.workunit.client.1.smithi142.stdout:7/866: chown d2/d8/da/d33/cca 39325 1 2024-06-15T01:07:10.749 INFO:tasks.workunit.client.0.smithi137.stdout:3/872: mkdir d0/d2f/dbe/d11d 0 2024-06-15T01:07:10.749 INFO:tasks.workunit.client.1.smithi142.stdout:1/896: creat d6/d12/d3c/d63/d8d/d97/f13b x:0 0 0 2024-06-15T01:07:10.751 INFO:tasks.workunit.client.1.smithi142.stdout:0/814: write d3/d39/d3e/d81/f8d [1046215,82914] 0 2024-06-15T01:07:10.752 INFO:tasks.workunit.client.0.smithi137.stdout:1/630: creat d0/d1f/d76/fda x:0 0 0 2024-06-15T01:07:10.753 INFO:tasks.workunit.client.1.smithi142.stdout:0/815: stat d3/d39/lcb 0 2024-06-15T01:07:10.754 INFO:tasks.workunit.client.0.smithi137.stdout:3/873: fsync d0/d77/f111 0 2024-06-15T01:07:10.754 INFO:tasks.workunit.client.1.smithi142.stdout:1/897: dread d6/d12/d80/dc8/fe7 [0,4194304] 0 2024-06-15T01:07:10.755 INFO:tasks.workunit.client.1.smithi142.stdout:8/868: write d0/d17/d40/d86/f9b [1017792,74296] 0 2024-06-15T01:07:10.756 INFO:tasks.workunit.client.0.smithi137.stdout:1/631: mknod d0/d10/d1d/cdb 0 2024-06-15T01:07:10.756 INFO:tasks.workunit.client.0.smithi137.stdout:3/874: chown d0/d56/d8f/d7b 9 1 2024-06-15T01:07:10.757 INFO:tasks.workunit.client.1.smithi142.stdout:3/898: mkdir d0/d3/d10e 0 2024-06-15T01:07:10.758 INFO:tasks.workunit.client.0.smithi137.stdout:1/632: mkdir d0/d1f/ddc 0 2024-06-15T01:07:10.758 INFO:tasks.workunit.client.1.smithi142.stdout:6/856: dwrite d1a/d3c/db0/feb [0,4194304] 0 2024-06-15T01:07:10.760 INFO:tasks.workunit.client.0.smithi137.stdout:3/875: unlink d0/d2/d8/d67/l71 0 2024-06-15T01:07:10.760 INFO:tasks.workunit.client.1.smithi142.stdout:1/898: dread d6/d12/d1c/d2c/d93/db9/fbb [0,4194304] 0 2024-06-15T01:07:10.761 INFO:tasks.workunit.client.1.smithi142.stdout:1/899: chown d6/d12/l42 65 1 2024-06-15T01:07:10.762 INFO:tasks.workunit.client.1.smithi142.stdout:8/869: creat d0/d25/d53/d9c/f101 x:0 0 0 2024-06-15T01:07:10.764 INFO:tasks.workunit.client.0.smithi137.stdout:3/876: symlink d0/d56/l11e 0 2024-06-15T01:07:10.766 INFO:tasks.workunit.client.1.smithi142.stdout:3/899: chown d0/d3/de2/d55/f57 4 1 2024-06-15T01:07:10.768 INFO:tasks.workunit.client.1.smithi142.stdout:6/857: creat d1a/d39/d3a/d51/d62/f12a x:0 0 0 2024-06-15T01:07:10.770 INFO:tasks.workunit.client.1.smithi142.stdout:1/900: stat d6/d12/d1c/d2c/c133 0 2024-06-15T01:07:10.772 INFO:tasks.workunit.client.1.smithi142.stdout:5/799: sync 2024-06-15T01:07:10.772 INFO:tasks.workunit.client.0.smithi137.stdout:7/490: dread df/d7d/d1c/d1d/d27/d33/d43/d6c/f37 [0,4194304] 0 2024-06-15T01:07:10.772 INFO:tasks.workunit.client.1.smithi142.stdout:8/870: unlink d0/d25/d5a/d8f/d98/da4/cc7 0 2024-06-15T01:07:10.775 INFO:tasks.workunit.client.1.smithi142.stdout:4/911: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f60 [511375,77360] 0 2024-06-15T01:07:10.775 INFO:tasks.workunit.client.0.smithi137.stdout:7/491: mknod df/d13/c99 0 2024-06-15T01:07:10.776 INFO:tasks.workunit.client.1.smithi142.stdout:7/867: dread d2/d8/da/d33/fd7 [0,4194304] 0 2024-06-15T01:07:10.776 INFO:tasks.workunit.client.1.smithi142.stdout:4/912: fdatasync d5/d9/f131 0 2024-06-15T01:07:10.777 INFO:tasks.workunit.client.1.smithi142.stdout:1/901: creat d6/d25/dfc/d132/db2/dc2/f13c x:0 0 0 2024-06-15T01:07:10.777 INFO:tasks.workunit.client.1.smithi142.stdout:1/902: fsync d6/f9 0 2024-06-15T01:07:10.779 INFO:tasks.workunit.client.0.smithi137.stdout:7/492: creat df/d13/d2e/d1a/d6a/f9a x:0 0 0 2024-06-15T01:07:10.779 INFO:tasks.workunit.client.1.smithi142.stdout:5/800: unlink d2/de/d19/d5f/l6f 0 2024-06-15T01:07:10.781 INFO:tasks.workunit.client.0.smithi137.stdout:1/633: dwrite d0/d3/d1b/d4c/d59/fa6 [0,4194304] 0 2024-06-15T01:07:10.781 INFO:tasks.workunit.client.1.smithi142.stdout:8/871: creat d0/d25/db9/f102 x:0 0 0 2024-06-15T01:07:10.782 INFO:tasks.workunit.client.1.smithi142.stdout:8/872: chown d0/d25/c2b 652 1 2024-06-15T01:07:10.782 INFO:tasks.workunit.client.0.smithi137.stdout:1/634: write d0/d2d/fc6 [1032952,82419] 0 2024-06-15T01:07:10.783 INFO:tasks.workunit.client.1.smithi142.stdout:0/816: rename d3/d39/d3e/d81/d44/f5a to d3/d39/d3e/d81/d1b/d38/f114 0 2024-06-15T01:07:10.784 INFO:tasks.workunit.client.1.smithi142.stdout:6/858: symlink d1a/d39/d42/df7/dfc/l12b 0 2024-06-15T01:07:10.789 INFO:tasks.workunit.client.0.smithi137.stdout:1/635: fdatasync d0/d3/d1b/f9d 0 2024-06-15T01:07:10.790 INFO:tasks.workunit.client.0.smithi137.stdout:7/493: getdents df/d7d/d3b 0 2024-06-15T01:07:10.791 INFO:tasks.workunit.client.0.smithi137.stdout:1/636: dread - d0/d1f/d76/fc4 zero size 2024-06-15T01:07:10.791 INFO:tasks.workunit.client.1.smithi142.stdout:3/900: creat d0/d1f/d20/f10f x:0 0 0 2024-06-15T01:07:10.793 INFO:tasks.workunit.client.1.smithi142.stdout:0/817: mkdir d3/d39/d3e/d81/d1b/d52/d115 0 2024-06-15T01:07:10.794 INFO:tasks.workunit.client.0.smithi137.stdout:1/637: symlink d0/d1f/d67/ldd 0 2024-06-15T01:07:10.795 INFO:tasks.workunit.client.1.smithi142.stdout:8/873: creat d0/d25/d53/da7/dcb/dfb/f103 x:0 0 0 2024-06-15T01:07:10.796 INFO:tasks.workunit.client.1.smithi142.stdout:4/913: dread d5/d1c/d6f/dc7/fab [0,4194304] 0 2024-06-15T01:07:10.796 INFO:tasks.workunit.client.0.smithi137.stdout:1/638: fdatasync d0/d3/d1b/d4c/d59/d66/fad 0 2024-06-15T01:07:10.797 INFO:tasks.workunit.client.1.smithi142.stdout:6/859: symlink d1a/d3c/d100/l12c 0 2024-06-15T01:07:10.797 INFO:tasks.workunit.client.0.smithi137.stdout:7/494: link df/d13/d3c/f6e df/d7d/d1c/d1d/d27/d33/f9b 0 2024-06-15T01:07:10.798 INFO:tasks.workunit.client.1.smithi142.stdout:3/901: creat d0/d95/dbc/f110 x:0 0 0 2024-06-15T01:07:10.799 INFO:tasks.workunit.client.1.smithi142.stdout:5/801: dread d2/de/d19/d5f/d68/f73 [4194304,4194304] 0 2024-06-15T01:07:10.799 INFO:tasks.workunit.client.0.smithi137.stdout:1/639: creat d0/d3/d47/dd7/fde x:0 0 0 2024-06-15T01:07:10.800 INFO:tasks.workunit.client.1.smithi142.stdout:5/802: dread - d2/de/d19/d5f/fd2 zero size 2024-06-15T01:07:10.801 INFO:tasks.workunit.client.0.smithi137.stdout:1/640: write d0/d10/daf/fb6 [422610,105892] 0 2024-06-15T01:07:10.802 INFO:tasks.workunit.client.0.smithi137.stdout:7/495: rename df/d13/d2e/d1a/f77 to df/d7d/d1c/f9c 0 2024-06-15T01:07:10.804 INFO:tasks.workunit.client.1.smithi142.stdout:6/860: chown d1a/d2d/d2e/l36 1884 1 2024-06-15T01:07:10.805 INFO:tasks.workunit.client.1.smithi142.stdout:6/861: readlink d1a/d2d/l35 0 2024-06-15T01:07:10.806 INFO:tasks.workunit.client.1.smithi142.stdout:6/862: dread - d1a/d83/d107/d11a/f11e zero size 2024-06-15T01:07:10.806 INFO:tasks.workunit.client.1.smithi142.stdout:4/914: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102/d116/f13e x:0 0 0 2024-06-15T01:07:10.807 INFO:tasks.workunit.client.0.smithi137.stdout:1/641: rmdir d0/d1f/d67/d81 0 2024-06-15T01:07:10.807 INFO:tasks.workunit.client.1.smithi142.stdout:3/902: symlink d0/d95/dbc/l111 0 2024-06-15T01:07:10.808 INFO:tasks.workunit.client.1.smithi142.stdout:8/874: dwrite d0/d17/f1f [0,4194304] 0 2024-06-15T01:07:10.810 INFO:tasks.workunit.client.1.smithi142.stdout:3/903: write d0/d3/f3d [2309450,123259] 0 2024-06-15T01:07:10.810 INFO:tasks.workunit.client.0.smithi137.stdout:1/642: rmdir d0/d3/d14/dba 0 2024-06-15T01:07:10.813 INFO:tasks.workunit.client.1.smithi142.stdout:0/818: truncate d3/d39/d3e/d81/d4c/faf 3651464 0 2024-06-15T01:07:10.814 INFO:tasks.workunit.client.0.smithi137.stdout:1/643: link d0/d1f/c51 d0/d3/d14/da0/cdf 0 2024-06-15T01:07:10.816 INFO:tasks.workunit.client.1.smithi142.stdout:1/903: write d6/d25/dfc/d132/db2/dbf/f104 [645515,7271] 0 2024-06-15T01:07:10.816 INFO:tasks.workunit.client.1.smithi142.stdout:5/803: write d2/fb7 [3206250,102468] 0 2024-06-15T01:07:10.816 INFO:tasks.workunit.client.0.smithi137.stdout:1/644: mkdir d0/d10/d1d/de0 0 2024-06-15T01:07:10.817 INFO:tasks.workunit.client.1.smithi142.stdout:5/804: stat d2/d18/d2e/f71 0 2024-06-15T01:07:10.817 INFO:tasks.workunit.client.0.smithi137.stdout:1/645: dread - d0/d3/d1b/d4c/d59/fb4 zero size 2024-06-15T01:07:10.819 INFO:tasks.workunit.client.1.smithi142.stdout:4/915: mknod d5/d1c/d6f/dc7/d30/d46/d50/d51/c13f 0 2024-06-15T01:07:10.819 INFO:tasks.workunit.client.0.smithi137.stdout:1/646: write d0/d1f/d76/fda [687599,48815] 0 2024-06-15T01:07:10.819 INFO:tasks.workunit.client.1.smithi142.stdout:4/916: readlink d5/d9/d23/l96 0 2024-06-15T01:07:10.820 INFO:tasks.workunit.client.1.smithi142.stdout:7/868: link d2/d6/d22/d47/l7e d2/d8/dd/l110 0 2024-06-15T01:07:10.822 INFO:tasks.workunit.client.0.smithi137.stdout:1/647: read d0/d1f/d76/f8f [720491,1482] 0 2024-06-15T01:07:10.823 INFO:tasks.workunit.client.1.smithi142.stdout:8/875: rename d0/d17/c5d to d0/d25/d53/da7/dcb/dfb/c104 0 2024-06-15T01:07:10.824 INFO:tasks.workunit.client.1.smithi142.stdout:3/904: rmdir d0/d3/da/d18 39 2024-06-15T01:07:10.828 INFO:tasks.workunit.client.1.smithi142.stdout:1/904: symlink d6/d12/d80/dae/l13d 0 2024-06-15T01:07:10.832 INFO:tasks.workunit.client.1.smithi142.stdout:1/905: dread d6/d12/f22 [0,4194304] 0 2024-06-15T01:07:10.832 INFO:tasks.workunit.client.1.smithi142.stdout:4/917: mknod d5/d9/d23/d32/c140 0 2024-06-15T01:07:10.833 INFO:tasks.workunit.client.1.smithi142.stdout:1/906: readlink d6/d12/d3c/d63/d65/l12a 0 2024-06-15T01:07:10.834 INFO:tasks.workunit.client.0.smithi137.stdout:1/648: dwrite d0/d3/d14/d34/f52 [0,4194304] 0 2024-06-15T01:07:10.837 INFO:tasks.workunit.client.1.smithi142.stdout:7/869: dread d2/d6/d22/d3c/f56 [0,4194304] 0 2024-06-15T01:07:10.837 INFO:tasks.workunit.client.1.smithi142.stdout:5/805: rename d2/de/d19/f36 to d2/de/d27/db2/ffc 0 2024-06-15T01:07:10.838 INFO:tasks.workunit.client.1.smithi142.stdout:3/905: rmdir d0/d3/da 39 2024-06-15T01:07:10.839 INFO:tasks.workunit.client.1.smithi142.stdout:3/906: chown d0/d1b/d6b/dbd 26436 1 2024-06-15T01:07:10.839 INFO:tasks.workunit.client.1.smithi142.stdout:5/806: write d2/de/d27/d86/dbd/fdf [577193,36958] 0 2024-06-15T01:07:10.841 INFO:tasks.workunit.client.1.smithi142.stdout:5/807: chown d2/d18/d2e/dae 51 1 2024-06-15T01:07:10.842 INFO:tasks.workunit.client.1.smithi142.stdout:8/876: mkdir d0/d17/d105 0 2024-06-15T01:07:10.846 INFO:tasks.workunit.client.1.smithi142.stdout:5/808: write d2/de/d19/d5f/d68/d76/d96/fde [825641,98736] 0 2024-06-15T01:07:10.848 INFO:tasks.workunit.client.1.smithi142.stdout:0/819: creat d3/d39/d9a/f116 x:0 0 0 2024-06-15T01:07:10.848 INFO:tasks.workunit.client.1.smithi142.stdout:3/907: creat d0/d95/f112 x:0 0 0 2024-06-15T01:07:10.849 INFO:tasks.workunit.client.1.smithi142.stdout:6/863: truncate d1a/d3c/d100/fb3 118491 0 2024-06-15T01:07:10.850 INFO:tasks.workunit.client.1.smithi142.stdout:7/870: dwrite d2/d8/da/d33/d25/d79/f95 [0,4194304] 0 2024-06-15T01:07:10.850 INFO:tasks.workunit.client.1.smithi142.stdout:4/918: mknod d5/c141 0 2024-06-15T01:07:10.852 INFO:tasks.workunit.client.1.smithi142.stdout:2/813: sync 2024-06-15T01:07:10.855 INFO:tasks.workunit.client.1.smithi142.stdout:5/809: dread d2/f14 [0,4194304] 0 2024-06-15T01:07:10.857 INFO:tasks.workunit.client.1.smithi142.stdout:7/871: creat d2/d6/d22/d3c/f111 x:0 0 0 2024-06-15T01:07:10.858 INFO:tasks.workunit.client.1.smithi142.stdout:8/877: link d0/l70 d0/d25/d53/d9c/l106 0 2024-06-15T01:07:10.860 INFO:tasks.workunit.client.1.smithi142.stdout:4/919: rename d5/d9/d41/f65 to d5/d9/da/d92/d98/dc1/d133/f142 0 2024-06-15T01:07:10.861 INFO:tasks.workunit.client.0.smithi137.stdout:1/649: creat d0/d3/d1b/d4c/d59/dd4/fe1 x:0 0 0 2024-06-15T01:07:10.861 INFO:tasks.workunit.client.1.smithi142.stdout:1/907: link d6/d12/c15 d6/d12/d80/dae/c13e 0 2024-06-15T01:07:10.861 INFO:tasks.workunit.client.0.smithi137.stdout:1/650: stat d0/d3/d89/fbc 0 2024-06-15T01:07:10.862 INFO:tasks.workunit.client.1.smithi142.stdout:2/814: unlink d8/df/d10/d94/db5/ded/fe8 0 2024-06-15T01:07:10.865 INFO:tasks.workunit.client.1.smithi142.stdout:2/815: chown d8/df/d10/d94/db5/ded 607920 1 2024-06-15T01:07:10.865 INFO:tasks.workunit.client.1.smithi142.stdout:0/820: dwrite d3/d39/d3e/d81/d1b/d38/da3/fed [0,4194304] 0 2024-06-15T01:07:10.866 INFO:tasks.workunit.client.1.smithi142.stdout:6/864: unlink d1a/d3c/d100/c95 0 2024-06-15T01:07:10.867 INFO:tasks.workunit.client.1.smithi142.stdout:7/872: symlink d2/d8/dd/d14/d2c/d70/l112 0 2024-06-15T01:07:10.868 INFO:tasks.workunit.client.0.smithi137.stdout:1/651: dwrite d0/d3/d14/f38 [0,4194304] 0 2024-06-15T01:07:10.868 INFO:tasks.workunit.client.1.smithi142.stdout:8/878: rmdir d0/d25/db9 39 2024-06-15T01:07:10.870 INFO:tasks.workunit.client.1.smithi142.stdout:4/920: mkdir d5/d9/da/d92/d98/dc1/d143 0 2024-06-15T01:07:10.871 INFO:tasks.workunit.client.1.smithi142.stdout:2/816: dread d8/df/d10/f87 [0,4194304] 0 2024-06-15T01:07:10.872 INFO:tasks.workunit.client.1.smithi142.stdout:4/921: write d5/d1c/d6f/dc7/d30/d46/d50/d51/f75 [1011589,92113] 0 2024-06-15T01:07:10.872 INFO:tasks.workunit.client.1.smithi142.stdout:5/810: unlink d2/d3e/d31/c57 0 2024-06-15T01:07:10.875 INFO:tasks.workunit.client.1.smithi142.stdout:7/873: creat d2/d8/da/d33/d25/d79/f113 x:0 0 0 2024-06-15T01:07:10.875 INFO:tasks.workunit.client.1.smithi142.stdout:4/922: readlink d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/l66 0 2024-06-15T01:07:10.876 INFO:tasks.workunit.client.1.smithi142.stdout:6/865: write d1a/d83/daa/fb6 [1419883,68073] 0 2024-06-15T01:07:10.876 INFO:tasks.workunit.client.1.smithi142.stdout:9/790: sync 2024-06-15T01:07:10.877 INFO:tasks.workunit.client.1.smithi142.stdout:4/923: chown d5/d1c/d6f/f10e 55137524 1 2024-06-15T01:07:10.879 INFO:tasks.workunit.client.1.smithi142.stdout:5/811: read d2/d3e/d74/da5/ffb [454001,120417] 0 2024-06-15T01:07:10.880 INFO:tasks.workunit.client.1.smithi142.stdout:9/791: read d6/dc/f19 [2631003,26072] 0 2024-06-15T01:07:10.881 INFO:tasks.workunit.client.1.smithi142.stdout:1/908: rename d6/d12/da0/db0 to d6/d12/d1c/d13f 0 2024-06-15T01:07:10.889 INFO:tasks.workunit.client.1.smithi142.stdout:5/812: rmdir d2/d18/d2e 39 2024-06-15T01:07:10.890 INFO:tasks.workunit.client.1.smithi142.stdout:5/813: stat d2/de/d27/f9e 0 2024-06-15T01:07:10.890 INFO:tasks.workunit.client.1.smithi142.stdout:1/909: stat d6/d46/l8f 0 2024-06-15T01:07:10.891 INFO:tasks.workunit.client.1.smithi142.stdout:4/924: mknod d5/d9/da/d92/d98/dc1/d143/c144 0 2024-06-15T01:07:10.895 INFO:tasks.workunit.client.1.smithi142.stdout:5/814: dwrite d2/d3e/d74/fcf [0,4194304] 0 2024-06-15T01:07:10.895 INFO:tasks.workunit.client.1.smithi142.stdout:1/910: creat d6/d12/d3c/d63/d8d/d97/dc6/f140 x:0 0 0 2024-06-15T01:07:10.897 INFO:tasks.workunit.client.0.smithi137.stdout:1/652: truncate d0/d3/d14/f19 2612053 0 2024-06-15T01:07:10.897 INFO:tasks.workunit.client.1.smithi142.stdout:0/821: rename d3/d39/d9a/lb7 to d3/d39/d3e/d81/d44/d7c/l117 0 2024-06-15T01:07:10.899 INFO:tasks.workunit.client.1.smithi142.stdout:6/866: getdents d1a 0 2024-06-15T01:07:10.903 INFO:tasks.workunit.client.1.smithi142.stdout:8/879: rename d0/d25/d5a/f85 to d0/d25/d5a/d84/dd0/f107 0 2024-06-15T01:07:10.904 INFO:tasks.workunit.client.1.smithi142.stdout:8/880: stat d0/d25/d5a/f4d 0 2024-06-15T01:07:10.905 INFO:tasks.workunit.client.1.smithi142.stdout:0/822: write d3/d39/d3e/d81/d4c/d61/fb5 [1124998,1250] 0 2024-06-15T01:07:10.906 INFO:tasks.workunit.client.1.smithi142.stdout:5/815: symlink d2/d3e/df2/lfd 0 2024-06-15T01:07:10.908 INFO:tasks.workunit.client.1.smithi142.stdout:6/867: creat d1a/d2d/d80/d8e/d110/f12d x:0 0 0 2024-06-15T01:07:10.909 INFO:tasks.workunit.client.1.smithi142.stdout:5/816: chown d2/de/d27/d86/db9 27067399 1 2024-06-15T01:07:10.911 INFO:tasks.workunit.client.1.smithi142.stdout:9/792: rename d6/dc/d1b/d6f/da1/caf to d6/d34/df3/c125 0 2024-06-15T01:07:10.912 INFO:tasks.workunit.client.1.smithi142.stdout:6/868: chown d1a/d3c/d100/d58/d87/f89 2 1 2024-06-15T01:07:10.913 INFO:tasks.workunit.client.1.smithi142.stdout:8/881: dwrite d0/d17/d40/fa2 [0,4194304] 0 2024-06-15T01:07:10.913 INFO:tasks.workunit.client.1.smithi142.stdout:1/911: creat d6/d3a/f141 x:0 0 0 2024-06-15T01:07:10.917 INFO:tasks.workunit.client.1.smithi142.stdout:7/874: write d2/f1a [478110,16116] 0 2024-06-15T01:07:10.917 INFO:tasks.workunit.client.1.smithi142.stdout:1/912: readlink d6/d25/l72 0 2024-06-15T01:07:10.917 INFO:tasks.workunit.client.1.smithi142.stdout:3/908: dwrite d0/d1f/d20/d5d/d82/f63 [0,4194304] 0 2024-06-15T01:07:10.917 INFO:tasks.workunit.client.1.smithi142.stdout:9/793: chown d6/dc/d1b/d25/d35/l121 41 1 2024-06-15T01:07:10.929 INFO:tasks.workunit.client.1.smithi142.stdout:5/817: dwrite d2/d3e/d98/fd0 [0,4194304] 0 2024-06-15T01:07:10.930 INFO:tasks.workunit.client.1.smithi142.stdout:3/909: dread d0/d3/f1a [0,4194304] 0 2024-06-15T01:07:10.932 INFO:tasks.workunit.client.1.smithi142.stdout:0/823: dread d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [0,4194304] 0 2024-06-15T01:07:10.932 INFO:tasks.workunit.client.1.smithi142.stdout:1/913: dwrite d6/d25/dfc/fd6 [0,4194304] 0 2024-06-15T01:07:10.933 INFO:tasks.workunit.client.1.smithi142.stdout:0/824: readlink d3/d39/d3e/d81/d4c/l89 0 2024-06-15T01:07:10.946 INFO:tasks.workunit.client.0.smithi137.stdout:5/597: sync 2024-06-15T01:07:10.948 INFO:tasks.workunit.client.0.smithi137.stdout:1/653: dread d0/d1f/f4d [0,4194304] 0 2024-06-15T01:07:10.949 INFO:tasks.workunit.client.0.smithi137.stdout:1/654: stat d0/d1f/d67/ldd 0 2024-06-15T01:07:10.951 INFO:tasks.workunit.client.1.smithi142.stdout:8/882: mknod d0/d25/d53/d9c/c108 0 2024-06-15T01:07:10.951 INFO:tasks.workunit.client.0.smithi137.stdout:5/598: fsync df/d44/d54/d95/d3b/d49/f77 0 2024-06-15T01:07:10.952 INFO:tasks.workunit.client.1.smithi142.stdout:2/817: dwrite d8/df/d10/d94/d25/da9/f6f [0,4194304] 0 2024-06-15T01:07:10.952 INFO:tasks.workunit.client.1.smithi142.stdout:9/794: symlink d6/dc/d1b/d25/d58/d74/l126 0 2024-06-15T01:07:10.955 INFO:tasks.workunit.client.0.smithi137.stdout:1/655: read d0/d3/d1b/f4f [878411,126953] 0 2024-06-15T01:07:10.955 INFO:tasks.workunit.client.0.smithi137.stdout:5/599: creat df/d44/d54/d95/d3b/d49/fca x:0 0 0 2024-06-15T01:07:10.956 INFO:tasks.workunit.client.1.smithi142.stdout:9/795: write d6/d34/d8a/dec/ff5 [885261,122492] 0 2024-06-15T01:07:10.956 INFO:tasks.workunit.client.1.smithi142.stdout:9/796: dread - d6/d31/f120 zero size 2024-06-15T01:07:10.957 INFO:tasks.workunit.client.1.smithi142.stdout:9/797: chown d6/dc/d1b/d6f/d103/df2 1942 1 2024-06-15T01:07:10.958 INFO:tasks.workunit.client.0.smithi137.stdout:5/600: fsync df/d11/f6d 0 2024-06-15T01:07:10.958 INFO:tasks.workunit.client.1.smithi142.stdout:0/825: dread d3/d39/d3e/d81/d44/d7c/f95 [0,4194304] 0 2024-06-15T01:07:10.961 INFO:tasks.workunit.client.0.smithi137.stdout:1/656: rename d0/cd1 to d0/d3/d1b/d4c/ce2 0 2024-06-15T01:07:10.962 INFO:tasks.workunit.client.0.smithi137.stdout:1/657: dread d0/f3b [0,4194304] 0 2024-06-15T01:07:10.966 INFO:tasks.workunit.client.1.smithi142.stdout:2/818: getdents d8/d19/da3 0 2024-06-15T01:07:10.967 INFO:tasks.workunit.client.1.smithi142.stdout:2/819: write d8/df/d10/d94/d25/da9/f6f [2546933,125946] 0 2024-06-15T01:07:10.969 INFO:tasks.workunit.client.1.smithi142.stdout:5/818: mknod d2/d18/d2e/dae/cfe 0 2024-06-15T01:07:10.971 INFO:tasks.workunit.client.1.smithi142.stdout:7/875: link d2/d8/dd/d14/d2c/d37/la9 d2/d8/dd/d14/d2c/l114 0 2024-06-15T01:07:10.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/820: write d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa4 [1407311,18] 0 2024-06-15T01:07:10.973 INFO:tasks.workunit.client.1.smithi142.stdout:9/798: getdents d6/dc/d1b/d25/d58/de2 0 2024-06-15T01:07:10.976 INFO:tasks.workunit.client.1.smithi142.stdout:2/821: symlink d8/df/d10/d94/d25/d22/lfc 0 2024-06-15T01:07:10.977 INFO:tasks.workunit.client.1.smithi142.stdout:9/799: symlink d6/d34/df3/l127 0 2024-06-15T01:07:10.977 INFO:tasks.workunit.client.1.smithi142.stdout:5/819: link d2/d18/d2e/dae/cfe d2/de/d19/d5f/cff 0 2024-06-15T01:07:10.980 INFO:tasks.workunit.client.1.smithi142.stdout:7/876: rmdir d2/d8/da/d33/d25/d79/de0 0 2024-06-15T01:07:10.981 INFO:tasks.workunit.client.1.smithi142.stdout:9/800: creat d6/dc/dd/f128 x:0 0 0 2024-06-15T01:07:10.982 INFO:tasks.workunit.client.1.smithi142.stdout:5/820: symlink d2/de/d19/d5f/l100 0 2024-06-15T01:07:10.984 INFO:tasks.workunit.client.1.smithi142.stdout:9/801: creat d6/dc/d1b/d25/d58/de2/f129 x:0 0 0 2024-06-15T01:07:10.985 INFO:tasks.workunit.client.1.smithi142.stdout:9/802: write d6/dc/d1b/d25/d35/ff1 [259232,61034] 0 2024-06-15T01:07:10.986 INFO:tasks.workunit.client.1.smithi142.stdout:8/883: rename d0/d25/d5a/d84/dd0 to d0/d25/d5a/d8f/df3/d109 0 2024-06-15T01:07:10.988 INFO:tasks.workunit.client.1.smithi142.stdout:7/877: mkdir d2/d8/dd/d8a/de2/d115 0 2024-06-15T01:07:10.988 INFO:tasks.workunit.client.0.smithi137.stdout:1/658: write d0/d1f/d60/f74 [330559,81052] 0 2024-06-15T01:07:10.990 INFO:tasks.workunit.client.0.smithi137.stdout:1/659: fsync d0/d3/d1b/d4c/f75 0 2024-06-15T01:07:10.991 INFO:tasks.workunit.client.0.smithi137.stdout:5/601: dwrite df/d11/d30/f82 [0,4194304] 0 2024-06-15T01:07:10.992 INFO:tasks.workunit.client.1.smithi142.stdout:3/910: rename d0/d1f/d20/d5d/d60/f10a to d0/d1f/d20/d59/d61/d96/f113 0 2024-06-15T01:07:10.993 INFO:tasks.workunit.client.1.smithi142.stdout:8/884: truncate d0/f6f 659305 0 2024-06-15T01:07:10.994 INFO:tasks.workunit.client.0.smithi137.stdout:5/602: symlink df/d76/d7c/lcb 0 2024-06-15T01:07:10.995 INFO:tasks.workunit.client.0.smithi137.stdout:1/660: rename d0/d1f/d60/d99/da3 to d0/d1f/d60/d99/de3 0 2024-06-15T01:07:10.996 INFO:tasks.workunit.client.0.smithi137.stdout:1/661: stat d0/d3/f2f 0 2024-06-15T01:07:10.996 INFO:tasks.workunit.client.0.smithi137.stdout:5/603: creat df/d44/d54/fcc x:0 0 0 2024-06-15T01:07:10.997 INFO:tasks.workunit.client.0.smithi137.stdout:1/662: chown d0/d1f/d60/da9 0 1 2024-06-15T01:07:10.998 INFO:tasks.workunit.client.1.smithi142.stdout:7/878: rename d2/d8/dd/d14/d2c/d37/d6b/d88 to d2/d8/dd/d14/d2c/d37/d6b/d116 0 2024-06-15T01:07:11.000 INFO:tasks.workunit.client.0.smithi137.stdout:1/663: symlink d0/d3/d1b/d4c/d59/d64/le4 0 2024-06-15T01:07:11.002 INFO:tasks.workunit.client.1.smithi142.stdout:3/911: creat d0/d3/de2/d55/dae/f114 x:0 0 0 2024-06-15T01:07:11.002 INFO:tasks.workunit.client.0.smithi137.stdout:1/664: mkdir d0/d3/d1b/d4c/de5 0 2024-06-15T01:07:11.003 INFO:tasks.workunit.client.1.smithi142.stdout:3/912: write d0/d1f/d20/d5d/ff9 [2790212,15568] 0 2024-06-15T01:07:11.006 INFO:tasks.workunit.client.0.smithi137.stdout:1/665: unlink d0/d1f/d67/ldd 0 2024-06-15T01:07:11.006 INFO:tasks.workunit.client.1.smithi142.stdout:7/879: creat d2/da2/f117 x:0 0 0 2024-06-15T01:07:11.007 INFO:tasks.workunit.client.0.smithi137.stdout:1/666: read d0/d3/d89/fb3 [1379589,99250] 0 2024-06-15T01:07:11.008 INFO:tasks.workunit.client.1.smithi142.stdout:5/821: getdents d2/de/d85 0 2024-06-15T01:07:11.009 INFO:tasks.workunit.client.1.smithi142.stdout:5/822: chown d2/de/d27/f46 9053302 1 2024-06-15T01:07:11.010 INFO:tasks.workunit.client.1.smithi142.stdout:1/914: truncate d6/d12/d80/fdf 1615395 0 2024-06-15T01:07:11.011 INFO:tasks.workunit.client.0.smithi137.stdout:1/667: dread d0/d3/d14/d34/f52 [0,4194304] 0 2024-06-15T01:07:11.013 INFO:tasks.workunit.client.1.smithi142.stdout:5/823: stat d2/de/d19/f22 0 2024-06-15T01:07:11.014 INFO:tasks.workunit.client.1.smithi142.stdout:0/826: write d3/d39/d3e/fe0 [963107,111235] 0 2024-06-15T01:07:11.014 INFO:tasks.workunit.client.1.smithi142.stdout:7/880: dread d2/d8/dd/dcf/ff0 [0,4194304] 0 2024-06-15T01:07:11.015 INFO:tasks.workunit.client.0.smithi137.stdout:1/668: creat d0/d3/d1b/fe6 x:0 0 0 2024-06-15T01:07:11.015 INFO:tasks.workunit.client.1.smithi142.stdout:7/881: write d2/d8/dd8/fed [366452,21208] 0 2024-06-15T01:07:11.016 INFO:tasks.workunit.client.0.smithi137.stdout:5/604: dread fe [0,4194304] 0 2024-06-15T01:07:11.017 INFO:tasks.workunit.client.0.smithi137.stdout:1/669: truncate d0/d3/d47/f92 479436 0 2024-06-15T01:07:11.018 INFO:tasks.workunit.client.0.smithi137.stdout:1/670: fdatasync d0/d3/d1b/f50 0 2024-06-15T01:07:11.019 INFO:tasks.workunit.client.1.smithi142.stdout:1/915: creat d6/de4/f142 x:0 0 0 2024-06-15T01:07:11.019 INFO:tasks.workunit.client.1.smithi142.stdout:9/803: rename d6/dc/d1b/d25/d58/d74/f102 to d6/f12a 0 2024-06-15T01:07:11.020 INFO:tasks.workunit.client.1.smithi142.stdout:9/804: fdatasync d6/dc/d1b/d25/d58/fd6 0 2024-06-15T01:07:11.021 INFO:tasks.workunit.client.1.smithi142.stdout:8/885: getdents d0/d25/d53/da7/dcb/dfb 0 2024-06-15T01:07:11.028 INFO:tasks.workunit.client.0.smithi137.stdout:1/671: write d0/d2d/d3e/f96 [83521,25789] 0 2024-06-15T01:07:11.030 INFO:tasks.workunit.client.0.smithi137.stdout:1/672: stat d0/d10/d1d/l43 0 2024-06-15T01:07:11.031 INFO:tasks.workunit.client.0.smithi137.stdout:5/605: write df/d44/d54/d95/d3b/d49/f77 [4952608,41401] 0 2024-06-15T01:07:11.035 INFO:tasks.workunit.client.0.smithi137.stdout:1/673: dread d0/d3/d1b/d4c/f8e [0,4194304] 0 2024-06-15T01:07:11.036 INFO:tasks.workunit.client.0.smithi137.stdout:5/606: read df/d76/f8f [524897,87059] 0 2024-06-15T01:07:11.036 INFO:tasks.workunit.client.0.smithi137.stdout:1/674: readlink d0/d2d/d3e/l5b 0 2024-06-15T01:07:11.038 INFO:tasks.workunit.client.0.smithi137.stdout:5/607: creat df/d11/d70/dbf/fcd x:0 0 0 2024-06-15T01:07:11.038 INFO:tasks.workunit.client.0.smithi137.stdout:1/675: fsync d0/d3/d1b/d4c/f8e 0 2024-06-15T01:07:11.040 INFO:tasks.workunit.client.0.smithi137.stdout:5/608: creat df/d11/d14/d6c/fce x:0 0 0 2024-06-15T01:07:11.041 INFO:tasks.workunit.client.0.smithi137.stdout:5/609: read df/d11/f13 [7209031,6042] 0 2024-06-15T01:07:11.041 INFO:tasks.workunit.client.1.smithi142.stdout:8/886: write d0/d17/d40/d86/fec [3323,95903] 0 2024-06-15T01:07:11.042 INFO:tasks.workunit.client.1.smithi142.stdout:5/824: fdatasync d2/de/d27/f77 0 2024-06-15T01:07:11.042 INFO:tasks.workunit.client.1.smithi142.stdout:7/882: symlink d2/d8/dd/d14/d2c/d70/l118 0 2024-06-15T01:07:11.042 INFO:tasks.workunit.client.0.smithi137.stdout:1/676: rename d0/d1f/d76/f8f to d0/d3/d1b/d4c/de5/fe7 0 2024-06-15T01:07:11.043 INFO:tasks.workunit.client.0.smithi137.stdout:1/677: write d0/d2d/f62 [451954,37749] 0 2024-06-15T01:07:11.046 INFO:tasks.workunit.client.1.smithi142.stdout:5/825: stat d2/de/d19/d5f/d68/d76/da6/dc5/fd5 0 2024-06-15T01:07:11.047 INFO:tasks.workunit.client.0.smithi137.stdout:1/678: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:07:11.047 INFO:tasks.workunit.client.1.smithi142.stdout:9/805: dread d6/dc/d51/f80 [0,4194304] 0 2024-06-15T01:07:11.049 INFO:tasks.workunit.client.0.smithi137.stdout:1/679: fdatasync d0/d2d/d3e/fa4 0 2024-06-15T01:07:11.051 INFO:tasks.workunit.client.1.smithi142.stdout:5/826: creat d2/de/d27/f101 x:0 0 0 2024-06-15T01:07:11.052 INFO:tasks.workunit.client.1.smithi142.stdout:9/806: write d6/dc/d1b/d6f/d103/f9f [1552520,114195] 0 2024-06-15T01:07:11.052 INFO:tasks.workunit.client.1.smithi142.stdout:3/913: rename d0/d3/de2/d55/l56 to d0/d1f/d20/d5d/d60/l115 0 2024-06-15T01:07:11.052 INFO:tasks.workunit.client.1.smithi142.stdout:1/916: dwrite d6/d12/d1c/d2c/d93/db9/df2/f126 [0,4194304] 0 2024-06-15T01:07:11.053 INFO:tasks.workunit.client.1.smithi142.stdout:8/887: creat d0/d17/d40/db8/f10a x:0 0 0 2024-06-15T01:07:11.055 INFO:tasks.workunit.client.0.smithi137.stdout:5/610: dwrite df/d44/d54/d95/f6f [0,4194304] 0 2024-06-15T01:07:11.055 INFO:tasks.workunit.client.1.smithi142.stdout:5/827: read - d2/de/d27/fab zero size 2024-06-15T01:07:11.056 INFO:tasks.workunit.client.1.smithi142.stdout:9/807: symlink d6/dc/dd/l12b 0 2024-06-15T01:07:11.057 INFO:tasks.workunit.client.1.smithi142.stdout:7/883: dread d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:07:11.057 INFO:tasks.workunit.client.1.smithi142.stdout:3/914: fdatasync d0/d95/ff2 0 2024-06-15T01:07:11.058 INFO:tasks.workunit.client.0.smithi137.stdout:5/611: symlink df/d11/d14/lcf 0 2024-06-15T01:07:11.059 INFO:tasks.workunit.client.1.smithi142.stdout:2/822: rename d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/fa4 to d8/df/d10/d94/d25/ffd 0 2024-06-15T01:07:11.060 INFO:tasks.workunit.client.0.smithi137.stdout:5/612: creat df/d11/d14/d6c/dc7/fd0 x:0 0 0 2024-06-15T01:07:11.063 INFO:tasks.workunit.client.1.smithi142.stdout:9/808: symlink d6/dc/d1b/d25/dcf/l12c 0 2024-06-15T01:07:11.066 INFO:tasks.workunit.client.1.smithi142.stdout:7/884: creat d2/d8/da/d33/d25/d79/f119 x:0 0 0 2024-06-15T01:07:11.066 INFO:tasks.workunit.client.1.smithi142.stdout:5/828: truncate d2/d18/f63 290040 0 2024-06-15T01:07:11.067 INFO:tasks.workunit.client.1.smithi142.stdout:3/915: dwrite d0/d1b/fe4 [0,4194304] 0 2024-06-15T01:07:11.067 INFO:tasks.workunit.client.1.smithi142.stdout:8/888: mknod d0/d25/d5a/d8f/d97/ded/c10b 0 2024-06-15T01:07:11.071 INFO:tasks.workunit.client.1.smithi142.stdout:2/823: creat d8/df/ffe x:0 0 0 2024-06-15T01:07:11.071 INFO:tasks.workunit.client.1.smithi142.stdout:2/824: stat d8/d19/deb/df8 0 2024-06-15T01:07:11.072 INFO:tasks.workunit.client.1.smithi142.stdout:5/829: symlink d2/de/d19/d93/l102 0 2024-06-15T01:07:11.073 INFO:tasks.workunit.client.1.smithi142.stdout:5/830: chown d2/d3e/d98/c9b 559136834 1 2024-06-15T01:07:11.075 INFO:tasks.workunit.client.1.smithi142.stdout:5/831: read - d2/de/d27/f101 zero size 2024-06-15T01:07:11.076 INFO:tasks.workunit.client.1.smithi142.stdout:9/809: link d6/dc/dd/le3 d6/dc/d4e/l12d 0 2024-06-15T01:07:11.077 INFO:tasks.workunit.client.0.smithi137.stdout:5/613: dread f7 [0,4194304] 0 2024-06-15T01:07:11.078 INFO:tasks.workunit.client.1.smithi142.stdout:8/889: creat d0/d25/d53/de2/f10c x:0 0 0 2024-06-15T01:07:11.080 INFO:tasks.workunit.client.1.smithi142.stdout:2/825: dwrite f4 [4194304,4194304] 0 2024-06-15T01:07:11.081 INFO:tasks.workunit.client.1.smithi142.stdout:9/810: readlink d6/dc/d1b/d25/d58/d74/l126 0 2024-06-15T01:07:11.081 INFO:tasks.workunit.client.0.smithi137.stdout:5/614: truncate df/d11/d14/f16 614789 0 2024-06-15T01:07:11.082 INFO:tasks.workunit.client.0.smithi137.stdout:5/615: truncate df/d11/d70/dbf/fcd 649826 0 2024-06-15T01:07:11.083 INFO:tasks.workunit.client.1.smithi142.stdout:9/811: write d6/dc/d1b/d6f/da1/df6/f118 [234946,110585] 0 2024-06-15T01:07:11.085 INFO:tasks.workunit.client.1.smithi142.stdout:2/826: stat d8/df/d10/d94/d25/d1a/la7 0 2024-06-15T01:07:11.085 INFO:tasks.workunit.client.1.smithi142.stdout:7/885: write d2/d8/dd/dcf/ff0 [780392,71348] 0 2024-06-15T01:07:11.087 INFO:tasks.workunit.client.1.smithi142.stdout:7/886: dread - d2/d8/da/d33/ffc zero size 2024-06-15T01:07:11.089 INFO:tasks.workunit.client.1.smithi142.stdout:5/832: link d2/de/d19/d5f/d68/d76/f7c d2/d3e/d31/f103 0 2024-06-15T01:07:11.090 INFO:tasks.workunit.client.1.smithi142.stdout:2/827: mknod d8/df/d10/d94/d25/d22/d43/d51/dd7/cff 0 2024-06-15T01:07:11.091 INFO:tasks.workunit.client.1.smithi142.stdout:2/828: chown d8/d19/d33/c62 4109916 1 2024-06-15T01:07:11.092 INFO:tasks.workunit.client.1.smithi142.stdout:7/887: unlink d2/d8/da/d33/d25/f67 0 2024-06-15T01:07:11.092 INFO:tasks.workunit.client.1.smithi142.stdout:2/829: write d8/df/d10/d94/d25/d76/fc6 [306923,30972] 0 2024-06-15T01:07:11.092 INFO:tasks.workunit.client.1.smithi142.stdout:4/925: sync 2024-06-15T01:07:11.093 INFO:tasks.workunit.client.1.smithi142.stdout:5/833: dread - d2/de/d27/db2/ffc zero size 2024-06-15T01:07:11.095 INFO:tasks.workunit.client.1.smithi142.stdout:4/926: creat d5/d9/da/f145 x:0 0 0 2024-06-15T01:07:11.095 INFO:tasks.workunit.client.1.smithi142.stdout:2/830: creat d8/df/d10/d94/d25/d22/d43/f100 x:0 0 0 2024-06-15T01:07:11.097 INFO:tasks.workunit.client.1.smithi142.stdout:5/834: mkdir d2/d3e/d74/da5/dbe/d104 0 2024-06-15T01:07:11.100 INFO:tasks.workunit.client.1.smithi142.stdout:2/831: unlink d8/df/d10/d94/d25/d22/d43/d52/dc1/ff1 0 2024-06-15T01:07:11.101 INFO:tasks.workunit.client.1.smithi142.stdout:5/835: mkdir d2/d3e/d31/d105 0 2024-06-15T01:07:11.103 INFO:tasks.workunit.client.1.smithi142.stdout:4/927: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117/f11b [0,4194304] 0 2024-06-15T01:07:11.104 INFO:tasks.workunit.client.1.smithi142.stdout:5/836: mkdir d2/de/d19/d5f/d68/d106 0 2024-06-15T01:07:11.105 INFO:tasks.workunit.client.1.smithi142.stdout:2/832: creat d8/df/d10/d94/d25/d22/d43/de3/f101 x:0 0 0 2024-06-15T01:07:11.106 INFO:tasks.workunit.client.1.smithi142.stdout:5/837: unlink d2/de/d19/d5f/c8e 0 2024-06-15T01:07:11.107 INFO:tasks.workunit.client.1.smithi142.stdout:5/838: stat d2/d3e/d98/c9b 0 2024-06-15T01:07:11.108 INFO:tasks.workunit.client.1.smithi142.stdout:4/928: getdents d5/d1c/d6f/db9 0 2024-06-15T01:07:11.108 INFO:tasks.workunit.client.1.smithi142.stdout:2/833: mknod d8/df/c102 0 2024-06-15T01:07:11.112 INFO:tasks.workunit.client.1.smithi142.stdout:4/929: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/fc2 [4713423,115431] 0 2024-06-15T01:07:11.112 INFO:tasks.workunit.client.1.smithi142.stdout:2/834: creat d8/d19/deb/df8/f103 x:0 0 0 2024-06-15T01:07:11.114 INFO:tasks.workunit.client.1.smithi142.stdout:4/930: mkdir d5/d9/d1a/dee/d146 0 2024-06-15T01:07:11.115 INFO:tasks.workunit.client.1.smithi142.stdout:2/835: mknod d8/df/d10/d1c/d7c/c104 0 2024-06-15T01:07:11.120 INFO:tasks.workunit.client.1.smithi142.stdout:5/839: dread d2/de/d19/f2a [0,4194304] 0 2024-06-15T01:07:11.125 INFO:tasks.workunit.client.0.smithi137.stdout:6/497: sync 2024-06-15T01:07:11.127 INFO:tasks.workunit.client.1.smithi142.stdout:4/931: unlink d5/c9c 0 2024-06-15T01:07:11.128 INFO:tasks.workunit.client.0.smithi137.stdout:6/498: fdatasync d0/d1a/f65 0 2024-06-15T01:07:11.129 INFO:tasks.workunit.client.1.smithi142.stdout:5/840: read - d2/d18/d79/fce zero size 2024-06-15T01:07:11.131 INFO:tasks.workunit.client.0.smithi137.stdout:5/616: dread df/d44/d54/d95/d35/f9b [0,4194304] 0 2024-06-15T01:07:11.132 INFO:tasks.workunit.client.1.smithi142.stdout:4/932: mkdir d5/d1c/d6f/d147 0 2024-06-15T01:07:11.132 INFO:tasks.workunit.client.0.smithi137.stdout:5/617: truncate df/d44/d54/d95/f6f 4203461 0 2024-06-15T01:07:11.134 INFO:tasks.workunit.client.1.smithi142.stdout:5/841: dread d2/de/d27/f9e [0,4194304] 0 2024-06-15T01:07:11.135 INFO:tasks.workunit.client.0.smithi137.stdout:6/499: write d0/d1a/f65 [1163479,82608] 0 2024-06-15T01:07:11.137 INFO:tasks.workunit.client.1.smithi142.stdout:2/836: dwrite d8/df/d10/f87 [0,4194304] 0 2024-06-15T01:07:11.140 INFO:tasks.workunit.client.0.smithi137.stdout:6/500: creat d0/d5/d9/f9b x:0 0 0 2024-06-15T01:07:11.140 INFO:tasks.workunit.client.1.smithi142.stdout:9/812: write f2 [4864168,130545] 0 2024-06-15T01:07:11.141 INFO:tasks.workunit.client.1.smithi142.stdout:8/890: dread d0/d17/d40/d6e/fc2 [0,4194304] 0 2024-06-15T01:07:11.143 INFO:tasks.workunit.client.1.smithi142.stdout:5/842: symlink d2/de/d19/d5f/d68/d76/d96/l107 0 2024-06-15T01:07:11.143 INFO:tasks.workunit.client.1.smithi142.stdout:3/916: dwrite d0/d3/d15/f77 [0,4194304] 0 2024-06-15T01:07:11.144 INFO:tasks.workunit.client.0.smithi137.stdout:6/501: creat d0/d5/d46/f9c x:0 0 0 2024-06-15T01:07:11.145 INFO:tasks.workunit.client.1.smithi142.stdout:3/917: fdatasync d0/d3/de2/d55/fe6 0 2024-06-15T01:07:11.148 INFO:tasks.workunit.client.1.smithi142.stdout:2/837: rmdir d8/df/d1f 39 2024-06-15T01:07:11.149 INFO:tasks.workunit.client.1.smithi142.stdout:7/888: truncate d2/d6/d22/d47/f50 1607647 0 2024-06-15T01:07:11.150 INFO:tasks.workunit.client.0.smithi137.stdout:6/502: truncate d0/f3c 499540 0 2024-06-15T01:07:11.151 INFO:tasks.workunit.client.0.smithi137.stdout:5/618: dwrite df/d7a/fb5 [0,4194304] 0 2024-06-15T01:07:11.152 INFO:tasks.workunit.client.1.smithi142.stdout:8/891: write d0/d25/db9/f102 [1015608,117624] 0 2024-06-15T01:07:11.153 INFO:tasks.workunit.client.1.smithi142.stdout:8/892: fdatasync d0/d25/d5a/d8f/d98/da4/ff6 0 2024-06-15T01:07:11.153 INFO:tasks.workunit.client.0.smithi137.stdout:6/503: truncate d0/d1a/d3e/d57/d32/d74/d5a/f69 5086814 0 2024-06-15T01:07:11.154 INFO:tasks.workunit.client.1.smithi142.stdout:9/813: symlink d6/dc/d1b/l12e 0 2024-06-15T01:07:11.155 INFO:tasks.workunit.client.0.smithi137.stdout:6/504: stat d0/d1a/d3e/d57/d1d/d2e/d62/d84 0 2024-06-15T01:07:11.158 INFO:tasks.workunit.client.1.smithi142.stdout:4/933: link d5/d9/d1a/fe5 d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f148 0 2024-06-15T01:07:11.158 INFO:tasks.workunit.client.1.smithi142.stdout:6/869: sync 2024-06-15T01:07:11.160 INFO:tasks.workunit.client.1.smithi142.stdout:6/870: write d1a/d39/d42/fbf [888921,101340] 0 2024-06-15T01:07:11.161 INFO:tasks.workunit.client.1.smithi142.stdout:6/871: stat d1a/d39/d42/c8d 0 2024-06-15T01:07:11.161 INFO:tasks.workunit.client.1.smithi142.stdout:4/934: dread d5/d9/f1f [0,4194304] 0 2024-06-15T01:07:11.164 INFO:tasks.workunit.client.1.smithi142.stdout:4/935: fsync d5/d1c/d6f/dc7/d30/d46/d50/f5e 0 2024-06-15T01:07:11.165 INFO:tasks.workunit.client.0.smithi137.stdout:6/505: link d0/d5/f90 d0/d5/d9/d29/d6b/f9d 0 2024-06-15T01:07:11.166 INFO:tasks.workunit.client.0.smithi137.stdout:5/619: rename df/d44/d54/d6b/f86 to df/d44/d54/d95/d3b/d49/d5c/fd1 0 2024-06-15T01:07:11.168 INFO:tasks.workunit.client.1.smithi142.stdout:7/889: fdatasync d2/d8/da/d33/d25/f59 0 2024-06-15T01:07:11.170 INFO:tasks.workunit.client.0.smithi137.stdout:6/506: creat d0/d1a/d3e/d57/d1d/d2e/d62/f9e x:0 0 0 2024-06-15T01:07:11.170 INFO:tasks.workunit.client.1.smithi142.stdout:8/893: mkdir d0/d25/d53/da7/d10d 0 2024-06-15T01:07:11.171 INFO:tasks.workunit.client.1.smithi142.stdout:8/894: chown d0/f3f 0 1 2024-06-15T01:07:11.172 INFO:tasks.workunit.client.1.smithi142.stdout:8/895: truncate d0/d25/d5a/d8f/d98/fc8 539564 0 2024-06-15T01:07:11.172 INFO:tasks.workunit.client.0.smithi137.stdout:6/507: creat d0/d1a/d68/f9f x:0 0 0 2024-06-15T01:07:11.173 INFO:tasks.workunit.client.0.smithi137.stdout:5/620: dwrite df/f12 [0,4194304] 0 2024-06-15T01:07:11.175 INFO:tasks.workunit.client.0.smithi137.stdout:5/621: chown df/d3d/f97 3 1 2024-06-15T01:07:11.179 INFO:tasks.workunit.client.1.smithi142.stdout:4/936: rmdir d5/d1c/d6f/db9 39 2024-06-15T01:07:11.179 INFO:tasks.workunit.client.1.smithi142.stdout:8/896: dread d0/d25/d5a/f69 [0,4194304] 0 2024-06-15T01:07:11.180 INFO:tasks.workunit.client.0.smithi137.stdout:5/622: symlink df/d11/ld2 0 2024-06-15T01:07:11.182 INFO:tasks.workunit.client.1.smithi142.stdout:2/838: mkdir d8/df/d10/d94/db5/ded/db6/d105 0 2024-06-15T01:07:11.185 INFO:tasks.workunit.client.1.smithi142.stdout:9/814: link d6/d31/f84 d6/d34/d8a/f12f 0 2024-06-15T01:07:11.186 INFO:tasks.workunit.client.1.smithi142.stdout:6/872: dwrite f0 [0,4194304] 0 2024-06-15T01:07:11.190 INFO:tasks.workunit.client.1.smithi142.stdout:4/937: rmdir d5/d9/d1a 39 2024-06-15T01:07:11.193 INFO:tasks.workunit.client.1.smithi142.stdout:3/918: fsync d0/d3/da/d18/f8a 0 2024-06-15T01:07:11.193 INFO:tasks.workunit.client.1.smithi142.stdout:0/827: sync 2024-06-15T01:07:11.193 INFO:tasks.workunit.client.1.smithi142.stdout:1/917: sync 2024-06-15T01:07:11.193 INFO:tasks.workunit.client.1.smithi142.stdout:3/919: fdatasync d0/d95/ff2 0 2024-06-15T01:07:11.194 INFO:tasks.workunit.client.0.smithi137.stdout:5/623: write df/d11/d30/f7b [1612046,120265] 0 2024-06-15T01:07:11.196 INFO:tasks.workunit.client.1.smithi142.stdout:3/920: write d0/d3/de2/d55/dae/f114 [843613,43218] 0 2024-06-15T01:07:11.196 INFO:tasks.workunit.client.1.smithi142.stdout:8/897: truncate d0/d25/f9e 23364 0 2024-06-15T01:07:11.197 INFO:tasks.workunit.client.1.smithi142.stdout:2/839: chown d8/df/d1f/c39 994 1 2024-06-15T01:07:11.199 INFO:tasks.workunit.client.1.smithi142.stdout:5/843: getdents d2/de/d27/d86/dbd 0 2024-06-15T01:07:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:5/624: dread df/d7a/fb5 [0,4194304] 0 2024-06-15T01:07:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:6/508: dread d0/d1a/d3e/d57/d32/d74/d5a/f69 [0,4194304] 0 2024-06-15T01:07:11.201 INFO:tasks.workunit.client.1.smithi142.stdout:6/873: symlink d1a/d2d/d3b/dab/dce/l12e 0 2024-06-15T01:07:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:6/509: stat d0/d1a/d3e/d57/d1d 0 2024-06-15T01:07:11.202 INFO:tasks.workunit.client.1.smithi142.stdout:6/874: write d1a/d3c/d105/f113 [523653,34477] 0 2024-06-15T01:07:11.202 INFO:tasks.workunit.client.1.smithi142.stdout:9/815: creat d6/dc/d1b/d25/d58/de2/f130 x:0 0 0 2024-06-15T01:07:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:6/510: dread - d0/d5/d46/f9c zero size 2024-06-15T01:07:11.203 INFO:tasks.workunit.client.1.smithi142.stdout:4/938: write d5/d9/d23/d68/f8e [1152001,21657] 0 2024-06-15T01:07:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:6/511: mkdir d0/d1a/da0 0 2024-06-15T01:07:11.205 INFO:tasks.workunit.client.0.smithi137.stdout:6/512: chown d0/d1a/d3e/d57/d1d 64567637 1 2024-06-15T01:07:11.206 INFO:tasks.workunit.client.1.smithi142.stdout:1/918: symlink d6/d12/d3c/d63/d65/l143 0 2024-06-15T01:07:11.208 INFO:tasks.workunit.client.0.smithi137.stdout:5/625: write df/d44/d54/d95/d3b/d49/f96 [5192613,46172] 0 2024-06-15T01:07:11.210 INFO:tasks.workunit.client.0.smithi137.stdout:6/513: getdents d0/d1a/d3e/d57/d1d/d2e 0 2024-06-15T01:07:11.211 INFO:tasks.workunit.client.1.smithi142.stdout:0/828: rename d3/d39/d3e/d81/d44/d7c/d22/d59/f9c to d3/d39/d3e/f118 0 2024-06-15T01:07:11.213 INFO:tasks.workunit.client.1.smithi142.stdout:7/890: dwrite d2/d8/dd/dcf/f2f [0,4194304] 0 2024-06-15T01:07:11.214 INFO:tasks.workunit.client.1.smithi142.stdout:7/891: chown d2/d8/f2d 159048536 1 2024-06-15T01:07:11.214 INFO:tasks.workunit.client.0.smithi137.stdout:6/514: link d0/d1a/d3e/d57/d1d/d2e/f8d d0/d1a/da0/fa1 0 2024-06-15T01:07:11.215 INFO:tasks.workunit.client.1.smithi142.stdout:8/898: creat d0/d25/d5a/d8f/df3/f10e x:0 0 0 2024-06-15T01:07:11.216 INFO:tasks.workunit.client.1.smithi142.stdout:6/875: mkdir d1a/d2d/d2e/d12f 0 2024-06-15T01:07:11.217 INFO:tasks.workunit.client.1.smithi142.stdout:3/921: dread d0/d3/da/d18/f87 [0,4194304] 0 2024-06-15T01:07:11.218 INFO:tasks.workunit.client.1.smithi142.stdout:4/939: creat d5/d9/d23/d68/f149 x:0 0 0 2024-06-15T01:07:11.218 INFO:tasks.workunit.client.0.smithi137.stdout:6/515: chown d0/d5/d9/d29/d64/f86 362994985 1 2024-06-15T01:07:11.219 INFO:tasks.workunit.client.1.smithi142.stdout:3/922: chown d0/d1f/d20/d5d/d60/cd6 2 1 2024-06-15T01:07:11.219 INFO:tasks.workunit.client.1.smithi142.stdout:1/919: rmdir d6/d46/dc3 39 2024-06-15T01:07:11.222 INFO:tasks.workunit.client.0.smithi137.stdout:5/626: dwrite df/d11/f6d [0,4194304] 0 2024-06-15T01:07:11.228 INFO:tasks.workunit.client.0.smithi137.stdout:5/627: fsync df/d11/d14/d6c/f90 0 2024-06-15T01:07:11.229 INFO:tasks.workunit.client.0.smithi137.stdout:6/516: creat d0/d1a/d3e/d57/d1d/fa2 x:0 0 0 2024-06-15T01:07:11.230 INFO:tasks.workunit.client.0.smithi137.stdout:5/628: fsync df/d44/d54/d95/d3b/d49/fca 0 2024-06-15T01:07:11.231 INFO:tasks.workunit.client.0.smithi137.stdout:6/517: chown d0/d1a/d3e/d57/d32/d74/f78 2472 1 2024-06-15T01:07:11.233 INFO:tasks.workunit.client.0.smithi137.stdout:5/629: creat df/d44/d54/fd3 x:0 0 0 2024-06-15T01:07:11.238 INFO:tasks.workunit.client.1.smithi142.stdout:6/876: rmdir d1a/d3c/d100/d58/d10f 39 2024-06-15T01:07:11.238 INFO:tasks.workunit.client.1.smithi142.stdout:5/844: dread d2/de/d19/f22 [0,4194304] 0 2024-06-15T01:07:11.238 INFO:tasks.workunit.client.1.smithi142.stdout:1/920: mkdir d6/d25/d144 0 2024-06-15T01:07:11.239 INFO:tasks.workunit.client.1.smithi142.stdout:0/829: getdents d3/d39/d9a/d9d/dcd 0 2024-06-15T01:07:11.241 INFO:tasks.workunit.client.0.smithi137.stdout:8/486: sync 2024-06-15T01:07:11.241 INFO:tasks.workunit.client.0.smithi137.stdout:6/518: dwrite d0/d1a/d3e/d57/d32/d74/f78 [0,4194304] 0 2024-06-15T01:07:11.243 INFO:tasks.workunit.client.1.smithi142.stdout:9/816: write d6/f9 [1384461,21263] 0 2024-06-15T01:07:11.243 INFO:tasks.workunit.client.1.smithi142.stdout:2/840: dwrite d8/f16 [0,4194304] 0 2024-06-15T01:07:11.244 INFO:tasks.workunit.client.0.smithi137.stdout:8/487: creat d9/d2a/d79/fa3 x:0 0 0 2024-06-15T01:07:11.245 INFO:tasks.workunit.client.1.smithi142.stdout:9/817: read d6/dc/d4e/d78/f9d [113182,39005] 0 2024-06-15T01:07:11.248 INFO:tasks.workunit.client.0.smithi137.stdout:6/519: getdents d0/d5/d9/d29/d64 0 2024-06-15T01:07:11.257 INFO:tasks.workunit.client.0.smithi137.stdout:8/488: dwrite d9/d2a/d2f/f90 [0,4194304] 0 2024-06-15T01:07:11.264 INFO:tasks.workunit.client.0.smithi137.stdout:5/630: dread df/d11/f24 [0,4194304] 0 2024-06-15T01:07:11.265 INFO:tasks.workunit.client.0.smithi137.stdout:5/631: chown fe 121366 1 2024-06-15T01:07:11.269 INFO:tasks.workunit.client.0.smithi137.stdout:5/632: mknod df/d11/cd4 0 2024-06-15T01:07:11.274 INFO:tasks.workunit.client.0.smithi137.stdout:6/520: dwrite d0/d5/d9/fe [0,4194304] 0 2024-06-15T01:07:11.280 INFO:tasks.workunit.client.1.smithi142.stdout:5/845: fdatasync d2/d18/fc9 0 2024-06-15T01:07:11.280 INFO:tasks.workunit.client.0.smithi137.stdout:6/521: link d0/d1a/d3e/d57/f80 d0/d1a/d99/fa3 0 2024-06-15T01:07:11.281 INFO:tasks.workunit.client.1.smithi142.stdout:5/846: dread - d2/d18/d2e/dae/ff4 zero size 2024-06-15T01:07:11.282 INFO:tasks.workunit.client.1.smithi142.stdout:7/892: write d2/d6/d22/d47/f7a [2159043,4866] 0 2024-06-15T01:07:11.284 INFO:tasks.workunit.client.0.smithi137.stdout:6/522: creat d0/fa4 x:0 0 0 2024-06-15T01:07:11.284 INFO:tasks.workunit.client.1.smithi142.stdout:0/830: mkdir d3/d39/d9a/d9d/dbf/d119 0 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:1/680: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:0/705: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:2/464: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:3/877: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:9/432: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:4/557: sync 2024-06-15T01:07:11.285 INFO:tasks.workunit.client.0.smithi137.stdout:7/496: sync 2024-06-15T01:07:11.289 INFO:tasks.workunit.client.1.smithi142.stdout:9/818: creat d6/d34/d8a/f131 x:0 0 0 2024-06-15T01:07:11.289 INFO:tasks.workunit.client.1.smithi142.stdout:6/877: dread d1a/d2d/d2e/f49 [0,4194304] 0 2024-06-15T01:07:11.290 INFO:tasks.workunit.client.1.smithi142.stdout:3/923: rmdir d0/d3/de2/d75/dfd 0 2024-06-15T01:07:11.291 INFO:tasks.workunit.client.1.smithi142.stdout:4/940: creat d5/d9/f14a x:0 0 0 2024-06-15T01:07:11.291 INFO:tasks.workunit.client.0.smithi137.stdout:7/497: fdatasync df/d13/d2e/f86 0 2024-06-15T01:07:11.293 INFO:tasks.workunit.client.1.smithi142.stdout:1/921: truncate d6/d46/dc3/f4b 304972 0 2024-06-15T01:07:11.295 INFO:tasks.workunit.client.0.smithi137.stdout:9/433: dread df/d12/f1f [4194304,4194304] 0 2024-06-15T01:07:11.295 INFO:tasks.workunit.client.1.smithi142.stdout:5/847: write d2/de/fb0 [332746,35580] 0 2024-06-15T01:07:11.296 INFO:tasks.workunit.client.0.smithi137.stdout:8/489: dwrite d9/d2a/f2d [0,4194304] 0 2024-06-15T01:07:11.297 INFO:tasks.workunit.client.1.smithi142.stdout:7/893: rmdir d2/d6/d22 39 2024-06-15T01:07:11.297 INFO:tasks.workunit.client.1.smithi142.stdout:5/848: truncate d2/d3e/d74/da5/fe2 1023630 0 2024-06-15T01:07:11.298 INFO:tasks.workunit.client.1.smithi142.stdout:5/849: dread - d2/d18/d2e/dae/ff4 zero size 2024-06-15T01:07:11.298 INFO:tasks.workunit.client.1.smithi142.stdout:0/831: stat d3/d39/d9a/d9d/dbf/lca 0 2024-06-15T01:07:11.299 INFO:tasks.workunit.client.1.smithi142.stdout:5/850: chown d2/d18/d2e/c8a 13229 1 2024-06-15T01:07:11.300 INFO:tasks.workunit.client.1.smithi142.stdout:2/841: symlink d8/df/d10/d94/l106 0 2024-06-15T01:07:11.301 INFO:tasks.workunit.client.1.smithi142.stdout:2/842: fdatasync d8/df/d10/d94/d25/d1a/d63/f95 0 2024-06-15T01:07:11.306 INFO:tasks.workunit.client.1.smithi142.stdout:4/941: unlink d5/d1c/d6f/dc7/d30/dcc/l85 0 2024-06-15T01:07:11.306 INFO:tasks.workunit.client.0.smithi137.stdout:9/434: readlink df/d12/l3b 0 2024-06-15T01:07:11.307 INFO:tasks.workunit.client.0.smithi137.stdout:7/498: rename df/d13/d2e/l3e to df/d13/d2e/d1a/d6a/l9d 0 2024-06-15T01:07:11.307 INFO:tasks.workunit.client.0.smithi137.stdout:3/878: dread d0/ddf/ff2 [0,4194304] 0 2024-06-15T01:07:11.308 INFO:tasks.workunit.client.1.smithi142.stdout:9/819: dread d6/dc/d1b/d25/d58/fd6 [4194304,4194304] 0 2024-06-15T01:07:11.309 INFO:tasks.workunit.client.0.smithi137.stdout:5/633: dwrite df/d44/d54/d6b/f8c [0,4194304] 0 2024-06-15T01:07:11.311 INFO:tasks.workunit.client.0.smithi137.stdout:1/681: truncate d0/d3/d1b/d4c/d59/dd4/fe1 829514 0 2024-06-15T01:07:11.313 INFO:tasks.workunit.client.1.smithi142.stdout:7/894: unlink d2/d8/dd/d14/d2c/d70/l118 0 2024-06-15T01:07:11.315 INFO:tasks.workunit.client.1.smithi142.stdout:5/851: mkdir d2/de/d27/db2/d108 0 2024-06-15T01:07:11.315 INFO:tasks.workunit.client.1.smithi142.stdout:0/832: write d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [411954,102733] 0 2024-06-15T01:07:11.315 INFO:tasks.workunit.client.1.smithi142.stdout:4/942: dwrite d5/f10 [0,4194304] 0 2024-06-15T01:07:11.317 INFO:tasks.workunit.client.0.smithi137.stdout:3/879: symlink d0/d2/d23/l11f 0 2024-06-15T01:07:11.318 INFO:tasks.workunit.client.1.smithi142.stdout:2/843: rmdir d8/df/d10/d94/d25/d22/d43/d52/dc1 39 2024-06-15T01:07:11.319 INFO:tasks.workunit.client.0.smithi137.stdout:9/435: truncate df/d12/d29/d4a/f6c 8645279 0 2024-06-15T01:07:11.324 INFO:tasks.workunit.client.1.smithi142.stdout:0/833: truncate d3/d39/d3e/d81/d1e/f2b 4527186 0 2024-06-15T01:07:11.324 INFO:tasks.workunit.client.1.smithi142.stdout:7/895: mknod d2/d8/dd/d8a/d94/c11a 0 2024-06-15T01:07:11.325 INFO:tasks.workunit.client.0.smithi137.stdout:5/634: mknod df/d44/cd5 0 2024-06-15T01:07:11.325 INFO:tasks.workunit.client.0.smithi137.stdout:3/880: chown d0/d2/d23/ddb/dde/d104 447 1 2024-06-15T01:07:11.326 INFO:tasks.workunit.client.1.smithi142.stdout:0/834: chown d3/d39/d3e/d81/d1b/d38/da3/cb3 2027858 1 2024-06-15T01:07:11.326 INFO:tasks.workunit.client.1.smithi142.stdout:4/943: unlink d5/d9/d41/c86 0 2024-06-15T01:07:11.327 INFO:tasks.workunit.client.1.smithi142.stdout:0/835: fsync d3/d39/d9a/d9d/dbf/f10d 0 2024-06-15T01:07:11.327 INFO:tasks.workunit.client.0.smithi137.stdout:4/558: dwrite dd/d15/d36/d45/d55/d8d/f70 [0,4194304] 0 2024-06-15T01:07:11.328 INFO:tasks.workunit.client.1.smithi142.stdout:5/852: rename d2/de/d19/c5e to d2/d3e/d98/c109 0 2024-06-15T01:07:11.334 INFO:tasks.workunit.client.1.smithi142.stdout:4/944: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/f14b x:0 0 0 2024-06-15T01:07:11.336 INFO:tasks.workunit.client.0.smithi137.stdout:0/706: dread d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:11.337 INFO:tasks.workunit.client.0.smithi137.stdout:6/523: dread d0/d5/f7 [0,4194304] 0 2024-06-15T01:07:11.337 INFO:tasks.workunit.client.0.smithi137.stdout:8/490: dread d9/de/f7b [0,4194304] 0 2024-06-15T01:07:11.337 INFO:tasks.workunit.client.0.smithi137.stdout:6/524: stat d0/d5/d9/c7a 0 2024-06-15T01:07:11.337 INFO:tasks.workunit.client.0.smithi137.stdout:7/499: truncate df/d7d/d1c/d1d/d27/d33/f9b 1134666 0 2024-06-15T01:07:11.339 INFO:tasks.workunit.client.1.smithi142.stdout:0/836: unlink d3/d39/d3e/d81/d1b/c2e 0 2024-06-15T01:07:11.339 INFO:tasks.workunit.client.1.smithi142.stdout:8/899: truncate d0/d17/f6b 1319184 0 2024-06-15T01:07:11.339 INFO:tasks.workunit.client.1.smithi142.stdout:5/853: truncate d2/de/d27/d86/db9/fd4 7423493 0 2024-06-15T01:07:11.339 INFO:tasks.workunit.client.0.smithi137.stdout:8/491: chown d9/d2a/d2f/d59/d82/c85 110 1 2024-06-15T01:07:11.340 INFO:tasks.workunit.client.0.smithi137.stdout:2/465: dread d0/de/d10/f2a [0,4194304] 0 2024-06-15T01:07:11.340 INFO:tasks.workunit.client.0.smithi137.stdout:4/559: creat dd/d15/d36/d45/d55/fd4 x:0 0 0 2024-06-15T01:07:11.340 INFO:tasks.workunit.client.0.smithi137.stdout:3/881: write d0/d56/f6a [1009744,82873] 0 2024-06-15T01:07:11.341 INFO:tasks.workunit.client.1.smithi142.stdout:7/896: mkdir d2/d6/d22/d47/d11b 0 2024-06-15T01:07:11.341 INFO:tasks.workunit.client.0.smithi137.stdout:2/466: write d0/da/f18 [3658470,61464] 0 2024-06-15T01:07:11.344 INFO:tasks.workunit.client.1.smithi142.stdout:4/945: dwrite d5/d9/d23/d32/ffa [0,4194304] 0 2024-06-15T01:07:11.345 INFO:tasks.workunit.client.0.smithi137.stdout:2/467: truncate d0/f1 554208 0 2024-06-15T01:07:11.345 INFO:tasks.workunit.client.0.smithi137.stdout:4/560: read - dd/d15/d36/d45/dcb/d5b/fc6 zero size 2024-06-15T01:07:11.345 INFO:tasks.workunit.client.0.smithi137.stdout:6/525: truncate d0/d1a/d3e/d57/d1d/f1f 1033956 0 2024-06-15T01:07:11.349 INFO:tasks.workunit.client.0.smithi137.stdout:3/882: creat d0/d2/d8/dbd/f120 x:0 0 0 2024-06-15T01:07:11.350 INFO:tasks.workunit.client.1.smithi142.stdout:8/900: unlink d0/d25/d66/f92 0 2024-06-15T01:07:11.350 INFO:tasks.workunit.client.1.smithi142.stdout:8/901: chown d0/d25/d5a/ld 3338 1 2024-06-15T01:07:11.353 INFO:tasks.workunit.client.0.smithi137.stdout:7/500: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/f9e x:0 0 0 2024-06-15T01:07:11.353 INFO:tasks.workunit.client.0.smithi137.stdout:1/682: write d0/d3/f13 [1588924,113837] 0 2024-06-15T01:07:11.354 INFO:tasks.workunit.client.0.smithi137.stdout:4/561: mknod dd/d15/d36/d45/d8f/cd5 0 2024-06-15T01:07:11.356 INFO:tasks.workunit.client.0.smithi137.stdout:7/501: write df/d13/d3c/f92 [315135,20207] 0 2024-06-15T01:07:11.357 INFO:tasks.workunit.client.0.smithi137.stdout:5/635: creat df/d44/d54/d95/d3b/d49/fd6 x:0 0 0 2024-06-15T01:07:11.358 INFO:tasks.workunit.client.1.smithi142.stdout:7/897: unlink d2/d6/d22/lf9 0 2024-06-15T01:07:11.359 INFO:tasks.workunit.client.0.smithi137.stdout:1/683: truncate d0/d3/d47/f88 4778804 0 2024-06-15T01:07:11.360 INFO:tasks.workunit.client.0.smithi137.stdout:8/492: getdents d9/de/d1e/d3f 0 2024-06-15T01:07:11.361 INFO:tasks.workunit.client.0.smithi137.stdout:5/636: fdatasync df/d11/f4c 0 2024-06-15T01:07:11.362 INFO:tasks.workunit.client.1.smithi142.stdout:1/922: dwrite d6/d12/d3c/d63/d8d/d97/dd3/f76 [0,4194304] 0 2024-06-15T01:07:11.362 INFO:tasks.workunit.client.1.smithi142.stdout:2/844: dread d8/df/d10/d94/d25/da9/fd0 [0,4194304] 0 2024-06-15T01:07:11.363 INFO:tasks.workunit.client.0.smithi137.stdout:9/436: truncate df/d12/d1b/d31/d42/f56 56278 0 2024-06-15T01:07:11.363 INFO:tasks.workunit.client.0.smithi137.stdout:5/637: truncate df/d11/d14/d6c/dc7/fd0 518497 0 2024-06-15T01:07:11.364 INFO:tasks.workunit.client.1.smithi142.stdout:3/924: truncate d0/d1f/d20/d5d/d82/f63 2876457 0 2024-06-15T01:07:11.364 INFO:tasks.workunit.client.1.smithi142.stdout:0/837: rmdir d3/d39/d3e/d81/d1b/d52/d115 0 2024-06-15T01:07:11.366 INFO:tasks.workunit.client.1.smithi142.stdout:1/923: dread - d6/d12/d1c/d2c/f62 zero size 2024-06-15T01:07:11.367 INFO:tasks.workunit.client.1.smithi142.stdout:2/845: stat d8/df/d10/d94/d25/d22/fbd 0 2024-06-15T01:07:11.368 INFO:tasks.workunit.client.1.smithi142.stdout:1/924: chown d6/d3a/dd4/l10b 288383 1 2024-06-15T01:07:11.371 INFO:tasks.workunit.client.1.smithi142.stdout:1/925: write d6/d3a/f99 [2121717,105799] 0 2024-06-15T01:07:11.371 INFO:tasks.workunit.client.0.smithi137.stdout:6/526: rename d0/d1a/d3e/d57/d32/d74/d5a to d0/d1a/d3e/d57/d1d/da5 0 2024-06-15T01:07:11.372 INFO:tasks.workunit.client.1.smithi142.stdout:7/898: creat d2/d8/dd/f11c x:0 0 0 2024-06-15T01:07:11.375 INFO:tasks.workunit.client.0.smithi137.stdout:1/684: mknod d0/d3/d14/ce8 0 2024-06-15T01:07:11.375 INFO:tasks.workunit.client.1.smithi142.stdout:8/902: rename d0/d25/d53/da7/dcb/fd9 to d0/d25/d5a/f10f 0 2024-06-15T01:07:11.377 INFO:tasks.workunit.client.1.smithi142.stdout:2/846: mkdir d8/df/d10/d94/d25/d1a/d63/d107 0 2024-06-15T01:07:11.378 INFO:tasks.workunit.client.1.smithi142.stdout:3/925: dwrite d0/d1f/d20/d5d/d60/fce [0,4194304] 0 2024-06-15T01:07:11.380 INFO:tasks.workunit.client.1.smithi142.stdout:5/854: dread d2/de/f1f [0,4194304] 0 2024-06-15T01:07:11.381 INFO:tasks.workunit.client.0.smithi137.stdout:7/502: rename df/d13/d2e/d1a/c7f to df/d7d/d1c/d1d/d27/d97/c9f 0 2024-06-15T01:07:11.381 INFO:tasks.workunit.client.0.smithi137.stdout:1/685: dread - d0/d1f/d67/f94 zero size 2024-06-15T01:07:11.381 INFO:tasks.workunit.client.1.smithi142.stdout:2/847: fdatasync d8/df/f2d 0 2024-06-15T01:07:11.381 INFO:tasks.workunit.client.1.smithi142.stdout:8/903: chown d0/d25/d66/cc4 5 1 2024-06-15T01:07:11.382 INFO:tasks.workunit.client.1.smithi142.stdout:7/899: truncate d2/d8/dd/d14/d2c/d70/ff7 512727 0 2024-06-15T01:07:11.383 INFO:tasks.workunit.client.0.smithi137.stdout:4/562: dwrite dd/d15/d36/d45/dcb/f52 [0,4194304] 0 2024-06-15T01:07:11.385 INFO:tasks.workunit.client.1.smithi142.stdout:7/900: write d2/db6/f106 [985549,99414] 0 2024-06-15T01:07:11.385 INFO:tasks.workunit.client.0.smithi137.stdout:4/563: stat dd/d15/d36/d45/f4b 0 2024-06-15T01:07:11.385 INFO:tasks.workunit.client.0.smithi137.stdout:9/437: creat df/d12/d15/d75/f7c x:0 0 0 2024-06-15T01:07:11.386 INFO:tasks.workunit.client.1.smithi142.stdout:1/926: dwrite d6/d46/f78 [0,4194304] 0 2024-06-15T01:07:11.387 INFO:tasks.workunit.client.1.smithi142.stdout:8/904: dread - d0/d25/d53/d9c/fac zero size 2024-06-15T01:07:11.391 INFO:tasks.workunit.client.1.smithi142.stdout:2/848: mknod d8/df/d1f/c108 0 2024-06-15T01:07:11.392 INFO:tasks.workunit.client.1.smithi142.stdout:2/849: read d8/df/d10/d94/d25/da9/fd0 [1619946,25063] 0 2024-06-15T01:07:11.393 INFO:tasks.workunit.client.1.smithi142.stdout:6/878: sync 2024-06-15T01:07:11.399 INFO:tasks.workunit.client.1.smithi142.stdout:5/855: dwrite d2/de/d27/f56 [4194304,4194304] 0 2024-06-15T01:07:11.400 INFO:tasks.workunit.client.1.smithi142.stdout:7/901: creat d2/db6/de6/f11d x:0 0 0 2024-06-15T01:07:11.400 INFO:tasks.workunit.client.0.smithi137.stdout:4/564: mkdir dd/d15/d36/d45/d55/d8d/d1f/d86/dd6 0 2024-06-15T01:07:11.401 INFO:tasks.workunit.client.1.smithi142.stdout:5/856: chown d2/d3e/feb 14531725 1 2024-06-15T01:07:11.402 INFO:tasks.workunit.client.0.smithi137.stdout:9/438: dwrite df/d1a/f76 [0,4194304] 0 2024-06-15T01:07:11.403 INFO:tasks.workunit.client.0.smithi137.stdout:2/468: dread d0/de/d10/f19 [0,4194304] 0 2024-06-15T01:07:11.404 INFO:tasks.workunit.client.1.smithi142.stdout:8/905: symlink d0/d25/d53/da7/dcb/dfb/l110 0 2024-06-15T01:07:11.407 INFO:tasks.workunit.client.1.smithi142.stdout:8/906: chown d0/d17/d40/d6e/fcf 6145878 1 2024-06-15T01:07:11.407 INFO:tasks.workunit.client.1.smithi142.stdout:3/926: symlink d0/d1f/d20/d5d/d60/l116 0 2024-06-15T01:07:11.408 INFO:tasks.workunit.client.0.smithi137.stdout:5/638: getdents df/d44/d54/d95/d3b/d4b 0 2024-06-15T01:07:11.408 INFO:tasks.workunit.client.0.smithi137.stdout:4/565: creat dd/d15/d36/d45/fd7 x:0 0 0 2024-06-15T01:07:11.409 INFO:tasks.workunit.client.1.smithi142.stdout:4/946: dwrite d5/d9/d18/f80 [8388608,4194304] 0 2024-06-15T01:07:11.409 INFO:tasks.workunit.client.1.smithi142.stdout:1/927: dread d6/d12/d52/f89 [0,4194304] 0 2024-06-15T01:07:11.409 INFO:tasks.workunit.client.0.smithi137.stdout:6/527: rename d0/d5/d46/f5d to d0/fa6 0 2024-06-15T01:07:11.410 INFO:tasks.workunit.client.1.smithi142.stdout:8/907: chown d0/d25/d53/d9c/f101 479805 1 2024-06-15T01:07:11.410 INFO:tasks.workunit.client.1.smithi142.stdout:2/850: fsync d8/df/d10/d1c/f97 0 2024-06-15T01:07:11.411 INFO:tasks.workunit.client.0.smithi137.stdout:4/566: fdatasync dd/d15/d36/d45/d55/f95 0 2024-06-15T01:07:11.412 INFO:tasks.workunit.client.0.smithi137.stdout:9/439: truncate df/d12/d15/d75/f7c 387989 0 2024-06-15T01:07:11.414 INFO:tasks.workunit.client.0.smithi137.stdout:6/528: chown d0/c97 6 1 2024-06-15T01:07:11.414 INFO:tasks.workunit.client.0.smithi137.stdout:9/440: chown df/d12/d29 3538284 1 2024-06-15T01:07:11.414 INFO:tasks.workunit.client.0.smithi137.stdout:2/469: link d0/de/d10/f33 d0/dc/d22/d23/d5c/f8b 0 2024-06-15T01:07:11.416 INFO:tasks.workunit.client.1.smithi142.stdout:3/927: dwrite d0/d3/de2/d75/ff3 [0,4194304] 0 2024-06-15T01:07:11.418 INFO:tasks.workunit.client.1.smithi142.stdout:0/838: write d3/d39/d3e/f5f [4844354,97827] 0 2024-06-15T01:07:11.419 INFO:tasks.workunit.client.1.smithi142.stdout:5/857: chown d2/d18/c9f 730880383 1 2024-06-15T01:07:11.422 INFO:tasks.workunit.client.1.smithi142.stdout:8/908: symlink d0/d25/d53/da7/dcb/l111 0 2024-06-15T01:07:11.423 INFO:tasks.workunit.client.0.smithi137.stdout:2/470: getdents d0/dc/d22/d83 0 2024-06-15T01:07:11.424 INFO:tasks.workunit.client.0.smithi137.stdout:2/471: chown d0/c88 2 1 2024-06-15T01:07:11.424 INFO:tasks.workunit.client.1.smithi142.stdout:7/902: symlink d2/l11e 0 2024-06-15T01:07:11.425 INFO:tasks.workunit.client.0.smithi137.stdout:2/472: chown d0/d42/d49/d57/f7e 15 1 2024-06-15T01:07:11.426 INFO:tasks.workunit.client.1.smithi142.stdout:3/928: rmdir d0/d3/de2/d55/dae 39 2024-06-15T01:07:11.426 INFO:tasks.workunit.client.1.smithi142.stdout:9/820: sync 2024-06-15T01:07:11.427 INFO:tasks.workunit.client.0.smithi137.stdout:2/473: chown d0/dc/d2e/c81 16682299 1 2024-06-15T01:07:11.428 INFO:tasks.workunit.client.1.smithi142.stdout:9/821: readlink d6/dc/d1b/d25/d35/l121 0 2024-06-15T01:07:11.428 INFO:tasks.workunit.client.0.smithi137.stdout:6/529: link d0/d1a/d3e/f77 d0/d1a/d3e/d57/d1d/d2e/fa7 0 2024-06-15T01:07:11.429 INFO:tasks.workunit.client.0.smithi137.stdout:6/530: chown d0/d1a/da0/fa1 111 1 2024-06-15T01:07:11.434 INFO:tasks.workunit.client.0.smithi137.stdout:2/474: dread d0/da/f11 [0,4194304] 0 2024-06-15T01:07:11.434 INFO:tasks.workunit.client.0.smithi137.stdout:6/531: rmdir d0/d1a/d68 39 2024-06-15T01:07:11.435 INFO:tasks.workunit.client.1.smithi142.stdout:3/929: dwrite d0/d1f/d20/f10f [0,4194304] 0 2024-06-15T01:07:11.437 INFO:tasks.workunit.client.1.smithi142.stdout:4/947: mknod d5/d1c/d6f/c14c 0 2024-06-15T01:07:11.439 INFO:tasks.workunit.client.1.smithi142.stdout:2/851: dread d8/df/d10/d94/d25/d22/d43/f46 [0,4194304] 0 2024-06-15T01:07:11.441 INFO:tasks.workunit.client.1.smithi142.stdout:6/879: dwrite d1a/f1e [0,4194304] 0 2024-06-15T01:07:11.446 INFO:tasks.workunit.client.1.smithi142.stdout:0/839: truncate d3/d39/d3e/d81/d1b/d52/f85 448386 0 2024-06-15T01:07:11.447 INFO:tasks.workunit.client.1.smithi142.stdout:1/928: creat d6/d12/d80/f145 x:0 0 0 2024-06-15T01:07:11.448 INFO:tasks.workunit.client.0.smithi137.stdout:1/686: dread d0/d1f/d60/f91 [0,4194304] 0 2024-06-15T01:07:11.449 INFO:tasks.workunit.client.1.smithi142.stdout:8/909: symlink d0/d25/d53/da7/d10d/l112 0 2024-06-15T01:07:11.450 INFO:tasks.workunit.client.1.smithi142.stdout:7/903: symlink d2/d8/da/d33/d25/d79/dd4/d108/l11f 0 2024-06-15T01:07:11.450 INFO:tasks.workunit.client.1.smithi142.stdout:5/858: creat d2/d18/f10a x:0 0 0 2024-06-15T01:07:11.453 INFO:tasks.workunit.client.1.smithi142.stdout:2/852: symlink d8/df/d10/d1c/dbb/l109 0 2024-06-15T01:07:11.457 INFO:tasks.workunit.client.1.smithi142.stdout:9/822: creat d6/f132 x:0 0 0 2024-06-15T01:07:11.458 INFO:tasks.workunit.client.1.smithi142.stdout:0/840: rename d3/d39/d3e/d81/d44/d7c/d22/f82 to d3/d39/d9a/d9d/dbf/f11a 0 2024-06-15T01:07:11.458 INFO:tasks.workunit.client.0.smithi137.stdout:1/687: dwrite d0/d3/d47/d73/fb9 [0,4194304] 0 2024-06-15T01:07:11.460 INFO:tasks.workunit.client.1.smithi142.stdout:7/904: unlink d2/lf5 0 2024-06-15T01:07:11.461 INFO:tasks.workunit.client.0.smithi137.stdout:9/441: dread f3 [0,4194304] 0 2024-06-15T01:07:11.461 INFO:tasks.workunit.client.0.smithi137.stdout:5/639: dread df/d11/d30/f7b [0,4194304] 0 2024-06-15T01:07:11.463 INFO:tasks.workunit.client.0.smithi137.stdout:1/688: truncate d0/fb0 314474 0 2024-06-15T01:07:11.465 INFO:tasks.workunit.client.1.smithi142.stdout:5/859: dwrite d2/d18/d2e/fe1 [0,4194304] 0 2024-06-15T01:07:11.465 INFO:tasks.workunit.client.0.smithi137.stdout:4/567: dread dd/d15/d36/d45/f4b [0,4194304] 0 2024-06-15T01:07:11.466 INFO:tasks.workunit.client.0.smithi137.stdout:4/568: stat dd/d15/lc0 0 2024-06-15T01:07:11.466 INFO:tasks.workunit.client.1.smithi142.stdout:2/853: dwrite d8/df/d10/d94/d25/f6a [4194304,4194304] 0 2024-06-15T01:07:11.469 INFO:tasks.workunit.client.1.smithi142.stdout:1/929: creat d6/d3a/dc4/f146 x:0 0 0 2024-06-15T01:07:11.470 INFO:tasks.workunit.client.1.smithi142.stdout:2/854: read d8/df/d10/d94/d25/d1a/d63/f95 [1705906,39590] 0 2024-06-15T01:07:11.471 INFO:tasks.workunit.client.0.smithi137.stdout:6/532: write d0/d1a/d3e/d57/d32/d74/f78 [4474566,3226] 0 2024-06-15T01:07:11.476 INFO:tasks.workunit.client.0.smithi137.stdout:5/640: unlink df/d44/d54/ca5 0 2024-06-15T01:07:11.477 INFO:tasks.workunit.client.0.smithi137.stdout:1/689: creat d0/d3/d14/da0/fe9 x:0 0 0 2024-06-15T01:07:11.477 INFO:tasks.workunit.client.1.smithi142.stdout:9/823: creat d6/dc/d1b/d25/d58/de2/de7/f133 x:0 0 0 2024-06-15T01:07:11.478 INFO:tasks.workunit.client.1.smithi142.stdout:4/948: write d5/d9/d41/f78 [5949130,75923] 0 2024-06-15T01:07:11.480 INFO:tasks.workunit.client.0.smithi137.stdout:9/442: dwrite df/d12/d1b/f74 [0,4194304] 0 2024-06-15T01:07:11.480 INFO:tasks.workunit.client.1.smithi142.stdout:7/905: mkdir d2/d6/daf/dce/d120 0 2024-06-15T01:07:11.482 INFO:tasks.workunit.client.1.smithi142.stdout:4/949: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f107 47747 1 2024-06-15T01:07:11.483 INFO:tasks.workunit.client.0.smithi137.stdout:9/443: mknod df/c7d 0 2024-06-15T01:07:11.484 INFO:tasks.workunit.client.1.smithi142.stdout:8/910: creat d0/d17/f113 x:0 0 0 2024-06-15T01:07:11.485 INFO:tasks.workunit.client.0.smithi137.stdout:1/690: write d0/d3/d1b/d4c/d59/f63 [604870,48310] 0 2024-06-15T01:07:11.485 INFO:tasks.workunit.client.1.smithi142.stdout:3/930: dwrite d0/d3/de2/fa1 [0,4194304] 0 2024-06-15T01:07:11.486 INFO:tasks.workunit.client.0.smithi137.stdout:9/444: mkdir df/d2a/d7e 0 2024-06-15T01:07:11.488 INFO:tasks.workunit.client.0.smithi137.stdout:1/691: dread - d0/d3/d1b/d4c/d59/fb7 zero size 2024-06-15T01:07:11.489 INFO:tasks.workunit.client.1.smithi142.stdout:5/860: unlink d2/f21 0 2024-06-15T01:07:11.489 INFO:tasks.workunit.client.0.smithi137.stdout:9/445: chown df/d12/d15/f73 14255 1 2024-06-15T01:07:11.490 INFO:tasks.workunit.client.1.smithi142.stdout:4/950: truncate d5/d9/f131 1464050 0 2024-06-15T01:07:11.490 INFO:tasks.workunit.client.0.smithi137.stdout:6/533: dwrite d0/d1a/d3e/d57/d32/d74/f78 [0,4194304] 0 2024-06-15T01:07:11.491 INFO:tasks.workunit.client.1.smithi142.stdout:1/930: symlink d6/d12/db5/l147 0 2024-06-15T01:07:11.492 INFO:tasks.workunit.client.0.smithi137.stdout:9/446: truncate df/d49/d4b/f57 179192 0 2024-06-15T01:07:11.492 INFO:tasks.workunit.client.1.smithi142.stdout:9/824: creat d6/dc/d1b/d6f/de1/d110/f134 x:0 0 0 2024-06-15T01:07:11.497 INFO:tasks.workunit.client.1.smithi142.stdout:6/880: dwrite d1a/d3c/d100/fb3 [0,4194304] 0 2024-06-15T01:07:11.497 INFO:tasks.workunit.client.1.smithi142.stdout:5/861: write d2/f28 [4400652,62693] 0 2024-06-15T01:07:11.498 INFO:tasks.workunit.client.1.smithi142.stdout:8/911: symlink d0/d25/d5a/d8f/d98/da4/l114 0 2024-06-15T01:07:11.501 INFO:tasks.workunit.client.0.smithi137.stdout:6/534: creat d0/d5/d9/d29/d6b/fa8 x:0 0 0 2024-06-15T01:07:11.502 INFO:tasks.workunit.client.1.smithi142.stdout:2/855: link d8/df/daf/fcd d8/f10a 0 2024-06-15T01:07:11.502 INFO:tasks.workunit.client.1.smithi142.stdout:4/951: creat d5/d9/d23/d32/d11f/f14d x:0 0 0 2024-06-15T01:07:11.503 INFO:tasks.workunit.client.1.smithi142.stdout:1/931: mknod d6/d12/d3c/c148 0 2024-06-15T01:07:11.504 INFO:tasks.workunit.client.0.smithi137.stdout:1/692: read d0/d3/d1b/f4f [3187366,30492] 0 2024-06-15T01:07:11.504 INFO:tasks.workunit.client.1.smithi142.stdout:0/841: rename d3/d39/d3e/d81/d44/d7c/d22/f65 to d3/d39/d3e/d81/d1b/f11b 0 2024-06-15T01:07:11.505 INFO:tasks.workunit.client.0.smithi137.stdout:6/535: rmdir d0/d1a/d3e 39 2024-06-15T01:07:11.505 INFO:tasks.workunit.client.1.smithi142.stdout:9/825: dwrite d6/dc/d4e/f11e [0,4194304] 0 2024-06-15T01:07:11.508 INFO:tasks.workunit.client.1.smithi142.stdout:9/826: readlink d6/dc/dd/d22/l33 0 2024-06-15T01:07:11.510 INFO:tasks.workunit.client.1.smithi142.stdout:6/881: mknod d1a/d39/d42/c130 0 2024-06-15T01:07:11.510 INFO:tasks.workunit.client.1.smithi142.stdout:9/827: chown d6/dc/d1b 2652 1 2024-06-15T01:07:11.511 INFO:tasks.workunit.client.0.smithi137.stdout:1/693: mkdir d0/d1f/d60/d99/de3/dea 0 2024-06-15T01:07:11.511 INFO:tasks.workunit.client.1.smithi142.stdout:5/862: mkdir d2/de/d19/d5f/d68/d76/d8c/d10b 0 2024-06-15T01:07:11.512 INFO:tasks.workunit.client.1.smithi142.stdout:6/882: chown d1a/l69 56 1 2024-06-15T01:07:11.516 INFO:tasks.workunit.client.1.smithi142.stdout:6/883: read - d1a/d39/d3a/d51/d62/f12a zero size 2024-06-15T01:07:11.517 INFO:tasks.workunit.client.1.smithi142.stdout:5/863: write d2/de/d19/d93/fef [357149,118697] 0 2024-06-15T01:07:11.518 INFO:tasks.workunit.client.0.smithi137.stdout:5/641: write df/d76/d7c/d9c/da8/f67 [4300191,41149] 0 2024-06-15T01:07:11.520 INFO:tasks.workunit.client.0.smithi137.stdout:4/569: dwrite dd/f3b [0,4194304] 0 2024-06-15T01:07:11.520 INFO:tasks.workunit.client.1.smithi142.stdout:7/906: creat d2/d6/f121 x:0 0 0 2024-06-15T01:07:11.520 INFO:tasks.workunit.client.1.smithi142.stdout:4/952: dwrite d5/d9/f14a [0,4194304] 0 2024-06-15T01:07:11.520 INFO:tasks.workunit.client.1.smithi142.stdout:0/842: mknod d3/d39/d3e/d81/d1b/d52/c11c 0 2024-06-15T01:07:11.521 INFO:tasks.workunit.client.1.smithi142.stdout:8/912: getdents d0/d25/d100 0 2024-06-15T01:07:11.523 INFO:tasks.workunit.client.1.smithi142.stdout:1/932: symlink d6/d12/d1c/d2c/d93/db9/l149 0 2024-06-15T01:07:11.523 INFO:tasks.workunit.client.0.smithi137.stdout:4/570: readlink dd/d15/d30/l31 0 2024-06-15T01:07:11.525 INFO:tasks.workunit.client.1.smithi142.stdout:9/828: dwrite d6/dc/d1b/d6f/d103/f9f [0,4194304] 0 2024-06-15T01:07:11.526 INFO:tasks.workunit.client.1.smithi142.stdout:8/913: truncate d0/d17/d40/d6e/ff5 258234 0 2024-06-15T01:07:11.527 INFO:tasks.workunit.client.0.smithi137.stdout:5/642: creat df/d11/d30/db1/fd7 x:0 0 0 2024-06-15T01:07:11.527 INFO:tasks.workunit.client.1.smithi142.stdout:8/914: fdatasync d0/d25/d53/d9c/f101 0 2024-06-15T01:07:11.528 INFO:tasks.workunit.client.1.smithi142.stdout:9/829: write d6/dc/dd/f46 [4867637,65295] 0 2024-06-15T01:07:11.528 INFO:tasks.workunit.client.0.smithi137.stdout:1/694: dwrite d0/d1f/d7d/fae [0,4194304] 0 2024-06-15T01:07:11.529 INFO:tasks.workunit.client.1.smithi142.stdout:8/915: fdatasync d0/d17/d40/d86/f9b 0 2024-06-15T01:07:11.530 INFO:tasks.workunit.client.1.smithi142.stdout:8/916: stat d0/d25/d5a/f4d 0 2024-06-15T01:07:11.531 INFO:tasks.workunit.client.1.smithi142.stdout:0/843: stat d3/d39/d3e/d81/d4c/da0/cd5 0 2024-06-15T01:07:11.534 INFO:tasks.workunit.client.0.smithi137.stdout:6/536: dwrite d0/d1a/d3e/d57/d1d/f45 [0,4194304] 0 2024-06-15T01:07:11.534 INFO:tasks.workunit.client.1.smithi142.stdout:0/844: chown d3/d39/d3e/d81/d1b/d38/f114 1 1 2024-06-15T01:07:11.534 INFO:tasks.workunit.client.1.smithi142.stdout:6/884: mkdir d1a/d3c/d100/d58/d10f/d131 0 2024-06-15T01:07:11.537 INFO:tasks.workunit.client.0.smithi137.stdout:9/447: dwrite df/d12/d1b/d31/d42/f56 [0,4194304] 0 2024-06-15T01:07:11.537 INFO:tasks.workunit.client.1.smithi142.stdout:6/885: fdatasync d1a/d2d/d2e/fd9 0 2024-06-15T01:07:11.538 INFO:tasks.workunit.client.1.smithi142.stdout:1/933: symlink d6/d12/d3c/d63/d8d/d97/dd3/l14a 0 2024-06-15T01:07:11.538 INFO:tasks.workunit.client.1.smithi142.stdout:7/907: getdents d2/db6/de6/df8 0 2024-06-15T01:07:11.539 INFO:tasks.workunit.client.0.smithi137.stdout:4/571: mknod dd/d15/d36/d45/dcb/d83/db2/d67/dbe/cd8 0 2024-06-15T01:07:11.539 INFO:tasks.workunit.client.1.smithi142.stdout:4/953: mknod d5/d1c/d6f/dc7/c14e 0 2024-06-15T01:07:11.540 INFO:tasks.workunit.client.1.smithi142.stdout:3/931: dread d0/d1f/d20/d59/f92 [0,4194304] 0 2024-06-15T01:07:11.543 INFO:tasks.workunit.client.1.smithi142.stdout:0/845: dread d3/d39/d3e/d81/d44/d7c/d22/d59/f80 [0,4194304] 0 2024-06-15T01:07:11.545 INFO:tasks.workunit.client.0.smithi137.stdout:9/448: rename df/d12/d4e/f6d to df/d12/d29/d4a/f7f 0 2024-06-15T01:07:11.546 INFO:tasks.workunit.client.0.smithi137.stdout:4/572: dread - dd/d15/d36/d45/d55/d8d/f90 zero size 2024-06-15T01:07:11.547 INFO:tasks.workunit.client.1.smithi142.stdout:5/864: link d2/d18/d79/f82 d2/de/d27/f10c 0 2024-06-15T01:07:11.547 INFO:tasks.workunit.client.1.smithi142.stdout:4/954: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/l72 358 1 2024-06-15T01:07:11.548 INFO:tasks.workunit.client.0.smithi137.stdout:1/695: dread d0/d3/d1b/d4c/f8e [0,4194304] 0 2024-06-15T01:07:11.548 INFO:tasks.workunit.client.0.smithi137.stdout:4/573: stat dd/d15/d21/d3a/d6e/d7b/d96 0 2024-06-15T01:07:11.549 INFO:tasks.workunit.client.1.smithi142.stdout:0/846: mknod d3/d39/d3e/d81/d1b/d38/da3/c11d 0 2024-06-15T01:07:11.549 INFO:tasks.workunit.client.0.smithi137.stdout:6/537: symlink d0/d5/d9/d29/la9 0 2024-06-15T01:07:11.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/574: write dd/d15/d21/d9c/d75/d99/fbd [3962830,74420] 0 2024-06-15T01:07:11.551 INFO:tasks.workunit.client.1.smithi142.stdout:0/847: stat d3/d39/d3e/d81/d44/d7c/fbd 0 2024-06-15T01:07:11.551 INFO:tasks.workunit.client.0.smithi137.stdout:6/538: write d0/d5/d9/f9b [814372,32569] 0 2024-06-15T01:07:11.553 INFO:tasks.workunit.client.1.smithi142.stdout:0/848: read d3/d39/d3e/d81/d44/f49 [199409,37867] 0 2024-06-15T01:07:11.553 INFO:tasks.workunit.client.0.smithi137.stdout:6/539: fsync d0/d1a/d3e/f71 0 2024-06-15T01:07:11.554 INFO:tasks.workunit.client.1.smithi142.stdout:5/865: creat d2/de/d19/f10d x:0 0 0 2024-06-15T01:07:11.557 INFO:tasks.workunit.client.1.smithi142.stdout:4/955: link d5/d9/da/da3/f123 d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/f14f 0 2024-06-15T01:07:11.557 INFO:tasks.workunit.client.0.smithi137.stdout:1/696: dread d0/d3/d1b/d4c/d59/dd4/fe1 [0,4194304] 0 2024-06-15T01:07:11.557 INFO:tasks.workunit.client.0.smithi137.stdout:4/575: symlink dd/d15/d36/d45/dcb/d83/db2/d67/dbe/ld9 0 2024-06-15T01:07:11.558 INFO:tasks.workunit.client.1.smithi142.stdout:4/956: chown d5/d9/d23/d68/f149 0 1 2024-06-15T01:07:11.559 INFO:tasks.workunit.client.0.smithi137.stdout:4/576: fdatasync dd/d15/d36/d45/fd7 0 2024-06-15T01:07:11.559 INFO:tasks.workunit.client.1.smithi142.stdout:0/849: mkdir d3/d39/d3e/d11e 0 2024-06-15T01:07:11.559 INFO:tasks.workunit.client.1.smithi142.stdout:4/957: write f2 [961723,93245] 0 2024-06-15T01:07:11.560 INFO:tasks.workunit.client.0.smithi137.stdout:6/540: symlink d0/d1a/d3e/d57/d32/laa 0 2024-06-15T01:07:11.561 INFO:tasks.workunit.client.1.smithi142.stdout:5/866: dwrite d2/d18/fc9 [0,4194304] 0 2024-06-15T01:07:11.563 INFO:tasks.workunit.client.0.smithi137.stdout:4/577: fsync dd/d15/d36/d45/dcb/d83/db2/d67/f6c 0 2024-06-15T01:07:11.564 INFO:tasks.workunit.client.1.smithi142.stdout:0/850: creat d3/d39/d3e/d81/d4c/d61/f11f x:0 0 0 2024-06-15T01:07:11.568 INFO:tasks.workunit.client.1.smithi142.stdout:5/867: write d2/d18/f55 [818041,96073] 0 2024-06-15T01:07:11.569 INFO:tasks.workunit.client.1.smithi142.stdout:4/958: creat d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d102/f150 x:0 0 0 2024-06-15T01:07:11.570 INFO:tasks.workunit.client.0.smithi137.stdout:6/541: dwrite d0/d1a/d68/f6a [0,4194304] 0 2024-06-15T01:07:11.570 INFO:tasks.workunit.client.0.smithi137.stdout:4/578: getdents dd/dbf 0 2024-06-15T01:07:11.571 INFO:tasks.workunit.client.1.smithi142.stdout:4/959: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f13d [487599,99164] 0 2024-06-15T01:07:11.572 INFO:tasks.workunit.client.1.smithi142.stdout:0/851: dread d3/d39/d3e/d81/d4c/f107 [0,4194304] 0 2024-06-15T01:07:11.572 INFO:tasks.workunit.client.0.smithi137.stdout:4/579: chown dd/d15/c44 2204705 1 2024-06-15T01:07:11.575 INFO:tasks.workunit.client.1.smithi142.stdout:2/856: mknod d8/df/d10/d94/d25/d22/d43/d52/c10b 0 2024-06-15T01:07:11.576 INFO:tasks.workunit.client.0.smithi137.stdout:6/542: symlink d0/d1a/d3e/d57/d1d/lab 0 2024-06-15T01:07:11.577 INFO:tasks.workunit.client.1.smithi142.stdout:0/852: creat d3/d39/d3e/f120 x:0 0 0 2024-06-15T01:07:11.577 INFO:tasks.workunit.client.1.smithi142.stdout:5/868: symlink d2/d3e/d74/da5/dbe/d104/l10e 0 2024-06-15T01:07:11.579 INFO:tasks.workunit.client.0.smithi137.stdout:6/543: mkdir d0/d1a/d3e/d57/d32/d92/dac 0 2024-06-15T01:07:11.579 INFO:tasks.workunit.client.1.smithi142.stdout:0/853: write d3/d39/d3e/d81/f73 [3963222,14305] 0 2024-06-15T01:07:11.580 INFO:tasks.workunit.client.0.smithi137.stdout:4/580: read dd/d15/f2e [775016,22970] 0 2024-06-15T01:07:11.580 INFO:tasks.workunit.client.0.smithi137.stdout:4/581: chown c1 262436999 1 2024-06-15T01:07:11.581 INFO:tasks.workunit.client.1.smithi142.stdout:5/869: readlink d2/de/d19/d5f/d68/d76/d8c/lee 0 2024-06-15T01:07:11.582 INFO:tasks.workunit.client.1.smithi142.stdout:4/960: fsync d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/f105 0 2024-06-15T01:07:11.582 INFO:tasks.workunit.client.0.smithi137.stdout:6/544: mknod d0/d1a/d3e/d57/d32/d74/cad 0 2024-06-15T01:07:11.585 INFO:tasks.workunit.client.1.smithi142.stdout:0/854: write d3/d39/d3e/d81/f17 [4650908,122096] 0 2024-06-15T01:07:11.585 INFO:tasks.workunit.client.1.smithi142.stdout:0/855: dread - d3/d39/d3e/f120 zero size 2024-06-15T01:07:11.585 INFO:tasks.workunit.client.0.smithi137.stdout:4/582: rmdir dd/d15/d36/d45/dcb/d83 39 2024-06-15T01:07:11.587 INFO:tasks.workunit.client.0.smithi137.stdout:6/545: truncate d0/d5/d9/d29/d64/f86 76065 0 2024-06-15T01:07:11.588 INFO:tasks.workunit.client.1.smithi142.stdout:2/857: rename d8/df/d10/d94/d25/d76/fc6 to d8/d19/da3/f10c 0 2024-06-15T01:07:11.588 INFO:tasks.workunit.client.0.smithi137.stdout:9/449: write df/d12/d29/f44 [1503496,2760] 0 2024-06-15T01:07:11.589 INFO:tasks.workunit.client.1.smithi142.stdout:2/858: chown d8/df/d1f/c39 1392558 1 2024-06-15T01:07:11.590 INFO:tasks.workunit.client.0.smithi137.stdout:1/697: write d0/d3/d89/fb3 [190029,89560] 0 2024-06-15T01:07:11.590 INFO:tasks.workunit.client.0.smithi137.stdout:9/450: write df/d1a/f76 [2986715,118759] 0 2024-06-15T01:07:11.590 INFO:tasks.workunit.client.1.smithi142.stdout:5/870: dwrite d2/de/d27/db2/ff0 [0,4194304] 0 2024-06-15T01:07:11.591 INFO:tasks.workunit.client.1.smithi142.stdout:0/856: mknod d3/d39/d9a/d9d/dcd/c121 0 2024-06-15T01:07:11.591 INFO:tasks.workunit.client.0.smithi137.stdout:1/698: chown d0/d3/d1b/d4c/d59/d64/lcb 123405018 1 2024-06-15T01:07:11.592 INFO:tasks.workunit.client.1.smithi142.stdout:7/908: dread d2/d6/f9 [0,4194304] 0 2024-06-15T01:07:11.593 INFO:tasks.workunit.client.0.smithi137.stdout:1/699: fsync d0/d10/daf/fb6 0 2024-06-15T01:07:11.593 INFO:tasks.workunit.client.0.smithi137.stdout:9/451: mknod df/c80 0 2024-06-15T01:07:11.594 INFO:tasks.workunit.client.0.smithi137.stdout:9/452: stat df/f4d 0 2024-06-15T01:07:11.596 INFO:tasks.workunit.client.0.smithi137.stdout:1/700: mkdir d0/d3/deb 0 2024-06-15T01:07:11.596 INFO:tasks.workunit.client.0.smithi137.stdout:9/453: write df/d12/f59 [340199,65047] 0 2024-06-15T01:07:11.602 INFO:tasks.workunit.client.0.smithi137.stdout:1/701: creat d0/d3/deb/fec x:0 0 0 2024-06-15T01:07:11.607 INFO:tasks.workunit.client.0.smithi137.stdout:1/702: link d0/d3/d1b/d4c/d59/dd4/fe1 d0/d3/d14/d34/fed 0 2024-06-15T01:07:11.611 INFO:tasks.workunit.client.0.smithi137.stdout:9/454: truncate df/d49/d4b/f57 9965 0 2024-06-15T01:07:11.612 INFO:tasks.workunit.client.0.smithi137.stdout:1/703: write d0/d10/fb5 [221249,26679] 0 2024-06-15T01:07:11.614 INFO:tasks.workunit.client.0.smithi137.stdout:1/704: write d0/d3/d1b/d4c/f8e [463918,18965] 0 2024-06-15T01:07:11.616 INFO:tasks.workunit.client.0.smithi137.stdout:1/705: write d0/d3/d1b/d4c/d59/d64/f6e [4083011,8002] 0 2024-06-15T01:07:11.618 INFO:tasks.workunit.client.0.smithi137.stdout:6/546: dread d0/d1a/f65 [4194304,4194304] 0 2024-06-15T01:07:11.621 INFO:tasks.workunit.client.0.smithi137.stdout:1/706: read d0/d1f/d60/f74 [2836117,37427] 0 2024-06-15T01:07:11.622 INFO:tasks.workunit.client.0.smithi137.stdout:1/707: chown d0/d3/f9f 630 1 2024-06-15T01:07:11.623 INFO:tasks.workunit.client.0.smithi137.stdout:6/547: creat d0/d5/fae x:0 0 0 2024-06-15T01:07:11.625 INFO:tasks.workunit.client.0.smithi137.stdout:9/455: dwrite df/d12/d1b/d31/d42/f46 [0,4194304] 0 2024-06-15T01:07:11.625 INFO:tasks.workunit.client.0.smithi137.stdout:1/708: mknod d0/d1f/d60/d99/de3/cee 0 2024-06-15T01:07:11.664 INFO:tasks.workunit.client.1.smithi142.stdout:8/917: write d0/d17/d40/f76 [237344,37363] 0 2024-06-15T01:07:11.665 INFO:tasks.workunit.client.1.smithi142.stdout:6/886: truncate d1a/d39/fbe 1204059 0 2024-06-15T01:07:11.665 INFO:tasks.workunit.client.1.smithi142.stdout:5/871: fdatasync d2/d18/d79/fce 0 2024-06-15T01:07:11.665 INFO:tasks.workunit.client.1.smithi142.stdout:2/859: read d8/d19/fe7 [4073,24047] 0 2024-06-15T01:07:11.665 INFO:tasks.workunit.client.1.smithi142.stdout:7/909: fdatasync d2/d8/dd/d14/f4b 0 2024-06-15T01:07:11.667 INFO:tasks.workunit.client.1.smithi142.stdout:6/887: stat d1a/d39/d42/df7/dfc 0 2024-06-15T01:07:11.668 INFO:tasks.workunit.client.1.smithi142.stdout:0/857: mknod d3/d39/d3e/d81/d44/d7c/d22/d59/c122 0 2024-06-15T01:07:11.669 INFO:tasks.workunit.client.1.smithi142.stdout:6/888: chown d1a/d39/d3a/d51/d62/f79 13 1 2024-06-15T01:07:11.669 INFO:tasks.workunit.client.1.smithi142.stdout:7/910: mknod d2/d8/dd/d8a/d94/c122 0 2024-06-15T01:07:11.669 INFO:tasks.workunit.client.1.smithi142.stdout:8/918: chown d0/d17/l51 233223 1 2024-06-15T01:07:11.673 INFO:tasks.workunit.client.1.smithi142.stdout:0/858: unlink d3/d39/d3e/d81/d1b/d38/f114 0 2024-06-15T01:07:11.674 INFO:tasks.workunit.client.1.smithi142.stdout:6/889: read d1a/d2d/d3b/fa9 [669853,35317] 0 2024-06-15T01:07:11.674 INFO:tasks.workunit.client.1.smithi142.stdout:7/911: creat d2/db6/de6/f123 x:0 0 0 2024-06-15T01:07:11.674 INFO:tasks.workunit.client.1.smithi142.stdout:0/859: chown d3/d39/d3e/d81/d1b/d38/da3/lab 14594 1 2024-06-15T01:07:11.677 INFO:tasks.workunit.client.1.smithi142.stdout:8/919: creat d0/d17/d105/f115 x:0 0 0 2024-06-15T01:07:11.677 INFO:tasks.workunit.client.1.smithi142.stdout:2/860: write d8/f9 [4410427,103400] 0 2024-06-15T01:07:11.679 INFO:tasks.workunit.client.1.smithi142.stdout:5/872: rename d2/de/d19/d5f/d68/fdc to d2/de/d19/f10f 0 2024-06-15T01:07:11.679 INFO:tasks.workunit.client.1.smithi142.stdout:8/920: write d0/d17/f61 [6307460,44949] 0 2024-06-15T01:07:11.683 INFO:tasks.workunit.client.1.smithi142.stdout:0/860: mkdir d3/d39/d3e/d81/d44/d7c/d123 0 2024-06-15T01:07:11.684 INFO:tasks.workunit.client.1.smithi142.stdout:7/912: dwrite d2/d8/dd/d14/f58 [4194304,4194304] 0 2024-06-15T01:07:11.684 INFO:tasks.workunit.client.1.smithi142.stdout:2/861: write d8/fc [1479784,66786] 0 2024-06-15T01:07:11.684 INFO:tasks.workunit.client.1.smithi142.stdout:5/873: mknod d2/d3e/d31/d105/c110 0 2024-06-15T01:07:11.687 INFO:tasks.workunit.client.1.smithi142.stdout:5/874: stat d2/d3e/c48 0 2024-06-15T01:07:11.691 INFO:tasks.workunit.client.1.smithi142.stdout:0/861: creat d3/d39/d3e/d81/d44/d7c/d123/f124 x:0 0 0 2024-06-15T01:07:11.692 INFO:tasks.workunit.client.1.smithi142.stdout:6/890: dwrite d1a/d3c/d100/d58/fae [0,4194304] 0 2024-06-15T01:07:11.692 INFO:tasks.workunit.client.1.smithi142.stdout:5/875: chown d2/d18/l43 508 1 2024-06-15T01:07:11.695 INFO:tasks.workunit.client.1.smithi142.stdout:2/862: creat d8/df/d10/d94/d25/d1a/d63/f10d x:0 0 0 2024-06-15T01:07:11.695 INFO:tasks.workunit.client.1.smithi142.stdout:5/876: chown d2/de/d19/d5f 819756 1 2024-06-15T01:07:11.695 INFO:tasks.workunit.client.1.smithi142.stdout:8/921: dwrite d0/f43 [0,4194304] 0 2024-06-15T01:07:11.696 INFO:tasks.workunit.client.1.smithi142.stdout:5/877: fsync d2/d18/f10a 0 2024-06-15T01:07:11.697 INFO:tasks.workunit.client.1.smithi142.stdout:9/830: sync 2024-06-15T01:07:11.698 INFO:tasks.workunit.client.1.smithi142.stdout:6/891: mkdir d1a/d3c/d132 0 2024-06-15T01:07:11.702 INFO:tasks.workunit.client.1.smithi142.stdout:9/831: stat d6/d34/ddd/fd7 0 2024-06-15T01:07:11.703 INFO:tasks.workunit.client.1.smithi142.stdout:5/878: write d2/d18/f10a [945520,111255] 0 2024-06-15T01:07:11.703 INFO:tasks.workunit.client.1.smithi142.stdout:8/922: mkdir d0/d17/d40/d5e/d116 0 2024-06-15T01:07:11.706 INFO:tasks.workunit.client.1.smithi142.stdout:9/832: creat d6/dc/d1b/d25/d58/f135 x:0 0 0 2024-06-15T01:07:11.707 INFO:tasks.workunit.client.1.smithi142.stdout:5/879: mknod d2/d18/d2e/c111 0 2024-06-15T01:07:11.708 INFO:tasks.workunit.client.1.smithi142.stdout:2/863: write d8/df/d10/d94/f5d [755726,38472] 0 2024-06-15T01:07:11.710 INFO:tasks.workunit.client.1.smithi142.stdout:6/892: write fc [2087931,104102] 0 2024-06-15T01:07:11.714 INFO:tasks.workunit.client.1.smithi142.stdout:6/893: read - d1a/d39/d42/f116 zero size 2024-06-15T01:07:11.716 INFO:tasks.workunit.client.1.smithi142.stdout:7/913: dwrite d2/d8/da/d33/d25/f59 [0,4194304] 0 2024-06-15T01:07:11.718 INFO:tasks.workunit.client.1.smithi142.stdout:8/923: dwrite d0/d25/d53/d9c/f101 [0,4194304] 0 2024-06-15T01:07:11.720 INFO:tasks.workunit.client.1.smithi142.stdout:0/862: dwrite d3/d39/d3e/d81/d44/d7c/d22/ffb [0,4194304] 0 2024-06-15T01:07:11.723 INFO:tasks.workunit.client.1.smithi142.stdout:0/863: fsync d3/d39/d3e/d81/d44/d7c/d123/f124 0 2024-06-15T01:07:11.724 INFO:tasks.workunit.client.1.smithi142.stdout:7/914: rename d2/d8/dd/l110 to d2/d6/daf/l124 0 2024-06-15T01:07:11.726 INFO:tasks.workunit.client.1.smithi142.stdout:0/864: write d3/d39/d9a/da2/fbc [5175576,4163] 0 2024-06-15T01:07:11.727 INFO:tasks.workunit.client.1.smithi142.stdout:5/880: write d2/f12 [696474,125214] 0 2024-06-15T01:07:11.728 INFO:tasks.workunit.client.1.smithi142.stdout:2/864: getdents d8/df/d10/d94/d25/d22/d43/d52/d7a 0 2024-06-15T01:07:11.730 INFO:tasks.workunit.client.1.smithi142.stdout:0/865: stat d3/d39/d3e/d81/d1b/d38/da3/lab 0 2024-06-15T01:07:11.730 INFO:tasks.workunit.client.1.smithi142.stdout:8/924: rename d0/d25/d5a/d8f/df3/d109/lf8 to d0/d25/d53/da7/l117 0 2024-06-15T01:07:11.732 INFO:tasks.workunit.client.1.smithi142.stdout:5/881: mknod d2/de/d19/d5f/d68/d76/da6/c112 0 2024-06-15T01:07:11.734 INFO:tasks.workunit.client.1.smithi142.stdout:8/925: mknod d0/d25/d53/d9c/dbf/c118 0 2024-06-15T01:07:11.735 INFO:tasks.workunit.client.1.smithi142.stdout:5/882: mknod d2/de/d19/c113 0 2024-06-15T01:07:11.736 INFO:tasks.workunit.client.1.smithi142.stdout:0/866: dread d3/d39/d3e/fc5 [0,4194304] 0 2024-06-15T01:07:11.739 INFO:tasks.workunit.client.1.smithi142.stdout:2/865: dwrite d8/df/d10/d94/d25/da9/fd0 [0,4194304] 0 2024-06-15T01:07:11.740 INFO:tasks.workunit.client.1.smithi142.stdout:0/867: creat d3/d39/d9a/d9d/dcd/f125 x:0 0 0 2024-06-15T01:07:11.740 INFO:tasks.workunit.client.1.smithi142.stdout:9/833: dread d6/dc/d1b/d25/d58/de2/fe8 [0,4194304] 0 2024-06-15T01:07:11.741 INFO:tasks.workunit.client.1.smithi142.stdout:5/883: fdatasync d2/de/d19/d5f/d68/d76/f7c 0 2024-06-15T01:07:11.742 INFO:tasks.workunit.client.1.smithi142.stdout:9/834: readlink d6/l42 0 2024-06-15T01:07:11.743 INFO:tasks.workunit.client.1.smithi142.stdout:8/926: link d0/d25/d53/da7/d10d/l112 d0/d25/d5a/d8f/l119 0 2024-06-15T01:07:11.744 INFO:tasks.workunit.client.1.smithi142.stdout:2/866: truncate d8/df/d10/d1c/d7c/fcc 140641 0 2024-06-15T01:07:11.746 INFO:tasks.workunit.client.1.smithi142.stdout:9/835: readlink d6/dc/d1b/d25/d35/l121 0 2024-06-15T01:07:11.746 INFO:tasks.workunit.client.1.smithi142.stdout:5/884: truncate d2/d18/d2e/f71 4478094 0 2024-06-15T01:07:11.747 INFO:tasks.workunit.client.1.smithi142.stdout:2/867: truncate d8/d19/fc8 490187 0 2024-06-15T01:07:11.748 INFO:tasks.workunit.client.1.smithi142.stdout:5/885: chown d2/de/d27/f9e 1112091191 1 2024-06-15T01:07:11.749 INFO:tasks.workunit.client.1.smithi142.stdout:9/836: rmdir d6/dc/d1b/d25/d58/de2 39 2024-06-15T01:07:11.752 INFO:tasks.workunit.client.0.smithi137.stdout:9/456: write df/d12/d29/d4a/f6c [5063970,19820] 0 2024-06-15T01:07:11.753 INFO:tasks.workunit.client.1.smithi142.stdout:0/868: truncate d3/d39/d9a/d9d/dbf/f11a 3200334 0 2024-06-15T01:07:11.753 INFO:tasks.workunit.client.1.smithi142.stdout:7/915: write d2/d8/dd/dc0/fc3 [1012826,59010] 0 2024-06-15T01:07:11.754 INFO:tasks.workunit.client.0.smithi137.stdout:1/709: unlink d0/d1f/d60/da9/laa 0 2024-06-15T01:07:11.754 INFO:tasks.workunit.client.0.smithi137.stdout:6/548: getdents d0/d1a/d3e/d87 0 2024-06-15T01:07:11.756 INFO:tasks.workunit.client.0.smithi137.stdout:1/710: write d0/d2d/d3e/f77 [622688,105770] 0 2024-06-15T01:07:11.758 INFO:tasks.workunit.client.1.smithi142.stdout:0/869: chown d3/d39/d9a/d9d/dbf/df4 247517 1 2024-06-15T01:07:11.759 INFO:tasks.workunit.client.1.smithi142.stdout:9/837: dwrite d6/dc/dd/d22/f10f [4194304,4194304] 0 2024-06-15T01:07:11.761 INFO:tasks.workunit.client.0.smithi137.stdout:6/549: mkdir d0/d1a/d3e/d57/d1d/d2e/d62/daf 0 2024-06-15T01:07:11.762 INFO:tasks.workunit.client.1.smithi142.stdout:9/838: read d6/dc/d1b/d6f/d103/d8d/fb9 [290276,57535] 0 2024-06-15T01:07:11.762 INFO:tasks.workunit.client.1.smithi142.stdout:0/870: fdatasync d3/f28 0 2024-06-15T01:07:11.762 INFO:tasks.workunit.client.1.smithi142.stdout:7/916: chown d2/d6/d22/d3c/f111 593882 1 2024-06-15T01:07:11.765 INFO:tasks.workunit.client.0.smithi137.stdout:1/711: getdents d0/d1f/d76 0 2024-06-15T01:07:11.765 INFO:tasks.workunit.client.1.smithi142.stdout:0/871: write d3/d39/d3e/d81/fb [490601,75015] 0 2024-06-15T01:07:11.767 INFO:tasks.workunit.client.1.smithi142.stdout:8/927: dread d0/d17/d40/d6e/fa9 [0,4194304] 0 2024-06-15T01:07:11.768 INFO:tasks.workunit.client.1.smithi142.stdout:8/928: fdatasync d0/d17/d40/d86/fec 0 2024-06-15T01:07:11.769 INFO:tasks.workunit.client.0.smithi137.stdout:6/550: creat d0/d5/fb0 x:0 0 0 2024-06-15T01:07:11.770 INFO:tasks.workunit.client.1.smithi142.stdout:8/929: stat d0/d25/d53/da7/dcb/dfb 0 2024-06-15T01:07:11.770 INFO:tasks.workunit.client.1.smithi142.stdout:9/839: mknod d6/dc/c136 0 2024-06-15T01:07:11.771 INFO:tasks.workunit.client.0.smithi137.stdout:1/712: mknod d0/d1f/d60/d99/de3/cef 0 2024-06-15T01:07:11.775 INFO:tasks.workunit.client.1.smithi142.stdout:0/872: mknod d3/d39/d3e/d81/d1b/d38/c126 0 2024-06-15T01:07:11.776 INFO:tasks.workunit.client.0.smithi137.stdout:9/457: dwrite df/d12/d15/f26 [4194304,4194304] 0 2024-06-15T01:07:11.776 INFO:tasks.workunit.client.1.smithi142.stdout:9/840: creat d6/dc/d1b/d6f/de1/d110/f137 x:0 0 0 2024-06-15T01:07:11.776 INFO:tasks.workunit.client.1.smithi142.stdout:5/886: dwrite d2/d3e/f5c [0,4194304] 0 2024-06-15T01:07:11.777 INFO:tasks.workunit.client.0.smithi137.stdout:1/713: mkdir d0/d3/d14/d34/df0 0 2024-06-15T01:07:11.777 INFO:tasks.workunit.client.1.smithi142.stdout:5/887: stat d2/d18/d79/c95 0 2024-06-15T01:07:11.779 INFO:tasks.workunit.client.1.smithi142.stdout:1/934: sync 2024-06-15T01:07:11.781 INFO:tasks.workunit.client.1.smithi142.stdout:5/888: truncate d2/de/d19/d5f/d68/d76/fc8 1734498 0 2024-06-15T01:07:11.782 INFO:tasks.workunit.client.1.smithi142.stdout:1/935: mkdir d6/d25/dfc/d14b 0 2024-06-15T01:07:11.782 INFO:tasks.workunit.client.0.smithi137.stdout:9/458: symlink df/d12/d15/d16/d3c/l81 0 2024-06-15T01:07:11.783 INFO:tasks.workunit.client.0.smithi137.stdout:9/459: chown df/d12/d15/f73 3671508 1 2024-06-15T01:07:11.783 INFO:tasks.workunit.client.1.smithi142.stdout:2/868: dread d8/f40 [0,4194304] 0 2024-06-15T01:07:11.784 INFO:tasks.workunit.client.1.smithi142.stdout:9/841: mknod d6/dc/d51/d11c/c138 0 2024-06-15T01:07:11.785 INFO:tasks.workunit.client.1.smithi142.stdout:1/936: write d6/d25/dfc/d132/db2/dc2/f114 [444278,11534] 0 2024-06-15T01:07:11.786 INFO:tasks.workunit.client.1.smithi142.stdout:1/937: fsync d6/d12/f22 0 2024-06-15T01:07:11.787 INFO:tasks.workunit.client.1.smithi142.stdout:7/917: truncate d2/db6/fec 2190381 0 2024-06-15T01:07:11.788 INFO:tasks.workunit.client.1.smithi142.stdout:1/938: stat d6/d25/dfc/d132/db2/dbf/f104 0 2024-06-15T01:07:11.789 INFO:tasks.workunit.client.0.smithi137.stdout:6/551: link d0/d5/d9/d29/f4b d0/d5/d9/fb1 0 2024-06-15T01:07:11.791 INFO:tasks.workunit.client.0.smithi137.stdout:9/460: creat df/d49/d72/f82 x:0 0 0 2024-06-15T01:07:11.791 INFO:tasks.workunit.client.1.smithi142.stdout:0/873: getdents d3/d39/d3e/d81/d1b/d38 0 2024-06-15T01:07:11.792 INFO:tasks.workunit.client.0.smithi137.stdout:6/552: creat d0/d1a/d3e/d57/d32/d74/fb2 x:0 0 0 2024-06-15T01:07:11.795 INFO:tasks.workunit.client.1.smithi142.stdout:2/869: creat d8/df/d10/d94/db5/f10e x:0 0 0 2024-06-15T01:07:11.797 INFO:tasks.workunit.client.0.smithi137.stdout:9/461: truncate df/d49/d4b/f57 816008 0 2024-06-15T01:07:11.797 INFO:tasks.workunit.client.1.smithi142.stdout:1/939: mknod d6/d25/dfc/d132/c14c 0 2024-06-15T01:07:11.798 INFO:tasks.workunit.client.0.smithi137.stdout:9/462: truncate df/d13/f22 1459844 0 2024-06-15T01:07:11.798 INFO:tasks.workunit.client.0.smithi137.stdout:6/553: creat d0/d1a/d3e/d57/d32/d92/dac/fb3 x:0 0 0 2024-06-15T01:07:11.799 INFO:tasks.workunit.client.1.smithi142.stdout:8/930: dread d0/f5f [0,4194304] 0 2024-06-15T01:07:11.799 INFO:tasks.workunit.client.1.smithi142.stdout:5/889: fsync d2/d18/f61 0 2024-06-15T01:07:11.799 INFO:tasks.workunit.client.0.smithi137.stdout:1/714: write d0/fb0 [858012,25964] 0 2024-06-15T01:07:11.800 INFO:tasks.workunit.client.1.smithi142.stdout:8/931: fdatasync d0/d25/d5a/f69 0 2024-06-15T01:07:11.801 INFO:tasks.workunit.client.1.smithi142.stdout:0/874: creat d3/d39/d3e/d8f/f127 x:0 0 0 2024-06-15T01:07:11.801 INFO:tasks.workunit.client.1.smithi142.stdout:9/842: fdatasync d6/f6a 0 2024-06-15T01:07:11.801 INFO:tasks.workunit.client.1.smithi142.stdout:2/870: symlink d8/df/d1f/l10f 0 2024-06-15T01:07:11.802 INFO:tasks.workunit.client.1.smithi142.stdout:0/875: write d3/f2a [5664357,25922] 0 2024-06-15T01:07:11.802 INFO:tasks.workunit.client.1.smithi142.stdout:7/918: mkdir d2/d6/d22/ddd/d85/d125 0 2024-06-15T01:07:11.803 INFO:tasks.workunit.client.1.smithi142.stdout:0/876: chown d3/d39/d9a/da2 64737155 1 2024-06-15T01:07:11.804 INFO:tasks.workunit.client.0.smithi137.stdout:1/715: truncate d0/d3/d1b/d4c/d59/dd4/fe1 1675676 0 2024-06-15T01:07:11.804 INFO:tasks.workunit.client.1.smithi142.stdout:5/890: mknod d2/d3e/d74/da5/c114 0 2024-06-15T01:07:11.806 INFO:tasks.workunit.client.1.smithi142.stdout:5/891: dread d2/d3e/d31/fc3 [0,4194304] 0 2024-06-15T01:07:11.807 INFO:tasks.workunit.client.1.smithi142.stdout:5/892: chown d2/de/d19/d5f/d68/d76/da6/dc5/fd5 386623208 1 2024-06-15T01:07:11.808 INFO:tasks.workunit.client.0.smithi137.stdout:1/716: unlink d0/f2 0 2024-06-15T01:07:11.809 INFO:tasks.workunit.client.1.smithi142.stdout:2/871: symlink d8/d19/deb/df8/l110 0 2024-06-15T01:07:11.809 INFO:tasks.workunit.client.0.smithi137.stdout:1/717: mkdir d0/d3/d1b/d4c/d59/d93/df1 0 2024-06-15T01:07:11.810 INFO:tasks.workunit.client.0.smithi137.stdout:1/718: write d0/d3/d1b/f50 [2524782,65303] 0 2024-06-15T01:07:11.812 INFO:tasks.workunit.client.0.smithi137.stdout:1/719: write d0/d3/fd6 [228650,81518] 0 2024-06-15T01:07:11.817 INFO:tasks.workunit.client.0.smithi137.stdout:9/463: dread df/d12/d15/f32 [0,4194304] 0 2024-06-15T01:07:11.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:11 smithi142 ceph-mon[37379]: pgmap v21: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 266 MiB/s wr, 487 op/s 2024-06-15T01:07:11.820 INFO:tasks.workunit.client.0.smithi137.stdout:9/464: mknod df/d12/d1b/d31/c83 0 2024-06-15T01:07:11.824 INFO:tasks.workunit.client.0.smithi137.stdout:9/465: mknod df/d12/d15/d16/d3c/d51/c84 0 2024-06-15T01:07:11.824 INFO:tasks.workunit.client.0.smithi137.stdout:9/466: chown df/d2a/f4c 1652 1 2024-06-15T01:07:11.825 INFO:tasks.workunit.client.0.smithi137.stdout:1/720: write d0/f3b [572397,73651] 0 2024-06-15T01:07:11.832 INFO:tasks.workunit.client.1.smithi142.stdout:1/940: mknod d6/d25/d144/c14d 0 2024-06-15T01:07:11.832 INFO:tasks.workunit.client.1.smithi142.stdout:0/877: symlink d3/d39/d9a/d9d/df8/l128 0 2024-06-15T01:07:11.833 INFO:tasks.workunit.client.0.smithi137.stdout:9/467: symlink df/d12/d1b/d31/d42/l85 0 2024-06-15T01:07:11.833 INFO:tasks.workunit.client.0.smithi137.stdout:1/721: unlink d0/d3/d47/f88 0 2024-06-15T01:07:11.833 INFO:tasks.workunit.client.1.smithi142.stdout:2/872: readlink d8/df/d10/d1c/lee 0 2024-06-15T01:07:11.834 INFO:tasks.workunit.client.1.smithi142.stdout:1/941: stat d6/d12/d3c/d63/d8d/d97/dc6/f101 0 2024-06-15T01:07:11.835 INFO:tasks.workunit.client.1.smithi142.stdout:5/893: write d2/de/d27/f54 [5566830,40861] 0 2024-06-15T01:07:11.836 INFO:tasks.workunit.client.1.smithi142.stdout:8/932: write d0/d25/f35 [3103503,105283] 0 2024-06-15T01:07:11.836 INFO:tasks.workunit.client.1.smithi142.stdout:2/873: mknod d8/d19/d8d/c111 0 2024-06-15T01:07:11.837 INFO:tasks.workunit.client.1.smithi142.stdout:0/878: creat d3/d39/d3e/d81/d4c/d61/f129 x:0 0 0 2024-06-15T01:07:11.839 INFO:tasks.workunit.client.1.smithi142.stdout:1/942: fdatasync d6/d12/d80/f100 0 2024-06-15T01:07:11.843 INFO:tasks.workunit.client.1.smithi142.stdout:5/894: dread d2/d3e/d65/f70 [0,4194304] 0 2024-06-15T01:07:11.844 INFO:tasks.workunit.client.1.smithi142.stdout:8/933: rename d0/d25/d53/d9c/fbd to d0/d25/d66/f11a 0 2024-06-15T01:07:11.847 INFO:tasks.workunit.client.1.smithi142.stdout:8/934: chown d0/d17/d40/d6e/fbc 496327 1 2024-06-15T01:07:11.847 INFO:tasks.workunit.client.1.smithi142.stdout:0/879: dwrite d3/d39/d3e/d81/d4c/da0/fb2 [0,4194304] 0 2024-06-15T01:07:11.849 INFO:tasks.workunit.client.1.smithi142.stdout:1/943: rename d6/d25/f3d to d6/d12/da0/f14e 0 2024-06-15T01:07:11.852 INFO:tasks.workunit.client.1.smithi142.stdout:7/919: dread d2/d8/da/fae [0,4194304] 0 2024-06-15T01:07:11.854 INFO:tasks.workunit.client.1.smithi142.stdout:5/895: dwrite d2/f8b [0,4194304] 0 2024-06-15T01:07:11.855 INFO:tasks.workunit.client.1.smithi142.stdout:0/880: write d3/d39/d3e/d81/d4c/da0/fb2 [6087855,122906] 0 2024-06-15T01:07:11.855 INFO:tasks.workunit.client.1.smithi142.stdout:7/920: fsync d2/d8/da/d33/ffc 0 2024-06-15T01:07:11.856 INFO:tasks.workunit.client.1.smithi142.stdout:2/874: dwrite d8/df/d10/d94/d25/d1a/f1d [0,4194304] 0 2024-06-15T01:07:11.857 INFO:tasks.workunit.client.1.smithi142.stdout:8/935: creat d0/d25/f11b x:0 0 0 2024-06-15T01:07:11.858 INFO:tasks.workunit.client.0.smithi137.stdout:9/468: truncate df/d12/d1b/d31/f33 3872014 0 2024-06-15T01:07:11.860 INFO:tasks.workunit.client.1.smithi142.stdout:1/944: rename d6/d12/da0/c121 to d6/d25/dfc/d14b/c14f 0 2024-06-15T01:07:11.863 INFO:tasks.workunit.client.1.smithi142.stdout:5/896: truncate d2/de/d19/d93/fef 648904 0 2024-06-15T01:07:11.864 INFO:tasks.workunit.client.1.smithi142.stdout:8/936: fdatasync d0/d17/d40/d5e/f94 0 2024-06-15T01:07:11.864 INFO:tasks.workunit.client.1.smithi142.stdout:7/921: mkdir d2/d6/d22/ddd/d126 0 2024-06-15T01:07:11.872 INFO:tasks.workunit.client.1.smithi142.stdout:8/937: chown d0/l70 2153740 1 2024-06-15T01:07:11.872 INFO:tasks.workunit.client.1.smithi142.stdout:5/897: dwrite d2/de/d27/db2/ff0 [4194304,4194304] 0 2024-06-15T01:07:11.874 INFO:tasks.workunit.client.1.smithi142.stdout:1/945: creat d6/d12/d1c/d2c/dfb/f150 x:0 0 0 2024-06-15T01:07:11.875 INFO:tasks.workunit.client.1.smithi142.stdout:2/875: creat d8/df/d10/d94/d25/d1a/d63/d107/f112 x:0 0 0 2024-06-15T01:07:11.877 INFO:tasks.workunit.client.1.smithi142.stdout:8/938: write d0/d25/d5a/d84/fcc [4667880,43846] 0 2024-06-15T01:07:11.878 INFO:tasks.workunit.client.1.smithi142.stdout:7/922: rename d2/db6/de6/f123 to d2/d8/dd/d14/d2c/f127 0 2024-06-15T01:07:11.880 INFO:tasks.workunit.client.1.smithi142.stdout:8/939: fsync d0/d25/d5a/f4d 0 2024-06-15T01:07:11.881 INFO:tasks.workunit.client.0.smithi137.stdout:1/722: write d0/d3/d1b/f9d [63054,95547] 0 2024-06-15T01:07:11.882 INFO:tasks.workunit.client.1.smithi142.stdout:6/894: sync 2024-06-15T01:07:11.882 INFO:tasks.workunit.client.1.smithi142.stdout:4/961: sync 2024-06-15T01:07:11.882 INFO:tasks.workunit.client.1.smithi142.stdout:3/932: sync 2024-06-15T01:07:11.882 INFO:tasks.workunit.client.1.smithi142.stdout:2/876: truncate d8/d19/deb/df8/f103 4547 0 2024-06-15T01:07:11.883 INFO:tasks.workunit.client.1.smithi142.stdout:5/898: rename d2/de/d19/d5f/d68/d76/f81 to d2/d3e/d98/f115 0 2024-06-15T01:07:11.887 INFO:tasks.workunit.client.1.smithi142.stdout:1/946: dwrite d6/d12/d1c/d13f/f115 [0,4194304] 0 2024-06-15T01:07:11.887 INFO:tasks.workunit.client.1.smithi142.stdout:6/895: write d1a/d2d/d3b/dab/f109 [204535,29995] 0 2024-06-15T01:07:11.887 INFO:tasks.workunit.client.1.smithi142.stdout:5/899: stat d2/c4f 0 2024-06-15T01:07:11.888 INFO:tasks.workunit.client.0.smithi137.stdout:1/723: symlink d0/d3/d1b/d4c/d59/d93/lf2 0 2024-06-15T01:07:11.888 INFO:tasks.workunit.client.1.smithi142.stdout:2/877: readlink d8/d19/l1b 0 2024-06-15T01:07:11.890 INFO:tasks.workunit.client.0.smithi137.stdout:1/724: chown d0/d3/d1b/d4c/d59/f71 409 1 2024-06-15T01:07:11.890 INFO:tasks.workunit.client.0.smithi137.stdout:9/469: dwrite f5 [0,4194304] 0 2024-06-15T01:07:11.890 INFO:tasks.workunit.client.1.smithi142.stdout:1/947: readlink d6/d12/d1c/l10d 0 2024-06-15T01:07:11.891 INFO:tasks.workunit.client.1.smithi142.stdout:4/962: truncate d5/d1c/f34 557394 0 2024-06-15T01:07:11.891 INFO:tasks.workunit.client.0.smithi137.stdout:1/725: stat d0/d3/d14/c3f 0 2024-06-15T01:07:11.893 INFO:tasks.workunit.client.0.smithi137.stdout:9/470: creat df/d12/d29/d66/f86 x:0 0 0 2024-06-15T01:07:11.895 INFO:tasks.workunit.client.0.smithi137.stdout:9/471: creat df/d1a/f87 x:0 0 0 2024-06-15T01:07:11.896 INFO:tasks.workunit.client.1.smithi142.stdout:4/963: chown d5/d9/d23/d32/la2 271032 1 2024-06-15T01:07:11.896 INFO:tasks.workunit.client.1.smithi142.stdout:5/900: creat d2/de/d27/f116 x:0 0 0 2024-06-15T01:07:11.898 INFO:tasks.workunit.client.1.smithi142.stdout:7/923: dread d2/f1a [0,4194304] 0 2024-06-15T01:07:11.898 INFO:tasks.workunit.client.1.smithi142.stdout:0/881: dread d3/d39/d9a/da2/fec [0,4194304] 0 2024-06-15T01:07:11.899 INFO:tasks.workunit.client.1.smithi142.stdout:5/901: chown d2/de/d19/f22 36 1 2024-06-15T01:07:11.899 INFO:tasks.workunit.client.1.smithi142.stdout:6/896: mknod d1a/d3c/d100/d58/c133 0 2024-06-15T01:07:11.900 INFO:tasks.workunit.client.0.smithi137.stdout:9/472: dread df/d12/d1b/d31/d42/f56 [0,4194304] 0 2024-06-15T01:07:11.903 INFO:tasks.workunit.client.0.smithi137.stdout:9/473: unlink df/f24 0 2024-06-15T01:07:11.904 INFO:tasks.workunit.client.1.smithi142.stdout:2/878: creat d8/df/d10/d94/d25/d22/f113 x:0 0 0 2024-06-15T01:07:11.905 INFO:tasks.workunit.client.0.smithi137.stdout:9/474: fsync df/d12/d1b/f35 0 2024-06-15T01:07:11.906 INFO:tasks.workunit.client.1.smithi142.stdout:7/924: creat d2/d8/da/d33/d25/f128 x:0 0 0 2024-06-15T01:07:11.907 INFO:tasks.workunit.client.1.smithi142.stdout:5/902: dread d2/de/d19/d5f/d68/d76/fc8 [0,4194304] 0 2024-06-15T01:07:11.909 INFO:tasks.workunit.client.0.smithi137.stdout:9/475: dread df/d12/d1b/d31/d42/f56 [0,4194304] 0 2024-06-15T01:07:11.911 INFO:tasks.workunit.client.0.smithi137.stdout:9/476: mknod df/d49/c88 0 2024-06-15T01:07:11.914 INFO:tasks.workunit.client.1.smithi142.stdout:7/925: creat d2/db6/f129 x:0 0 0 2024-06-15T01:07:11.915 INFO:tasks.workunit.client.0.smithi137.stdout:9/477: rename df/d12/f59 to df/f89 0 2024-06-15T01:07:11.915 INFO:tasks.workunit.client.1.smithi142.stdout:8/940: truncate d0/f8e 961734 0 2024-06-15T01:07:11.916 INFO:tasks.workunit.client.0.smithi137.stdout:9/478: fsync df/d12/d15/f26 0 2024-06-15T01:07:11.916 INFO:tasks.workunit.client.1.smithi142.stdout:5/903: write d2/d3e/d65/f70 [349744,34903] 0 2024-06-15T01:07:11.917 INFO:tasks.workunit.client.0.smithi137.stdout:9/479: dread - df/d12/d4e/f5c zero size 2024-06-15T01:07:11.922 INFO:tasks.workunit.client.1.smithi142.stdout:4/964: symlink d5/d1c/d6f/db9/d138/l151 0 2024-06-15T01:07:11.923 INFO:tasks.workunit.client.0.smithi137.stdout:1/726: dwrite d0/d2d/d3e/fa4 [0,4194304] 0 2024-06-15T01:07:11.923 INFO:tasks.workunit.client.1.smithi142.stdout:7/926: symlink d2/d8/da/d33/d25/d81/l12a 0 2024-06-15T01:07:11.925 INFO:tasks.workunit.client.1.smithi142.stdout:0/882: write d3/d39/d3e/d81/d1b/d38/f9b [4364331,116480] 0 2024-06-15T01:07:11.928 INFO:tasks.workunit.client.0.smithi137.stdout:1/727: symlink d0/d3/d47/d73/dc5/lf3 0 2024-06-15T01:07:11.928 INFO:tasks.workunit.client.1.smithi142.stdout:8/941: dwrite d0/f43 [0,4194304] 0 2024-06-15T01:07:11.928 INFO:tasks.workunit.client.0.smithi137.stdout:9/480: write df/d13/f30 [235475,110016] 0 2024-06-15T01:07:11.929 INFO:tasks.workunit.client.1.smithi142.stdout:2/879: write d8/d19/f56 [368071,38321] 0 2024-06-15T01:07:11.931 INFO:tasks.workunit.client.1.smithi142.stdout:5/904: dwrite d2/d3e/f5c [4194304,4194304] 0 2024-06-15T01:07:11.931 INFO:tasks.workunit.client.0.smithi137.stdout:1/728: getdents d0/d1f/d7d 0 2024-06-15T01:07:11.931 INFO:tasks.workunit.client.0.smithi137.stdout:1/729: dread - d0/d3/d47/fd3 zero size 2024-06-15T01:07:11.934 INFO:tasks.workunit.client.1.smithi142.stdout:1/948: dread d6/d12/f3e [0,4194304] 0 2024-06-15T01:07:11.935 INFO:tasks.workunit.client.1.smithi142.stdout:4/965: mkdir d5/d9/d23/d152 0 2024-06-15T01:07:11.935 INFO:tasks.workunit.client.0.smithi137.stdout:9/481: link df/f4d df/d12/d15/d75/f8a 0 2024-06-15T01:07:11.936 INFO:tasks.workunit.client.1.smithi142.stdout:7/927: creat d2/d6/d22/d47/de5/f12b x:0 0 0 2024-06-15T01:07:11.936 INFO:tasks.workunit.client.1.smithi142.stdout:0/883: rmdir d3/d39/d3e/d81/d4c 39 2024-06-15T01:07:11.937 INFO:tasks.workunit.client.1.smithi142.stdout:5/905: readlink d2/de/d85/la1 0 2024-06-15T01:07:11.938 INFO:tasks.workunit.client.1.smithi142.stdout:8/942: read d0/d17/d40/d6e/ff5 [65023,49257] 0 2024-06-15T01:07:11.938 INFO:tasks.workunit.client.0.smithi137.stdout:9/482: getdents df/d12/d29/d79 0 2024-06-15T01:07:11.940 INFO:tasks.workunit.client.0.smithi137.stdout:9/483: getdents df/d12/d29/d79 0 2024-06-15T01:07:11.945 INFO:tasks.workunit.client.1.smithi142.stdout:6/897: dread d1a/d39/ded/f101 [0,4194304] 0 2024-06-15T01:07:11.945 INFO:tasks.workunit.client.1.smithi142.stdout:5/906: fsync d2/de/d27/f56 0 2024-06-15T01:07:11.947 INFO:tasks.workunit.client.1.smithi142.stdout:6/898: mknod d1a/d3c/d100/d58/d87/c134 0 2024-06-15T01:07:11.948 INFO:tasks.workunit.client.1.smithi142.stdout:2/880: chown d8/df/d10/d94/d25/d22/d43/d51/cd8 51371181 1 2024-06-15T01:07:11.948 INFO:tasks.workunit.client.1.smithi142.stdout:4/966: dread d5/d1c/d6f/dc7/d30/dcc/f56 [0,4194304] 0 2024-06-15T01:07:11.950 INFO:tasks.workunit.client.1.smithi142.stdout:0/884: rename d3/d39/d9a/da2/lad to d3/d39/d3e/d81/d4c/da0/l12a 0 2024-06-15T01:07:11.952 INFO:tasks.workunit.client.1.smithi142.stdout:1/949: getdents d6/d12/d3c/d63/d8d/d97/dc6 0 2024-06-15T01:07:11.955 INFO:tasks.workunit.client.1.smithi142.stdout:2/881: mknod d8/df/d10/d94/d25/d22/d43/d51/c114 0 2024-06-15T01:07:11.956 INFO:tasks.workunit.client.1.smithi142.stdout:8/943: dwrite d0/d25/d53/da7/fe8 [0,4194304] 0 2024-06-15T01:07:11.956 INFO:tasks.workunit.client.1.smithi142.stdout:9/843: sync 2024-06-15T01:07:11.956 INFO:tasks.workunit.client.1.smithi142.stdout:5/907: dwrite d2/d3e/d74/fcf [0,4194304] 0 2024-06-15T01:07:11.956 INFO:tasks.workunit.client.1.smithi142.stdout:4/967: symlink d5/d9/da/d92/d98/dc1/l153 0 2024-06-15T01:07:11.959 INFO:tasks.workunit.client.1.smithi142.stdout:4/968: chown d5/d1c/d6f/db9/d138 3875631 1 2024-06-15T01:07:11.961 INFO:tasks.workunit.client.1.smithi142.stdout:8/944: stat d0/d25/d53/da7/dcb/ld3 0 2024-06-15T01:07:11.963 INFO:tasks.workunit.client.1.smithi142.stdout:9/844: unlink d6/d34/fad 0 2024-06-15T01:07:11.965 INFO:tasks.workunit.client.1.smithi142.stdout:5/908: rename d2/de/d85/la1 to d2/de/d19/d5f/d68/d76/d8c/d10b/l117 0 2024-06-15T01:07:11.965 INFO:tasks.workunit.client.1.smithi142.stdout:4/969: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6/d117/d11e 6254968 1 2024-06-15T01:07:11.966 INFO:tasks.workunit.client.1.smithi142.stdout:0/885: link d3/d39/d3e/d81/ld d3/d39/d3e/d81/d44/df1/l12b 0 2024-06-15T01:07:11.967 INFO:tasks.workunit.client.0.smithi137.stdout:1/730: write d0/d3/d1b/d4c/d59/dd4/fe1 [1926933,33924] 0 2024-06-15T01:07:11.970 INFO:tasks.workunit.client.1.smithi142.stdout:9/845: unlink d6/dc/d1b/d6f/d103/d8d/cd5 0 2024-06-15T01:07:11.971 INFO:tasks.workunit.client.0.smithi137.stdout:1/731: rmdir d0/d1f/d7d 39 2024-06-15T01:07:11.972 INFO:tasks.workunit.client.1.smithi142.stdout:4/970: read d5/d9/da/d92/fc5 [1201985,125180] 0 2024-06-15T01:07:11.974 INFO:tasks.workunit.client.0.smithi137.stdout:9/484: dwrite df/d49/d4b/f4f [0,4194304] 0 2024-06-15T01:07:11.974 INFO:tasks.workunit.client.1.smithi142.stdout:8/945: symlink d0/d25/d5a/l11c 0 2024-06-15T01:07:11.974 INFO:tasks.workunit.client.1.smithi142.stdout:5/909: rmdir d2/d18/d2e 39 2024-06-15T01:07:11.975 INFO:tasks.workunit.client.1.smithi142.stdout:5/910: read d2/de/d19/d5f/d68/d76/d96/fde [446634,52396] 0 2024-06-15T01:07:11.975 INFO:tasks.workunit.client.0.smithi137.stdout:9/485: write df/d12/d15/f63 [1461942,10467] 0 2024-06-15T01:07:11.976 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:11 smithi137 ceph-mon[29354]: pgmap v21: 65 pgs: 65 active+clean; 2.4 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 266 MiB/s wr, 487 op/s 2024-06-15T01:07:11.977 INFO:tasks.workunit.client.0.smithi137.stdout:1/732: mknod d0/d3/d14/d34/df0/cf4 0 2024-06-15T01:07:11.977 INFO:tasks.workunit.client.0.smithi137.stdout:1/733: readlink d0/d10/daf/lb8 0 2024-06-15T01:07:11.977 INFO:tasks.workunit.client.1.smithi142.stdout:0/886: symlink d3/d39/d3e/d81/d1b/d38/da3/l12c 0 2024-06-15T01:07:11.980 INFO:tasks.workunit.client.0.smithi137.stdout:1/734: creat d0/d10/d41/d7b/ff5 x:0 0 0 2024-06-15T01:07:11.984 INFO:tasks.workunit.client.1.smithi142.stdout:6/899: truncate d1a/d39/d42/f45 3685603 0 2024-06-15T01:07:11.984 INFO:tasks.workunit.client.0.smithi137.stdout:9/486: dwrite df/d12/d15/d75/f7c [0,4194304] 0 2024-06-15T01:07:11.987 INFO:tasks.workunit.client.1.smithi142.stdout:1/950: truncate d6/d12/f3e 2282422 0 2024-06-15T01:07:11.989 INFO:tasks.workunit.client.1.smithi142.stdout:8/946: link d0/f14 d0/d25/d53/f11d 0 2024-06-15T01:07:11.990 INFO:tasks.workunit.client.1.smithi142.stdout:0/887: symlink d3/d39/d3e/d11e/l12d 0 2024-06-15T01:07:11.994 INFO:tasks.workunit.client.0.smithi137.stdout:9/487: read fd [2183493,101339] 0 2024-06-15T01:07:11.995 INFO:tasks.workunit.client.1.smithi142.stdout:6/900: dread d1a/d39/fb5 [0,4194304] 0 2024-06-15T01:07:11.999 INFO:tasks.workunit.client.1.smithi142.stdout:1/951: dwrite d6/d12/d3c/d63/f109 [0,4194304] 0 2024-06-15T01:07:12.000 INFO:tasks.workunit.client.1.smithi142.stdout:2/882: dwrite d8/df/d10/d1c/f97 [0,4194304] 0 2024-06-15T01:07:12.001 INFO:tasks.workunit.client.0.smithi137.stdout:1/735: dwrite d0/d3/d47/f5d [0,4194304] 0 2024-06-15T01:07:12.002 INFO:tasks.workunit.client.1.smithi142.stdout:0/888: rename d3/d39/d3e/d81/lc8 to d3/d39/d3e/d81/d44/df1/l12e 0 2024-06-15T01:07:12.003 INFO:tasks.workunit.client.1.smithi142.stdout:4/971: write d5/d9/da/da3/f123 [378971,58786] 0 2024-06-15T01:07:12.004 INFO:tasks.workunit.client.1.smithi142.stdout:1/952: mkdir d6/d12/d1c/d2c/d151 0 2024-06-15T01:07:12.006 INFO:tasks.workunit.client.1.smithi142.stdout:2/883: mknod d8/df/d10/d94/d25/d1a/d63/d107/c115 0 2024-06-15T01:07:12.011 INFO:tasks.workunit.client.0.smithi137.stdout:1/736: dwrite d0/d3/d1b/d4c/d59/d93/fd5 [0,4194304] 0 2024-06-15T01:07:12.015 INFO:tasks.workunit.client.1.smithi142.stdout:1/953: dwrite d6/d12/d80/dc8/f11d [0,4194304] 0 2024-06-15T01:07:12.015 INFO:tasks.workunit.client.1.smithi142.stdout:9/846: truncate d6/d34/d8a/fbf 320461 0 2024-06-15T01:07:12.015 INFO:tasks.workunit.client.1.smithi142.stdout:4/972: unlink d5/d9/d18/f80 0 2024-06-15T01:07:12.018 INFO:tasks.workunit.client.1.smithi142.stdout:6/901: link d1a/d3c/db0/ce9 d1a/d39/d3a/d51/d81/c135 0 2024-06-15T01:07:12.020 INFO:tasks.workunit.client.1.smithi142.stdout:2/884: creat d8/df/d10/d94/d25/d1a/d63/f116 x:0 0 0 2024-06-15T01:07:12.020 INFO:tasks.workunit.client.1.smithi142.stdout:1/954: mkdir d6/d12/da0/d152 0 2024-06-15T01:07:12.021 INFO:tasks.workunit.client.0.smithi137.stdout:1/737: dread d0/d10/fb5 [0,4194304] 0 2024-06-15T01:07:12.025 INFO:tasks.workunit.client.1.smithi142.stdout:9/847: truncate d6/d34/ddd/fa2 3355044 0 2024-06-15T01:07:12.025 INFO:tasks.workunit.client.0.smithi137.stdout:1/738: mkdir d0/d3/d1b/d4c/d59/d93/df1/df6 0 2024-06-15T01:07:12.027 INFO:tasks.workunit.client.1.smithi142.stdout:1/955: chown d6/d12/d1c/d2c/d93/dba/feb 115070 1 2024-06-15T01:07:12.028 INFO:tasks.workunit.client.1.smithi142.stdout:2/885: write d8/df/d10/d94/d25/ff7 [4687137,60722] 0 2024-06-15T01:07:12.029 INFO:tasks.workunit.client.0.smithi137.stdout:1/739: getdents d0/d1f/d60/d99/de3/dea 0 2024-06-15T01:07:12.030 INFO:tasks.workunit.client.0.smithi137.stdout:1/740: dread - d0/d3/d89/fbc zero size 2024-06-15T01:07:12.030 INFO:tasks.workunit.client.1.smithi142.stdout:4/973: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47/f14f [0,4194304] 0 2024-06-15T01:07:12.031 INFO:tasks.workunit.client.0.smithi137.stdout:1/741: chown d0/d3/d1b/d4c/d59/d93/lf2 1 1 2024-06-15T01:07:12.032 INFO:tasks.workunit.client.0.smithi137.stdout:1/742: write d0/d1f/d60/f55 [4618426,28990] 0 2024-06-15T01:07:12.033 INFO:tasks.workunit.client.1.smithi142.stdout:1/956: dread - d6/d12/fa1 zero size 2024-06-15T01:07:12.035 INFO:tasks.workunit.client.0.smithi137.stdout:1/743: mknod d0/d3/d1b/d4c/d59/d64/cf7 0 2024-06-15T01:07:12.036 INFO:tasks.workunit.client.1.smithi142.stdout:8/947: dwrite d0/d25/d5a/f10f [0,4194304] 0 2024-06-15T01:07:12.037 INFO:tasks.workunit.client.0.smithi137.stdout:9/488: dread fd [0,4194304] 0 2024-06-15T01:07:12.039 INFO:tasks.workunit.client.1.smithi142.stdout:9/848: getdents d6/dc/dd 0 2024-06-15T01:07:12.039 INFO:tasks.workunit.client.1.smithi142.stdout:1/957: mknod d6/d12/d3c/c153 0 2024-06-15T01:07:12.041 INFO:tasks.workunit.client.1.smithi142.stdout:8/948: write d0/d25/d5a/d8f/d98/da4/ff6 [564427,36814] 0 2024-06-15T01:07:12.041 INFO:tasks.workunit.client.1.smithi142.stdout:2/886: dwrite d8/df/d1f/f3e [0,4194304] 0 2024-06-15T01:07:12.042 INFO:tasks.workunit.client.0.smithi137.stdout:1/744: rename d0/d3/d14/f38 to d0/d3/d1b/d4c/ff8 0 2024-06-15T01:07:12.043 INFO:tasks.workunit.client.0.smithi137.stdout:9/489: dread f3 [0,4194304] 0 2024-06-15T01:07:12.044 INFO:tasks.workunit.client.0.smithi137.stdout:9/490: fsync df/d12/d15/f5f 0 2024-06-15T01:07:12.049 INFO:tasks.workunit.client.0.smithi137.stdout:1/745: link d0/d3/d14/ce8 d0/d10/d41/d7b/cf9 0 2024-06-15T01:07:12.049 INFO:tasks.workunit.client.0.smithi137.stdout:1/746: chown d0/d1f/f4d 5 1 2024-06-15T01:07:12.050 INFO:tasks.workunit.client.0.smithi137.stdout:1/747: chown d0/d3/f13 210 1 2024-06-15T01:07:12.051 INFO:tasks.workunit.client.0.smithi137.stdout:9/491: getdents df/d49/d4b 0 2024-06-15T01:07:12.051 INFO:tasks.workunit.client.1.smithi142.stdout:2/887: rmdir d8/df/d10/d94/d25/d1a/d63/d107 39 2024-06-15T01:07:12.054 INFO:tasks.workunit.client.1.smithi142.stdout:4/974: rename d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/da6 to d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/d154 0 2024-06-15T01:07:12.054 INFO:tasks.workunit.client.0.smithi137.stdout:1/748: rename d0/d10/c78 to d0/d3/d1b/d4c/d59/cfa 0 2024-06-15T01:07:12.055 INFO:tasks.workunit.client.1.smithi142.stdout:1/958: dwrite d6/d12/d80/dae/f108 [8388608,4194304] 0 2024-06-15T01:07:12.055 INFO:tasks.workunit.client.1.smithi142.stdout:8/949: dread d0/d17/d40/d6e/fa9 [0,4194304] 0 2024-06-15T01:07:12.055 INFO:tasks.workunit.client.0.smithi137.stdout:9/492: truncate df/d12/d29/d4a/f5d 318628 0 2024-06-15T01:07:12.056 INFO:tasks.workunit.client.1.smithi142.stdout:8/950: dread - d0/d25/db9/ff0 zero size 2024-06-15T01:07:12.056 INFO:tasks.workunit.client.0.smithi137.stdout:1/749: truncate d0/d3/d47/d73/fc7 68380 0 2024-06-15T01:07:12.059 INFO:tasks.workunit.client.1.smithi142.stdout:0/889: dread d3/d39/d3e/d81/d1b/f30 [0,4194304] 0 2024-06-15T01:07:12.060 INFO:tasks.workunit.client.1.smithi142.stdout:0/890: readlink d3/d39/d3e/d81/d1b/d38/da3/lab 0 2024-06-15T01:07:12.061 INFO:tasks.workunit.client.0.smithi137.stdout:1/750: unlink d0/d1f/d67/l6b 0 2024-06-15T01:07:12.063 INFO:tasks.workunit.client.1.smithi142.stdout:2/888: creat d8/df/d10/d1c/f117 x:0 0 0 2024-06-15T01:07:12.064 INFO:tasks.workunit.client.0.smithi137.stdout:1/751: write d0/d3/d14/da0/fe9 [720270,57489] 0 2024-06-15T01:07:12.064 INFO:tasks.workunit.client.1.smithi142.stdout:2/889: fsync d8/df/d10/d1c/dbb/ff9 0 2024-06-15T01:07:12.064 INFO:tasks.workunit.client.1.smithi142.stdout:3/933: sync 2024-06-15T01:07:12.065 INFO:tasks.workunit.client.0.smithi137.stdout:9/493: dwrite df/d12/d4e/f5c [0,4194304] 0 2024-06-15T01:07:12.067 INFO:tasks.workunit.client.1.smithi142.stdout:8/951: link d0/d25/d53/fbb d0/d17/d40/d5e/d116/f11e 0 2024-06-15T01:07:12.068 INFO:tasks.workunit.client.1.smithi142.stdout:0/891: creat d3/d39/d3e/d81/d1b/d52/f12f x:0 0 0 2024-06-15T01:07:12.069 INFO:tasks.workunit.client.0.smithi137.stdout:1/752: symlink d0/d2d/d3e/lfb 0 2024-06-15T01:07:12.072 INFO:tasks.workunit.client.0.smithi137.stdout:9/494: getdents df/d12 0 2024-06-15T01:07:12.073 INFO:tasks.workunit.client.0.smithi137.stdout:1/753: creat d0/d3/d1b/d4c/d59/d93/df1/ffc x:0 0 0 2024-06-15T01:07:12.073 INFO:tasks.workunit.client.0.smithi137.stdout:9/495: chown df/d2a/c60 9942140 1 2024-06-15T01:07:12.073 INFO:tasks.workunit.client.1.smithi142.stdout:3/934: creat d0/d3/d15/f117 x:0 0 0 2024-06-15T01:07:12.073 INFO:tasks.workunit.client.1.smithi142.stdout:6/902: truncate d1a/d3c/db0/f104 4023420 0 2024-06-15T01:07:12.074 INFO:tasks.workunit.client.1.smithi142.stdout:2/890: creat d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf/f118 x:0 0 0 2024-06-15T01:07:12.075 INFO:tasks.workunit.client.0.smithi137.stdout:1/754: symlink d0/d3/d47/d73/lfd 0 2024-06-15T01:07:12.076 INFO:tasks.workunit.client.0.smithi137.stdout:1/755: write d0/d3/d47/f5d [647048,89655] 0 2024-06-15T01:07:12.076 INFO:tasks.workunit.client.1.smithi142.stdout:8/952: truncate d0/d17/d40/d6e/f93 805567 0 2024-06-15T01:07:12.078 INFO:tasks.workunit.client.0.smithi137.stdout:1/756: rmdir d0/d1f/d60/d99 39 2024-06-15T01:07:12.080 INFO:tasks.workunit.client.1.smithi142.stdout:6/903: rename fc to d1a/d39/d3a/d51/d81/d122/f136 0 2024-06-15T01:07:12.080 INFO:tasks.workunit.client.0.smithi137.stdout:1/757: write d0/d3/f22 [3224778,35278] 0 2024-06-15T01:07:12.081 INFO:tasks.workunit.client.1.smithi142.stdout:4/975: getdents d5/d9/d23 0 2024-06-15T01:07:12.082 INFO:tasks.workunit.client.1.smithi142.stdout:2/891: fdatasync d8/df/d10/d94/d25/f18 0 2024-06-15T01:07:12.084 INFO:tasks.workunit.client.0.smithi137.stdout:1/758: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:07:12.084 INFO:tasks.workunit.client.1.smithi142.stdout:8/953: mkdir d0/d25/d5a/d8f/d97/ded/d11f 0 2024-06-15T01:07:12.086 INFO:tasks.workunit.client.0.smithi137.stdout:1/759: dread d0/d3/d47/d73/fc7 [0,4194304] 0 2024-06-15T01:07:12.087 INFO:tasks.workunit.client.1.smithi142.stdout:6/904: creat d1a/d3c/db0/dd8/f137 x:0 0 0 2024-06-15T01:07:12.088 INFO:tasks.workunit.client.1.smithi142.stdout:9/849: dwrite d6/dc/d1b/d25/d58/de2/fe8 [0,4194304] 0 2024-06-15T01:07:12.091 INFO:tasks.workunit.client.0.smithi137.stdout:1/760: creat d0/d1f/d60/d99/de3/dea/ffe x:0 0 0 2024-06-15T01:07:12.092 INFO:tasks.workunit.client.0.smithi137.stdout:1/761: write d0/d2d/f79 [1628833,52471] 0 2024-06-15T01:07:12.094 INFO:tasks.workunit.client.1.smithi142.stdout:4/976: rename d5/d9/d23/l96 to d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/l155 0 2024-06-15T01:07:12.095 INFO:tasks.workunit.client.0.smithi137.stdout:1/762: mkdir d0/d2d/d3e/dff 0 2024-06-15T01:07:12.096 INFO:tasks.workunit.client.1.smithi142.stdout:2/892: truncate d8/df/d10/f35 303588 0 2024-06-15T01:07:12.096 INFO:tasks.workunit.client.0.smithi137.stdout:1/763: readlink d0/d10/l4a 0 2024-06-15T01:07:12.097 INFO:tasks.workunit.client.1.smithi142.stdout:2/893: readlink d8/df/d10/d94/l106 0 2024-06-15T01:07:12.097 INFO:tasks.workunit.client.1.smithi142.stdout:2/894: chown f4 9222503 1 2024-06-15T01:07:12.097 INFO:tasks.workunit.client.0.smithi137.stdout:1/764: fsync d0/d1f/d76/fda 0 2024-06-15T01:07:12.097 INFO:tasks.workunit.client.1.smithi142.stdout:6/905: truncate d1a/d39/ded/f108 609508 0 2024-06-15T01:07:12.100 INFO:tasks.workunit.client.0.smithi137.stdout:1/765: dread d0/d3/fd6 [0,4194304] 0 2024-06-15T01:07:12.101 INFO:tasks.workunit.client.1.smithi142.stdout:8/954: mkdir d0/d25/d5a/d8f/d97/d120 0 2024-06-15T01:07:12.101 INFO:tasks.workunit.client.0.smithi137.stdout:1/766: write d0/d3/d1b/f9d [1414200,104887] 0 2024-06-15T01:07:12.102 INFO:tasks.workunit.client.1.smithi142.stdout:4/977: mkdir d5/d9/da/d92/d98/dc1/d156 0 2024-06-15T01:07:12.103 INFO:tasks.workunit.client.0.smithi137.stdout:1/767: symlink d0/d1f/d67/l100 0 2024-06-15T01:07:12.105 INFO:tasks.workunit.client.0.smithi137.stdout:1/768: write d0/d3/d47/dd7/fde [57531,101548] 0 2024-06-15T01:07:12.105 INFO:tasks.workunit.client.1.smithi142.stdout:6/906: rmdir d1a/d39/ded 39 2024-06-15T01:07:12.107 INFO:tasks.workunit.client.1.smithi142.stdout:2/895: symlink d8/df/d10/d94/d25/l119 0 2024-06-15T01:07:12.108 INFO:tasks.workunit.client.0.smithi137.stdout:1/769: creat d0/d2d/d3e/f101 x:0 0 0 2024-06-15T01:07:12.108 INFO:tasks.workunit.client.1.smithi142.stdout:2/896: chown d8/d19/d8d/f93 39495 1 2024-06-15T01:07:12.111 INFO:tasks.workunit.client.1.smithi142.stdout:8/955: mknod d0/d25/d53/da7/c121 0 2024-06-15T01:07:12.114 INFO:tasks.workunit.client.0.smithi137.stdout:1/770: rename d0/d3/d47/d73/dc5 to d0/d3/d1b/d4c/d59/d93/df1/df6/d102 0 2024-06-15T01:07:12.114 INFO:tasks.workunit.client.1.smithi142.stdout:8/956: fsync d0/d25/d5a/d8f/d98/ff2 0 2024-06-15T01:07:12.114 INFO:tasks.workunit.client.1.smithi142.stdout:3/935: write d0/d1f/d20/d59/d61/fb7 [1547207,98555] 0 2024-06-15T01:07:12.114 INFO:tasks.workunit.client.1.smithi142.stdout:3/936: readlink d0/d1b/l50 0 2024-06-15T01:07:12.115 INFO:tasks.workunit.client.1.smithi142.stdout:3/937: readlink d0/d3/lac 0 2024-06-15T01:07:12.116 INFO:tasks.workunit.client.0.smithi137.stdout:1/771: creat d0/d10/d41/f103 x:0 0 0 2024-06-15T01:07:12.118 INFO:tasks.workunit.client.0.smithi137.stdout:1/772: read d0/d3/d1b/d4c/f5c [4130196,28686] 0 2024-06-15T01:07:12.119 INFO:tasks.workunit.client.0.smithi137.stdout:1/773: write d0/d10/f23 [2519526,118997] 0 2024-06-15T01:07:12.124 INFO:tasks.workunit.client.1.smithi142.stdout:8/957: mknod d0/d25/d5a/d8f/d97/c122 0 2024-06-15T01:07:12.124 INFO:tasks.workunit.client.1.smithi142.stdout:2/897: dread d8/df/d10/f87 [0,4194304] 0 2024-06-15T01:07:12.126 INFO:tasks.workunit.client.1.smithi142.stdout:4/978: getdents d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d20/d154/db0 0 2024-06-15T01:07:12.127 INFO:tasks.workunit.client.0.smithi137.stdout:1/774: dwrite d0/d3/d14/da0/fe9 [0,4194304] 0 2024-06-15T01:07:12.128 INFO:tasks.workunit.client.1.smithi142.stdout:2/898: rmdir d8/df/d10/d1c/d7c 39 2024-06-15T01:07:12.130 INFO:tasks.workunit.client.1.smithi142.stdout:9/850: truncate d6/d31/f32 1036664 0 2024-06-15T01:07:12.130 INFO:tasks.workunit.client.0.smithi137.stdout:1/775: mkdir d0/d3/d1b/d4c/d59/d93/d104 0 2024-06-15T01:07:12.132 INFO:tasks.workunit.client.1.smithi142.stdout:4/979: truncate d5/fd1 1477413 0 2024-06-15T01:07:12.132 INFO:tasks.workunit.client.1.smithi142.stdout:6/907: write d1a/d39/d3a/d51/f63 [4980393,34982] 0 2024-06-15T01:07:12.135 INFO:tasks.workunit.client.1.smithi142.stdout:8/958: mknod d0/d25/d53/da7/dcb/dfe/c123 0 2024-06-15T01:07:12.135 INFO:tasks.workunit.client.1.smithi142.stdout:2/899: rmdir d8/df/d10/d94/d25/d1a/d63/d107 39 2024-06-15T01:07:12.136 INFO:tasks.workunit.client.1.smithi142.stdout:5/911: sync 2024-06-15T01:07:12.136 INFO:tasks.workunit.client.1.smithi142.stdout:7/928: sync 2024-06-15T01:07:12.138 INFO:tasks.workunit.client.1.smithi142.stdout:8/959: mknod d0/d17/d40/d5e/c124 0 2024-06-15T01:07:12.139 INFO:tasks.workunit.client.1.smithi142.stdout:3/938: truncate d0/d1b/f91 1580220 0 2024-06-15T01:07:12.139 INFO:tasks.workunit.client.1.smithi142.stdout:6/908: mknod d1a/d83/c138 0 2024-06-15T01:07:12.142 INFO:tasks.workunit.client.1.smithi142.stdout:7/929: mknod d2/d8/da/d33/d25/d81/c12c 0 2024-06-15T01:07:12.142 INFO:tasks.workunit.client.1.smithi142.stdout:9/851: dread d6/dc/dd/d22/f54 [0,4194304] 0 2024-06-15T01:07:12.143 INFO:tasks.workunit.client.1.smithi142.stdout:4/980: dwrite d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f60 [0,4194304] 0 2024-06-15T01:07:12.145 INFO:tasks.workunit.client.1.smithi142.stdout:3/939: mknod d0/d3/da/d18/c118 0 2024-06-15T01:07:12.146 INFO:tasks.workunit.client.1.smithi142.stdout:5/912: symlink d2/d3e/d31/l118 0 2024-06-15T01:07:12.147 INFO:tasks.workunit.client.0.smithi137.stdout:1/776: truncate d0/d2d/d3e/f96 63307 0 2024-06-15T01:07:12.149 INFO:tasks.workunit.client.0.smithi137.stdout:1/777: write d0/d10/fb5 [1073727,1385] 0 2024-06-15T01:07:12.150 INFO:tasks.workunit.client.1.smithi142.stdout:8/960: rename d0/d25/d53/d9c to d0/d17/d40/d86/d125 0 2024-06-15T01:07:12.151 INFO:tasks.workunit.client.1.smithi142.stdout:7/930: dwrite d2/d8/da/d33/d25/d79/f95 [0,4194304] 0 2024-06-15T01:07:12.152 INFO:tasks.workunit.client.0.smithi137.stdout:1/778: creat d0/d1f/d76/f105 x:0 0 0 2024-06-15T01:07:12.152 INFO:tasks.workunit.client.1.smithi142.stdout:8/961: write d0/d25/d5a/d84/fcc [2119566,86299] 0 2024-06-15T01:07:12.154 INFO:tasks.workunit.client.0.smithi137.stdout:1/779: truncate d0/d3/d1b/f9d 2280086 0 2024-06-15T01:07:12.154 INFO:tasks.workunit.client.1.smithi142.stdout:7/931: write d2/d8/da/d33/d25/f59 [1712096,6480] 0 2024-06-15T01:07:12.156 INFO:tasks.workunit.client.1.smithi142.stdout:8/962: fsync d0/d25/d53/de2/f10c 0 2024-06-15T01:07:12.156 INFO:tasks.workunit.client.1.smithi142.stdout:7/932: readlink d2/d6/d22/d3c/l6f 0 2024-06-15T01:07:12.157 INFO:tasks.workunit.client.1.smithi142.stdout:2/900: dread d8/df/d10/f3b [0,4194304] 0 2024-06-15T01:07:12.157 INFO:tasks.workunit.client.1.smithi142.stdout:9/852: unlink d6/dc/d1b/d25/d58/d74/lc8 0 2024-06-15T01:07:12.157 INFO:tasks.workunit.client.0.smithi137.stdout:1/780: creat d0/d3/d1b/d4c/d59/f106 x:0 0 0 2024-06-15T01:07:12.161 INFO:tasks.workunit.client.0.smithi137.stdout:1/781: read d0/d10/d41/fa7 [3570648,98819] 0 2024-06-15T01:07:12.161 INFO:tasks.workunit.client.1.smithi142.stdout:2/901: write d8/df/d10/d94/d25/d22/d43/d51/dd7/fe4 [719538,852] 0 2024-06-15T01:07:12.162 INFO:tasks.workunit.client.0.smithi137.stdout:1/782: dread - d0/d3/d1b/d4c/d59/d93/df1/ffc zero size 2024-06-15T01:07:12.163 INFO:tasks.workunit.client.1.smithi142.stdout:2/902: write d8/d19/fc8 [1409823,90345] 0 2024-06-15T01:07:12.163 INFO:tasks.workunit.client.1.smithi142.stdout:6/909: write d1a/f46 [924562,453] 0 2024-06-15T01:07:12.163 INFO:tasks.workunit.client.1.smithi142.stdout:1/959: sync 2024-06-15T01:07:12.164 INFO:tasks.workunit.client.1.smithi142.stdout:0/892: sync 2024-06-15T01:07:12.165 INFO:tasks.workunit.client.1.smithi142.stdout:8/963: mknod d0/d17/d40/d86/d9d/c126 0 2024-06-15T01:07:12.166 INFO:tasks.workunit.client.0.smithi137.stdout:1/783: unlink d0/d3/d1b/d4c/d59/f71 0 2024-06-15T01:07:12.167 INFO:tasks.workunit.client.1.smithi142.stdout:5/913: creat d2/d18/d79/da2/f119 x:0 0 0 2024-06-15T01:07:12.168 INFO:tasks.workunit.client.0.smithi137.stdout:1/784: mknod d0/d2d/d3e/dff/c107 0 2024-06-15T01:07:12.172 INFO:tasks.workunit.client.1.smithi142.stdout:7/933: unlink d2/d8/da/d33/d25/d79/f92 0 2024-06-15T01:07:12.172 INFO:tasks.workunit.client.0.smithi137.stdout:1/785: getdents d0/d3/d1b 0 2024-06-15T01:07:12.173 INFO:tasks.workunit.client.0.smithi137.stdout:1/786: chown d0/d1f/d67/ccf 130201 1 2024-06-15T01:07:12.176 INFO:tasks.workunit.client.0.smithi137.stdout:1/787: mkdir d0/d3/d1b/d4c/d59/d93/df1/d108 0 2024-06-15T01:07:12.178 INFO:tasks.workunit.client.0.smithi137.stdout:1/788: write d0/d3/d1b/d4c/d59/d64/f6e [1414482,269] 0 2024-06-15T01:07:12.179 INFO:tasks.workunit.client.1.smithi142.stdout:3/940: write d0/f2c [3022508,30113] 0 2024-06-15T01:07:12.179 INFO:tasks.workunit.client.1.smithi142.stdout:8/964: dread d0/f8e [0,4194304] 0 2024-06-15T01:07:12.182 INFO:tasks.workunit.client.1.smithi142.stdout:9/853: truncate d6/dc/d1b/d25/d58/fd6 6524332 0 2024-06-15T01:07:12.182 INFO:tasks.workunit.client.1.smithi142.stdout:4/981: write d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d15/f21 [1598701,116896] 0 2024-06-15T01:07:12.183 INFO:tasks.workunit.client.1.smithi142.stdout:5/914: creat d2/de/d19/d5f/d68/d76/da6/f11a x:0 0 0 2024-06-15T01:07:12.185 INFO:tasks.workunit.client.1.smithi142.stdout:7/934: chown d2/d6/d22/lab 776 1 2024-06-15T01:07:12.190 INFO:tasks.workunit.client.1.smithi142.stdout:2/903: symlink d8/df/d10/d94/d25/d22/d43/d52/dc1/l11a 0 2024-06-15T01:07:12.191 INFO:tasks.workunit.client.1.smithi142.stdout:1/960: write d6/d12/f3e [1500762,15113] 0 2024-06-15T01:07:12.192 INFO:tasks.workunit.client.1.smithi142.stdout:1/961: chown d6/fc 66028 1 2024-06-15T01:07:12.193 INFO:tasks.workunit.client.1.smithi142.stdout:0/893: dwrite d3/d39/d9a/d9d/dcd/fcf [0,4194304] 0 2024-06-15T01:07:12.194 INFO:tasks.workunit.client.1.smithi142.stdout:9/854: symlink d6/dc/d1b/d6f/d103/df2/l139 0 2024-06-15T01:07:12.198 INFO:tasks.workunit.client.1.smithi142.stdout:1/962: readlink d6/d12/d1c/d2c/d93/l117 0 2024-06-15T01:07:12.198 INFO:tasks.workunit.client.1.smithi142.stdout:0/894: truncate d3/d39/d3e/fc5 1372598 0 2024-06-15T01:07:12.200 INFO:tasks.workunit.client.1.smithi142.stdout:4/982: rmdir d5/d9/d1a/dee/d146 0 2024-06-15T01:07:12.203 INFO:tasks.workunit.client.1.smithi142.stdout:7/935: truncate d2/d8/dd/d14/d2c/d37/d6b/f96 2655604 0 2024-06-15T01:07:12.203 INFO:tasks.workunit.client.1.smithi142.stdout:6/910: dwrite d1a/d39/d3a/d51/d62/fd1 [0,4194304] 0 2024-06-15T01:07:12.204 INFO:tasks.workunit.client.1.smithi142.stdout:1/963: mkdir d6/d12/d3c/d63/d8d/d97/dd3/d106/d154 0 2024-06-15T01:07:12.205 INFO:tasks.workunit.client.1.smithi142.stdout:7/936: fdatasync d2/d6/d22/d47/f7a 0 2024-06-15T01:07:12.205 INFO:tasks.workunit.client.1.smithi142.stdout:7/937: readlink d2/d6/lb4 0 2024-06-15T01:07:12.210 INFO:tasks.workunit.client.1.smithi142.stdout:1/964: dread d6/d25/dfc/d132/db2/dc2/f114 [0,4194304] 0 2024-06-15T01:07:12.212 INFO:tasks.workunit.client.1.smithi142.stdout:5/915: dread d2/d3e/d31/d7e/fa8 [0,4194304] 0 2024-06-15T01:07:12.213 INFO:tasks.workunit.client.1.smithi142.stdout:5/916: chown d2/de/d19/d5f/d68/c89 2 1 2024-06-15T01:07:12.213 INFO:tasks.workunit.client.1.smithi142.stdout:5/917: chown d2/d18/d79/fce 30231554 1 2024-06-15T01:07:12.218 INFO:tasks.workunit.client.1.smithi142.stdout:9/855: dread d6/f21 [0,4194304] 0 2024-06-15T01:07:12.219 INFO:tasks.workunit.client.1.smithi142.stdout:9/856: fdatasync d6/dc/d4e/f94 0 2024-06-15T01:07:12.306 INFO:tasks.workunit.client.1.smithi142.stdout:8/965: sync 2024-06-15T01:07:12.306 INFO:tasks.workunit.client.1.smithi142.stdout:3/941: sync 2024-06-15T01:07:12.306 INFO:tasks.workunit.client.1.smithi142.stdout:2/904: sync 2024-06-15T01:07:12.306 INFO:tasks.workunit.client.1.smithi142.stdout:8/966: dread - d0/d25/d5a/f8c zero size 2024-06-15T01:07:12.311 INFO:tasks.workunit.client.1.smithi142.stdout:3/942: dwrite d0/d3/d15/f117 [0,4194304] 0 2024-06-15T01:07:12.311 INFO:tasks.workunit.client.1.smithi142.stdout:8/967: dwrite d0/f87 [0,4194304] 0 2024-06-15T01:07:12.312 INFO:tasks.workunit.client.1.smithi142.stdout:3/943: readlink d0/d3/d15/l52 0 2024-06-15T01:07:12.320 INFO:tasks.workunit.client.1.smithi142.stdout:8/968: dwrite d0/d25/d5a/f8d [0,4194304] 0 2024-06-15T01:07:12.321 INFO:tasks.workunit.client.1.smithi142.stdout:7/938: sync 2024-06-15T01:07:12.322 INFO:tasks.workunit.client.1.smithi142.stdout:7/939: chown d2/d8/da/d33/d25/d81/df3 3354808 1 2024-06-15T01:07:12.323 INFO:tasks.workunit.client.1.smithi142.stdout:7/940: write d2/d8/f42 [5652474,84162] 0 2024-06-15T01:07:12.346 INFO:tasks.workunit.client.1.smithi142.stdout:4/983: readlink d5/d1c/d6f/dc7/de1/l12c 0 2024-06-15T01:07:12.346 INFO:tasks.workunit.client.1.smithi142.stdout:0/895: fdatasync d3/d39/d3e/d81/d44/d7c/d22/d59/f71 0 2024-06-15T01:07:12.347 INFO:tasks.workunit.client.1.smithi142.stdout:4/984: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/fd0 480479 1 2024-06-15T01:07:12.403 INFO:tasks.workunit.client.0.smithi137.stdout:0/707: sync 2024-06-15T01:07:12.404 INFO:tasks.workunit.client.1.smithi142.stdout:5/918: mknod d2/d3e/d31/d7e/c11b 0 2024-06-15T01:07:12.413 INFO:tasks.workunit.client.1.smithi142.stdout:6/911: dwrite d1a/d39/f57 [0,4194304] 0 2024-06-15T01:07:12.415 INFO:tasks.workunit.client.0.smithi137.stdout:0/708: dwrite d1/d2/dc/d10/d14/d15/d17/f95 [0,4194304] 0 2024-06-15T01:07:12.418 INFO:tasks.workunit.client.0.smithi137.stdout:0/709: fdatasync d1/f32 0 2024-06-15T01:07:12.437 INFO:tasks.workunit.client.1.smithi142.stdout:2/905: creat d8/df/d10/d94/d25/d22/d43/d52/dc1/f11b x:0 0 0 2024-06-15T01:07:12.440 INFO:tasks.workunit.client.1.smithi142.stdout:8/969: symlink d0/d17/d40/d6e/l127 0 2024-06-15T01:07:12.444 INFO:tasks.workunit.client.1.smithi142.stdout:1/965: rename d6/d12/d1c/d2c/cd8 to d6/d12/d3c/d63/d8d/d97/dd3/d106/d154/c155 0 2024-06-15T01:07:12.445 INFO:tasks.workunit.client.0.smithi137.stdout:0/710: dread d1/d2/d13/f40 [0,4194304] 0 2024-06-15T01:07:12.446 INFO:tasks.workunit.client.1.smithi142.stdout:5/919: creat d2/d18/d79/da2/f11c x:0 0 0 2024-06-15T01:07:12.449 INFO:tasks.workunit.client.0.smithi137.stdout:0/711: rmdir d1/d2/dc/d10/d14/d52 39 2024-06-15T01:07:12.449 INFO:tasks.workunit.client.1.smithi142.stdout:6/912: symlink d1a/d2d/d80/d8e/d110/l139 0 2024-06-15T01:07:12.450 INFO:tasks.workunit.client.0.smithi137.stdout:0/712: unlink d1/d2/d13/d66/l9e 0 2024-06-15T01:07:12.451 INFO:tasks.workunit.client.0.smithi137.stdout:0/713: stat d1/d2/d13/d66/c6d 0 2024-06-15T01:07:12.451 INFO:tasks.workunit.client.1.smithi142.stdout:2/906: rmdir d8/df/d10/d94/d25/d76 39 2024-06-15T01:07:12.453 INFO:tasks.workunit.client.0.smithi137.stdout:0/714: truncate d1/d2/d13/f33 91671 0 2024-06-15T01:07:12.456 INFO:tasks.workunit.client.1.smithi142.stdout:8/970: fdatasync d0/d25/d5a/d8f/d98/da4/fa6 0 2024-06-15T01:07:12.457 INFO:tasks.workunit.client.0.smithi137.stdout:0/715: truncate d1/d2/dc/d10/f96 3060170 0 2024-06-15T01:07:12.457 INFO:tasks.workunit.client.1.smithi142.stdout:0/896: symlink d3/d39/d3e/d81/d44/d7c/l130 0 2024-06-15T01:07:12.458 INFO:tasks.workunit.client.0.smithi137.stdout:0/716: chown d1/d2/dc/d10/d22/db7 1562 1 2024-06-15T01:07:12.458 INFO:tasks.workunit.client.1.smithi142.stdout:4/985: stat d5/d9/d23/lb3 0 2024-06-15T01:07:12.460 INFO:tasks.workunit.client.1.smithi142.stdout:4/986: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/dca/cd5 48663 1 2024-06-15T01:07:12.462 INFO:tasks.workunit.client.1.smithi142.stdout:9/857: creat d6/dc/d82/f13a x:0 0 0 2024-06-15T01:07:12.463 INFO:tasks.workunit.client.1.smithi142.stdout:9/858: read - d6/dc/d82/f13a zero size 2024-06-15T01:07:12.467 INFO:tasks.workunit.client.1.smithi142.stdout:2/907: mknod d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/c11c 0 2024-06-15T01:07:12.469 INFO:tasks.workunit.client.0.smithi137.stdout:0/717: dwrite d1/f79 [0,4194304] 0 2024-06-15T01:07:12.470 INFO:tasks.workunit.client.1.smithi142.stdout:0/897: mknod d3/d39/d3e/d8f/c131 0 2024-06-15T01:07:12.471 INFO:tasks.workunit.client.0.smithi137.stdout:0/718: chown d1/d2/dc/d10/d14/d15/d17/c4d 967792 1 2024-06-15T01:07:12.471 INFO:tasks.workunit.client.1.smithi142.stdout:0/898: readlink d3/l2f 0 2024-06-15T01:07:12.474 INFO:tasks.workunit.client.1.smithi142.stdout:2/908: dwrite d8/df/d10/d94/d25/f27 [0,4194304] 0 2024-06-15T01:07:12.475 INFO:tasks.workunit.client.1.smithi142.stdout:3/944: getdents d0/d3/de2/d55 0 2024-06-15T01:07:12.476 INFO:tasks.workunit.client.1.smithi142.stdout:2/909: write d8/d19/f70 [1644226,48995] 0 2024-06-15T01:07:12.477 INFO:tasks.workunit.client.1.smithi142.stdout:1/966: creat d6/d12/d80/f156 x:0 0 0 2024-06-15T01:07:12.479 INFO:tasks.workunit.client.1.smithi142.stdout:4/987: dread d5/d1c/d6f/dc7/d30/d46/f76 [0,4194304] 0 2024-06-15T01:07:12.480 INFO:tasks.workunit.client.1.smithi142.stdout:5/920: getdents d2/d3e/d31/d105 0 2024-06-15T01:07:12.482 INFO:tasks.workunit.client.1.smithi142.stdout:4/988: dread - d5/d1c/d6f/dc7/d30/d46/d50/d51/f14b zero size 2024-06-15T01:07:12.482 INFO:tasks.workunit.client.1.smithi142.stdout:3/945: rmdir d0/d1b/d6b/dbd 39 2024-06-15T01:07:12.484 INFO:tasks.workunit.client.1.smithi142.stdout:3/946: read - d0/d3/da/db8/ff5 zero size 2024-06-15T01:07:12.485 INFO:tasks.workunit.client.1.smithi142.stdout:0/899: rename d3/d39/d9a/d9d/df8/l110 to d3/d39/d3e/d81/d44/l132 0 2024-06-15T01:07:12.486 INFO:tasks.workunit.client.1.smithi142.stdout:2/910: mkdir d8/df/d10/d94/db5/ded/db6/d105/d11d 0 2024-06-15T01:07:12.487 INFO:tasks.workunit.client.1.smithi142.stdout:2/911: write d8/d19/d33/f83 [706711,62424] 0 2024-06-15T01:07:12.487 INFO:tasks.workunit.client.1.smithi142.stdout:4/989: truncate d5/d9/d23/d32/f8f 4586754 0 2024-06-15T01:07:12.491 INFO:tasks.workunit.client.1.smithi142.stdout:5/921: link d2/f12 d2/de/d19/d5f/d68/d76/d96/dac/f11d 0 2024-06-15T01:07:12.492 INFO:tasks.workunit.client.1.smithi142.stdout:1/967: creat d6/d12/d1c/d13f/d98/f157 x:0 0 0 2024-06-15T01:07:12.494 INFO:tasks.workunit.client.1.smithi142.stdout:2/912: symlink d8/df/d10/d94/db5/ded/db6/l11e 0 2024-06-15T01:07:12.495 INFO:tasks.workunit.client.1.smithi142.stdout:4/990: creat d5/d9/da/da3/f157 x:0 0 0 2024-06-15T01:07:12.497 INFO:tasks.workunit.client.1.smithi142.stdout:4/991: chown d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d12/d47 4303086 1 2024-06-15T01:07:12.498 INFO:tasks.workunit.client.1.smithi142.stdout:4/992: chown d5/d9/fd 6693729 1 2024-06-15T01:07:12.498 INFO:tasks.workunit.client.0.smithi137.stdout:0/719: dread d1/d2/dc/d10/f96 [0,4194304] 0 2024-06-15T01:07:12.499 INFO:tasks.workunit.client.0.smithi137.stdout:0/720: chown d1/d2/f21 55668 1 2024-06-15T01:07:12.500 INFO:tasks.workunit.client.0.smithi137.stdout:0/721: chown d1/d7e/d87/da0/dc4/de4 25 1 2024-06-15T01:07:12.500 INFO:tasks.workunit.client.1.smithi142.stdout:3/947: link d0/d1f/d20/d59/f92 d0/d1f/f119 0 2024-06-15T01:07:12.501 INFO:tasks.workunit.client.1.smithi142.stdout:7/941: truncate d2/f49 3379764 0 2024-06-15T01:07:12.504 INFO:tasks.workunit.client.1.smithi142.stdout:1/968: truncate d6/d12/d3c/d63/d8d/d97/dd3/fac 1820074 0 2024-06-15T01:07:12.504 INFO:tasks.workunit.client.0.smithi137.stdout:0/722: rename d1/d2/c2e to d1/d2/dc/d6c/cef 0 2024-06-15T01:07:12.504 INFO:tasks.workunit.client.0.smithi137.stdout:0/723: chown d1/d2/dc/d10/d14/d56/cba 83313080 1 2024-06-15T01:07:12.506 INFO:tasks.workunit.client.1.smithi142.stdout:4/993: mkdir d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/d158 0 2024-06-15T01:07:12.507 INFO:tasks.workunit.client.0.smithi137.stdout:0/724: rmdir d1/d2/dc/d10/d14/d15/d17/d54/d7a 39 2024-06-15T01:07:12.508 INFO:tasks.workunit.client.1.smithi142.stdout:7/942: dwrite d2/d6/d22/d47/f7a [4194304,4194304] 0 2024-06-15T01:07:12.508 INFO:tasks.workunit.client.1.smithi142.stdout:5/922: dread d2/d3e/d98/fd0 [0,4194304] 0 2024-06-15T01:07:12.508 INFO:tasks.workunit.client.1.smithi142.stdout:2/913: mknod d8/df/d10/d94/db5/ded/db6/d105/d11d/c11f 0 2024-06-15T01:07:12.511 INFO:tasks.workunit.client.1.smithi142.stdout:1/969: rename d6/d12/d1c/d2c/d93/lf5 to d6/d12/d1c/d2c/d93/db9/df2/l158 0 2024-06-15T01:07:12.513 INFO:tasks.workunit.client.1.smithi142.stdout:2/914: chown d8/df/d10/d94/d25/d22/d43/l86 5 1 2024-06-15T01:07:12.514 INFO:tasks.workunit.client.1.smithi142.stdout:4/994: dread d5/d1c/d6f/dc7/d30/d46/d50/d51/d5b/d2d/f9d [0,4194304] 0 2024-06-15T01:07:12.514 INFO:tasks.workunit.client.1.smithi142.stdout:6/913: dwrite d1a/f61 [0,4194304] 0 2024-06-15T01:07:12.515 INFO:tasks.workunit.client.1.smithi142.stdout:3/948: creat d0/d3/f11a x:0 0 0 2024-06-15T01:07:12.516 INFO:tasks.workunit.client.1.smithi142.stdout:7/943: write d2/d6/d22/ddd/f9c [908087,10851] 0 2024-06-15T01:07:12.516 INFO:tasks.workunit.client.0.smithi137.stdout:0/725: dwrite d1/d2/dc/d10/d14/d52/fcc [0,4194304] 0 2024-06-15T01:07:12.517 INFO:tasks.workunit.client.1.smithi142.stdout:5/923: creat d2/de/d19/d5f/d68/d76/da6/f11e x:0 0 0 2024-06-15T01:07:12.517 INFO:tasks.workunit.client.1.smithi142.stdout:1/970: read d6/d12/d1c/d2c/d47/f5f [353929,43300] 0 2024-06-15T01:07:12.522 INFO:tasks.workunit.client.0.smithi137.stdout:0/726: getdents d1/d7e 0 2024-06-15T01:07:12.525 INFO:tasks.workunit.client.1.smithi142.stdout:8/971: dwrite d0/d17/d40/d86/fc6 [0,4194304] 0 2024-06-15T01:07:12.527 INFO:tasks.workunit.client.0.smithi137.stdout:0/727: mknod d1/d2/dc/d10/d14/d15/d17/d54/cf0 0 2024-06-15T01:07:12.530 INFO:tasks.workunit.client.1.smithi142.stdout:0/900: write d3/d39/d3e/d81/d1e/f10e [190339,128106] 0 2024-06-15T01:07:12.530 INFO:tasks.workunit.client.1.smithi142.stdout:4/995: fdatasync d5/d9/da/d92/d98/dc1/d133/f142 0 2024-06-15T01:07:12.532 INFO:tasks.workunit.client.1.smithi142.stdout:5/924: mkdir d2/de/d27/d11f 0 2024-06-15T01:07:12.532 INFO:tasks.workunit.client.1.smithi142.stdout:4/996: dread - d5/d9/da/da3/f157 zero size 2024-06-15T01:07:12.532 INFO:tasks.workunit.client.1.smithi142.stdout:0/901: chown d3/d39/d9a/d9d/de6/cf5 7708 1 2024-06-15T01:07:12.533 INFO:tasks.workunit.client.1.smithi142.stdout:1/971: rmdir d6/d12/da0 39 2024-06-15T01:07:12.534 INFO:tasks.workunit.client.1.smithi142.stdout:2/915: rmdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98 39 2024-06-15T01:07:12.535 INFO:tasks.workunit.client.1.smithi142.stdout:6/914: dwrite d1a/d3c/d100/fb3 [0,4194304] 0 2024-06-15T01:07:12.535 INFO:tasks.workunit.client.1.smithi142.stdout:1/972: stat d6/d25/dfc/fd9 0 2024-06-15T01:07:12.536 INFO:tasks.workunit.client.1.smithi142.stdout:0/902: read d3/d39/d3e/d81/d4c/da0/fb2 [3540381,22624] 0 2024-06-15T01:07:12.537 INFO:tasks.workunit.client.1.smithi142.stdout:8/972: chown d0/d17/d40/d5e/cde 3 1 2024-06-15T01:07:12.537 INFO:tasks.workunit.client.0.smithi137.stdout:0/728: dwrite d1/d2/dc/d6c/d90/fc6 [0,4194304] 0 2024-06-15T01:07:12.537 INFO:tasks.workunit.client.1.smithi142.stdout:7/944: mknod d2/d6/d22/c12d 0 2024-06-15T01:07:12.538 INFO:tasks.workunit.client.1.smithi142.stdout:4/997: unlink d5/d9/d18/l25 0 2024-06-15T01:07:12.541 INFO:tasks.workunit.client.1.smithi142.stdout:5/925: symlink d2/de/d19/d5f/d68/l120 0 2024-06-15T01:07:12.543 INFO:tasks.workunit.client.1.smithi142.stdout:2/916: unlink d8/df/d10/d94/l106 0 2024-06-15T01:07:12.544 INFO:tasks.workunit.client.0.smithi137.stdout:0/729: creat d1/d2/dc/d10/d14/d15/d17/d54/d7a/dd0/ff1 x:0 0 0 2024-06-15T01:07:12.546 INFO:tasks.workunit.client.1.smithi142.stdout:6/915: creat d1a/d2d/d3b/dab/dce/f13a x:0 0 0 2024-06-15T01:07:12.547 INFO:tasks.workunit.client.1.smithi142.stdout:1/973: dread d6/d25/dfc/f5d [0,4194304] 0 2024-06-15T01:07:12.547 INFO:tasks.workunit.client.0.smithi137.stdout:0/730: mknod d1/d2/dc/d6c/d88/d91/cf2 0 2024-06-15T01:07:12.548 INFO:tasks.workunit.client.1.smithi142.stdout:9/859: sync 2024-06-15T01:07:12.548 INFO:tasks.workunit.client.0.smithi137.stdout:0/731: stat d1/d2/dc/d10/d53/da2/fa5 0 2024-06-15T01:07:12.550 INFO:tasks.workunit.client.1.smithi142.stdout:1/974: write d6/d12/d80/dc8/fda [239311,129506] 0 2024-06-15T01:07:12.551 INFO:tasks.workunit.client.0.smithi137.stdout:0/732: write d1/d2/dc/d10/f75 [4972229,41428] 0 2024-06-15T01:07:12.552 INFO:tasks.workunit.client.1.smithi142.stdout:1/975: dread - d6/d3a/dc4/f146 zero size 2024-06-15T01:07:12.552 INFO:tasks.workunit.client.1.smithi142.stdout:2/917: dread d8/df/d10/d94/d25/f27 [0,4194304] 0 2024-06-15T01:07:12.555 INFO:tasks.workunit.client.1.smithi142.stdout:7/945: rename d2/d6/d22/d47/de5/f100 to d2/d8/dd/d8a/d10e/f12e 0 2024-06-15T01:07:12.555 INFO:tasks.workunit.client.1.smithi142.stdout:4/998: creat d5/d9/da/d92/d98/dc1/d143/f159 x:0 0 0 2024-06-15T01:07:12.556 INFO:tasks.workunit.client.0.smithi137.stdout:0/733: dread d1/d2/dc/d6c/d90/fc6 [0,4194304] 0 2024-06-15T01:07:12.557 INFO:tasks.workunit.client.1.smithi142.stdout:4/999: write d5/d9/d23/d32/ffa [484883,33162] 0 2024-06-15T01:07:12.560 INFO:tasks.workunit.client.0.smithi137.stdout:0/734: unlink d1/d2/dc/d10/f75 0 2024-06-15T01:07:12.561 INFO:tasks.workunit.client.0.smithi137.stdout:0/735: write d1/d2/d55/f6e [2407001,119721] 0 2024-06-15T01:07:12.562 INFO:tasks.workunit.client.0.smithi137.stdout:0/736: chown d1/d2/dc/d10/d14/d15/c36 5 1 2024-06-15T01:07:12.563 INFO:tasks.workunit.client.1.smithi142.stdout:9/860: symlink d6/dc/dd/l13b 0 2024-06-15T01:07:12.563 INFO:tasks.workunit.client.1.smithi142.stdout:3/949: write d0/d3/f90 [4744736,9267] 0 2024-06-15T01:07:12.564 INFO:tasks.workunit.client.0.smithi137.stdout:0/737: chown d1/d7e/d87/da0/dc4/dd7/de2 64 1 2024-06-15T01:07:12.564 INFO:tasks.workunit.client.1.smithi142.stdout:9/861: fdatasync d6/d34/df3/f100 0 2024-06-15T01:07:12.565 INFO:tasks.workunit.client.1.smithi142.stdout:9/862: dread - d6/dc/d1b/d25/d58/de2/f129 zero size 2024-06-15T01:07:12.568 INFO:tasks.workunit.client.1.smithi142.stdout:5/926: rename d2/f8b to d2/d3e/d65/dd6/f121 0 2024-06-15T01:07:12.569 INFO:tasks.workunit.client.0.smithi137.stdout:0/738: rename d1/d2/dc/d6c/d90/fc6 to d1/ff3 0 2024-06-15T01:07:12.571 INFO:tasks.workunit.client.1.smithi142.stdout:7/946: unlink d2/d8/dd/d14/d2c/d37/fc7 0 2024-06-15T01:07:12.572 INFO:tasks.workunit.client.1.smithi142.stdout:2/918: rmdir d8/df/d10/d94/db5/ded/db6/d105 39 2024-06-15T01:07:12.573 INFO:tasks.workunit.client.0.smithi137.stdout:0/739: mkdir d1/d7e/d87/da0/dc4/df4 0 2024-06-15T01:07:12.573 INFO:tasks.workunit.client.1.smithi142.stdout:2/919: chown d8/df/d1f 1327892373 1 2024-06-15T01:07:12.574 INFO:tasks.workunit.client.1.smithi142.stdout:5/927: dread d2/de/d27/d86/dbd/fcd [0,4194304] 0 2024-06-15T01:07:12.575 INFO:tasks.workunit.client.1.smithi142.stdout:6/916: mknod d1a/d39/c13b 0 2024-06-15T01:07:12.576 INFO:tasks.workunit.client.0.smithi137.stdout:0/740: symlink d1/d7e/d87/da0/dc4/dd7/de2/d8d/lf5 0 2024-06-15T01:07:12.576 INFO:tasks.workunit.client.1.smithi142.stdout:8/973: dread d0/d25/d53/da7/ff1 [0,4194304] 0 2024-06-15T01:07:12.578 INFO:tasks.workunit.client.0.smithi137.stdout:0/741: write d1/d2/d13/d66/f60 [968390,46878] 0 2024-06-15T01:07:12.582 INFO:tasks.workunit.client.1.smithi142.stdout:6/917: dwrite d1a/f2c [4194304,4194304] 0 2024-06-15T01:07:12.582 INFO:tasks.workunit.client.1.smithi142.stdout:7/947: rmdir d2/da2 39 2024-06-15T01:07:12.583 INFO:tasks.workunit.client.0.smithi137.stdout:0/742: dread d1/d2/dc/d10/f4f [0,4194304] 0 2024-06-15T01:07:12.585 INFO:tasks.workunit.client.0.smithi137.stdout:0/743: truncate d1/d7e/d87/da0/fcf 773095 0 2024-06-15T01:07:12.586 INFO:tasks.workunit.client.1.smithi142.stdout:5/928: read d2/de/d27/f10c [248254,128063] 0 2024-06-15T01:07:12.587 INFO:tasks.workunit.client.1.smithi142.stdout:3/950: symlink d0/d1f/d20/d5d/d82/d6e/l11b 0 2024-06-15T01:07:12.588 INFO:tasks.workunit.client.1.smithi142.stdout:6/918: dwrite d1a/d2d/d80/fee [0,4194304] 0 2024-06-15T01:07:12.593 INFO:tasks.workunit.client.1.smithi142.stdout:8/974: creat d0/d17/d40/d6e/f128 x:0 0 0 2024-06-15T01:07:12.593 INFO:tasks.workunit.client.1.smithi142.stdout:8/975: chown d0/d17/d40/db8 7258 1 2024-06-15T01:07:12.594 INFO:tasks.workunit.client.1.smithi142.stdout:8/976: dread - d0/d17/d40/d6e/fe3 zero size 2024-06-15T01:07:12.598 INFO:tasks.workunit.client.1.smithi142.stdout:0/903: truncate d3/d39/d3e/d81/d44/fd6 3728262 0 2024-06-15T01:07:12.599 INFO:tasks.workunit.client.1.smithi142.stdout:0/904: fdatasync d3/d39/d3e/d81/f103 0 2024-06-15T01:07:12.601 INFO:tasks.workunit.client.1.smithi142.stdout:5/929: rename d2/de/d27/f72 to d2/de/d19/d5f/d68/d76/d8c/dc4/f122 0 2024-06-15T01:07:12.603 INFO:tasks.workunit.client.1.smithi142.stdout:1/976: write d6/d3a/f88 [2534743,35091] 0 2024-06-15T01:07:12.604 INFO:tasks.workunit.client.1.smithi142.stdout:1/977: chown d6/d12/d1c/d13f 384960977 1 2024-06-15T01:07:12.605 INFO:tasks.workunit.client.1.smithi142.stdout:2/920: symlink d8/df/d10/d1c/d7c/l120 0 2024-06-15T01:07:12.606 INFO:tasks.workunit.client.1.smithi142.stdout:0/905: dwrite d3/d39/d3e/d81/d4c/d61/f77 [0,4194304] 0 2024-06-15T01:07:12.607 INFO:tasks.workunit.client.1.smithi142.stdout:2/921: readlink d8/df/d10/d1c/d7c/l120 0 2024-06-15T01:07:12.609 INFO:tasks.workunit.client.1.smithi142.stdout:7/948: rename d2/d6/l3b to d2/d6/l12f 0 2024-06-15T01:07:12.612 INFO:tasks.workunit.client.1.smithi142.stdout:0/906: creat d3/f133 x:0 0 0 2024-06-15T01:07:12.613 INFO:tasks.workunit.client.0.smithi137.stdout:0/744: dread d1/d2/f2f [0,4194304] 0 2024-06-15T01:07:12.615 INFO:tasks.workunit.client.1.smithi142.stdout:2/922: creat d8/df/d10/d94/d25/d22/d43/d51/f121 x:0 0 0 2024-06-15T01:07:12.616 INFO:tasks.workunit.client.0.smithi137.stdout:0/745: mkdir d1/df6 0 2024-06-15T01:07:12.617 INFO:tasks.workunit.client.1.smithi142.stdout:9/863: dwrite d6/dc/f10c [0,4194304] 0 2024-06-15T01:07:12.617 INFO:tasks.workunit.client.1.smithi142.stdout:6/919: write d1a/d2d/d3b/fa9 [955075,37666] 0 2024-06-15T01:07:12.621 INFO:tasks.workunit.client.1.smithi142.stdout:9/864: write d6/dc/dd/f91 [599296,103787] 0 2024-06-15T01:07:12.622 INFO:tasks.workunit.client.1.smithi142.stdout:7/949: dwrite d2/d6/fc5 [0,4194304] 0 2024-06-15T01:07:12.625 INFO:tasks.workunit.client.1.smithi142.stdout:9/865: chown d6/dc/d1b/le6 1528 1 2024-06-15T01:07:12.625 INFO:tasks.workunit.client.1.smithi142.stdout:5/930: creat d2/de/d19/d5f/d68/d76/f123 x:0 0 0 2024-06-15T01:07:12.625 INFO:tasks.workunit.client.1.smithi142.stdout:5/931: chown d2/de/la9 1675 1 2024-06-15T01:07:12.626 INFO:tasks.workunit.client.1.smithi142.stdout:0/907: creat d3/d39/d3e/d81/d4c/da0/f134 x:0 0 0 2024-06-15T01:07:12.629 INFO:tasks.workunit.client.1.smithi142.stdout:7/950: chown d2/d8/da/d33/d25/d79 150746143 1 2024-06-15T01:07:12.629 INFO:tasks.workunit.client.1.smithi142.stdout:1/978: rename d6/d3a/c111 to d6/d46/c159 0 2024-06-15T01:07:12.633 INFO:tasks.workunit.client.0.smithi137.stdout:0/746: dwrite d1/d7e/fae [0,4194304] 0 2024-06-15T01:07:12.633 INFO:tasks.workunit.client.1.smithi142.stdout:8/977: dread d0/f4c [0,4194304] 0 2024-06-15T01:07:12.634 INFO:tasks.workunit.client.0.smithi137.stdout:0/747: write d1/d2/d13/d66/f60 [1314001,42707] 0 2024-06-15T01:07:12.635 INFO:tasks.workunit.client.1.smithi142.stdout:3/951: dwrite d0/d1f/d20/f32 [0,4194304] 0 2024-06-15T01:07:12.638 INFO:tasks.workunit.client.0.smithi137.stdout:0/748: symlink d1/d2/dc/d6c/lf7 0 2024-06-15T01:07:12.638 INFO:tasks.workunit.client.1.smithi142.stdout:9/866: rename d6/dc/d1b/d25 to d6/dc/d1b/d6f/d103/df2/d13c 0 2024-06-15T01:07:12.640 INFO:tasks.workunit.client.0.smithi137.stdout:0/749: rmdir d1/d2/dc/d10/d14/d52 39 2024-06-15T01:07:12.641 INFO:tasks.workunit.client.1.smithi142.stdout:7/951: creat d2/d6/d22/d3c/f130 x:0 0 0 2024-06-15T01:07:12.642 INFO:tasks.workunit.client.1.smithi142.stdout:7/952: write d2/d6/fe8 [181943,121969] 0 2024-06-15T01:07:12.645 INFO:tasks.workunit.client.1.smithi142.stdout:3/952: chown d0/f66 15716138 1 2024-06-15T01:07:12.646 INFO:tasks.workunit.client.0.smithi137.stdout:0/750: rename d1/d2/dc/d10/d14/d15 to d1/d2/dc/d10/d22/db7/df8 0 2024-06-15T01:07:12.646 INFO:tasks.workunit.client.1.smithi142.stdout:6/920: link d1a/d2d/d3b/dab/dce/f13a d1a/d3c/d100/f13c 0 2024-06-15T01:07:12.648 INFO:tasks.workunit.client.1.smithi142.stdout:1/979: creat d6/d12/db5/d10c/f15a x:0 0 0 2024-06-15T01:07:12.648 INFO:tasks.workunit.client.1.smithi142.stdout:1/980: chown d6/d3a 614710 1 2024-06-15T01:07:12.650 INFO:tasks.workunit.client.1.smithi142.stdout:2/923: getdents d8/df 0 2024-06-15T01:07:12.651 INFO:tasks.workunit.client.1.smithi142.stdout:7/953: truncate d2/d8/dd/dcf/f7b 2218114 0 2024-06-15T01:07:12.651 INFO:tasks.workunit.client.0.smithi137.stdout:0/751: link d1/d2/dc/d10/d22/fc7 d1/d2/dc/d10/d14/d52/ff9 0 2024-06-15T01:07:12.652 INFO:tasks.workunit.client.1.smithi142.stdout:8/978: mkdir d0/d17/d129 0 2024-06-15T01:07:12.656 INFO:tasks.workunit.client.1.smithi142.stdout:6/921: dwrite d1a/d2d/d2e/fd9 [0,4194304] 0 2024-06-15T01:07:12.656 INFO:tasks.workunit.client.1.smithi142.stdout:1/981: mkdir d6/d12/db5/d10c/d15b 0 2024-06-15T01:07:12.657 INFO:tasks.workunit.client.1.smithi142.stdout:5/932: write d2/de/d27/db2/ffc [359915,1991] 0 2024-06-15T01:07:12.659 INFO:tasks.workunit.client.1.smithi142.stdout:2/924: mkdir d8/df/d10/d94/d25/d22/d43/d51/dd7/d122 0 2024-06-15T01:07:12.664 INFO:tasks.workunit.client.1.smithi142.stdout:1/982: fsync d6/d12/d1c/d2c/d93/dba/feb 0 2024-06-15T01:07:12.666 INFO:tasks.workunit.client.1.smithi142.stdout:0/908: dwrite d3/f28 [0,4194304] 0 2024-06-15T01:07:12.666 INFO:tasks.workunit.client.1.smithi142.stdout:3/953: link d0/d3/de2/cc4 d0/d3/de2/d55/dae/c11c 0 2024-06-15T01:07:12.668 INFO:tasks.workunit.client.1.smithi142.stdout:9/867: truncate d6/dc/dd/d22/f10f 3342378 0 2024-06-15T01:07:12.668 INFO:tasks.workunit.client.1.smithi142.stdout:3/954: readlink d0/d1f/d20/l3b 0 2024-06-15T01:07:12.669 INFO:tasks.workunit.client.1.smithi142.stdout:1/983: write d6/d12/d1c/d2c/d47/f5a [5756725,106661] 0 2024-06-15T01:07:12.675 INFO:tasks.workunit.client.1.smithi142.stdout:6/922: write d1a/d39/d3a/d51/f9b [807922,104548] 0 2024-06-15T01:07:12.676 INFO:tasks.workunit.client.1.smithi142.stdout:1/984: dread d6/d12/d3c/d63/d8d/d97/dc6/f101 [0,4194304] 0 2024-06-15T01:07:12.680 INFO:tasks.workunit.client.1.smithi142.stdout:2/925: getdents d8/df/d10/d94/d25/d22 0 2024-06-15T01:07:12.682 INFO:tasks.workunit.client.1.smithi142.stdout:0/909: rename d3/d39/d3e/d81/d44/d7c/d123/f124 to d3/d39/d3e/d81/d1b/f135 0 2024-06-15T01:07:12.682 INFO:tasks.workunit.client.1.smithi142.stdout:1/985: write d6/d25/dfc/d132/db2/dbf/fe5 [765030,1025] 0 2024-06-15T01:07:12.682 INFO:tasks.workunit.client.1.smithi142.stdout:3/955: dwrite d0/d1b/fe4 [0,4194304] 0 2024-06-15T01:07:12.684 INFO:tasks.workunit.client.1.smithi142.stdout:1/986: chown d6/d12/d1c/d2c/d47 200350 1 2024-06-15T01:07:12.686 INFO:tasks.workunit.client.1.smithi142.stdout:2/926: mkdir d8/df/daf/d123 0 2024-06-15T01:07:12.686 INFO:tasks.workunit.client.1.smithi142.stdout:3/956: chown d0/d1f/d20/l3b 485853 1 2024-06-15T01:07:12.690 INFO:tasks.workunit.client.1.smithi142.stdout:2/927: chown d8/df/d10/d94/d25/d22/lfc 258 1 2024-06-15T01:07:12.690 INFO:tasks.workunit.client.1.smithi142.stdout:6/923: dwrite d1a/d3c/d100/d58/f7f [0,4194304] 0 2024-06-15T01:07:12.692 INFO:tasks.workunit.client.1.smithi142.stdout:0/910: rmdir d3/d39/d9a/d9d/dbf/d119 0 2024-06-15T01:07:12.696 INFO:tasks.workunit.client.1.smithi142.stdout:8/979: dread d0/d17/fb2 [0,4194304] 0 2024-06-15T01:07:12.696 INFO:tasks.workunit.client.1.smithi142.stdout:3/957: symlink d0/d1f/d20/d5d/d60/l11d 0 2024-06-15T01:07:12.696 INFO:tasks.workunit.client.1.smithi142.stdout:6/924: read d1a/d3c/fc4 [3300421,120485] 0 2024-06-15T01:07:12.697 INFO:tasks.workunit.client.0.smithi137.stdout:0/752: dread d1/d2/d55/f6e [0,4194304] 0 2024-06-15T01:07:12.697 INFO:tasks.workunit.client.1.smithi142.stdout:5/933: dread d2/de/d19/d5f/d68/d76/d96/dac/fc2 [0,4194304] 0 2024-06-15T01:07:12.697 INFO:tasks.workunit.client.1.smithi142.stdout:2/928: unlink d8/c3d 0 2024-06-15T01:07:12.698 INFO:tasks.workunit.client.1.smithi142.stdout:6/925: readlink d1a/l29 0 2024-06-15T01:07:12.699 INFO:tasks.workunit.client.0.smithi137.stdout:0/753: truncate d1/d2/dc/d10/d22/db7/df8/f30 741499 0 2024-06-15T01:07:12.702 INFO:tasks.workunit.client.0.smithi137.stdout:3/883: sync 2024-06-15T01:07:12.703 INFO:tasks.workunit.client.1.smithi142.stdout:0/911: mkdir d3/d39/d3e/d81/d1b/d86/dfe/d136 0 2024-06-15T01:07:12.705 INFO:tasks.workunit.client.0.smithi137.stdout:3/884: unlink d0/d2f/faa 0 2024-06-15T01:07:12.706 INFO:tasks.workunit.client.1.smithi142.stdout:5/934: unlink d2/d18/c9f 0 2024-06-15T01:07:12.708 INFO:tasks.workunit.client.1.smithi142.stdout:6/926: creat d1a/d3c/db0/f13d x:0 0 0 2024-06-15T01:07:12.710 INFO:tasks.workunit.client.1.smithi142.stdout:0/912: creat d3/d39/d3e/d81/d4c/d61/dea/f137 x:0 0 0 2024-06-15T01:07:12.711 INFO:tasks.workunit.client.1.smithi142.stdout:5/935: mknod d2/d18/d79/c124 0 2024-06-15T01:07:12.712 INFO:tasks.workunit.client.1.smithi142.stdout:2/929: read d8/df/d10/d1c/d7c/f90 [219505,54580] 0 2024-06-15T01:07:12.714 INFO:tasks.workunit.client.1.smithi142.stdout:6/927: rename d1a/d39/d42/f74 to d1a/d39/d42/f13e 0 2024-06-15T01:07:12.714 INFO:tasks.workunit.client.1.smithi142.stdout:6/928: chown d1a/f28 3061335 1 2024-06-15T01:07:12.715 INFO:tasks.workunit.client.1.smithi142.stdout:6/929: chown d1a/d3c/d100/d58/f6a 213950714 1 2024-06-15T01:07:12.715 INFO:tasks.workunit.client.1.smithi142.stdout:5/936: mkdir d2/de/d19/d5f/d68/d125 0 2024-06-15T01:07:12.716 INFO:tasks.workunit.client.1.smithi142.stdout:6/930: fdatasync d1a/d3c/d100/fb3 0 2024-06-15T01:07:12.721 INFO:tasks.workunit.client.1.smithi142.stdout:0/913: creat d3/d39/f138 x:0 0 0 2024-06-15T01:07:12.721 INFO:tasks.workunit.client.1.smithi142.stdout:2/930: chown d8/df/d10/d94/d25/d22/d43/d52/d7a/cae 1 1 2024-06-15T01:07:12.723 INFO:tasks.workunit.client.0.smithi137.stdout:0/754: dread d1/d2/dc/d10/d14/f39 [4194304,4194304] 0 2024-06-15T01:07:12.726 INFO:tasks.workunit.client.0.smithi137.stdout:0/755: write d1/d2/dc/f73 [4191125,104521] 0 2024-06-15T01:07:12.726 INFO:tasks.workunit.client.1.smithi142.stdout:0/914: creat d3/d39/d9a/d9d/df8/f139 x:0 0 0 2024-06-15T01:07:12.727 INFO:tasks.workunit.client.0.smithi137.stdout:0/756: symlink d1/d2/dc/d6c/d88/d91/lfa 0 2024-06-15T01:07:12.727 INFO:tasks.workunit.client.1.smithi142.stdout:5/937: dread d2/d18/f63 [0,4194304] 0 2024-06-15T01:07:12.729 INFO:tasks.workunit.client.1.smithi142.stdout:6/931: dwrite d1a/d3c/f48 [0,4194304] 0 2024-06-15T01:07:12.729 INFO:tasks.workunit.client.1.smithi142.stdout:3/958: dwrite d0/d3/de2/d55/f94 [0,4194304] 0 2024-06-15T01:07:12.731 INFO:tasks.workunit.client.1.smithi142.stdout:3/959: readlink d0/d3/de2/d55/dae/ldb 0 2024-06-15T01:07:12.732 INFO:tasks.workunit.client.1.smithi142.stdout:0/915: chown d3/d39/d3e/d81/d4c/d61/fb5 230422 1 2024-06-15T01:07:12.736 INFO:tasks.workunit.client.1.smithi142.stdout:8/980: dread d0/d25/d5a/f2c [0,4194304] 0 2024-06-15T01:07:12.736 INFO:tasks.workunit.client.1.smithi142.stdout:6/932: dread - d1a/d3c/d100/f13c zero size 2024-06-15T01:07:12.737 INFO:tasks.workunit.client.1.smithi142.stdout:2/931: rename d8/df/d10/d94/d25/d22/d43/d52/dc1/fcb to d8/df/daf/d123/f124 0 2024-06-15T01:07:12.739 INFO:tasks.workunit.client.1.smithi142.stdout:2/932: write d8/df/d10/d94/d25/d1a/d63/f95 [938808,2902] 0 2024-06-15T01:07:12.740 INFO:tasks.workunit.client.1.smithi142.stdout:6/933: symlink d1a/d39/d42/df7/l13f 0 2024-06-15T01:07:12.744 INFO:tasks.workunit.client.1.smithi142.stdout:5/938: rename d2/d3e/d31/c91 to d2/de/d19/d5f/d68/d76/da6/c126 0 2024-06-15T01:07:12.748 INFO:tasks.workunit.client.0.smithi137.stdout:3/885: dread d0/d13/d37/d66/fae [0,4194304] 0 2024-06-15T01:07:12.750 INFO:tasks.workunit.client.1.smithi142.stdout:5/939: creat d2/d3e/d74/da5/dbe/d104/f127 x:0 0 0 2024-06-15T01:07:12.750 INFO:tasks.workunit.client.1.smithi142.stdout:3/960: getdents d0/d1f/d20/d59/d61/d96 0 2024-06-15T01:07:12.751 INFO:tasks.workunit.client.1.smithi142.stdout:2/933: fsync d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 0 2024-06-15T01:07:12.751 INFO:tasks.workunit.client.0.smithi137.stdout:3/886: mknod d0/d56/d8f/d7b/c121 0 2024-06-15T01:07:12.753 INFO:tasks.workunit.client.0.smithi137.stdout:3/887: write d0/d56/d8f/fcf [5170285,123851] 0 2024-06-15T01:07:12.753 INFO:tasks.workunit.client.1.smithi142.stdout:3/961: dread - d0/d1f/d20/d59/d61/d96/fde zero size 2024-06-15T01:07:12.755 INFO:tasks.workunit.client.1.smithi142.stdout:5/940: unlink d2/d18/f37 0 2024-06-15T01:07:12.756 INFO:tasks.workunit.client.1.smithi142.stdout:5/941: chown d2/de/d19/d5f/d68 36284 1 2024-06-15T01:07:12.756 INFO:tasks.workunit.client.1.smithi142.stdout:2/934: rmdir d8/df/d10/d94/d25/d22/d43/de3 39 2024-06-15T01:07:12.757 INFO:tasks.workunit.client.0.smithi137.stdout:0/757: dread d1/d2/dc/d10/d22/db7/df8/d1c/f1d [0,4194304] 0 2024-06-15T01:07:12.759 INFO:tasks.workunit.client.0.smithi137.stdout:0/758: write d1/d2/dc/d6c/d88/d91/f9a [3954024,86522] 0 2024-06-15T01:07:12.760 INFO:tasks.workunit.client.1.smithi142.stdout:5/942: creat d2/d3e/d31/d7e/f128 x:0 0 0 2024-06-15T01:07:12.761 INFO:tasks.workunit.client.0.smithi137.stdout:0/759: mkdir d1/d7e/dfb 0 2024-06-15T01:07:12.761 INFO:tasks.workunit.client.1.smithi142.stdout:2/935: mkdir d8/df/d10/d94/d25/da9/d125 0 2024-06-15T01:07:12.762 INFO:tasks.workunit.client.1.smithi142.stdout:8/981: dread d0/f14 [0,4194304] 0 2024-06-15T01:07:12.763 INFO:tasks.workunit.client.1.smithi142.stdout:7/954: sync 2024-06-15T01:07:12.763 INFO:tasks.workunit.client.1.smithi142.stdout:5/943: write d2/d3e/d31/d7e/f128 [933245,40795] 0 2024-06-15T01:07:12.764 INFO:tasks.workunit.client.1.smithi142.stdout:3/962: fdatasync d0/d3/de2/d75/fee 0 2024-06-15T01:07:12.768 INFO:tasks.workunit.client.1.smithi142.stdout:0/916: dwrite d3/d39/d3e/d81/d44/d7c/fbd [4194304,4194304] 0 2024-06-15T01:07:12.769 INFO:tasks.workunit.client.0.smithi137.stdout:3/888: write d0/d2/d23/de2/f101 [5496643,1068] 0 2024-06-15T01:07:12.769 INFO:tasks.workunit.client.1.smithi142.stdout:6/934: dread d1a/d83/f7c [0,4194304] 0 2024-06-15T01:07:12.770 INFO:tasks.workunit.client.1.smithi142.stdout:7/955: write d2/d8/da/d33/d25/f128 [388366,37771] 0 2024-06-15T01:07:12.770 INFO:tasks.workunit.client.1.smithi142.stdout:2/936: rename d8/df/d10/d1c/d7c/f90 to d8/df/f126 0 2024-06-15T01:07:12.772 INFO:tasks.workunit.client.1.smithi142.stdout:0/917: chown d3/d39/d3e/f5f 15 1 2024-06-15T01:07:12.772 INFO:tasks.workunit.client.1.smithi142.stdout:8/982: write d0/d25/d5a/f48 [3834311,86712] 0 2024-06-15T01:07:12.772 INFO:tasks.workunit.client.0.smithi137.stdout:3/889: mknod d0/d2f/c122 0 2024-06-15T01:07:12.774 INFO:tasks.workunit.client.1.smithi142.stdout:3/963: creat d0/d3/de2/d55/f11e x:0 0 0 2024-06-15T01:07:12.776 INFO:tasks.workunit.client.1.smithi142.stdout:5/944: write d2/d3e/d65/f70 [1852581,42152] 0 2024-06-15T01:07:12.777 INFO:tasks.workunit.client.1.smithi142.stdout:6/935: mkdir d1a/d3c/d140 0 2024-06-15T01:07:12.780 INFO:tasks.workunit.client.1.smithi142.stdout:0/918: rename d3/d39/d3e/d81/d44/f4b to d3/d39/d3e/d81/d44/df1/f13a 0 2024-06-15T01:07:12.782 INFO:tasks.workunit.client.0.smithi137.stdout:3/890: dwrite d0/d2f/f69 [4194304,4194304] 0 2024-06-15T01:07:12.782 INFO:tasks.workunit.client.1.smithi142.stdout:2/937: read d8/df/d10/f11 [859287,120032] 0 2024-06-15T01:07:12.783 INFO:tasks.workunit.client.1.smithi142.stdout:6/936: creat d1a/d39/d42/df7/dfc/f141 x:0 0 0 2024-06-15T01:07:12.783 INFO:tasks.workunit.client.0.smithi137.stdout:3/891: chown d0/d2/d22/d63/f6d 1713897 1 2024-06-15T01:07:12.783 INFO:tasks.workunit.client.1.smithi142.stdout:6/937: readlink d1a/d2d/d3b/le7 0 2024-06-15T01:07:12.785 INFO:tasks.workunit.client.0.smithi137.stdout:3/892: symlink d0/d2/d1e/l123 0 2024-06-15T01:07:12.786 INFO:tasks.workunit.client.1.smithi142.stdout:3/964: rename d0/d95/fdd to d0/d3/da/d18/f11f 0 2024-06-15T01:07:12.787 INFO:tasks.workunit.client.0.smithi137.stdout:3/893: chown d0/d2/d8/f7a 24 1 2024-06-15T01:07:12.787 INFO:tasks.workunit.client.1.smithi142.stdout:8/983: creat d0/d17/d40/f12a x:0 0 0 2024-06-15T01:07:12.789 INFO:tasks.workunit.client.1.smithi142.stdout:8/984: chown d0/d17/d40/d86/fc6 13121 1 2024-06-15T01:07:12.792 INFO:tasks.workunit.client.1.smithi142.stdout:5/945: rename d2/de/d19/d5f/d68/d76/f7c to d2/de/d27/f129 0 2024-06-15T01:07:12.795 INFO:tasks.workunit.client.0.smithi137.stdout:3/894: dwrite d0/d2/d23/f48 [4194304,4194304] 0 2024-06-15T01:07:12.798 INFO:tasks.workunit.client.1.smithi142.stdout:2/938: rename d8/df/d10/d94/d25/d1a/d63/d107/c115 to d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/c127 0 2024-06-15T01:07:12.800 INFO:tasks.workunit.client.0.smithi137.stdout:3/895: mknod d0/d2/d8/dbd/d106/d2e/df0/c124 0 2024-06-15T01:07:12.801 INFO:tasks.workunit.client.1.smithi142.stdout:5/946: truncate d2/de/d27/d86/dbd/fe0 381902 0 2024-06-15T01:07:12.803 INFO:tasks.workunit.client.0.smithi137.stdout:3/896: mkdir d0/d2/d8/dbd/d106/d2e/d92/d125 0 2024-06-15T01:07:12.804 INFO:tasks.workunit.client.1.smithi142.stdout:3/965: link d0/d1f/d20/d5d/d60/dc1/cf4 d0/d3/da/c120 0 2024-06-15T01:07:12.804 INFO:tasks.workunit.client.0.smithi137.stdout:3/897: dread - d0/d2/d8/dbd/fd8 zero size 2024-06-15T01:07:12.804 INFO:tasks.workunit.client.1.smithi142.stdout:8/985: dread d0/d17/d40/d6e/f73 [0,4194304] 0 2024-06-15T01:07:12.806 INFO:tasks.workunit.client.1.smithi142.stdout:2/939: rename d8/df/d10/d94/db5/ded/db6 to d8/df/d10/d94/d25/d22/d43/d51/d128 0 2024-06-15T01:07:12.806 INFO:tasks.workunit.client.1.smithi142.stdout:7/956: dread d2/d8/da/d33/f52 [0,4194304] 0 2024-06-15T01:07:12.807 INFO:tasks.workunit.client.1.smithi142.stdout:2/940: readlink d8/df/d10/d1c/l2f 0 2024-06-15T01:07:12.811 INFO:tasks.workunit.client.0.smithi137.stdout:3/898: dwrite d0/d13/d37/f43 [8388608,4194304] 0 2024-06-15T01:07:12.813 INFO:tasks.workunit.client.1.smithi142.stdout:8/986: rename d0/d25/d53/da7/dcb/dfb/l110 to d0/d17/d129/l12b 0 2024-06-15T01:07:12.813 INFO:tasks.workunit.client.0.smithi137.stdout:3/899: dread - d0/d2/d22/d63/f115 zero size 2024-06-15T01:07:12.815 INFO:tasks.workunit.client.0.smithi137.stdout:3/900: unlink d0/d2/d8/dbd/d106/l11b 0 2024-06-15T01:07:12.817 INFO:tasks.workunit.client.0.smithi137.stdout:3/901: truncate d0/f1f 2268259 0 2024-06-15T01:07:12.821 INFO:tasks.workunit.client.1.smithi142.stdout:0/919: dread d3/d39/d3e/d81/d44/df1/f13a [0,4194304] 0 2024-06-15T01:07:12.821 INFO:tasks.workunit.client.1.smithi142.stdout:1/987: sync 2024-06-15T01:07:12.821 INFO:tasks.workunit.client.1.smithi142.stdout:9/868: sync 2024-06-15T01:07:12.823 INFO:tasks.workunit.client.1.smithi142.stdout:2/941: truncate d8/df/d10/d1c/fad 1309418 0 2024-06-15T01:07:12.823 INFO:tasks.workunit.client.0.smithi137.stdout:3/902: dread d0/d2/d23/d68/fda [0,4194304] 0 2024-06-15T01:07:12.824 INFO:tasks.workunit.client.1.smithi142.stdout:5/947: getdents d2/de/d27/db2 0 2024-06-15T01:07:12.824 INFO:tasks.workunit.client.1.smithi142.stdout:6/938: dread d1a/d3c/d100/d58/fae [0,4194304] 0 2024-06-15T01:07:12.825 INFO:tasks.workunit.client.0.smithi137.stdout:3/903: write d0/d2/d23/f3f [4684839,46299] 0 2024-06-15T01:07:12.825 INFO:tasks.workunit.client.1.smithi142.stdout:3/966: read d0/d3/de2/d55/dae/fcd [1547070,69632] 0 2024-06-15T01:07:12.826 INFO:tasks.workunit.client.1.smithi142.stdout:0/920: symlink d3/l13b 0 2024-06-15T01:07:12.827 INFO:tasks.workunit.client.1.smithi142.stdout:9/869: rmdir d6/dc/d4e/dae 39 2024-06-15T01:07:12.828 INFO:tasks.workunit.client.1.smithi142.stdout:9/870: dread - d6/dc/f123 zero size 2024-06-15T01:07:12.828 INFO:tasks.workunit.client.1.smithi142.stdout:1/988: mknod d6/d12/d3c/d63/d65/c15c 0 2024-06-15T01:07:12.829 INFO:tasks.workunit.client.1.smithi142.stdout:2/942: write d8/df/d10/d94/d25/d22/d43/f46 [8768168,76358] 0 2024-06-15T01:07:12.830 INFO:tasks.workunit.client.1.smithi142.stdout:7/957: link d2/f49 d2/d8/da/f131 0 2024-06-15T01:07:12.832 INFO:tasks.workunit.client.0.smithi137.stdout:3/904: dwrite d0/d2/f118 [0,4194304] 0 2024-06-15T01:07:12.832 INFO:tasks.workunit.client.1.smithi142.stdout:7/958: readlink d2/d8/dd/d14/d2c/l6a 0 2024-06-15T01:07:12.833 INFO:tasks.workunit.client.1.smithi142.stdout:6/939: mknod d1a/d39/d3a/d51/d81/c142 0 2024-06-15T01:07:12.834 INFO:tasks.workunit.client.1.smithi142.stdout:3/967: fsync d0/d3/d15/f83 0 2024-06-15T01:07:12.837 INFO:tasks.workunit.client.1.smithi142.stdout:0/921: creat d3/d39/d3e/d81/d1e/d98/f13c x:0 0 0 2024-06-15T01:07:12.838 INFO:tasks.workunit.client.1.smithi142.stdout:8/987: dread d0/d25/d66/fe1 [0,4194304] 0 2024-06-15T01:07:12.840 INFO:tasks.workunit.client.1.smithi142.stdout:2/943: mkdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/dcf/d129 0 2024-06-15T01:07:12.843 INFO:tasks.workunit.client.1.smithi142.stdout:1/989: dwrite d6/d12/d80/dc8/f110 [0,4194304] 0 2024-06-15T01:07:12.844 INFO:tasks.workunit.client.1.smithi142.stdout:0/922: write d3/d39/d3e/d81/d4c/da0/fff [1603142,77176] 0 2024-06-15T01:07:12.845 INFO:tasks.workunit.client.1.smithi142.stdout:2/944: write d8/df/d10/ffb [1039084,25765] 0 2024-06-15T01:07:12.845 INFO:tasks.workunit.client.1.smithi142.stdout:5/948: creat d2/d18/f12a x:0 0 0 2024-06-15T01:07:12.847 INFO:tasks.workunit.client.1.smithi142.stdout:7/959: creat d2/d8/da/d33/d25/d79/dbc/f132 x:0 0 0 2024-06-15T01:07:12.851 INFO:tasks.workunit.client.1.smithi142.stdout:3/968: rmdir d0/d3/da/d18/dd9 39 2024-06-15T01:07:12.852 INFO:tasks.workunit.client.1.smithi142.stdout:0/923: chown d3/d39/d3e/d81/d44/d7c/d22/d59/f80 12686 1 2024-06-15T01:07:12.852 INFO:tasks.workunit.client.1.smithi142.stdout:9/871: rename d6/d34/df3/dfb to d6/dc/d4e/dae/d13d 0 2024-06-15T01:07:12.857 INFO:tasks.workunit.client.1.smithi142.stdout:2/945: dread d8/df/d10/ffb [0,4194304] 0 2024-06-15T01:07:12.859 INFO:tasks.workunit.client.1.smithi142.stdout:5/949: creat d2/d3e/d74/da5/dbe/f12b x:0 0 0 2024-06-15T01:07:12.859 INFO:tasks.workunit.client.0.smithi137.stdout:3/905: dread d0/d2/d8/d67/d7e/f86 [0,4194304] 0 2024-06-15T01:07:12.859 INFO:tasks.workunit.client.1.smithi142.stdout:1/990: dwrite d6/d12/d80/dc8/f11d [0,4194304] 0 2024-06-15T01:07:12.860 INFO:tasks.workunit.client.0.smithi137.stdout:3/906: write d0/d42/ff7 [577626,126208] 0 2024-06-15T01:07:12.861 INFO:tasks.workunit.client.1.smithi142.stdout:5/950: write d2/de/d27/f116 [756620,128213] 0 2024-06-15T01:07:12.861 INFO:tasks.workunit.client.1.smithi142.stdout:1/991: chown d6/d12/c79 4926436 1 2024-06-15T01:07:12.867 INFO:tasks.workunit.client.1.smithi142.stdout:6/940: link d1a/l128 d1a/d3c/db0/l143 0 2024-06-15T01:07:12.869 INFO:tasks.workunit.client.1.smithi142.stdout:5/951: mknod d2/d3e/d65/dd6/c12c 0 2024-06-15T01:07:12.872 INFO:tasks.workunit.client.1.smithi142.stdout:5/952: read d2/d3e/d74/fcf [784947,12053] 0 2024-06-15T01:07:12.872 INFO:tasks.workunit.client.1.smithi142.stdout:6/941: mknod d1a/d39/d3a/d51/d62/da8/c144 0 2024-06-15T01:07:12.876 INFO:tasks.workunit.client.1.smithi142.stdout:1/992: link d6/d46/f78 d6/d12/d3c/d63/d8d/d97/dd3/d106/f15d 0 2024-06-15T01:07:12.877 INFO:tasks.workunit.client.1.smithi142.stdout:5/953: creat d2/d3e/d31/d7e/f12d x:0 0 0 2024-06-15T01:07:12.878 INFO:tasks.workunit.client.1.smithi142.stdout:1/993: readlink d6/d12/d1c/d2c/d47/l68 0 2024-06-15T01:07:12.879 INFO:tasks.workunit.client.0.smithi137.stdout:3/907: creat d0/d2/d8/dbd/d106/f126 x:0 0 0 2024-06-15T01:07:12.881 INFO:tasks.workunit.client.1.smithi142.stdout:3/969: dread d0/d1f/d20/d5d/d82/dc9/fda [0,4194304] 0 2024-06-15T01:07:12.881 INFO:tasks.workunit.client.0.smithi137.stdout:3/908: getdents d0/d13/d37/de8 0 2024-06-15T01:07:12.882 INFO:tasks.workunit.client.1.smithi142.stdout:1/994: write d6/d12/d5b/f8a [942943,93592] 0 2024-06-15T01:07:12.882 INFO:tasks.workunit.client.0.smithi137.stdout:3/909: mkdir d0/d2/d23/ddb/d127 0 2024-06-15T01:07:12.884 INFO:tasks.workunit.client.0.smithi137.stdout:3/910: symlink d0/d42/l128 0 2024-06-15T01:07:12.885 INFO:tasks.workunit.client.0.smithi137.stdout:3/911: getdents d0/d4c 0 2024-06-15T01:07:12.886 INFO:tasks.workunit.client.1.smithi142.stdout:7/960: dwrite d2/d6/d22/ddd/f36 [0,4194304] 0 2024-06-15T01:07:12.886 INFO:tasks.workunit.client.0.smithi137.stdout:3/912: readlink d0/d2/l32 0 2024-06-15T01:07:12.889 INFO:tasks.workunit.client.1.smithi142.stdout:1/995: symlink d6/d4d/da2/l15e 0 2024-06-15T01:07:12.889 INFO:tasks.workunit.client.1.smithi142.stdout:9/872: dread d6/dc/dd/fac [0,4194304] 0 2024-06-15T01:07:12.891 INFO:tasks.workunit.client.1.smithi142.stdout:0/924: dwrite f1 [0,4194304] 0 2024-06-15T01:07:12.893 INFO:tasks.workunit.client.1.smithi142.stdout:7/961: write d2/d8/dd/dc0/f104 [633381,97163] 0 2024-06-15T01:07:12.894 INFO:tasks.workunit.client.1.smithi142.stdout:8/988: dwrite d0/d17/d40/d6e/f93 [0,4194304] 0 2024-06-15T01:07:12.897 INFO:tasks.workunit.client.1.smithi142.stdout:8/989: readlink d0/d25/d53/l75 0 2024-06-15T01:07:12.898 INFO:tasks.workunit.client.1.smithi142.stdout:5/954: dwrite d2/de/d27/f77 [0,4194304] 0 2024-06-15T01:07:12.900 INFO:tasks.workunit.client.0.smithi137.stdout:8/493: sync 2024-06-15T01:07:12.900 INFO:tasks.workunit.client.0.smithi137.stdout:7/503: sync 2024-06-15T01:07:12.903 INFO:tasks.workunit.client.0.smithi137.stdout:8/494: chown d9/de/l17 1 1 2024-06-15T01:07:12.904 INFO:tasks.workunit.client.1.smithi142.stdout:9/873: creat d6/d31/f13e x:0 0 0 2024-06-15T01:07:12.904 INFO:tasks.workunit.client.1.smithi142.stdout:7/962: rmdir d2/d6/d22 39 2024-06-15T01:07:12.904 INFO:tasks.workunit.client.1.smithi142.stdout:0/925: creat d3/d39/d9a/d9d/dcd/f13d x:0 0 0 2024-06-15T01:07:12.907 INFO:tasks.workunit.client.0.smithi137.stdout:7/504: unlink df/d7d/d1c/d1d/d27/d33/d43/d6c/f8b 0 2024-06-15T01:07:12.907 INFO:tasks.workunit.client.1.smithi142.stdout:0/926: dread - d3/d39/d3e/d81/f103 zero size 2024-06-15T01:07:12.907 INFO:tasks.workunit.client.1.smithi142.stdout:5/955: unlink d2/de/d27/f38 0 2024-06-15T01:07:12.909 INFO:tasks.workunit.client.0.smithi137.stdout:8/495: mknod d9/d2a/ca4 0 2024-06-15T01:07:12.910 INFO:tasks.workunit.client.1.smithi142.stdout:8/990: unlink d0/f6f 0 2024-06-15T01:07:12.912 INFO:tasks.workunit.client.0.smithi137.stdout:7/505: mknod df/d13/d2e/d87/ca0 0 2024-06-15T01:07:12.912 INFO:tasks.workunit.client.1.smithi142.stdout:9/874: creat d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f13f x:0 0 0 2024-06-15T01:07:12.913 INFO:tasks.workunit.client.0.smithi137.stdout:7/506: chown df/d7d/d3b/d5b 903839 1 2024-06-15T01:07:12.913 INFO:tasks.workunit.client.1.smithi142.stdout:7/963: creat d2/d6/d22/ddd/d85/f133 x:0 0 0 2024-06-15T01:07:12.914 INFO:tasks.workunit.client.0.smithi137.stdout:8/496: mkdir d9/de/d1e/d3f/da5 0 2024-06-15T01:07:12.914 INFO:tasks.workunit.client.1.smithi142.stdout:5/956: symlink d2/d3e/d74/da5/dbe/l12e 0 2024-06-15T01:07:12.915 INFO:tasks.workunit.client.0.smithi137.stdout:7/507: rmdir df/d7d/d1c/d1d 39 2024-06-15T01:07:12.916 INFO:tasks.workunit.client.1.smithi142.stdout:9/875: mkdir d6/dc/d1b/d6f/d103/df2/d140 0 2024-06-15T01:07:12.917 INFO:tasks.workunit.client.0.smithi137.stdout:8/497: creat d9/de/d1e/d67/fa6 x:0 0 0 2024-06-15T01:07:12.918 INFO:tasks.workunit.client.0.smithi137.stdout:7/508: stat df/d7d/c5c 0 2024-06-15T01:07:12.919 INFO:tasks.workunit.client.0.smithi137.stdout:8/498: fsync d9/de/d1e/d3f/d84/f86 0 2024-06-15T01:07:12.919 INFO:tasks.workunit.client.1.smithi142.stdout:5/957: symlink d2/de/d27/db2/l12f 0 2024-06-15T01:07:12.921 INFO:tasks.workunit.client.1.smithi142.stdout:9/876: write d6/dc/d1b/d6f/d103/df2/d13c/d58/f63 [870077,10732] 0 2024-06-15T01:07:12.922 INFO:tasks.workunit.client.0.smithi137.stdout:7/509: read - df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/f9e zero size 2024-06-15T01:07:12.922 INFO:tasks.workunit.client.1.smithi142.stdout:9/877: readlink d6/dc/d1b/le6 0 2024-06-15T01:07:12.922 INFO:tasks.workunit.client.1.smithi142.stdout:9/878: chown d6/dc/l13 1119636007 1 2024-06-15T01:07:12.923 INFO:tasks.workunit.client.1.smithi142.stdout:7/964: truncate d2/d6/d22/f4d 4948760 0 2024-06-15T01:07:12.924 INFO:tasks.workunit.client.1.smithi142.stdout:9/879: chown d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f133 32403960 1 2024-06-15T01:07:12.924 INFO:tasks.workunit.client.0.smithi137.stdout:8/499: creat d9/de/d1e/d3f/da5/fa7 x:0 0 0 2024-06-15T01:07:12.925 INFO:tasks.workunit.client.1.smithi142.stdout:5/958: unlink d2/d3e/d65/dd6/fdb 0 2024-06-15T01:07:12.925 INFO:tasks.workunit.client.1.smithi142.stdout:0/927: getdents d3/d39/d3e/d81/d4c/d61/dea 0 2024-06-15T01:07:12.926 INFO:tasks.workunit.client.1.smithi142.stdout:5/959: write d2/d3e/d98/fbf [2240416,109277] 0 2024-06-15T01:07:12.926 INFO:tasks.workunit.client.1.smithi142.stdout:0/928: stat d3/d39/fc4 0 2024-06-15T01:07:12.927 INFO:tasks.workunit.client.0.smithi137.stdout:8/500: write d9/d2a/f69 [1356370,85370] 0 2024-06-15T01:07:12.928 INFO:tasks.workunit.client.1.smithi142.stdout:0/929: fsync d3/d39/d3e/d81/d44/d7c/fbd 0 2024-06-15T01:07:12.929 INFO:tasks.workunit.client.1.smithi142.stdout:9/880: rename d6/dc/f61 to d6/d34/d8a/dec/f141 0 2024-06-15T01:07:12.930 INFO:tasks.workunit.client.1.smithi142.stdout:0/930: write d3/f6e [5264489,121485] 0 2024-06-15T01:07:12.930 INFO:tasks.workunit.client.1.smithi142.stdout:9/881: chown d6/d34/ddd 96283257 1 2024-06-15T01:07:12.934 INFO:tasks.workunit.client.1.smithi142.stdout:3/970: truncate d0/d1f/d20/d5d/d60/fb0 3521755 0 2024-06-15T01:07:12.934 INFO:tasks.workunit.client.0.smithi137.stdout:8/501: creat d9/d22/fa8 x:0 0 0 2024-06-15T01:07:12.934 INFO:tasks.workunit.client.0.smithi137.stdout:7/510: link df/d13/d3c/l55 df/d7d/d1c/d1d/d27/la1 0 2024-06-15T01:07:12.937 INFO:tasks.workunit.client.1.smithi142.stdout:0/931: rename d3/d39/d3e/d81/d44/d7c to d3/d39/d3e/d81/d1b/d52/d13e 0 2024-06-15T01:07:12.938 INFO:tasks.workunit.client.0.smithi137.stdout:3/913: truncate d0/d2/d4e/fd9 3922190 0 2024-06-15T01:07:12.939 INFO:tasks.workunit.client.1.smithi142.stdout:9/882: mknod d6/dc/d1b/d6f/d103/d8d/d10b/c142 0 2024-06-15T01:07:12.939 INFO:tasks.workunit.client.0.smithi137.stdout:3/914: dread - d0/d2/d8/dbd/f120 zero size 2024-06-15T01:07:12.940 INFO:tasks.workunit.client.1.smithi142.stdout:1/996: dwrite d6/d3a/fee [0,4194304] 0 2024-06-15T01:07:12.944 INFO:tasks.workunit.client.1.smithi142.stdout:3/971: dwrite d0/d3/f11a [0,4194304] 0 2024-06-15T01:07:12.944 INFO:tasks.workunit.client.0.smithi137.stdout:3/915: truncate d0/d2/d8/d67/d7e/f53 1157910 0 2024-06-15T01:07:12.946 INFO:tasks.workunit.client.0.smithi137.stdout:5/643: sync 2024-06-15T01:07:12.946 INFO:tasks.workunit.client.0.smithi137.stdout:4/583: sync 2024-06-15T01:07:12.946 INFO:tasks.workunit.client.0.smithi137.stdout:2/475: sync 2024-06-15T01:07:12.946 INFO:tasks.workunit.client.0.smithi137.stdout:9/496: sync 2024-06-15T01:07:12.946 INFO:tasks.workunit.client.0.smithi137.stdout:1/789: sync 2024-06-15T01:07:12.947 INFO:tasks.workunit.client.0.smithi137.stdout:6/554: sync 2024-06-15T01:07:12.947 INFO:tasks.workunit.client.0.smithi137.stdout:0/760: sync 2024-06-15T01:07:12.947 INFO:tasks.workunit.client.1.smithi142.stdout:0/932: mkdir d3/d39/d9a/d9d/d13f 0 2024-06-15T01:07:12.948 INFO:tasks.workunit.client.1.smithi142.stdout:5/960: dread d2/de/f50 [0,4194304] 0 2024-06-15T01:07:12.951 INFO:tasks.workunit.client.1.smithi142.stdout:1/997: creat d6/d12/d3c/d63/d8d/d97/dc6/f15f x:0 0 0 2024-06-15T01:07:12.951 INFO:tasks.workunit.client.0.smithi137.stdout:9/497: write df/d12/d29/f44 [2427432,114887] 0 2024-06-15T01:07:12.952 INFO:tasks.workunit.client.1.smithi142.stdout:1/998: fsync d6/d12/d80/f123 0 2024-06-15T01:07:12.952 INFO:tasks.workunit.client.0.smithi137.stdout:1/790: write d0/d3/d47/f92 [597881,63666] 0 2024-06-15T01:07:12.954 INFO:tasks.workunit.client.0.smithi137.stdout:3/916: rename d0/d2/d23/ddb to d0/d2/d8/d67/d129 0 2024-06-15T01:07:12.957 INFO:tasks.workunit.client.1.smithi142.stdout:8/991: truncate d0/f87 4118235 0 2024-06-15T01:07:12.958 INFO:tasks.workunit.client.1.smithi142.stdout:8/992: chown d0/d17/d40/d86/fff 5136478 1 2024-06-15T01:07:12.959 INFO:tasks.workunit.client.1.smithi142.stdout:1/999: unlink d6/d12/d3c/d63/fdd 0 2024-06-15T01:07:12.959 INFO:tasks.workunit.client.0.smithi137.stdout:8/502: dread d9/d22/f33 [0,4194304] 0 2024-06-15T01:07:12.959 INFO:tasks.workunit.client.1.smithi142.stdout:2/946: sync 2024-06-15T01:07:12.959 INFO:tasks.workunit.client.1.smithi142.stdout:6/942: sync 2024-06-15T01:07:12.960 INFO:tasks.workunit.client.0.smithi137.stdout:6/555: symlink d0/d5/d9/d3f/lb4 0 2024-06-15T01:07:12.961 INFO:tasks.workunit.client.0.smithi137.stdout:1/791: chown d0/d10/d1d/l43 1420 1 2024-06-15T01:07:12.961 INFO:tasks.workunit.client.0.smithi137.stdout:9/498: symlink df/d49/d72/l8b 0 2024-06-15T01:07:12.963 INFO:tasks.workunit.client.0.smithi137.stdout:7/511: dread f3 [0,4194304] 0 2024-06-15T01:07:12.963 INFO:tasks.workunit.client.0.smithi137.stdout:4/584: creat dd/d15/d36/d45/dcb/d5b/fda x:0 0 0 2024-06-15T01:07:12.964 INFO:tasks.workunit.client.1.smithi142.stdout:5/961: mkdir d2/d3e/d65/d130 0 2024-06-15T01:07:12.964 INFO:tasks.workunit.client.1.smithi142.stdout:3/972: dwrite d0/d3/de2/d55/f7a [4194304,4194304] 0 2024-06-15T01:07:12.964 INFO:tasks.workunit.client.1.smithi142.stdout:7/965: write d2/d8/da/d33/d25/d79/fb7 [1183241,90002] 0 2024-06-15T01:07:12.964 INFO:tasks.workunit.client.1.smithi142.stdout:0/933: dread d3/d39/d3e/d81/d1e/d98/fd9 [0,4194304] 0 2024-06-15T01:07:12.964 INFO:tasks.workunit.client.0.smithi137.stdout:8/503: readlink d9/de/d1e/l49 0 2024-06-15T01:07:12.969 INFO:tasks.workunit.client.1.smithi142.stdout:2/947: mkdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/d12a 0 2024-06-15T01:07:12.969 INFO:tasks.workunit.client.0.smithi137.stdout:9/499: creat df/d49/d4b/f8c x:0 0 0 2024-06-15T01:07:12.972 INFO:tasks.workunit.client.1.smithi142.stdout:9/883: dread d6/dc/d4e/f57 [0,4194304] 0 2024-06-15T01:07:12.975 INFO:tasks.workunit.client.1.smithi142.stdout:3/973: unlink d0/d3/fcb 0 2024-06-15T01:07:12.976 INFO:tasks.workunit.client.1.smithi142.stdout:9/884: dread d6/dc/d51/f80 [0,4194304] 0 2024-06-15T01:07:12.977 INFO:tasks.workunit.client.1.smithi142.stdout:5/962: symlink d2/de/d85/l131 0 2024-06-15T01:07:12.978 INFO:tasks.workunit.client.1.smithi142.stdout:9/885: write d6/d34/d8a/f131 [585052,96877] 0 2024-06-15T01:07:12.979 INFO:tasks.workunit.client.1.smithi142.stdout:6/943: truncate d1a/d2d/d3b/dab/dce/f13a 703617 0 2024-06-15T01:07:12.979 INFO:tasks.workunit.client.0.smithi137.stdout:8/504: readlink d9/de/l19 0 2024-06-15T01:07:12.980 INFO:tasks.workunit.client.0.smithi137.stdout:1/792: dwrite d0/d3/fd6 [0,4194304] 0 2024-06-15T01:07:12.980 INFO:tasks.workunit.client.1.smithi142.stdout:0/934: rmdir d3/d39/d3e/d8f 39 2024-06-15T01:07:12.984 INFO:tasks.workunit.client.0.smithi137.stdout:7/512: dwrite f3 [4194304,4194304] 0 2024-06-15T01:07:12.985 INFO:tasks.workunit.client.1.smithi142.stdout:5/963: dwrite d2/d3e/d65/fc7 [0,4194304] 0 2024-06-15T01:07:12.986 INFO:tasks.workunit.client.0.smithi137.stdout:8/505: dread - d9/de/d1e/d3f/d84/f86 zero size 2024-06-15T01:07:12.986 INFO:tasks.workunit.client.0.smithi137.stdout:1/793: dread - d0/d3/d1b/d4c/d59/d66/fad zero size 2024-06-15T01:07:12.988 INFO:tasks.workunit.client.0.smithi137.stdout:0/761: creat d1/d7e/d87/da0/dc4/dd7/de2/d8d/ffc x:0 0 0 2024-06-15T01:07:12.990 INFO:tasks.workunit.client.0.smithi137.stdout:9/500: mkdir df/d49/d8d 0 2024-06-15T01:07:12.993 INFO:tasks.workunit.client.1.smithi142.stdout:3/974: rename d0/d3/f100 to d0/d1f/d20/d59/d61/f121 0 2024-06-15T01:07:12.994 INFO:tasks.workunit.client.0.smithi137.stdout:3/917: rename d0/d42/la3 to d0/d56/l12a 0 2024-06-15T01:07:12.994 INFO:tasks.workunit.client.0.smithi137.stdout:2/476: dwrite d0/d1c/f66 [0,4194304] 0 2024-06-15T01:07:12.998 INFO:tasks.workunit.client.1.smithi142.stdout:9/886: creat d6/dc/d4e/dae/f143 x:0 0 0 2024-06-15T01:07:12.999 INFO:tasks.workunit.client.0.smithi137.stdout:7/513: rmdir df/d7d/d1c/d1d/d27 39 2024-06-15T01:07:13.000 INFO:tasks.workunit.client.1.smithi142.stdout:6/944: mkdir d1a/d2d/d3b/d145 0 2024-06-15T01:07:13.002 INFO:tasks.workunit.client.0.smithi137.stdout:3/918: read d0/ddf/ff2 [3350287,36] 0 2024-06-15T01:07:13.002 INFO:tasks.workunit.client.0.smithi137.stdout:3/919: dread - d0/d2/d23/f119 zero size 2024-06-15T01:07:13.004 INFO:tasks.workunit.client.0.smithi137.stdout:0/762: rmdir d1/d7e/d87/da0/dc4/dd7/de2/d77 39 2024-06-15T01:07:13.007 INFO:tasks.workunit.client.1.smithi142.stdout:2/948: mkdir d8/df/d10/d1c/d7c/d12b 0 2024-06-15T01:07:13.009 INFO:tasks.workunit.client.0.smithi137.stdout:7/514: creat df/d13/d2e/fa2 x:0 0 0 2024-06-15T01:07:13.012 INFO:tasks.workunit.client.0.smithi137.stdout:4/585: getdents dd/d15/d21 0 2024-06-15T01:07:13.013 INFO:tasks.workunit.client.1.smithi142.stdout:9/887: creat d6/dc/d1b/d6f/de1/d110/f144 x:0 0 0 2024-06-15T01:07:13.014 INFO:tasks.workunit.client.0.smithi137.stdout:0/763: rename d1/d2/dc/d6c/d88/d91/f9a to d1/d2/dc/d10/d14/d56/ffd 0 2024-06-15T01:07:13.015 INFO:tasks.workunit.client.1.smithi142.stdout:9/888: dread - d6/dc/d1b/d6f/d103/df2/d13c/d35/f70 zero size 2024-06-15T01:07:13.016 INFO:tasks.workunit.client.0.smithi137.stdout:1/794: mkdir d0/d1f/d76/d109 0 2024-06-15T01:07:13.017 INFO:tasks.workunit.client.0.smithi137.stdout:3/920: mkdir d0/d13/d37/d12b 0 2024-06-15T01:07:13.017 INFO:tasks.workunit.client.0.smithi137.stdout:1/795: stat d0/d10/d1d/de0 0 2024-06-15T01:07:13.017 INFO:tasks.workunit.client.1.smithi142.stdout:6/945: mknod d1a/d3c/d100/d58/d10f/d131/c146 0 2024-06-15T01:07:13.018 INFO:tasks.workunit.client.1.smithi142.stdout:7/966: dwrite d2/d8/dd/d14/f65 [0,4194304] 0 2024-06-15T01:07:13.018 INFO:tasks.workunit.client.1.smithi142.stdout:0/935: link d3/l13b d3/d39/d3e/db4/l140 0 2024-06-15T01:07:13.020 INFO:tasks.workunit.client.1.smithi142.stdout:9/889: write d6/dc/d1b/d6f/de1/d110/f144 [891491,52853] 0 2024-06-15T01:07:13.021 INFO:tasks.workunit.client.0.smithi137.stdout:7/515: truncate df/d13/d2e/f4f 423690 0 2024-06-15T01:07:13.022 INFO:tasks.workunit.client.1.smithi142.stdout:5/964: creat d2/f132 x:0 0 0 2024-06-15T01:07:13.024 INFO:tasks.workunit.client.1.smithi142.stdout:7/967: creat d2/d8/da/f134 x:0 0 0 2024-06-15T01:07:13.024 INFO:tasks.workunit.client.1.smithi142.stdout:5/965: readlink d2/de/d19/d5f/d68/d76/d8c/le4 0 2024-06-15T01:07:13.025 INFO:tasks.workunit.client.0.smithi137.stdout:1/796: dread d0/f3b [0,4194304] 0 2024-06-15T01:07:13.026 INFO:tasks.workunit.client.1.smithi142.stdout:2/949: dwrite d8/df/ff0 [0,4194304] 0 2024-06-15T01:07:13.028 INFO:tasks.workunit.client.0.smithi137.stdout:5/644: dread df/d44/d54/d95/d3b/d49/f77 [0,4194304] 0 2024-06-15T01:07:13.029 INFO:tasks.workunit.client.1.smithi142.stdout:8/993: truncate d0/d17/f61 4193014 0 2024-06-15T01:07:13.033 INFO:tasks.workunit.client.0.smithi137.stdout:5/645: stat df/d11/f24 0 2024-06-15T01:07:13.035 INFO:tasks.workunit.client.1.smithi142.stdout:7/968: rmdir d2/db6/de6 39 2024-06-15T01:07:13.035 INFO:tasks.workunit.client.0.smithi137.stdout:5/646: chown df/d11/d30/ca3 84 1 2024-06-15T01:07:13.036 INFO:tasks.workunit.client.1.smithi142.stdout:8/994: dwrite d0/d17/d40/d5e/fad [0,4194304] 0 2024-06-15T01:07:13.038 INFO:tasks.workunit.client.1.smithi142.stdout:5/966: symlink d2/de/d85/l133 0 2024-06-15T01:07:13.041 INFO:tasks.workunit.client.0.smithi137.stdout:0/764: dwrite d1/d7e/fae [0,4194304] 0 2024-06-15T01:07:13.042 INFO:tasks.workunit.client.1.smithi142.stdout:3/975: write d0/d3/d15/f8c [91069,47402] 0 2024-06-15T01:07:13.042 INFO:tasks.workunit.client.1.smithi142.stdout:0/936: rmdir d3/d39/d3e/db4 39 2024-06-15T01:07:13.043 INFO:tasks.workunit.client.1.smithi142.stdout:2/950: stat d8/d19/d8d/f93 0 2024-06-15T01:07:13.044 INFO:tasks.workunit.client.0.smithi137.stdout:3/921: symlink d0/l12c 0 2024-06-15T01:07:13.047 INFO:tasks.workunit.client.1.smithi142.stdout:0/937: fdatasync d3/d39/d3e/d81/d1b/d52/d13e/d22/ffb 0 2024-06-15T01:07:13.048 INFO:tasks.workunit.client.0.smithi137.stdout:2/477: write d0/dc/d22/d23/d5c/f82 [479150,124367] 0 2024-06-15T01:07:13.048 INFO:tasks.workunit.client.0.smithi137.stdout:6/556: dwrite d0/d5/d9/fb1 [0,4194304] 0 2024-06-15T01:07:13.048 INFO:tasks.workunit.client.1.smithi142.stdout:5/967: dwrite d2/de/d85/ff7 [0,4194304] 0 2024-06-15T01:07:13.049 INFO:tasks.workunit.client.0.smithi137.stdout:8/506: dread d9/de/d1e/d35/d5f/f74 [0,4194304] 0 2024-06-15T01:07:13.050 INFO:tasks.workunit.client.1.smithi142.stdout:9/890: dread d6/dc/d1b/d6f/d103/f9f [0,4194304] 0 2024-06-15T01:07:13.055 INFO:tasks.workunit.client.1.smithi142.stdout:8/995: unlink d0/d17/d40/d5e/ff4 0 2024-06-15T01:07:13.058 INFO:tasks.workunit.client.0.smithi137.stdout:7/516: mknod df/ca3 0 2024-06-15T01:07:13.058 INFO:tasks.workunit.client.1.smithi142.stdout:3/976: creat d0/d1f/d20/d5d/d82/d6e/f122 x:0 0 0 2024-06-15T01:07:13.061 INFO:tasks.workunit.client.1.smithi142.stdout:9/891: dwrite d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/f129 [0,4194304] 0 2024-06-15T01:07:13.061 INFO:tasks.workunit.client.1.smithi142.stdout:2/951: mknod d8/df/d10/d94/d25/d22/d43/d51/c12c 0 2024-06-15T01:07:13.063 INFO:tasks.workunit.client.0.smithi137.stdout:4/586: write dd/f8e [1699,80844] 0 2024-06-15T01:07:13.063 INFO:tasks.workunit.client.1.smithi142.stdout:5/968: readlink d2/d3e/d31/l5d 0 2024-06-15T01:07:13.064 INFO:tasks.workunit.client.1.smithi142.stdout:0/938: readlink d3/d39/d3e/d81/d4c/l58 0 2024-06-15T01:07:13.066 INFO:tasks.workunit.client.0.smithi137.stdout:1/797: creat d0/d1f/d76/da1/f10a x:0 0 0 2024-06-15T01:07:13.067 INFO:tasks.workunit.client.1.smithi142.stdout:5/969: dread d2/de/d27/f46 [0,4194304] 0 2024-06-15T01:07:13.069 INFO:tasks.workunit.client.1.smithi142.stdout:8/996: mknod d0/d17/d40/d86/d125/c12c 0 2024-06-15T01:07:13.069 INFO:tasks.workunit.client.1.smithi142.stdout:6/946: getdents d1a/d39/d3a/d51/d62/da8 0 2024-06-15T01:07:13.070 INFO:tasks.workunit.client.0.smithi137.stdout:2/478: symlink d0/dc/d2e/d72/l8c 0 2024-06-15T01:07:13.074 INFO:tasks.workunit.client.0.smithi137.stdout:8/507: mknod d9/d55/ca9 0 2024-06-15T01:07:13.074 INFO:tasks.workunit.client.1.smithi142.stdout:7/969: creat d2/d8/da/d33/f135 x:0 0 0 2024-06-15T01:07:13.075 INFO:tasks.workunit.client.0.smithi137.stdout:0/765: dwrite d1/ff3 [0,4194304] 0 2024-06-15T01:07:13.076 INFO:tasks.workunit.client.0.smithi137.stdout:6/557: creat d0/d1a/d68/fb5 x:0 0 0 2024-06-15T01:07:13.078 INFO:tasks.workunit.client.1.smithi142.stdout:5/970: creat d2/d3e/d65/dd6/f134 x:0 0 0 2024-06-15T01:07:13.079 INFO:tasks.workunit.client.0.smithi137.stdout:4/587: creat dd/d15/d21/d3a/fdb x:0 0 0 2024-06-15T01:07:13.081 INFO:tasks.workunit.client.1.smithi142.stdout:2/952: mkdir d8/df/d10/d94/d25/d22/d43/d52/d7a/d12d 0 2024-06-15T01:07:13.082 INFO:tasks.workunit.client.0.smithi137.stdout:1/798: mknod d0/d3/d47/d73/c10b 0 2024-06-15T01:07:13.082 INFO:tasks.workunit.client.0.smithi137.stdout:5/647: dwrite df/f41 [0,4194304] 0 2024-06-15T01:07:13.082 INFO:tasks.workunit.client.1.smithi142.stdout:7/970: mknod d2/d6/d22/d3c/c136 0 2024-06-15T01:07:13.083 INFO:tasks.workunit.client.0.smithi137.stdout:0/766: stat d1/d2/d13/d66/f60 0 2024-06-15T01:07:13.083 INFO:tasks.workunit.client.1.smithi142.stdout:7/971: readlink d2/l11e 0 2024-06-15T01:07:13.086 INFO:tasks.workunit.client.1.smithi142.stdout:7/972: chown d2/d8/dd8/fed 802703510 1 2024-06-15T01:07:13.086 INFO:tasks.workunit.client.0.smithi137.stdout:4/588: dread - dd/d15/d21/d9c/d75/d99/fa9 zero size 2024-06-15T01:07:13.087 INFO:tasks.workunit.client.0.smithi137.stdout:6/558: fsync d0/d5/d9/d3f/f67 0 2024-06-15T01:07:13.088 INFO:tasks.workunit.client.1.smithi142.stdout:8/997: dwrite d0/f14 [4194304,4194304] 0 2024-06-15T01:07:13.088 INFO:tasks.workunit.client.1.smithi142.stdout:7/973: chown d2/f2e 5509841 1 2024-06-15T01:07:13.091 INFO:tasks.workunit.client.0.smithi137.stdout:6/559: chown d0/d1a/d3e/d57/d1d/d2e/d62/l72 31961 1 2024-06-15T01:07:13.094 INFO:tasks.workunit.client.0.smithi137.stdout:6/560: dread - d0/d1a/da0/fa1 zero size 2024-06-15T01:07:13.095 INFO:tasks.workunit.client.1.smithi142.stdout:6/947: rename d1a/d2d/d80/lc5 to d1a/d3c/d100/l147 0 2024-06-15T01:07:13.096 INFO:tasks.workunit.client.1.smithi142.stdout:2/953: rmdir d8/df/d10/d1c/dbb 39 2024-06-15T01:07:13.097 INFO:tasks.workunit.client.0.smithi137.stdout:2/479: getdents d0/d1c 0 2024-06-15T01:07:13.098 INFO:tasks.workunit.client.1.smithi142.stdout:9/892: getdents d6/dc/d4e/dae 0 2024-06-15T01:07:13.103 INFO:tasks.workunit.client.1.smithi142.stdout:7/974: mknod d2/da2/c137 0 2024-06-15T01:07:13.103 INFO:tasks.workunit.client.1.smithi142.stdout:8/998: dwrite d0/d25/d5a/f48 [0,4194304] 0 2024-06-15T01:07:13.103 INFO:tasks.workunit.client.0.smithi137.stdout:7/517: write df/d13/d2e/d1a/f40 [3772006,65122] 0 2024-06-15T01:07:13.104 INFO:tasks.workunit.client.1.smithi142.stdout:2/954: write d8/df/f2d [808221,85749] 0 2024-06-15T01:07:13.104 INFO:tasks.workunit.client.0.smithi137.stdout:4/589: dwrite dd/d15/d36/d45/d55/f62 [0,4194304] 0 2024-06-15T01:07:13.105 INFO:tasks.workunit.client.0.smithi137.stdout:2/480: chown d0/dc/d22/d23/d5c/c6b 1013304 1 2024-06-15T01:07:13.106 INFO:tasks.workunit.client.1.smithi142.stdout:3/977: dread d0/d1f/d20/d5d/d60/fce [0,4194304] 0 2024-06-15T01:07:13.106 INFO:tasks.workunit.client.0.smithi137.stdout:0/767: write d1/d2/dc/d10/d14/d52/f58 [2999499,114419] 0 2024-06-15T01:07:13.110 INFO:tasks.workunit.client.1.smithi142.stdout:9/893: rmdir d6/d34/d8a 39 2024-06-15T01:07:13.112 INFO:tasks.workunit.client.0.smithi137.stdout:4/590: fdatasync dd/d15/d21/d9c/d75/d99/fbd 0 2024-06-15T01:07:13.114 INFO:tasks.workunit.client.1.smithi142.stdout:7/975: fdatasync d2/d8/da/d33/d25/f16 0 2024-06-15T01:07:13.117 INFO:tasks.workunit.client.1.smithi142.stdout:3/978: mknod d0/d1f/d20/d5d/d60/d72/c123 0 2024-06-15T01:07:13.118 INFO:tasks.workunit.client.0.smithi137.stdout:3/922: dread d0/d2/d22/d24/d5e/f85 [0,4194304] 0 2024-06-15T01:07:13.119 INFO:tasks.workunit.client.0.smithi137.stdout:4/591: symlink dd/ldc 0 2024-06-15T01:07:13.119 INFO:tasks.workunit.client.1.smithi142.stdout:0/939: dwrite d3/d39/d3e/d81/d4c/fc1 [0,4194304] 0 2024-06-15T01:07:13.119 INFO:tasks.workunit.client.0.smithi137.stdout:3/923: dread - d0/d2/d23/f119 zero size 2024-06-15T01:07:13.120 INFO:tasks.workunit.client.1.smithi142.stdout:8/999: symlink d0/d25/d100/l12d 0 2024-06-15T01:07:13.121 INFO:tasks.workunit.client.0.smithi137.stdout:0/768: fsync d1/d2/dc/d10/d22/fc7 0 2024-06-15T01:07:13.123 INFO:tasks.workunit.client.0.smithi137.stdout:0/769: chown d1/d2/dc/d10/d14/d56/l94 62767498 1 2024-06-15T01:07:13.124 INFO:tasks.workunit.client.0.smithi137.stdout:7/518: rename df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/f9e to df/d13/d2e/d1a/fa4 0 2024-06-15T01:07:13.125 INFO:tasks.workunit.client.1.smithi142.stdout:6/948: getdents d1a/d39/d42 0 2024-06-15T01:07:13.127 INFO:tasks.workunit.client.1.smithi142.stdout:5/971: dwrite d2/de/d19/d5f/d68/d76/d8c/fa7 [0,4194304] 0 2024-06-15T01:07:13.128 INFO:tasks.workunit.client.1.smithi142.stdout:7/976: creat d2/d8/da/d33/d25/d81/df3/f138 x:0 0 0 2024-06-15T01:07:13.129 INFO:tasks.workunit.client.1.smithi142.stdout:9/894: mkdir d6/dc/d1b/d6f/de5/d145 0 2024-06-15T01:07:13.130 INFO:tasks.workunit.client.1.smithi142.stdout:3/979: creat d0/d1b/d6b/f124 x:0 0 0 2024-06-15T01:07:13.131 INFO:tasks.workunit.client.1.smithi142.stdout:7/977: truncate d2/d8/da/f134 475863 0 2024-06-15T01:07:13.131 INFO:tasks.workunit.client.1.smithi142.stdout:6/949: creat d1a/d39/d42/df7/f148 x:0 0 0 2024-06-15T01:07:13.132 INFO:tasks.workunit.client.0.smithi137.stdout:5/648: dread df/d11/f4c [4194304,4194304] 0 2024-06-15T01:07:13.133 INFO:tasks.workunit.client.0.smithi137.stdout:8/508: dread d9/d22/f37 [0,4194304] 0 2024-06-15T01:07:13.134 INFO:tasks.workunit.client.1.smithi142.stdout:5/972: rename d2/f14 to d2/de/d27/d86/f135 0 2024-06-15T01:07:13.134 INFO:tasks.workunit.client.1.smithi142.stdout:9/895: rmdir d6/dc/d1b/d6f/d103/df2/d13c/d58/d74 39 2024-06-15T01:07:13.136 INFO:tasks.workunit.client.1.smithi142.stdout:6/950: chown d1a/d39/d3a/dd2/ld6 26 1 2024-06-15T01:07:13.139 INFO:tasks.workunit.client.1.smithi142.stdout:9/896: read d6/dc/d1b/d6f/de1/d110/f144 [907726,62494] 0 2024-06-15T01:07:13.139 INFO:tasks.workunit.client.1.smithi142.stdout:3/980: link d0/d3/da/db8/ff5 d0/d3/d10e/f125 0 2024-06-15T01:07:13.140 INFO:tasks.workunit.client.1.smithi142.stdout:3/981: read - d0/d3/de2/d75/ff0 zero size 2024-06-15T01:07:13.141 INFO:tasks.workunit.client.0.smithi137.stdout:5/649: dwrite df/f41 [0,4194304] 0 2024-06-15T01:07:13.142 INFO:tasks.workunit.client.1.smithi142.stdout:3/982: fsync d0/d1f/d20/d5d/d82/dc9/f102 0 2024-06-15T01:07:13.146 INFO:tasks.workunit.client.1.smithi142.stdout:6/951: creat d1a/d2d/d80/d127/f149 x:0 0 0 2024-06-15T01:07:13.147 INFO:tasks.workunit.client.1.smithi142.stdout:9/897: truncate d6/d31/f84 2730576 0 2024-06-15T01:07:13.149 INFO:tasks.workunit.client.1.smithi142.stdout:3/983: rename d0/d3/da/db8/ceb to d0/d1f/d20/d59/c126 0 2024-06-15T01:07:13.152 INFO:tasks.workunit.client.1.smithi142.stdout:6/952: rename d1a/d39/f82 to d1a/d39/d3a/d51/d81/f14a 0 2024-06-15T01:07:13.152 INFO:tasks.workunit.client.1.smithi142.stdout:3/984: dread d0/d3/da/d18/fd4 [0,4194304] 0 2024-06-15T01:07:13.152 INFO:tasks.workunit.client.1.smithi142.stdout:5/973: dread d2/de/d19/d5f/d68/d76/d96/fde [0,4194304] 0 2024-06-15T01:07:13.157 INFO:tasks.workunit.client.1.smithi142.stdout:9/898: read d6/d34/d8a/fbf [139562,49124] 0 2024-06-15T01:07:13.160 INFO:tasks.workunit.client.1.smithi142.stdout:5/974: rmdir d2/de/d19/d5f/d68/d76/d8c/d10b 39 2024-06-15T01:07:13.160 INFO:tasks.workunit.client.1.smithi142.stdout:6/953: read d1a/d39/fbe [69547,36933] 0 2024-06-15T01:07:13.161 INFO:tasks.workunit.client.1.smithi142.stdout:6/954: stat d1a/d2d/d80/d8e/d110/da0/ccf 0 2024-06-15T01:07:13.161 INFO:tasks.workunit.client.1.smithi142.stdout:9/899: mkdir d6/dc/d1b/d6f/d103/df2/d13c/dcf/d146 0 2024-06-15T01:07:13.165 INFO:tasks.workunit.client.1.smithi142.stdout:0/940: dread d3/d39/fc4 [0,4194304] 0 2024-06-15T01:07:13.166 INFO:tasks.workunit.client.1.smithi142.stdout:6/955: creat d1a/d39/d42/f14b x:0 0 0 2024-06-15T01:07:13.167 INFO:tasks.workunit.client.1.smithi142.stdout:6/956: fdatasync d1a/d2d/d80/fee 0 2024-06-15T01:07:13.168 INFO:tasks.workunit.client.1.smithi142.stdout:0/941: creat d3/d39/d3e/d81/d4c/da0/f141 x:0 0 0 2024-06-15T01:07:13.169 INFO:tasks.workunit.client.1.smithi142.stdout:6/957: read d1a/d2d/d2e/fd9 [1660803,119333] 0 2024-06-15T01:07:13.171 INFO:tasks.workunit.client.1.smithi142.stdout:6/958: write d1a/d3c/d100/d58/d87/fa4 [1705056,78320] 0 2024-06-15T01:07:13.173 INFO:tasks.workunit.client.1.smithi142.stdout:6/959: chown d1a/d3c/db0/dd8 57 1 2024-06-15T01:07:13.175 INFO:tasks.workunit.client.1.smithi142.stdout:0/942: creat d3/d39/d3e/d81/f142 x:0 0 0 2024-06-15T01:07:13.177 INFO:tasks.workunit.client.1.smithi142.stdout:6/960: rmdir d1a/d3c/d100/d58/d10f 39 2024-06-15T01:07:13.180 INFO:tasks.workunit.client.1.smithi142.stdout:6/961: creat d1a/d39/d3a/f14c x:0 0 0 2024-06-15T01:07:13.180 INFO:tasks.workunit.client.1.smithi142.stdout:0/943: write d3/d39/d9a/d9d/dbf/f104 [196193,91994] 0 2024-06-15T01:07:13.182 INFO:tasks.workunit.client.1.smithi142.stdout:3/985: dread d0/d1b/f41 [0,4194304] 0 2024-06-15T01:07:13.185 INFO:tasks.workunit.client.0.smithi137.stdout:0/770: creat d1/d2/d13/ffe x:0 0 0 2024-06-15T01:07:13.186 INFO:tasks.workunit.client.0.smithi137.stdout:1/799: dwrite d0/d3/d1b/d4c/ff8 [0,4194304] 0 2024-06-15T01:07:13.187 INFO:tasks.workunit.client.1.smithi142.stdout:6/962: dread d1a/d3c/d100/d58/f7f [0,4194304] 0 2024-06-15T01:07:13.188 INFO:tasks.workunit.client.0.smithi137.stdout:7/519: creat df/d7d/d1c/d1d/d27/d33/fa5 x:0 0 0 2024-06-15T01:07:13.189 INFO:tasks.workunit.client.0.smithi137.stdout:6/561: write d0/d1a/d3e/f77 [66530,93346] 0 2024-06-15T01:07:13.191 INFO:tasks.workunit.client.1.smithi142.stdout:3/986: mknod d0/d1f/d20/d5d/c127 0 2024-06-15T01:07:13.191 INFO:tasks.workunit.client.1.smithi142.stdout:3/987: dread - d0/d95/dbc/f110 zero size 2024-06-15T01:07:13.191 INFO:tasks.workunit.client.1.smithi142.stdout:5/975: dread d2/d18/d2e/f71 [0,4194304] 0 2024-06-15T01:07:13.191 INFO:tasks.workunit.client.1.smithi142.stdout:0/944: creat d3/d39/d3e/d81/d1b/d52/d13e/d22/f143 x:0 0 0 2024-06-15T01:07:13.193 INFO:tasks.workunit.client.0.smithi137.stdout:8/509: rename d9/de/d1e/d35/fa2 to d9/d2a/d79/faa 0 2024-06-15T01:07:13.193 INFO:tasks.workunit.client.1.smithi142.stdout:6/963: mknod d1a/d83/d107/c14d 0 2024-06-15T01:07:13.196 INFO:tasks.workunit.client.1.smithi142.stdout:9/900: dread d6/d34/ddd/f97 [0,4194304] 0 2024-06-15T01:07:13.196 INFO:tasks.workunit.client.0.smithi137.stdout:5/650: mknod df/d76/d7c/d9c/cd8 0 2024-06-15T01:07:13.197 INFO:tasks.workunit.client.1.smithi142.stdout:9/901: write d6/dc/d82/f13a [569794,65287] 0 2024-06-15T01:07:13.198 INFO:tasks.workunit.client.0.smithi137.stdout:2/481: dwrite d0/dc/d2e/d72/f59 [0,4194304] 0 2024-06-15T01:07:13.199 INFO:tasks.workunit.client.0.smithi137.stdout:2/482: stat d0/de/d2b/c68 0 2024-06-15T01:07:13.201 INFO:tasks.workunit.client.1.smithi142.stdout:0/945: creat d3/d39/d3e/d81/d1b/dae/f144 x:0 0 0 2024-06-15T01:07:13.202 INFO:tasks.workunit.client.0.smithi137.stdout:3/924: creat d0/d2/d8/dbd/d106/d2e/d92/d125/f12d x:0 0 0 2024-06-15T01:07:13.203 INFO:tasks.workunit.client.1.smithi142.stdout:6/964: creat d1a/d2d/d80/d127/f14e x:0 0 0 2024-06-15T01:07:13.204 INFO:tasks.workunit.client.0.smithi137.stdout:0/771: dread - d1/d2/dc/fbf zero size 2024-06-15T01:07:13.205 INFO:tasks.workunit.client.1.smithi142.stdout:3/988: creat d0/d1b/d6b/dbd/f128 x:0 0 0 2024-06-15T01:07:13.206 INFO:tasks.workunit.client.1.smithi142.stdout:5/976: mknod d2/de/d27/db2/d108/c136 0 2024-06-15T01:07:13.206 INFO:tasks.workunit.client.1.smithi142.stdout:6/965: stat d1a/d3c/d100/d58/f7a 0 2024-06-15T01:07:13.207 INFO:tasks.workunit.client.1.smithi142.stdout:2/955: sync 2024-06-15T01:07:13.208 INFO:tasks.workunit.client.0.smithi137.stdout:7/520: mkdir df/d7d/d3b/d5b/da6 0 2024-06-15T01:07:13.210 INFO:tasks.workunit.client.0.smithi137.stdout:3/925: dwrite d0/d42/ff7 [0,4194304] 0 2024-06-15T01:07:13.212 INFO:tasks.workunit.client.1.smithi142.stdout:7/978: dwrite d2/d8/dd/dcf/f7b [0,4194304] 0 2024-06-15T01:07:13.213 INFO:tasks.workunit.client.0.smithi137.stdout:1/800: creat d0/d3/d1b/d4c/d59/f10c x:0 0 0 2024-06-15T01:07:13.213 INFO:tasks.workunit.client.1.smithi142.stdout:0/946: rename d3/d39/d9a/d9d/dcd/fe4 to d3/d39/d3e/d81/d4c/f145 0 2024-06-15T01:07:13.214 INFO:tasks.workunit.client.1.smithi142.stdout:0/947: dread - d3/d39/d3e/d81/d1b/df6/f112 zero size 2024-06-15T01:07:13.215 INFO:tasks.workunit.client.1.smithi142.stdout:6/966: mknod d1a/d2d/d80/c14f 0 2024-06-15T01:07:13.215 INFO:tasks.workunit.client.0.smithi137.stdout:1/801: dread - d0/d3/d1b/d4c/d59/d93/df1/ffc zero size 2024-06-15T01:07:13.215 INFO:tasks.workunit.client.0.smithi137.stdout:6/562: chown d0/d1a/d3e/d57/d1d/d2e/f36 492 1 2024-06-15T01:07:13.217 INFO:tasks.workunit.client.0.smithi137.stdout:5/651: chown df/d44/d54/l9f 126993 1 2024-06-15T01:07:13.217 INFO:tasks.workunit.client.1.smithi142.stdout:3/989: creat d0/d1f/d20/d59/d61/d96/f129 x:0 0 0 2024-06-15T01:07:13.219 INFO:tasks.workunit.client.1.smithi142.stdout:3/990: chown d0/d3/de2/d75/fa7 57842119 1 2024-06-15T01:07:13.220 INFO:tasks.workunit.client.1.smithi142.stdout:3/991: truncate d0/d95/f112 453065 0 2024-06-15T01:07:13.222 INFO:tasks.workunit.client.0.smithi137.stdout:1/802: dread d0/d3/d14/d34/f52 [0,4194304] 0 2024-06-15T01:07:13.223 INFO:tasks.workunit.client.1.smithi142.stdout:5/977: creat d2/de/d19/d5f/d68/f137 x:0 0 0 2024-06-15T01:07:13.224 INFO:tasks.workunit.client.1.smithi142.stdout:2/956: fdatasync d8/df/d10/d1c/dbb/ff9 0 2024-06-15T01:07:13.225 INFO:tasks.workunit.client.1.smithi142.stdout:2/957: fdatasync d8/d19/d33/f83 0 2024-06-15T01:07:13.231 INFO:tasks.workunit.client.0.smithi137.stdout:1/803: dwrite d0/d3/f13 [4194304,4194304] 0 2024-06-15T01:07:13.232 INFO:tasks.workunit.client.0.smithi137.stdout:6/563: chown d0/d5/l50 29 1 2024-06-15T01:07:13.240 INFO:tasks.workunit.client.0.smithi137.stdout:1/804: dwrite d0/d1f/d60/f55 [4194304,4194304] 0 2024-06-15T01:07:13.243 INFO:tasks.workunit.client.0.smithi137.stdout:0/772: mknod d1/d2/dc/d10/d53/cff 0 2024-06-15T01:07:13.248 INFO:tasks.workunit.client.0.smithi137.stdout:5/652: creat df/d11/fd9 x:0 0 0 2024-06-15T01:07:13.251 INFO:tasks.workunit.client.1.smithi142.stdout:9/902: write d6/dc/d4e/d78/f9d [914807,47097] 0 2024-06-15T01:07:13.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/773: mknod d1/d2/dc/d10/d22/db7/dbd/c100 0 2024-06-15T01:07:13.254 INFO:tasks.workunit.client.0.smithi137.stdout:5/653: fdatasync df/d44/d54/d95/d35/f81 0 2024-06-15T01:07:13.255 INFO:tasks.workunit.client.0.smithi137.stdout:1/805: getdents d0/d1f/d7d 0 2024-06-15T01:07:13.259 INFO:tasks.workunit.client.0.smithi137.stdout:9/501: sync 2024-06-15T01:07:13.261 INFO:tasks.workunit.client.0.smithi137.stdout:1/806: symlink d0/d1f/d76/da1/l10d 0 2024-06-15T01:07:13.262 INFO:tasks.workunit.client.0.smithi137.stdout:9/502: fsync df/d12/d1b/d31/d42/f56 0 2024-06-15T01:07:13.262 INFO:tasks.workunit.client.0.smithi137.stdout:1/807: chown d0/d1f/d67/ccf 36358 1 2024-06-15T01:07:13.265 INFO:tasks.workunit.client.0.smithi137.stdout:8/510: unlink d9/de/d1e/d3f/c64 0 2024-06-15T01:07:13.265 INFO:tasks.workunit.client.1.smithi142.stdout:3/992: write d0/d1f/d20/f26 [8496771,69790] 0 2024-06-15T01:07:13.267 INFO:tasks.workunit.client.0.smithi137.stdout:9/503: mknod df/d12/d15/c8e 0 2024-06-15T01:07:13.268 INFO:tasks.workunit.client.1.smithi142.stdout:0/948: write d3/d39/d3e/d81/d4c/faf [2248842,95296] 0 2024-06-15T01:07:13.268 INFO:tasks.workunit.client.0.smithi137.stdout:1/808: symlink d0/d3/d1b/d4c/d59/dd4/l10e 0 2024-06-15T01:07:13.270 INFO:tasks.workunit.client.0.smithi137.stdout:1/809: write d0/d3/d47/f5d [372833,63082] 0 2024-06-15T01:07:13.273 INFO:tasks.workunit.client.0.smithi137.stdout:9/504: mkdir df/d12/d15/d16/d3c/d51/d8f 0 2024-06-15T01:07:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:8/511: getdents d9/de 0 2024-06-15T01:07:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:8/512: readlink d9/de/l19 0 2024-06-15T01:07:13.293 INFO:tasks.workunit.client.1.smithi142.stdout:2/958: rename d8/df/daf/lb4 to d8/d19/da3/l12e 0 2024-06-15T01:07:13.296 INFO:tasks.workunit.client.1.smithi142.stdout:0/949: fsync d3/d39/d3e/d81/d1b/d52/d13e/d22/d59/f80 0 2024-06-15T01:07:13.298 INFO:tasks.workunit.client.1.smithi142.stdout:7/979: getdents d2/d8/dd/d8a 0 2024-06-15T01:07:13.298 INFO:tasks.workunit.client.1.smithi142.stdout:7/980: read - d2/d8/da/d33/d25/d79/dbc/f132 zero size 2024-06-15T01:07:13.299 INFO:tasks.workunit.client.1.smithi142.stdout:2/959: creat d8/d19/deb/df8/f12f x:0 0 0 2024-06-15T01:07:13.300 INFO:tasks.workunit.client.1.smithi142.stdout:3/993: mkdir d0/d3/da/d18/dd9/d12a 0 2024-06-15T01:07:13.301 INFO:tasks.workunit.client.1.smithi142.stdout:0/950: creat d3/d39/d3e/d81/d1b/d52/d13e/d22/d59/f146 x:0 0 0 2024-06-15T01:07:13.305 INFO:tasks.workunit.client.1.smithi142.stdout:2/960: write d8/d19/d8d/f93 [3632195,82553] 0 2024-06-15T01:07:13.305 INFO:tasks.workunit.client.0.smithi137.stdout:5/654: rmdir df/d44/d54/d95/d3b/d4b 39 2024-06-15T01:07:13.308 INFO:tasks.workunit.client.1.smithi142.stdout:3/994: dwrite d0/d3/d15/f77 [0,4194304] 0 2024-06-15T01:07:13.308 INFO:tasks.workunit.client.0.smithi137.stdout:5/655: mkdir df/d11/d70/dbf/dda 0 2024-06-15T01:07:13.308 INFO:tasks.workunit.client.1.smithi142.stdout:0/951: mknod d3/d39/d9a/da2/c147 0 2024-06-15T01:07:13.310 INFO:tasks.workunit.client.1.smithi142.stdout:7/981: creat d2/d6/f139 x:0 0 0 2024-06-15T01:07:13.311 INFO:tasks.workunit.client.1.smithi142.stdout:2/961: dread d8/df/d10/d94/d25/d22/fbd [0,4194304] 0 2024-06-15T01:07:13.312 INFO:tasks.workunit.client.0.smithi137.stdout:7/521: write df/d13/d2e/f58 [710979,107955] 0 2024-06-15T01:07:13.313 INFO:tasks.workunit.client.0.smithi137.stdout:3/926: write d0/d13/d37/d66/f88 [1989524,77609] 0 2024-06-15T01:07:13.313 INFO:tasks.workunit.client.0.smithi137.stdout:5/656: read df/d7a/fb5 [1398356,110812] 0 2024-06-15T01:07:13.313 INFO:tasks.workunit.client.0.smithi137.stdout:7/522: chown df 31155 1 2024-06-15T01:07:13.314 INFO:tasks.workunit.client.1.smithi142.stdout:0/952: chown d3/d39/d3e/d8f 1647 1 2024-06-15T01:07:13.314 INFO:tasks.workunit.client.1.smithi142.stdout:2/962: read d8/df/d10/d94/d25/f74 [3671656,26066] 0 2024-06-15T01:07:13.315 INFO:tasks.workunit.client.0.smithi137.stdout:5/657: chown df/d44/d54/d95/d35/f81 3459835 1 2024-06-15T01:07:13.315 INFO:tasks.workunit.client.0.smithi137.stdout:5/658: chown df/l1b 30 1 2024-06-15T01:07:13.316 INFO:tasks.workunit.client.1.smithi142.stdout:9/903: dread d6/dc/d1b/f76 [0,4194304] 0 2024-06-15T01:07:13.317 INFO:tasks.workunit.client.0.smithi137.stdout:7/523: stat df/d7d/d1c/f9c 0 2024-06-15T01:07:13.317 INFO:tasks.workunit.client.1.smithi142.stdout:3/995: readlink d0/d3/de2/d55/l68 0 2024-06-15T01:07:13.319 INFO:tasks.workunit.client.0.smithi137.stdout:5/659: fsync df/d44/d54/d95/d35/f3f 0 2024-06-15T01:07:13.320 INFO:tasks.workunit.client.1.smithi142.stdout:3/996: write d0/d1f/d20/d59/d61/fb7 [2179180,34653] 0 2024-06-15T01:07:13.321 INFO:tasks.workunit.client.1.smithi142.stdout:0/953: dwrite d3/d39/d3e/d8f/fd2 [0,4194304] 0 2024-06-15T01:07:13.322 INFO:tasks.workunit.client.0.smithi137.stdout:5/660: stat df/d11/d30/db1/fbb 0 2024-06-15T01:07:13.323 INFO:tasks.workunit.client.1.smithi142.stdout:0/954: chown d3/d39/d3e/d81 429664662 1 2024-06-15T01:07:13.325 INFO:tasks.workunit.client.0.smithi137.stdout:6/564: dwrite d0/d1a/d68/f82 [0,4194304] 0 2024-06-15T01:07:13.325 INFO:tasks.workunit.client.0.smithi137.stdout:0/774: write d1/d2/d13/d66/f3f [1182144,24940] 0 2024-06-15T01:07:13.326 INFO:tasks.workunit.client.1.smithi142.stdout:3/997: dwrite d0/d3/f11a [0,4194304] 0 2024-06-15T01:07:13.327 INFO:tasks.workunit.client.0.smithi137.stdout:4/592: symlink dd/d15/d36/d45/dcb/d83/ldd 0 2024-06-15T01:07:13.328 INFO:tasks.workunit.client.1.smithi142.stdout:2/963: mkdir d8/df/d10/d94/db5/d130 0 2024-06-15T01:07:13.329 INFO:tasks.workunit.client.0.smithi137.stdout:0/775: unlink d1/c20 0 2024-06-15T01:07:13.332 INFO:tasks.workunit.client.1.smithi142.stdout:9/904: creat d6/dc/d51/d11c/f147 x:0 0 0 2024-06-15T01:07:13.334 INFO:tasks.workunit.client.0.smithi137.stdout:5/661: rename df/d76/d7c/d9c/da8/f67 to df/d11/d14/d6c/fdb 0 2024-06-15T01:07:13.336 INFO:tasks.workunit.client.0.smithi137.stdout:5/662: write df/d11/d14/d6c/fce [869073,98930] 0 2024-06-15T01:07:13.338 INFO:tasks.workunit.client.0.smithi137.stdout:5/663: write df/d11/d14/d6c/fce [1965019,77635] 0 2024-06-15T01:07:13.338 INFO:tasks.workunit.client.0.smithi137.stdout:4/593: truncate dd/f28 5082948 0 2024-06-15T01:07:13.339 INFO:tasks.workunit.client.1.smithi142.stdout:6/967: sync 2024-06-15T01:07:13.340 INFO:tasks.workunit.client.1.smithi142.stdout:6/968: dread - d1a/d39/d42/f116 zero size 2024-06-15T01:07:13.340 INFO:tasks.workunit.client.0.smithi137.stdout:5/664: write df/d11/f4c [838069,26228] 0 2024-06-15T01:07:13.341 INFO:tasks.workunit.client.1.smithi142.stdout:0/955: symlink d3/d39/d3e/d81/d1b/d86/l148 0 2024-06-15T01:07:13.342 INFO:tasks.workunit.client.0.smithi137.stdout:0/776: rename d1/d2/dc/d10/d22/db7/df8/d1c/c3d to d1/d2/dc/d10/d14/d56/c101 0 2024-06-15T01:07:13.342 INFO:tasks.workunit.client.1.smithi142.stdout:0/956: write d3/d39/d3e/d81/f103 [503040,55754] 0 2024-06-15T01:07:13.343 INFO:tasks.workunit.client.1.smithi142.stdout:7/982: symlink d2/db6/de6/df8/l13a 0 2024-06-15T01:07:13.345 INFO:tasks.workunit.client.1.smithi142.stdout:9/905: mkdir d6/dc/d1b/d6f/da1/df6/d148 0 2024-06-15T01:07:13.347 INFO:tasks.workunit.client.0.smithi137.stdout:4/594: unlink dd/d15/d36/d45/d55/fa8 0 2024-06-15T01:07:13.348 INFO:tasks.workunit.client.1.smithi142.stdout:6/969: write d1a/d3c/d100/d58/d87/f89 [592764,37028] 0 2024-06-15T01:07:13.349 INFO:tasks.workunit.client.0.smithi137.stdout:9/505: creat df/f90 x:0 0 0 2024-06-15T01:07:13.350 INFO:tasks.workunit.client.0.smithi137.stdout:0/777: dwrite d1/d7e/d87/da0/dc4/dd7/de2/d8d/ffc [0,4194304] 0 2024-06-15T01:07:13.351 INFO:tasks.workunit.client.1.smithi142.stdout:7/983: unlink d2/d8/dd/d14/d2c/d70/l73 0 2024-06-15T01:07:13.353 INFO:tasks.workunit.client.1.smithi142.stdout:2/964: creat d8/df/f131 x:0 0 0 2024-06-15T01:07:13.353 INFO:tasks.workunit.client.0.smithi137.stdout:4/595: dread - dd/d15/d36/d45/fb1 zero size 2024-06-15T01:07:13.355 INFO:tasks.workunit.client.1.smithi142.stdout:0/957: creat d3/d39/d3e/d8f/df0/f149 x:0 0 0 2024-06-15T01:07:13.356 INFO:tasks.workunit.client.0.smithi137.stdout:9/506: write df/d12/d1b/f2f [4656848,28547] 0 2024-06-15T01:07:13.357 INFO:tasks.workunit.client.1.smithi142.stdout:0/958: write d3/d39/d9a/d9d/dcd/f125 [279998,74366] 0 2024-06-15T01:07:13.357 INFO:tasks.workunit.client.1.smithi142.stdout:6/970: creat d1a/d2d/d80/d127/f150 x:0 0 0 2024-06-15T01:07:13.358 INFO:tasks.workunit.client.0.smithi137.stdout:0/778: fdatasync d1/d2/f47 0 2024-06-15T01:07:13.359 INFO:tasks.workunit.client.1.smithi142.stdout:7/984: rename d2/d6/d22/d3c/cde to d2/d8/dd/dcf/c13b 0 2024-06-15T01:07:13.360 INFO:tasks.workunit.client.0.smithi137.stdout:4/596: mknod dd/d15/d36/d45/d55/cde 0 2024-06-15T01:07:13.362 INFO:tasks.workunit.client.1.smithi142.stdout:9/906: link d6/dc/d4e/d78/f101 d6/dc/d1b/d6f/de1/d110/f149 0 2024-06-15T01:07:13.363 INFO:tasks.workunit.client.0.smithi137.stdout:8/513: dwrite d9/de/f13 [0,4194304] 0 2024-06-15T01:07:13.364 INFO:tasks.workunit.client.0.smithi137.stdout:9/507: chown df/d1a/c36 0 1 2024-06-15T01:07:13.365 INFO:tasks.workunit.client.1.smithi142.stdout:6/971: truncate d1a/d2d/f33 1491369 0 2024-06-15T01:07:13.365 INFO:tasks.workunit.client.0.smithi137.stdout:9/508: fdatasync df/d12/d4e/f5c 0 2024-06-15T01:07:13.366 INFO:tasks.workunit.client.0.smithi137.stdout:0/779: rmdir d1/d2/dc/d6c/d88 39 2024-06-15T01:07:13.367 INFO:tasks.workunit.client.1.smithi142.stdout:6/972: read d1a/d39/d3a/d51/d62/f79 [155035,111736] 0 2024-06-15T01:07:13.367 INFO:tasks.workunit.client.1.smithi142.stdout:2/965: mkdir d8/df/d10/d94/d25/da9/d125/d132 0 2024-06-15T01:07:13.368 INFO:tasks.workunit.client.1.smithi142.stdout:6/973: chown d1a/d2d/d80/fee 6636 1 2024-06-15T01:07:13.369 INFO:tasks.workunit.client.0.smithi137.stdout:8/514: mkdir d9/de/d1e/d3f/dab 0 2024-06-15T01:07:13.371 INFO:tasks.workunit.client.1.smithi142.stdout:7/985: mkdir d2/db6/d13c 0 2024-06-15T01:07:13.373 INFO:tasks.workunit.client.1.smithi142.stdout:6/974: mkdir d1a/d2d/d3b/d151 0 2024-06-15T01:07:13.376 INFO:tasks.workunit.client.1.smithi142.stdout:2/966: unlink d8/df/d10/d1c/d7c/cb1 0 2024-06-15T01:07:13.377 INFO:tasks.workunit.client.1.smithi142.stdout:3/998: dread d0/d1f/d20/d5d/d60/fb0 [0,4194304] 0 2024-06-15T01:07:13.378 INFO:tasks.workunit.client.1.smithi142.stdout:6/975: symlink d1a/d2d/d3b/dab/dce/l152 0 2024-06-15T01:07:13.378 INFO:tasks.workunit.client.1.smithi142.stdout:5/978: write d2/de/d19/d5f/d68/d76/d96/dac/f11d [1932842,84028] 0 2024-06-15T01:07:13.379 INFO:tasks.workunit.client.0.smithi137.stdout:8/515: rename d9/de/ff to d9/de/fac 0 2024-06-15T01:07:13.380 INFO:tasks.workunit.client.0.smithi137.stdout:9/509: fdatasync df/d12/d1b/d31/f33 0 2024-06-15T01:07:13.381 INFO:tasks.workunit.client.0.smithi137.stdout:4/597: dread dd/f8e [0,4194304] 0 2024-06-15T01:07:13.382 INFO:tasks.workunit.client.0.smithi137.stdout:2/483: rmdir d0/dc/d22/d83 0 2024-06-15T01:07:13.383 INFO:tasks.workunit.client.0.smithi137.stdout:9/510: truncate df/d49/d72/f82 412686 0 2024-06-15T01:07:13.383 INFO:tasks.workunit.client.1.smithi142.stdout:2/967: mkdir d8/df/d10/d94/d25/da9/d133 0 2024-06-15T01:07:13.383 INFO:tasks.workunit.client.1.smithi142.stdout:3/999: chown d0/d3/de2/d55/f57 1990 1 2024-06-15T01:07:13.385 INFO:tasks.workunit.client.1.smithi142.stdout:6/976: dwrite d1a/d39/d42/f14b [0,4194304] 0 2024-06-15T01:07:13.387 INFO:tasks.workunit.client.1.smithi142.stdout:0/959: dread d3/f9 [0,4194304] 0 2024-06-15T01:07:13.388 INFO:tasks.workunit.client.0.smithi137.stdout:2/484: fdatasync d0/de/d2b/f32 0 2024-06-15T01:07:13.389 INFO:tasks.workunit.client.0.smithi137.stdout:5/665: dread df/d44/d54/d95/f6f [0,4194304] 0 2024-06-15T01:07:13.389 INFO:tasks.workunit.client.1.smithi142.stdout:5/979: fsync d2/d18/fba 0 2024-06-15T01:07:13.390 INFO:tasks.workunit.client.1.smithi142.stdout:5/980: chown d2/d3e/d31/d105 6 1 2024-06-15T01:07:13.390 INFO:tasks.workunit.client.1.smithi142.stdout:2/968: mkdir d8/df/d10/d94/d25/d134 0 2024-06-15T01:07:13.391 INFO:tasks.workunit.client.0.smithi137.stdout:9/511: symlink df/d12/d1b/d31/l91 0 2024-06-15T01:07:13.391 INFO:tasks.workunit.client.1.smithi142.stdout:2/969: readlink d8/d19/d8d/lb2 0 2024-06-15T01:07:13.391 INFO:tasks.workunit.client.1.smithi142.stdout:5/981: read - d2/d3e/d31/d7e/ff6 zero size 2024-06-15T01:07:13.392 INFO:tasks.workunit.client.1.smithi142.stdout:2/970: chown d8/d19/d33/f83 15900 1 2024-06-15T01:07:13.392 INFO:tasks.workunit.client.1.smithi142.stdout:5/982: read - d2/d3e/feb zero size 2024-06-15T01:07:13.392 INFO:tasks.workunit.client.1.smithi142.stdout:6/977: dread d1a/fdc [0,4194304] 0 2024-06-15T01:07:13.394 INFO:tasks.workunit.client.1.smithi142.stdout:0/960: mkdir d3/d39/d3e/d81/d1b/d52/d13e/d123/d14a 0 2024-06-15T01:07:13.394 INFO:tasks.workunit.client.1.smithi142.stdout:0/961: chown d3/d39/d9a/da2/cd3 0 1 2024-06-15T01:07:13.398 INFO:tasks.workunit.client.1.smithi142.stdout:2/971: symlink d8/df/d10/d94/d25/d1a/d63/l135 0 2024-06-15T01:07:13.398 INFO:tasks.workunit.client.1.smithi142.stdout:0/962: mknod d3/d39/d3e/d11e/c14b 0 2024-06-15T01:07:13.399 INFO:tasks.workunit.client.1.smithi142.stdout:2/972: stat d8/df/d10/f3b 0 2024-06-15T01:07:13.402 INFO:tasks.workunit.client.1.smithi142.stdout:5/983: rename d2/de/d27/db2/ce8 to d2/d3e/d65/dd6/c138 0 2024-06-15T01:07:13.403 INFO:tasks.workunit.client.0.smithi137.stdout:4/598: symlink dd/d15/d21/d9c/d75/ldf 0 2024-06-15T01:07:13.403 INFO:tasks.workunit.client.0.smithi137.stdout:9/512: dwrite df/d13/f22 [0,4194304] 0 2024-06-15T01:07:13.405 INFO:tasks.workunit.client.1.smithi142.stdout:6/978: rename c19 to d1a/d2d/d2e/d12f/c153 0 2024-06-15T01:07:13.405 INFO:tasks.workunit.client.0.smithi137.stdout:6/565: write d0/fa6 [1151418,81485] 0 2024-06-15T01:07:13.408 INFO:tasks.workunit.client.0.smithi137.stdout:0/780: dread d1/d2/dc/d10/d14/d56/f82 [0,4194304] 0 2024-06-15T01:07:13.409 INFO:tasks.workunit.client.0.smithi137.stdout:3/927: dwrite d0/d2/d1e/f81 [0,4194304] 0 2024-06-15T01:07:13.412 INFO:tasks.workunit.client.1.smithi142.stdout:6/979: mkdir d1a/d83/d107/d154 0 2024-06-15T01:07:13.413 INFO:tasks.workunit.client.1.smithi142.stdout:6/980: readlink d1a/d83/lfb 0 2024-06-15T01:07:13.418 INFO:tasks.workunit.client.0.smithi137.stdout:5/666: link df/d11/d30/lc6 df/d44/d54/d80/ldc 0 2024-06-15T01:07:13.419 INFO:tasks.workunit.client.0.smithi137.stdout:3/928: chown d0/d2/d23/de2/lfa 16976 1 2024-06-15T01:07:13.419 INFO:tasks.workunit.client.0.smithi137.stdout:0/781: truncate d1/d2/dc/d10/d22/db7/df8/f50 4259344 0 2024-06-15T01:07:13.420 INFO:tasks.workunit.client.1.smithi142.stdout:6/981: readlink d1a/d2d/d80/l9e 0 2024-06-15T01:07:13.421 INFO:tasks.workunit.client.0.smithi137.stdout:4/599: symlink dd/d15/d36/d45/dcb/d83/db2/d72/d81/le0 0 2024-06-15T01:07:13.425 INFO:tasks.workunit.client.1.smithi142.stdout:0/963: link d3/d39/d3e/d81/d44/df1/l12e d3/d39/d9a/l14c 0 2024-06-15T01:07:13.426 INFO:tasks.workunit.client.1.smithi142.stdout:2/973: dread d8/df/d10/d94/d25/d22/d43/f46 [4194304,4194304] 0 2024-06-15T01:07:13.426 INFO:tasks.workunit.client.1.smithi142.stdout:6/982: creat d1a/d3c/d100/d58/d10f/d131/f155 x:0 0 0 2024-06-15T01:07:13.427 INFO:tasks.workunit.client.0.smithi137.stdout:3/929: dwrite d0/d13/d37/d83/fe1 [0,4194304] 0 2024-06-15T01:07:13.428 INFO:tasks.workunit.client.0.smithi137.stdout:3/930: fdatasync d0/d2/d23/f3d 0 2024-06-15T01:07:13.430 INFO:tasks.workunit.client.0.smithi137.stdout:0/782: fdatasync d1/d2/dc/d10/f96 0 2024-06-15T01:07:13.432 INFO:tasks.workunit.client.0.smithi137.stdout:3/931: chown d0/d2/d8/dbd/d106/d2e/d92 1 1 2024-06-15T01:07:13.433 INFO:tasks.workunit.client.0.smithi137.stdout:3/932: chown d0/d2/d8/d67/d7e/lc3 70 1 2024-06-15T01:07:13.434 INFO:tasks.workunit.client.1.smithi142.stdout:2/974: write d8/df/d10/d94/d25/d22/d43/f46 [1175675,59043] 0 2024-06-15T01:07:13.439 INFO:tasks.workunit.client.1.smithi142.stdout:2/975: mknod d8/df/d10/d94/d25/d134/c136 0 2024-06-15T01:07:13.440 INFO:tasks.workunit.client.0.smithi137.stdout:8/516: truncate d9/de/f3c 64737 0 2024-06-15T01:07:13.440 INFO:tasks.workunit.client.1.smithi142.stdout:6/983: dread d1a/d2d/d2e/f31 [0,4194304] 0 2024-06-15T01:07:13.441 INFO:tasks.workunit.client.0.smithi137.stdout:0/783: truncate d1/d2/d13/fc0 609287 0 2024-06-15T01:07:13.442 INFO:tasks.workunit.client.1.smithi142.stdout:9/907: dwrite d6/dc/d1b/f76 [4194304,4194304] 0 2024-06-15T01:07:13.443 INFO:tasks.workunit.client.1.smithi142.stdout:6/984: write d1a/d39/d3a/dd2/ff8 [44420,13853] 0 2024-06-15T01:07:13.444 INFO:tasks.workunit.client.1.smithi142.stdout:2/976: creat d8/d19/d8d/f137 x:0 0 0 2024-06-15T01:07:13.446 INFO:tasks.workunit.client.0.smithi137.stdout:3/933: rename d0/d2/daf to d0/d2/d23/de2/d12e 0 2024-06-15T01:07:13.446 INFO:tasks.workunit.client.0.smithi137.stdout:9/513: link df/f4d df/d12/d15/d16/f92 0 2024-06-15T01:07:13.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/514: fsync df/d12/d1b/f35 0 2024-06-15T01:07:13.453 INFO:tasks.workunit.client.0.smithi137.stdout:9/515: readlink df/d13/l2b 0 2024-06-15T01:07:13.454 INFO:tasks.workunit.client.0.smithi137.stdout:8/517: unlink d9/de/l19 0 2024-06-15T01:07:13.455 INFO:tasks.workunit.client.0.smithi137.stdout:6/566: write d0/d5/f83 [318969,105028] 0 2024-06-15T01:07:13.461 INFO:tasks.workunit.client.0.smithi137.stdout:4/600: dwrite dd/f18 [4194304,4194304] 0 2024-06-15T01:07:13.462 INFO:tasks.workunit.client.0.smithi137.stdout:0/784: rename d1/d2/dc/d10/c42 to d1/d2/dc/d10/d22/db7/df8/c102 0 2024-06-15T01:07:13.464 INFO:tasks.workunit.client.0.smithi137.stdout:6/567: mkdir d0/d1a/d3e/d57/db6 0 2024-06-15T01:07:13.466 INFO:tasks.workunit.client.0.smithi137.stdout:4/601: chown dd/d15/d21/d9c/d75/la5 1906906 1 2024-06-15T01:07:13.467 INFO:tasks.workunit.client.1.smithi142.stdout:7/986: sync 2024-06-15T01:07:13.467 INFO:tasks.workunit.client.1.smithi142.stdout:7/987: fdatasync d2/d6/f139 0 2024-06-15T01:07:13.470 INFO:tasks.workunit.client.0.smithi137.stdout:9/516: getdents df/d12/d1b 0 2024-06-15T01:07:13.470 INFO:tasks.workunit.client.0.smithi137.stdout:0/785: creat d1/d7e/d87/da0/dc4/dd7/de2/d8d/dd9/f103 x:0 0 0 2024-06-15T01:07:13.473 INFO:tasks.workunit.client.0.smithi137.stdout:9/517: stat df/d12/d1b/d31/f67 0 2024-06-15T01:07:13.474 INFO:tasks.workunit.client.0.smithi137.stdout:8/518: rename d9/de/d1e/l92 to d9/lad 0 2024-06-15T01:07:13.477 INFO:tasks.workunit.client.0.smithi137.stdout:5/667: dread df/d44/d54/d95/d3b/d49/d5c/f91 [0,4194304] 0 2024-06-15T01:07:13.479 INFO:tasks.workunit.client.0.smithi137.stdout:6/568: dwrite d0/d1a/d3e/f77 [0,4194304] 0 2024-06-15T01:07:13.482 INFO:tasks.workunit.client.0.smithi137.stdout:3/934: truncate d0/d42/fc9 894263 0 2024-06-15T01:07:13.482 INFO:tasks.workunit.client.0.smithi137.stdout:4/602: dwrite dd/d15/d36/d45/fd7 [0,4194304] 0 2024-06-15T01:07:13.483 INFO:tasks.workunit.client.0.smithi137.stdout:6/569: readlink d0/d1a/d3e/d57/d1d/lab 0 2024-06-15T01:07:13.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/519: unlink d9/d2a/d2f/d59/f87 0 2024-06-15T01:07:13.489 INFO:tasks.workunit.client.0.smithi137.stdout:3/935: read - d0/d42/fcb zero size 2024-06-15T01:07:13.493 INFO:tasks.workunit.client.0.smithi137.stdout:4/603: mknod dd/d15/d36/d45/dcb/d83/db2/d67/ce1 0 2024-06-15T01:07:13.498 INFO:tasks.workunit.client.0.smithi137.stdout:6/570: dwrite d0/d1a/d3e/d57/f5e [0,4194304] 0 2024-06-15T01:07:13.499 INFO:tasks.workunit.client.0.smithi137.stdout:6/571: chown d0/d5/d9/d3f/lb4 1633611344 1 2024-06-15T01:07:13.528 INFO:tasks.workunit.client.0.smithi137.stdout:3/936: mkdir d0/d56/d8f/d12f 0 2024-06-15T01:07:13.528 INFO:tasks.workunit.client.0.smithi137.stdout:6/572: dread d0/d5/d9/d29/d64/f86 [0,4194304] 0 2024-06-15T01:07:13.530 INFO:tasks.workunit.client.1.smithi142.stdout:2/977: rename d8/c20 to d8/df/d10/d94/d25/da9/d125/c138 0 2024-06-15T01:07:13.530 INFO:tasks.workunit.client.1.smithi142.stdout:9/908: link d6/dc/dd/fac d6/dc/d51/f14a 0 2024-06-15T01:07:13.531 INFO:tasks.workunit.client.0.smithi137.stdout:3/937: getdents d0/d13/d37/d12b 0 2024-06-15T01:07:13.533 INFO:tasks.workunit.client.0.smithi137.stdout:3/938: dread - d0/d56/d8f/fa8 zero size 2024-06-15T01:07:13.535 INFO:tasks.workunit.client.1.smithi142.stdout:5/984: dwrite d2/de/d19/f22 [0,4194304] 0 2024-06-15T01:07:13.536 INFO:tasks.workunit.client.1.smithi142.stdout:2/978: rename d8/d19/fc8 to d8/df/daf/f139 0 2024-06-15T01:07:13.536 INFO:tasks.workunit.client.1.smithi142.stdout:5/985: write d2/d3e/d65/fc7 [1763277,48701] 0 2024-06-15T01:07:13.537 INFO:tasks.workunit.client.1.smithi142.stdout:0/964: write d3/f4 [1224393,38180] 0 2024-06-15T01:07:13.538 INFO:tasks.workunit.client.1.smithi142.stdout:2/979: dread - d8/df/d10/d94/d25/d22/d43/f100 zero size 2024-06-15T01:07:13.539 INFO:tasks.workunit.client.1.smithi142.stdout:2/980: readlink d8/df/d10/d94/d25/l78 0 2024-06-15T01:07:13.542 INFO:tasks.workunit.client.1.smithi142.stdout:9/909: truncate d6/dc/d1b/d6f/d103/df2/d13c/d58/f64 2030291 0 2024-06-15T01:07:13.546 INFO:tasks.workunit.client.1.smithi142.stdout:6/985: write f18 [1981545,115889] 0 2024-06-15T01:07:13.547 INFO:tasks.workunit.client.1.smithi142.stdout:9/910: readlink d6/d31/l105 0 2024-06-15T01:07:13.547 INFO:tasks.workunit.client.0.smithi137.stdout:8/520: write d9/d22/f3e [2915604,57053] 0 2024-06-15T01:07:13.547 INFO:tasks.workunit.client.0.smithi137.stdout:5/668: write df/d44/d54/d95/d3b/d4b/fc0 [818451,52324] 0 2024-06-15T01:07:13.547 INFO:tasks.workunit.client.0.smithi137.stdout:0/786: truncate d1/d2/dc/d10/d22/db7/df8/f59 1759557 0 2024-06-15T01:07:13.549 INFO:tasks.workunit.client.1.smithi142.stdout:2/981: creat d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/f13a x:0 0 0 2024-06-15T01:07:13.549 INFO:tasks.workunit.client.0.smithi137.stdout:3/939: dwrite d0/d2f/dbe/fe9 [0,4194304] 0 2024-06-15T01:07:13.550 INFO:tasks.workunit.client.0.smithi137.stdout:8/521: dread - d9/de/d1e/d3f/da5/fa7 zero size 2024-06-15T01:07:13.551 INFO:tasks.workunit.client.1.smithi142.stdout:5/986: mkdir d2/de/d19/d5f/d68/d76/d139 0 2024-06-15T01:07:13.552 INFO:tasks.workunit.client.0.smithi137.stdout:9/518: dread f5 [0,4194304] 0 2024-06-15T01:07:13.553 INFO:tasks.workunit.client.1.smithi142.stdout:6/986: mknod d1a/d39/d42/df7/c156 0 2024-06-15T01:07:13.553 INFO:tasks.workunit.client.0.smithi137.stdout:3/940: fsync d0/d2/d22/d63/f115 0 2024-06-15T01:07:13.554 INFO:tasks.workunit.client.0.smithi137.stdout:4/604: dread dd/f17 [0,4194304] 0 2024-06-15T01:07:13.557 INFO:tasks.workunit.client.0.smithi137.stdout:3/941: readlink d0/ddf/da1/lbb 0 2024-06-15T01:07:13.558 INFO:tasks.workunit.client.0.smithi137.stdout:0/787: mknod d1/d2/dc/d10/d14/d56/c104 0 2024-06-15T01:07:13.558 INFO:tasks.workunit.client.0.smithi137.stdout:9/519: truncate df/d49/d72/f82 1375602 0 2024-06-15T01:07:13.558 INFO:tasks.workunit.client.0.smithi137.stdout:3/942: stat d0/ddf/da1/lbb 0 2024-06-15T01:07:13.559 INFO:tasks.workunit.client.0.smithi137.stdout:3/943: chown d0/d56/d8f/d9c 12590 1 2024-06-15T01:07:13.559 INFO:tasks.workunit.client.0.smithi137.stdout:4/605: creat dd/d15/d36/d45/d55/d8d/fe2 x:0 0 0 2024-06-15T01:07:13.561 INFO:tasks.workunit.client.0.smithi137.stdout:0/788: mkdir d1/d2/dc/d10/d14/d56/d105 0 2024-06-15T01:07:13.561 INFO:tasks.workunit.client.0.smithi137.stdout:4/606: stat dd/d15/d36/d45/dcb/f93 0 2024-06-15T01:07:13.561 INFO:tasks.workunit.client.0.smithi137.stdout:3/944: write d0/ddf/ffd [129429,24788] 0 2024-06-15T01:07:13.561 INFO:tasks.workunit.client.1.smithi142.stdout:6/987: dwrite d1a/d3c/d100/fb3 [4194304,4194304] 0 2024-06-15T01:07:13.563 INFO:tasks.workunit.client.1.smithi142.stdout:6/988: dread - d1a/d2d/d80/d8e/d110/f12d zero size 2024-06-15T01:07:13.563 INFO:tasks.workunit.client.0.smithi137.stdout:5/669: rename df/d44/d54/c93 to df/d44/d54/d95/d3b/d49/cdd 0 2024-06-15T01:07:13.564 INFO:tasks.workunit.client.0.smithi137.stdout:4/607: symlink dd/d15/d21/le3 0 2024-06-15T01:07:13.566 INFO:tasks.workunit.client.0.smithi137.stdout:3/945: creat d0/d2/d23/de2/f130 x:0 0 0 2024-06-15T01:07:13.569 INFO:tasks.workunit.client.0.smithi137.stdout:3/946: write d0/d2/d22/d63/f6d [3923986,111790] 0 2024-06-15T01:07:13.569 INFO:tasks.workunit.client.0.smithi137.stdout:0/789: mkdir d1/d106 0 2024-06-15T01:07:13.576 INFO:tasks.workunit.client.0.smithi137.stdout:0/790: write d1/d2/d55/f6e [1101338,89737] 0 2024-06-15T01:07:13.578 INFO:tasks.workunit.client.0.smithi137.stdout:8/522: rename d9/de/d1e/d67/f7c to d9/fae 0 2024-06-15T01:07:13.579 INFO:tasks.workunit.client.1.smithi142.stdout:7/988: dread d2/d6/f1d [0,4194304] 0 2024-06-15T01:07:13.580 INFO:tasks.workunit.client.0.smithi137.stdout:8/523: write d9/de/d1e/d35/f9f [486591,66091] 0 2024-06-15T01:07:13.583 INFO:tasks.workunit.client.0.smithi137.stdout:0/791: creat d1/d7e/d87/da0/dc4/de4/f107 x:0 0 0 2024-06-15T01:07:13.584 INFO:tasks.workunit.client.0.smithi137.stdout:8/524: mknod d9/de/d1e/d3f/d84/caf 0 2024-06-15T01:07:13.586 INFO:tasks.workunit.client.0.smithi137.stdout:0/792: readlink d1/d7e/lad 0 2024-06-15T01:07:13.589 INFO:tasks.workunit.client.0.smithi137.stdout:8/525: symlink d9/de/d1e/d35/lb0 0 2024-06-15T01:07:13.596 INFO:tasks.workunit.client.0.smithi137.stdout:0/793: dwrite d1/d7e/d87/da0/dc4/dd7/de2/d8d/fde [0,4194304] 0 2024-06-15T01:07:13.596 INFO:tasks.workunit.client.1.smithi142.stdout:5/987: truncate d2/d3e/d65/dd6/ff1 1768495 0 2024-06-15T01:07:13.598 INFO:tasks.workunit.client.1.smithi142.stdout:5/988: truncate d2/de/d27/d86/dbd/fe5 236003 0 2024-06-15T01:07:13.599 INFO:tasks.workunit.client.1.smithi142.stdout:5/989: stat d2/d3e/df2 0 2024-06-15T01:07:13.600 INFO:tasks.workunit.client.1.smithi142.stdout:5/990: chown d2/d18/d2e/dae 17904 1 2024-06-15T01:07:13.601 INFO:tasks.workunit.client.0.smithi137.stdout:8/526: dwrite d9/de/d1e/d3f/da5/fa7 [0,4194304] 0 2024-06-15T01:07:13.608 INFO:tasks.workunit.client.0.smithi137.stdout:8/527: fsync d9/de/f3c 0 2024-06-15T01:07:13.610 INFO:tasks.workunit.client.0.smithi137.stdout:0/794: dwrite d1/d2/dc/ddc/fee [0,4194304] 0 2024-06-15T01:07:13.611 INFO:tasks.workunit.client.1.smithi142.stdout:5/991: dwrite d2/d3e/d98/fbf [0,4194304] 0 2024-06-15T01:07:13.611 INFO:tasks.workunit.client.1.smithi142.stdout:7/989: dread d2/d6/d22/d47/f7c [0,4194304] 0 2024-06-15T01:07:13.615 INFO:tasks.workunit.client.0.smithi137.stdout:3/947: dread d0/d2/d23/d68/f3c [0,4194304] 0 2024-06-15T01:07:13.617 INFO:tasks.workunit.client.0.smithi137.stdout:3/948: dread - d0/d13/d37/d83/f109 zero size 2024-06-15T01:07:13.622 INFO:tasks.workunit.client.0.smithi137.stdout:3/949: symlink d0/d2/d8/dbd/d106/d2e/d92/d125/l131 0 2024-06-15T01:07:13.624 INFO:tasks.workunit.client.0.smithi137.stdout:3/950: write d0/d56/d8f/f112 [118173,120411] 0 2024-06-15T01:07:13.625 INFO:tasks.workunit.client.0.smithi137.stdout:8/528: dread d9/d2a/f69 [0,4194304] 0 2024-06-15T01:07:13.628 INFO:tasks.workunit.client.0.smithi137.stdout:4/608: dread f5 [0,4194304] 0 2024-06-15T01:07:13.632 INFO:tasks.workunit.client.0.smithi137.stdout:4/609: stat dd/c87 0 2024-06-15T01:07:13.632 INFO:tasks.workunit.client.0.smithi137.stdout:3/951: mkdir d0/d2/d8/d67/d7e/de3/d132 0 2024-06-15T01:07:13.634 INFO:tasks.workunit.client.0.smithi137.stdout:8/529: mkdir d9/de/d1e/db1 0 2024-06-15T01:07:13.634 INFO:tasks.workunit.client.0.smithi137.stdout:4/610: chown dd/d15/d36/d45/dcb/d83/lc7 32 1 2024-06-15T01:07:13.636 INFO:tasks.workunit.client.0.smithi137.stdout:5/670: dwrite df/d11/d14/f43 [0,4194304] 0 2024-06-15T01:07:13.636 INFO:tasks.workunit.client.0.smithi137.stdout:3/952: chown d0/d56/d8f/d7b 1585 1 2024-06-15T01:07:13.641 INFO:tasks.workunit.client.0.smithi137.stdout:5/671: mknod df/d44/d54/d80/cde 0 2024-06-15T01:07:13.642 INFO:tasks.workunit.client.0.smithi137.stdout:8/530: mknod d9/de/d1e/db1/cb2 0 2024-06-15T01:07:13.646 INFO:tasks.workunit.client.0.smithi137.stdout:5/672: creat df/d76/d8e/fdf x:0 0 0 2024-06-15T01:07:13.647 INFO:tasks.workunit.client.0.smithi137.stdout:4/611: getdents dd/d15/d21/d3a/d6e/d7b/d96 0 2024-06-15T01:07:13.654 INFO:tasks.workunit.client.0.smithi137.stdout:5/673: rename df/d44/d54/cc1 to df/d44/d54/d80/ce0 0 2024-06-15T01:07:13.658 INFO:tasks.workunit.client.0.smithi137.stdout:5/674: creat df/d44/d54/d95/d3b/d4b/fe1 x:0 0 0 2024-06-15T01:07:13.665 INFO:tasks.workunit.client.0.smithi137.stdout:0/795: dread d1/d2/dc/d10/d22/db7/df8/d17/f18 [0,4194304] 0 2024-06-15T01:07:13.665 INFO:tasks.workunit.client.1.smithi142.stdout:6/989: mkdir d1a/d2d/d2e/d12f/d157 0 2024-06-15T01:07:13.671 INFO:tasks.workunit.client.1.smithi142.stdout:7/990: rmdir d2/d8/dd/d8a/d10e 39 2024-06-15T01:07:13.671 INFO:tasks.workunit.client.0.smithi137.stdout:3/953: write d0/d2/d8/dbd/ff6 [915829,123595] 0 2024-06-15T01:07:13.679 INFO:tasks.workunit.client.0.smithi137.stdout:5/675: write df/d44/d54/d95/d3b/d4b/fe1 [598203,111662] 0 2024-06-15T01:07:13.679 INFO:tasks.workunit.client.0.smithi137.stdout:4/612: dwrite dd/d15/d21/d9c/d75/fc2 [0,4194304] 0 2024-06-15T01:07:13.681 INFO:tasks.workunit.client.0.smithi137.stdout:3/954: rename d0/d2/d8/d67/d7e/f53 to d0/d4c/f133 0 2024-06-15T01:07:13.681 INFO:tasks.workunit.client.0.smithi137.stdout:0/796: truncate d1/d2/dc/d10/d14/d56/ffd 719652 0 2024-06-15T01:07:13.684 INFO:tasks.workunit.client.0.smithi137.stdout:4/613: fdatasync dd/d15/d36/d45/d55/d8d/d1f/d86/fab 0 2024-06-15T01:07:13.685 INFO:tasks.workunit.client.0.smithi137.stdout:0/797: chown d1/d2/dc/d6c/d88/d91 6316 1 2024-06-15T01:07:13.688 INFO:tasks.workunit.client.0.smithi137.stdout:0/798: unlink d1/cb4 0 2024-06-15T01:07:13.689 INFO:tasks.workunit.client.0.smithi137.stdout:0/799: chown d1/f32 0 1 2024-06-15T01:07:13.694 INFO:tasks.workunit.client.0.smithi137.stdout:0/800: fdatasync d1/f9 0 2024-06-15T01:07:13.696 INFO:tasks.workunit.client.1.smithi142.stdout:6/990: getdents d1a/d3c 0 2024-06-15T01:07:13.697 INFO:tasks.workunit.client.1.smithi142.stdout:6/991: dread - d1a/d2d/d80/d127/f150 zero size 2024-06-15T01:07:13.698 INFO:tasks.workunit.client.0.smithi137.stdout:3/955: creat d0/d77/f134 x:0 0 0 2024-06-15T01:07:13.700 INFO:tasks.workunit.client.0.smithi137.stdout:3/956: dread - d0/d2/d22/d24/d5e/f116 zero size 2024-06-15T01:07:13.703 INFO:tasks.workunit.client.1.smithi142.stdout:6/992: dwrite d1a/f2c [4194304,4194304] 0 2024-06-15T01:07:13.704 INFO:tasks.workunit.client.0.smithi137.stdout:0/801: dwrite d1/d2/dc/d10/d22/db7/df8/d17/d54/d7a/f7d [0,4194304] 0 2024-06-15T01:07:13.706 INFO:tasks.workunit.client.0.smithi137.stdout:3/957: mkdir d0/d2/d8/dbd/d106/d2e/df0/dff/d135 0 2024-06-15T01:07:13.709 INFO:tasks.workunit.client.0.smithi137.stdout:0/802: creat d1/d2/dc/d10/d22/db7/dbd/f108 x:0 0 0 2024-06-15T01:07:13.710 INFO:tasks.workunit.client.0.smithi137.stdout:3/958: mknod d0/d2/d1e/c136 0 2024-06-15T01:07:13.719 INFO:tasks.workunit.client.0.smithi137.stdout:0/803: creat d1/d2/dc/d10/d14/d56/f109 x:0 0 0 2024-06-15T01:07:13.725 INFO:tasks.workunit.client.0.smithi137.stdout:4/614: dwrite dd/d15/d21/d9c/d75/fae [0,4194304] 0 2024-06-15T01:07:13.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:13 smithi137 ceph-mon[29354]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 135 MiB/s rd, 394 MiB/s wr, 762 op/s 2024-06-15T01:07:13.725 INFO:tasks.workunit.client.0.smithi137.stdout:3/959: dwrite d0/d13/d37/d66/fae [0,4194304] 0 2024-06-15T01:07:13.732 INFO:tasks.workunit.client.0.smithi137.stdout:4/615: creat dd/d15/d36/d45/dcb/d83/db2/d67/dbe/fe4 x:0 0 0 2024-06-15T01:07:13.740 INFO:tasks.workunit.client.0.smithi137.stdout:0/804: write d1/d2/dc/d10/d14/d56/f82 [3946786,9889] 0 2024-06-15T01:07:13.741 INFO:tasks.workunit.client.0.smithi137.stdout:0/805: readlink d1/d2/dc/d6c/lf7 0 2024-06-15T01:07:13.744 INFO:tasks.workunit.client.0.smithi137.stdout:0/806: chown d1/d7e/d87/da0/le9 13 1 2024-06-15T01:07:13.750 INFO:tasks.workunit.client.0.smithi137.stdout:0/807: creat d1/d2/dc/d6c/f10a x:0 0 0 2024-06-15T01:07:13.753 INFO:tasks.workunit.client.0.smithi137.stdout:0/808: rmdir d1/d7e/d87/da0/dc4/df4 0 2024-06-15T01:07:13.753 INFO:tasks.workunit.client.0.smithi137.stdout:0/809: chown d1/df6 28 1 2024-06-15T01:07:13.755 INFO:tasks.workunit.client.1.smithi142.stdout:7/991: write d2/f2e [1579206,27361] 0 2024-06-15T01:07:13.755 INFO:tasks.workunit.client.0.smithi137.stdout:0/810: symlink d1/df6/l10b 0 2024-06-15T01:07:13.756 INFO:tasks.workunit.client.1.smithi142.stdout:7/992: write d2/d8/dd/f76 [2847200,117604] 0 2024-06-15T01:07:13.757 INFO:tasks.workunit.client.1.smithi142.stdout:5/992: truncate d2/d3e/d98/f115 4038712 0 2024-06-15T01:07:13.758 INFO:tasks.workunit.client.0.smithi137.stdout:0/811: creat d1/d2/dc/d10/d53/da2/f10c x:0 0 0 2024-06-15T01:07:13.760 INFO:tasks.workunit.client.0.smithi137.stdout:0/812: chown d1/d2/dc/d10/d14/d56/c101 14271 1 2024-06-15T01:07:13.761 INFO:tasks.workunit.client.0.smithi137.stdout:0/813: chown d1/d2/dc/d10/d14/d52/ff9 0 1 2024-06-15T01:07:13.763 INFO:tasks.workunit.client.0.smithi137.stdout:3/960: write d0/d42/fcb [984011,5297] 0 2024-06-15T01:07:13.766 INFO:tasks.workunit.client.0.smithi137.stdout:0/814: mknod d1/d2/dc/d10/d22/db7/df8/d17/d54/d7a/c10d 0 2024-06-15T01:07:13.767 INFO:tasks.workunit.client.1.smithi142.stdout:0/965: rename d3/d39/d3e/d81/d1e/d98/le3 to d3/d39/d3e/d81/l14d 0 2024-06-15T01:07:13.770 INFO:tasks.workunit.client.1.smithi142.stdout:9/911: rename d6/dc/d1b/d6f/d103/df2/d13c/d58/d74/l126 to d6/dc/d1b/d6f/da1/l14b 0 2024-06-15T01:07:13.772 INFO:tasks.workunit.client.0.smithi137.stdout:3/961: dwrite d0/d2f/dbe/fe0 [0,4194304] 0 2024-06-15T01:07:13.773 INFO:tasks.workunit.client.1.smithi142.stdout:9/912: chown d6/d34/d8a/f12f 1 1 2024-06-15T01:07:13.775 INFO:tasks.workunit.client.0.smithi137.stdout:4/616: dwrite dd/f28 [0,4194304] 0 2024-06-15T01:07:13.775 INFO:tasks.workunit.client.0.smithi137.stdout:3/962: fsync d0/d2/d8/dbd/fd8 0 2024-06-15T01:07:13.776 INFO:tasks.workunit.client.1.smithi142.stdout:9/913: creat d6/d34/ddd/f14c x:0 0 0 2024-06-15T01:07:13.777 INFO:tasks.workunit.client.1.smithi142.stdout:9/914: chown d6/dc/d1b/d6f/d103/df2/d13c/d35/ff1 0 1 2024-06-15T01:07:13.777 INFO:tasks.workunit.client.1.smithi142.stdout:2/982: link d8/df/d1f/cc5 d8/c13b 0 2024-06-15T01:07:13.778 INFO:tasks.workunit.client.1.smithi142.stdout:9/915: readlink d6/d34/d8a/lb5 0 2024-06-15T01:07:13.778 INFO:tasks.workunit.client.0.smithi137.stdout:3/963: truncate d0/d2/d8/f7f 636478 0 2024-06-15T01:07:13.779 INFO:tasks.workunit.client.1.smithi142.stdout:2/983: dread d8/df/d10/ffb [0,4194304] 0 2024-06-15T01:07:13.782 INFO:tasks.workunit.client.0.smithi137.stdout:4/617: mkdir dd/d15/d36/d45/d8f/dac/de5 0 2024-06-15T01:07:13.785 INFO:tasks.workunit.client.0.smithi137.stdout:3/964: creat d0/d2/f137 x:0 0 0 2024-06-15T01:07:13.785 INFO:tasks.workunit.client.0.smithi137.stdout:4/618: truncate dd/d15/d30/f46 651521 0 2024-06-15T01:07:13.789 INFO:tasks.workunit.client.0.smithi137.stdout:4/619: rmdir dd/d15/d36/d45/d55/d8d 39 2024-06-15T01:07:13.790 INFO:tasks.workunit.client.0.smithi137.stdout:4/620: chown dd/d15/d36/d45/d55/fd4 104808 1 2024-06-15T01:07:13.792 INFO:tasks.workunit.client.0.smithi137.stdout:0/815: dread d1/d2/f21 [0,4194304] 0 2024-06-15T01:07:13.794 INFO:tasks.workunit.client.0.smithi137.stdout:4/621: mknod dd/d15/d36/d45/d55/d8d/ce6 0 2024-06-15T01:07:13.795 INFO:tasks.workunit.client.0.smithi137.stdout:4/622: stat dd/d15/d21/d3a/f9f 0 2024-06-15T01:07:13.797 INFO:tasks.workunit.client.0.smithi137.stdout:0/816: rename d1/fb6 to d1/d106/f10e 0 2024-06-15T01:07:13.801 INFO:tasks.workunit.client.0.smithi137.stdout:3/965: dwrite d0/d2/d23/de2/d12e/fe6 [0,4194304] 0 2024-06-15T01:07:13.802 INFO:tasks.workunit.client.1.smithi142.stdout:6/993: creat d1a/d2d/f158 x:0 0 0 2024-06-15T01:07:13.802 INFO:tasks.workunit.client.1.smithi142.stdout:9/916: rename d6/dc/dd/f4f to d6/dc/d1b/d6f/d103/df2/d140/f14d 0 2024-06-15T01:07:13.803 INFO:tasks.workunit.client.0.smithi137.stdout:3/966: fsync d0/d2/f15 0 2024-06-15T01:07:13.809 INFO:tasks.workunit.client.0.smithi137.stdout:3/967: mknod d0/d56/d8f/d9c/d9f/c138 0 2024-06-15T01:07:13.810 INFO:tasks.workunit.client.0.smithi137.stdout:0/817: link d1/d2/dc/d10/d14/d56/ffd d1/d2/dc/d10/d14/d56/f10f 0 2024-06-15T01:07:13.813 INFO:tasks.workunit.client.0.smithi137.stdout:3/968: symlink d0/ddf/l139 0 2024-06-15T01:07:13.814 INFO:tasks.workunit.client.0.smithi137.stdout:3/969: stat d0/d2/d1e/f81 0 2024-06-15T01:07:13.814 INFO:tasks.workunit.client.0.smithi137.stdout:0/818: creat d1/d7e/d87/da0/dc4/dd7/de2/f110 x:0 0 0 2024-06-15T01:07:13.815 INFO:tasks.workunit.client.0.smithi137.stdout:3/970: fsync d0/d2/d22/d24/f54 0 2024-06-15T01:07:13.816 INFO:tasks.workunit.client.0.smithi137.stdout:3/971: readlink d0/d2/d23/ld6 0 2024-06-15T01:07:13.816 INFO:tasks.workunit.client.0.smithi137.stdout:3/972: fdatasync d0/d42/f60 0 2024-06-15T01:07:13.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:13 smithi142 ceph-mon[37379]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 135 MiB/s rd, 394 MiB/s wr, 762 op/s 2024-06-15T01:07:13.818 INFO:tasks.workunit.client.0.smithi137.stdout:0/819: getdents d1/d2/dc/d10/d14/d56/d105 0 2024-06-15T01:07:13.819 INFO:tasks.workunit.client.0.smithi137.stdout:4/623: dread f8 [0,4194304] 0 2024-06-15T01:07:13.819 INFO:tasks.workunit.client.0.smithi137.stdout:3/973: mkdir d0/d2/d22/d63/d76/d13a 0 2024-06-15T01:07:13.821 INFO:tasks.workunit.client.0.smithi137.stdout:3/974: chown d0/d2/d8/d67/d7e/de3/f10c 183 1 2024-06-15T01:07:13.824 INFO:tasks.workunit.client.0.smithi137.stdout:4/624: mknod dd/d15/d21/d3a/ce7 0 2024-06-15T01:07:13.825 INFO:tasks.workunit.client.0.smithi137.stdout:0/820: creat d1/d2/dc/d10/d22/db7/de3/f111 x:0 0 0 2024-06-15T01:07:13.826 INFO:tasks.workunit.client.0.smithi137.stdout:0/821: stat d1/d2/dc/d10/d53/lb8 0 2024-06-15T01:07:13.827 INFO:tasks.workunit.client.0.smithi137.stdout:3/975: mknod d0/d2/d8/dbd/d106/d2e/df0/c13b 0 2024-06-15T01:07:13.829 INFO:tasks.workunit.client.0.smithi137.stdout:4/625: symlink dd/d15/d21/d3a/d6e/d7b/le8 0 2024-06-15T01:07:13.831 INFO:tasks.workunit.client.0.smithi137.stdout:0/822: creat d1/d2/dc/d10/d22/db7/f112 x:0 0 0 2024-06-15T01:07:13.832 INFO:tasks.workunit.client.0.smithi137.stdout:3/976: creat d0/d2/d8/d67/d7e/f13c x:0 0 0 2024-06-15T01:07:13.834 INFO:tasks.workunit.client.0.smithi137.stdout:0/823: fsync d1/d2/d13/d66/f4a 0 2024-06-15T01:07:13.835 INFO:tasks.workunit.client.0.smithi137.stdout:0/824: chown d1/d2/dc/d10/d14/l23 250 1 2024-06-15T01:07:13.836 INFO:tasks.workunit.client.0.smithi137.stdout:4/626: truncate dd/d15/d36/d45/d55/f95 3941237 0 2024-06-15T01:07:13.836 INFO:tasks.workunit.client.1.smithi142.stdout:6/994: mknod d1a/d39/d3a/dd2/c159 0 2024-06-15T01:07:13.839 INFO:tasks.workunit.client.1.smithi142.stdout:9/917: creat d6/f14e x:0 0 0 2024-06-15T01:07:13.841 INFO:tasks.workunit.client.1.smithi142.stdout:6/995: rmdir d1a/d3c/d100 39 2024-06-15T01:07:13.841 INFO:tasks.workunit.client.1.smithi142.stdout:5/993: dwrite d2/de/d27/d86/db9/fd4 [0,4194304] 0 2024-06-15T01:07:13.842 INFO:tasks.workunit.client.1.smithi142.stdout:7/993: dwrite d2/d8/f84 [0,4194304] 0 2024-06-15T01:07:13.844 INFO:tasks.workunit.client.0.smithi137.stdout:4/627: dread - dd/d15/d36/d45/d55/d8d/fe2 zero size 2024-06-15T01:07:13.846 INFO:tasks.workunit.client.1.smithi142.stdout:9/918: fsync d6/d34/ddd/fa5 0 2024-06-15T01:07:13.847 INFO:tasks.workunit.client.1.smithi142.stdout:2/984: dwrite d8/d19/d8d/f137 [0,4194304] 0 2024-06-15T01:07:13.848 INFO:tasks.workunit.client.1.smithi142.stdout:5/994: truncate d2/de/d27/d86/dbd/fdf 755831 0 2024-06-15T01:07:13.850 INFO:tasks.workunit.client.1.smithi142.stdout:7/994: mkdir d2/db6/d13c/d13d 0 2024-06-15T01:07:13.850 INFO:tasks.workunit.client.1.smithi142.stdout:5/995: dread - d2/de/d19/d5f/fd2 zero size 2024-06-15T01:07:13.851 INFO:tasks.workunit.client.1.smithi142.stdout:6/996: mkdir d1a/d39/d3a/d51/d81/d122/d15a 0 2024-06-15T01:07:13.852 INFO:tasks.workunit.client.1.smithi142.stdout:9/919: mknod d6/dc/d51/c14f 0 2024-06-15T01:07:13.856 INFO:tasks.workunit.client.1.smithi142.stdout:2/985: rename d8/d19/d8d to d8/df/daf/d123/d13c 0 2024-06-15T01:07:13.860 INFO:tasks.workunit.client.1.smithi142.stdout:6/997: mkdir d1a/d39/d3a/d51/d81/d122/d15b 0 2024-06-15T01:07:13.862 INFO:tasks.workunit.client.1.smithi142.stdout:9/920: mknod d6/dc/d51/c150 0 2024-06-15T01:07:13.863 INFO:tasks.workunit.client.0.smithi137.stdout:0/825: rename d1/d7e/d87/da0/dc4/de4/f107 to d1/d2/dc/d10/d14/d56/d105/f113 0 2024-06-15T01:07:13.864 INFO:tasks.workunit.client.1.smithi142.stdout:2/986: rmdir d8/df/d10/d94/db5 39 2024-06-15T01:07:13.865 INFO:tasks.workunit.client.0.smithi137.stdout:4/628: creat dd/d15/d21/fe9 x:0 0 0 2024-06-15T01:07:13.865 INFO:tasks.workunit.client.1.smithi142.stdout:9/921: write d6/dc/f10c [257382,59582] 0 2024-06-15T01:07:13.865 INFO:tasks.workunit.client.0.smithi137.stdout:4/629: stat f9 0 2024-06-15T01:07:13.865 INFO:tasks.workunit.client.0.smithi137.stdout:3/977: dread d0/d2/d22/d63/f6d [0,4194304] 0 2024-06-15T01:07:13.866 INFO:tasks.workunit.client.0.smithi137.stdout:0/826: creat d1/d2/dc/d10/d53/da2/f114 x:0 0 0 2024-06-15T01:07:13.867 INFO:tasks.workunit.client.0.smithi137.stdout:3/978: creat d0/d2/d23/de2/f13d x:0 0 0 2024-06-15T01:07:13.868 INFO:tasks.workunit.client.0.smithi137.stdout:3/979: chown d0/d56/d8f/fa8 152679 1 2024-06-15T01:07:13.869 INFO:tasks.workunit.client.1.smithi142.stdout:6/998: dwrite d1a/d2d/d80/fee [4194304,4194304] 0 2024-06-15T01:07:13.869 INFO:tasks.workunit.client.0.smithi137.stdout:4/630: truncate dd/d15/d36/d45/d55/f82 1102610 0 2024-06-15T01:07:13.870 INFO:tasks.workunit.client.1.smithi142.stdout:9/922: creat d6/dc/dd/d22/f151 x:0 0 0 2024-06-15T01:07:13.870 INFO:tasks.workunit.client.1.smithi142.stdout:6/999: truncate d1a/d39/d42/df7/f148 394252 0 2024-06-15T01:07:13.871 INFO:tasks.workunit.client.1.smithi142.stdout:5/996: getdents d2/de/d27/db2 0 2024-06-15T01:07:13.871 INFO:tasks.workunit.client.1.smithi142.stdout:9/923: write d6/d34/d8a/dec/ffc [736688,21647] 0 2024-06-15T01:07:13.871 INFO:tasks.workunit.client.1.smithi142.stdout:5/997: chown d2/d3e/d65/d130 2174460 1 2024-06-15T01:07:13.873 INFO:tasks.workunit.client.1.smithi142.stdout:5/998: chown d2/de/d27/db2/ffc 50921 1 2024-06-15T01:07:13.875 INFO:tasks.workunit.client.1.smithi142.stdout:2/987: symlink d8/df/d10/d94/db5/d130/l13d 0 2024-06-15T01:07:13.878 INFO:tasks.workunit.client.1.smithi142.stdout:5/999: dread d2/d3e/d31/fc3 [0,4194304] 0 2024-06-15T01:07:13.878 INFO:tasks.workunit.client.1.smithi142.stdout:7/995: dwrite d2/d6/d22/d3c/fc1 [0,4194304] 0 2024-06-15T01:07:13.880 INFO:tasks.workunit.client.1.smithi142.stdout:9/924: creat d6/dc/d1b/d6f/f152 x:0 0 0 2024-06-15T01:07:13.880 INFO:tasks.workunit.client.1.smithi142.stdout:7/996: truncate d2/d8/da/d33/d25/d79/dbc/f132 457308 0 2024-06-15T01:07:13.881 INFO:tasks.workunit.client.1.smithi142.stdout:9/925: chown d6/dc/dd/c68 3 1 2024-06-15T01:07:13.884 INFO:tasks.workunit.client.1.smithi142.stdout:9/926: symlink d6/d34/df3/d116/l153 0 2024-06-15T01:07:13.887 INFO:tasks.workunit.client.1.smithi142.stdout:9/927: write d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f106 [893876,41970] 0 2024-06-15T01:07:13.887 INFO:tasks.workunit.client.1.smithi142.stdout:7/997: dwrite d2/d6/d22/d47/f7a [0,4194304] 0 2024-06-15T01:07:13.891 INFO:tasks.workunit.client.1.smithi142.stdout:7/998: creat d2/d8/dd/d14/d2c/d37/d6b/d116/f13e x:0 0 0 2024-06-15T01:07:13.893 INFO:tasks.workunit.client.1.smithi142.stdout:9/928: dwrite d6/dc/d82/f13a [0,4194304] 0 2024-06-15T01:07:13.894 INFO:tasks.workunit.client.1.smithi142.stdout:7/999: mkdir d2/d6/daf/dce/d13f 0 2024-06-15T01:07:13.901 INFO:tasks.workunit.client.1.smithi142.stdout:2/988: dread d8/f40 [0,4194304] 0 2024-06-15T01:07:13.924 INFO:tasks.workunit.client.0.smithi137.stdout:3/980: creat d0/d2/d22/d63/f13e x:0 0 0 2024-06-15T01:07:13.925 INFO:tasks.workunit.client.0.smithi137.stdout:3/981: write d0/d13/d37/d66/f88 [2822494,79916] 0 2024-06-15T01:07:13.927 INFO:tasks.workunit.client.0.smithi137.stdout:3/982: stat d0/d2/d8/dbd/d106/d2e 0 2024-06-15T01:07:13.927 INFO:tasks.workunit.client.0.smithi137.stdout:4/631: read - dd/d15/d36/d45/dcb/f85 zero size 2024-06-15T01:07:13.928 INFO:tasks.workunit.client.0.smithi137.stdout:4/632: stat dd/d15/d30 0 2024-06-15T01:07:13.930 INFO:tasks.workunit.client.0.smithi137.stdout:4/633: dread - dd/d15/d21/d9c/d75/d99/fa9 zero size 2024-06-15T01:07:13.931 INFO:tasks.workunit.client.0.smithi137.stdout:3/983: mkdir d0/d2/d23/de2/d12e/d13f 0 2024-06-15T01:07:13.932 INFO:tasks.workunit.client.0.smithi137.stdout:4/634: rmdir dd/d15/d36/d45/d8f 39 2024-06-15T01:07:13.933 INFO:tasks.workunit.client.1.smithi142.stdout:0/966: sync 2024-06-15T01:07:13.933 INFO:tasks.workunit.client.0.smithi137.stdout:4/635: chown dd/d15/c6a 6143798 1 2024-06-15T01:07:13.935 INFO:tasks.workunit.client.0.smithi137.stdout:4/636: chown dd/d15/c2d 51758142 1 2024-06-15T01:07:13.938 INFO:tasks.workunit.client.1.smithi142.stdout:0/967: link d3/d39/d9a/d9d/f9e d3/d39/d9a/d9d/d13f/f14e 0 2024-06-15T01:07:13.940 INFO:tasks.workunit.client.0.smithi137.stdout:4/637: mkdir dd/d15/d36/d45/dcb/d83/db2/dea 0 2024-06-15T01:07:13.940 INFO:tasks.workunit.client.0.smithi137.stdout:4/638: chown dd/d15 24608111 1 2024-06-15T01:07:13.943 INFO:tasks.workunit.client.1.smithi142.stdout:0/968: rename d3/d39/d3e/d81/d1e/f2b to d3/d39/d3e/d81/d1b/d52/d13e/d22/d59/f14f 0 2024-06-15T01:07:13.946 INFO:tasks.workunit.client.0.smithi137.stdout:0/827: dread d1/d2/d13/f40 [0,4194304] 0 2024-06-15T01:07:13.946 INFO:tasks.workunit.client.1.smithi142.stdout:0/969: creat d3/d39/d3e/d81/d1b/d38/da3/f150 x:0 0 0 2024-06-15T01:07:13.954 INFO:tasks.workunit.client.0.smithi137.stdout:0/828: dwrite d1/d2/dc/d10/d53/da2/f114 [0,4194304] 0 2024-06-15T01:07:13.954 INFO:tasks.workunit.client.1.smithi142.stdout:9/929: truncate d6/dc/d1b/d6f/d103/df2/d13c/d58/d74/f92 54952 0 2024-06-15T01:07:13.958 INFO:tasks.workunit.client.0.smithi137.stdout:0/829: truncate d1/d2/dc/d10/d53/da2/f114 4675874 0 2024-06-15T01:07:13.958 INFO:tasks.workunit.client.1.smithi142.stdout:2/989: write d8/df/d10/f11 [1232876,123524] 0 2024-06-15T01:07:13.958 INFO:tasks.workunit.client.0.smithi137.stdout:0/830: readlink d1/lbb 0 2024-06-15T01:07:13.959 INFO:tasks.workunit.client.1.smithi142.stdout:2/990: dread d8/f40 [4194304,4194304] 0 2024-06-15T01:07:13.960 INFO:tasks.workunit.client.1.smithi142.stdout:9/930: creat d6/dc/d1b/d6f/f154 x:0 0 0 2024-06-15T01:07:13.961 INFO:tasks.workunit.client.0.smithi137.stdout:0/831: creat d1/d2/d55/f115 x:0 0 0 2024-06-15T01:07:13.962 INFO:tasks.workunit.client.1.smithi142.stdout:2/991: write d8/df/d10/d94/d25/fda [381569,20920] 0 2024-06-15T01:07:13.962 INFO:tasks.workunit.client.0.smithi137.stdout:0/832: dread - d1/d7e/d87/da0/dc4/dd7/de2/f110 zero size 2024-06-15T01:07:13.963 INFO:tasks.workunit.client.0.smithi137.stdout:4/639: dread dd/d15/d36/d45/f4b [0,4194304] 0 2024-06-15T01:07:13.964 INFO:tasks.workunit.client.0.smithi137.stdout:0/833: fsync d1/d2/d55/faf 0 2024-06-15T01:07:13.965 INFO:tasks.workunit.client.1.smithi142.stdout:2/992: stat d8/df/d10/d94/d25/d22/d43/d52/dc1/f11b 0 2024-06-15T01:07:13.966 INFO:tasks.workunit.client.0.smithi137.stdout:0/834: read - d1/d2/dc/d10/d22/db7/f112 zero size 2024-06-15T01:07:13.967 INFO:tasks.workunit.client.1.smithi142.stdout:0/970: dwrite d3/d39/d3e/d81/d1b/d52/d13e/d22/f2d [0,4194304] 0 2024-06-15T01:07:13.968 INFO:tasks.workunit.client.1.smithi142.stdout:0/971: chown d3/d39/f138 3871132 1 2024-06-15T01:07:13.969 INFO:tasks.workunit.client.0.smithi137.stdout:4/640: rename dd/c53 to dd/d15/d36/d45/d8f/dac/de5/ceb 0 2024-06-15T01:07:13.970 INFO:tasks.workunit.client.0.smithi137.stdout:4/641: chown dd/d15/d36/d45/d8f 1786 1 2024-06-15T01:07:13.972 INFO:tasks.workunit.client.1.smithi142.stdout:2/993: rename d8/df/d10/d94/d25/d22/d43/d52/d7a/d98/lab to d8/df/daf/d123/d13c/l13e 0 2024-06-15T01:07:13.973 INFO:tasks.workunit.client.0.smithi137.stdout:4/642: write dd/d15/d21/f29 [2637162,123980] 0 2024-06-15T01:07:13.976 INFO:tasks.workunit.client.1.smithi142.stdout:0/972: creat d3/f151 x:0 0 0 2024-06-15T01:07:13.978 INFO:tasks.workunit.client.1.smithi142.stdout:2/994: dread d8/df/d10/f9d [0,4194304] 0 2024-06-15T01:07:13.982 INFO:tasks.workunit.client.0.smithi137.stdout:3/984: dwrite d0/d2/d22/d24/d5e/f85 [0,4194304] 0 2024-06-15T01:07:13.985 INFO:tasks.workunit.client.1.smithi142.stdout:0/973: rename d3/d39/d3e/d81/d1b/d52/d13e/l16 to d3/d39/d3e/db4/l152 0 2024-06-15T01:07:13.987 INFO:tasks.workunit.client.0.smithi137.stdout:3/985: creat d0/d2/d1e/f140 x:0 0 0 2024-06-15T01:07:13.988 INFO:tasks.workunit.client.0.smithi137.stdout:4/643: dwrite dd/d15/d36/d45/dcb/d5b/fb0 [0,4194304] 0 2024-06-15T01:07:13.988 INFO:tasks.workunit.client.1.smithi142.stdout:0/974: creat d3/d39/d3e/d81/d1b/d52/f153 x:0 0 0 2024-06-15T01:07:13.991 INFO:tasks.workunit.client.0.smithi137.stdout:3/986: symlink d0/d13/d37/d12b/l141 0 2024-06-15T01:07:13.994 INFO:tasks.workunit.client.0.smithi137.stdout:0/835: dread d1/d2/dc/d10/f96 [0,4194304] 0 2024-06-15T01:07:13.996 INFO:tasks.workunit.client.0.smithi137.stdout:3/987: unlink d0/d2/f15 0 2024-06-15T01:07:13.996 INFO:tasks.workunit.client.0.smithi137.stdout:3/988: readlink d0/d2/d8/l108 0 2024-06-15T01:07:13.998 INFO:tasks.workunit.client.0.smithi137.stdout:4/644: link dd/d15/d36/d45/dcb/d5b/fc6 dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/fec 0 2024-06-15T01:07:13.999 INFO:tasks.workunit.client.0.smithi137.stdout:3/989: rmdir d0/d42 39 2024-06-15T01:07:14.000 INFO:tasks.workunit.client.0.smithi137.stdout:4/645: rmdir dd/d15/d21/d3a/d6e 39 2024-06-15T01:07:14.001 INFO:tasks.workunit.client.0.smithi137.stdout:4/646: getdents dd/dbf 0 2024-06-15T01:07:14.003 INFO:tasks.workunit.client.0.smithi137.stdout:3/990: read d0/d2/d23/de2/f101 [11897087,68732] 0 2024-06-15T01:07:14.004 INFO:tasks.workunit.client.0.smithi137.stdout:3/991: readlink d0/d56/l11e 0 2024-06-15T01:07:14.006 INFO:tasks.workunit.client.1.smithi142.stdout:2/995: dread d8/df/d10/d94/d25/d22/d43/d52/d7a/f81 [0,4194304] 0 2024-06-15T01:07:14.007 INFO:tasks.workunit.client.0.smithi137.stdout:3/992: creat d0/ddf/da1/f142 x:0 0 0 2024-06-15T01:07:14.008 INFO:tasks.workunit.client.0.smithi137.stdout:4/647: dwrite dd/d15/d21/d3a/fdb [0,4194304] 0 2024-06-15T01:07:14.011 INFO:tasks.workunit.client.0.smithi137.stdout:3/993: symlink d0/d13/d37/d66/l143 0 2024-06-15T01:07:14.013 INFO:tasks.workunit.client.0.smithi137.stdout:4/648: symlink dd/d15/d36/d45/dcb/d83/db2/d72/led 0 2024-06-15T01:07:14.014 INFO:tasks.workunit.client.1.smithi142.stdout:0/975: dread d3/d39/d3e/d81/d4c/d61/f91 [0,4194304] 0 2024-06-15T01:07:14.014 INFO:tasks.workunit.client.1.smithi142.stdout:2/996: dwrite d8/df/ff0 [0,4194304] 0 2024-06-15T01:07:14.015 INFO:tasks.workunit.client.1.smithi142.stdout:2/997: chown d8/df/d10/d94/db5 33477 1 2024-06-15T01:07:14.020 INFO:tasks.workunit.client.1.smithi142.stdout:0/976: creat d3/d39/d3e/d81/d1b/d38/f154 x:0 0 0 2024-06-15T01:07:14.021 INFO:tasks.workunit.client.0.smithi137.stdout:0/836: dwrite d1/d7e/d87/da0/dc4/dd7/de2/f6b [0,4194304] 0 2024-06-15T01:07:14.021 INFO:tasks.workunit.client.1.smithi142.stdout:2/998: fsync d8/d19/d33/fba 0 2024-06-15T01:07:14.025 INFO:tasks.workunit.client.1.smithi142.stdout:0/977: truncate d3/d39/d3e/d81/d44/fa7 26953 0 2024-06-15T01:07:14.025 INFO:tasks.workunit.client.0.smithi137.stdout:0/837: creat d1/d7e/d87/da0/dc4/dd7/f116 x:0 0 0 2024-06-15T01:07:14.029 INFO:tasks.workunit.client.1.smithi142.stdout:2/999: dwrite d8/df/d10/d1c/dbb/ff9 [0,4194304] 0 2024-06-15T01:07:14.030 INFO:tasks.workunit.client.1.smithi142.stdout:0/978: mknod d3/d39/d9a/c155 0 2024-06-15T01:07:14.030 INFO:tasks.workunit.client.0.smithi137.stdout:0/838: rename d1/d2/dc/d10/d22/db7/df8 to d1/d2/dc/d10/d14/d56/d117 0 2024-06-15T01:07:14.032 INFO:tasks.workunit.client.0.smithi137.stdout:3/994: truncate d0/d56/d8f/d9c/d9f/fb7 3664147 0 2024-06-15T01:07:14.033 INFO:tasks.workunit.client.0.smithi137.stdout:0/839: chown d1/d2/dc/d10/d53/da2/f10c 162 1 2024-06-15T01:07:14.034 INFO:tasks.workunit.client.1.smithi142.stdout:0/979: fsync d3/d39/d3e/d81/d44/df1/f13a 0 2024-06-15T01:07:14.036 INFO:tasks.workunit.client.1.smithi142.stdout:0/980: creat d3/d39/d3e/d81/d1b/d52/d13e/d123/f156 x:0 0 0 2024-06-15T01:07:14.036 INFO:tasks.workunit.client.0.smithi137.stdout:4/649: dwrite dd/d15/d36/d45/d55/d8d/f39 [0,4194304] 0 2024-06-15T01:07:14.037 INFO:tasks.workunit.client.1.smithi142.stdout:0/981: fdatasync d3/d39/d3e/d81/fb 0 2024-06-15T01:07:14.040 INFO:tasks.workunit.client.0.smithi137.stdout:3/995: getdents d0/d2/d22/d63/d76 0 2024-06-15T01:07:14.040 INFO:tasks.workunit.client.1.smithi142.stdout:9/931: sync 2024-06-15T01:07:14.042 INFO:tasks.workunit.client.1.smithi142.stdout:9/932: write d6/d34/df3/f100 [327296,116483] 0 2024-06-15T01:07:14.042 INFO:tasks.workunit.client.1.smithi142.stdout:0/982: unlink d3/d39/d3e/d81/d1b/d52/f153 0 2024-06-15T01:07:14.045 INFO:tasks.workunit.client.0.smithi137.stdout:0/840: dread d1/d2/d13/d66/f86 [0,4194304] 0 2024-06-15T01:07:14.048 INFO:tasks.workunit.client.0.smithi137.stdout:0/841: creat d1/d7e/d87/da0/dc4/de4/f118 x:0 0 0 2024-06-15T01:07:14.051 INFO:tasks.workunit.client.1.smithi142.stdout:9/933: rename d6/dc/d1b/d6f/d103/df2/d13c/d58/fbb to d6/dc/d4e/d78/f155 0 2024-06-15T01:07:14.052 INFO:tasks.workunit.client.0.smithi137.stdout:0/842: unlink d1/d2/d13/d66/lb2 0 2024-06-15T01:07:14.053 INFO:tasks.workunit.client.1.smithi142.stdout:9/934: chown d6/d31/c7e 32893397 1 2024-06-15T01:07:14.053 INFO:tasks.workunit.client.0.smithi137.stdout:4/650: truncate f9 168938 0 2024-06-15T01:07:14.056 INFO:tasks.workunit.client.0.smithi137.stdout:4/651: fsync dd/d15/d36/d45/dcb/fc9 0 2024-06-15T01:07:14.057 INFO:tasks.workunit.client.1.smithi142.stdout:9/935: creat d6/dc/d1b/d6f/da1/df6/d148/f156 x:0 0 0 2024-06-15T01:07:14.057 INFO:tasks.workunit.client.0.smithi137.stdout:0/843: mknod d1/d7e/d87/da0/dc4/de4/c119 0 2024-06-15T01:07:14.060 INFO:tasks.workunit.client.0.smithi137.stdout:3/996: dwrite d0/d2/d23/ff5 [0,4194304] 0 2024-06-15T01:07:14.061 INFO:tasks.workunit.client.1.smithi142.stdout:0/983: dwrite d3/d39/d3e/d81/d1b/d52/d13e/faa [0,4194304] 0 2024-06-15T01:07:14.062 INFO:tasks.workunit.client.0.smithi137.stdout:4/652: mkdir dd/d15/d21/d3a/d6e/d7b/dee 0 2024-06-15T01:07:14.066 INFO:tasks.workunit.client.0.smithi137.stdout:3/997: creat d0/d2f/dbe/f144 x:0 0 0 2024-06-15T01:07:14.068 INFO:tasks.workunit.client.1.smithi142.stdout:0/984: getdents d3/d39/d3e/d81/d1e/d98 0 2024-06-15T01:07:14.069 INFO:tasks.workunit.client.1.smithi142.stdout:0/985: chown d3/d39/d3e/d81/d4c/l99 3 1 2024-06-15T01:07:14.072 INFO:tasks.workunit.client.0.smithi137.stdout:3/998: rmdir d0/ddf 39 2024-06-15T01:07:14.074 INFO:tasks.workunit.client.1.smithi142.stdout:0/986: creat d3/d39/d3e/db4/f157 x:0 0 0 2024-06-15T01:07:14.074 INFO:tasks.workunit.client.0.smithi137.stdout:4/653: dwrite dd/d15/d36/d45/d55/f62 [0,4194304] 0 2024-06-15T01:07:14.075 INFO:tasks.workunit.client.1.smithi142.stdout:0/987: dread - d3/d39/d3e/d81/d1b/d52/d13e/d22/d59/f146 zero size 2024-06-15T01:07:14.076 INFO:tasks.workunit.client.0.smithi137.stdout:0/844: write d1/d2/dc/d10/d14/d56/d117/f59 [1910499,87123] 0 2024-06-15T01:07:14.077 INFO:tasks.workunit.client.0.smithi137.stdout:3/999: write d0/d2/d23/d68/f3c [2678823,15237] 0 2024-06-15T01:07:14.078 INFO:tasks.workunit.client.0.smithi137.stdout:4/654: dread - dd/d15/d36/d45/dcb/d83/db2/d67/dbe/fe4 zero size 2024-06-15T01:07:14.080 INFO:tasks.workunit.client.1.smithi142.stdout:9/936: dwrite d6/d34/f8b [0,4194304] 0 2024-06-15T01:07:14.081 INFO:tasks.workunit.client.0.smithi137.stdout:0/845: symlink d1/d2/dc/d10/d22/db7/dbd/l11a 0 2024-06-15T01:07:14.084 INFO:tasks.workunit.client.0.smithi137.stdout:4/655: dread dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:07:14.084 INFO:tasks.workunit.client.0.smithi137.stdout:0/846: mkdir d1/df6/d11b 0 2024-06-15T01:07:14.087 INFO:tasks.workunit.client.1.smithi142.stdout:9/937: dread d6/dc/d4e/d78/f9d [0,4194304] 0 2024-06-15T01:07:14.088 INFO:tasks.workunit.client.0.smithi137.stdout:0/847: getdents d1/d2/d13/d66/dc3 0 2024-06-15T01:07:14.089 INFO:tasks.workunit.client.0.smithi137.stdout:0/848: rmdir d1/d106 39 2024-06-15T01:07:14.091 INFO:tasks.workunit.client.0.smithi137.stdout:0/849: fdatasync d1/d2/dc/d10/f96 0 2024-06-15T01:07:14.093 INFO:tasks.workunit.client.1.smithi142.stdout:9/938: dwrite d6/dc/d1b/f76 [4194304,4194304] 0 2024-06-15T01:07:14.094 INFO:tasks.workunit.client.0.smithi137.stdout:0/850: unlink d1/d106/f10e 0 2024-06-15T01:07:14.096 INFO:tasks.workunit.client.1.smithi142.stdout:9/939: creat d6/dc/d1b/d6f/de1/d110/f157 x:0 0 0 2024-06-15T01:07:14.097 INFO:tasks.workunit.client.0.smithi137.stdout:0/851: unlink d1/d7e/d87/da0/dc4/dd7/de2/d8d/l92 0 2024-06-15T01:07:14.100 INFO:tasks.workunit.client.1.smithi142.stdout:9/940: truncate d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f133 1045814 0 2024-06-15T01:07:14.100 INFO:tasks.workunit.client.0.smithi137.stdout:0/852: fdatasync d1/d2/dc/d10/d14/f1b 0 2024-06-15T01:07:14.103 INFO:tasks.workunit.client.1.smithi142.stdout:9/941: mkdir d6/dc/d4e/dae/d158 0 2024-06-15T01:07:14.103 INFO:tasks.workunit.client.0.smithi137.stdout:0/853: mknod d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/dd0/c11c 0 2024-06-15T01:07:14.104 INFO:tasks.workunit.client.1.smithi142.stdout:0/988: dwrite d3/d39/d3e/d81/d44/fd6 [0,4194304] 0 2024-06-15T01:07:14.106 INFO:tasks.workunit.client.1.smithi142.stdout:9/942: truncate d6/dc/d1b/d6f/da1/df6/d148/f156 968409 0 2024-06-15T01:07:14.106 INFO:tasks.workunit.client.1.smithi142.stdout:0/989: creat d3/d39/d9a/da2/f158 x:0 0 0 2024-06-15T01:07:14.107 INFO:tasks.workunit.client.0.smithi137.stdout:4/656: rename f9 to dd/d15/d36/d45/d8f/dac/de5/fef 0 2024-06-15T01:07:14.109 INFO:tasks.workunit.client.1.smithi142.stdout:9/943: mknod d6/dc/d108/c159 0 2024-06-15T01:07:14.110 INFO:tasks.workunit.client.1.smithi142.stdout:9/944: chown d6/dc/fdf 4723 1 2024-06-15T01:07:14.111 INFO:tasks.workunit.client.1.smithi142.stdout:9/945: fsync d6/f71 0 2024-06-15T01:07:14.111 INFO:tasks.workunit.client.1.smithi142.stdout:0/990: mkdir d3/d39/d3e/d81/d1b/d159 0 2024-06-15T01:07:14.111 INFO:tasks.workunit.client.0.smithi137.stdout:4/657: creat dd/d15/d36/ff0 x:0 0 0 2024-06-15T01:07:14.112 INFO:tasks.workunit.client.1.smithi142.stdout:0/991: read d3/d39/d3e/d81/d44/fd6 [2614008,54175] 0 2024-06-15T01:07:14.116 INFO:tasks.workunit.client.1.smithi142.stdout:0/992: creat d3/d39/d3e/d81/d1b/d52/f15a x:0 0 0 2024-06-15T01:07:14.116 INFO:tasks.workunit.client.1.smithi142.stdout:0/993: dread - d3/d39/d9a/da2/f158 zero size 2024-06-15T01:07:14.117 INFO:tasks.workunit.client.0.smithi137.stdout:4/658: dread dd/d15/d36/d45/d55/d8d/f39 [0,4194304] 0 2024-06-15T01:07:14.117 INFO:tasks.workunit.client.1.smithi142.stdout:0/994: write d3/d39/d3e/d81/d4c/da0/f10c [1433859,77572] 0 2024-06-15T01:07:14.119 INFO:tasks.workunit.client.0.smithi137.stdout:4/659: rmdir dd/d15 39 2024-06-15T01:07:14.120 INFO:tasks.workunit.client.1.smithi142.stdout:0/995: fdatasync d3/d39/d3e/d81/d1b/d52/d13e/f95 0 2024-06-15T01:07:14.121 INFO:tasks.workunit.client.0.smithi137.stdout:4/660: readlink dd/d15/d21/d3a/laa 0 2024-06-15T01:07:14.122 INFO:tasks.workunit.client.1.smithi142.stdout:0/996: chown d3/d39/d3e/d81/d4c 2172767 1 2024-06-15T01:07:14.122 INFO:tasks.workunit.client.0.smithi137.stdout:4/661: fdatasync dd/d15/d36/d45/fd7 0 2024-06-15T01:07:14.123 INFO:tasks.workunit.client.0.smithi137.stdout:0/854: dwrite d1/d2/dc/d10/d14/f1b [0,4194304] 0 2024-06-15T01:07:14.128 INFO:tasks.workunit.client.0.smithi137.stdout:4/662: unlink dd/d15/d36/d45/d55/d8d/d1f/l2c 0 2024-06-15T01:07:14.128 INFO:tasks.workunit.client.1.smithi142.stdout:0/997: dwrite d3/d39/d3e/d81/d1b/d52/d13e/faa [4194304,4194304] 0 2024-06-15T01:07:14.130 INFO:tasks.workunit.client.0.smithi137.stdout:4/663: chown dd/d15/d36/d45/d55/d8d/ce6 11 1 2024-06-15T01:07:14.131 INFO:tasks.workunit.client.1.smithi142.stdout:0/998: stat d3/d39/d3e/d81/d1b/f30 0 2024-06-15T01:07:14.131 INFO:tasks.workunit.client.1.smithi142.stdout:9/946: dread d6/dc/d1b/d6f/d103/df2/d13c/d58/fd6 [4194304,4194304] 0 2024-06-15T01:07:14.131 INFO:tasks.workunit.client.1.smithi142.stdout:0/999: fsync d3/d39/d3e/d81/d1b/df6/f112 0 2024-06-15T01:07:14.132 INFO:tasks.workunit.client.0.smithi137.stdout:0/855: dwrite d1/d2/dc/d10/d14/d56/f109 [0,4194304] 0 2024-06-15T01:07:14.134 INFO:tasks.workunit.client.1.smithi142.stdout:9/947: creat d6/dc/d1b/f15a x:0 0 0 2024-06-15T01:07:14.137 INFO:tasks.workunit.client.1.smithi142.stdout:9/948: read d6/dc/d1b/f40 [1476467,53357] 0 2024-06-15T01:07:14.139 INFO:tasks.workunit.client.1.smithi142.stdout:9/949: write d6/f71 [2481643,125431] 0 2024-06-15T01:07:14.142 INFO:tasks.workunit.client.0.smithi137.stdout:4/664: link dd/d15/d21/d3a/d6e/f7e dd/d15/d21/d9c/d75/ff1 0 2024-06-15T01:07:14.145 INFO:tasks.workunit.client.1.smithi142.stdout:9/950: rename d6/dc/d108 to d6/dc/dd/d15b 0 2024-06-15T01:07:14.145 INFO:tasks.workunit.client.1.smithi142.stdout:9/951: write d6/dc/d1b/d6f/f152 [674972,37753] 0 2024-06-15T01:07:14.145 INFO:tasks.workunit.client.0.smithi137.stdout:4/665: creat dd/d15/d36/d45/d55/d8d/d1f/ff2 x:0 0 0 2024-06-15T01:07:14.148 INFO:tasks.workunit.client.1.smithi142.stdout:9/952: creat d6/d34/df3/d116/f15c x:0 0 0 2024-06-15T01:07:14.150 INFO:tasks.workunit.client.1.smithi142.stdout:9/953: rmdir d6/dc/d51/d11c 39 2024-06-15T01:07:14.152 INFO:tasks.workunit.client.1.smithi142.stdout:9/954: creat d6/d34/d8a/f15d x:0 0 0 2024-06-15T01:07:14.153 INFO:tasks.workunit.client.1.smithi142.stdout:9/955: stat d6/dc/d51 0 2024-06-15T01:07:14.153 INFO:tasks.workunit.client.0.smithi137.stdout:0/856: dwrite d1/d2/dc/d10/f96 [0,4194304] 0 2024-06-15T01:07:14.157 INFO:tasks.workunit.client.1.smithi142.stdout:9/956: dwrite d6/f45 [0,4194304] 0 2024-06-15T01:07:14.158 INFO:tasks.workunit.client.0.smithi137.stdout:0/857: getdents d1/d7e/d87/da0 0 2024-06-15T01:07:14.162 INFO:tasks.workunit.client.0.smithi137.stdout:4/666: write dd/d15/d36/d45/d55/f95 [1617175,21183] 0 2024-06-15T01:07:14.163 INFO:tasks.workunit.client.0.smithi137.stdout:4/667: fsync dd/d15/d30/fa7 0 2024-06-15T01:07:14.166 INFO:tasks.workunit.client.1.smithi142.stdout:9/957: dwrite d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f13f [0,4194304] 0 2024-06-15T01:07:14.168 INFO:tasks.workunit.client.1.smithi142.stdout:9/958: chown d6/dc/d1b/d6f/d103/df2/d13c/f3f 61220478 1 2024-06-15T01:07:14.177 INFO:tasks.workunit.client.0.smithi137.stdout:4/668: link dd/d15/d36/d45/d55/d8d/c8b dd/d15/d21/d3a/d6e/d7b/d96/cf3 0 2024-06-15T01:07:14.181 INFO:tasks.workunit.client.0.smithi137.stdout:4/669: rename dd/d15/d36/ff0 to dd/d15/d36/d45/dcb/d83/db2/d67/dbe/ff4 0 2024-06-15T01:07:14.182 INFO:tasks.workunit.client.0.smithi137.stdout:4/670: dread - dd/d15/d21/d3a/f73 zero size 2024-06-15T01:07:14.182 INFO:tasks.workunit.client.0.smithi137.stdout:4/671: read - dd/d15/d30/fa7 zero size 2024-06-15T01:07:14.184 INFO:tasks.workunit.client.0.smithi137.stdout:4/672: write dd/d15/d30/fa7 [715450,41445] 0 2024-06-15T01:07:14.187 INFO:tasks.workunit.client.0.smithi137.stdout:0/858: dread d1/d2/dc/d10/d14/d56/d117/d17/f95 [0,4194304] 0 2024-06-15T01:07:14.189 INFO:tasks.workunit.client.0.smithi137.stdout:0/859: truncate d1/d2/dc/d10/d14/d56/d117/f59 2640279 0 2024-06-15T01:07:14.193 INFO:tasks.workunit.client.0.smithi137.stdout:0/860: fdatasync d1/d2/d13/fc0 0 2024-06-15T01:07:14.194 INFO:tasks.workunit.client.1.smithi142.stdout:9/959: dwrite d6/f12a [0,4194304] 0 2024-06-15T01:07:14.196 INFO:tasks.workunit.client.1.smithi142.stdout:9/960: truncate d6/dc/fd9 1191548 0 2024-06-15T01:07:14.197 INFO:tasks.workunit.client.0.smithi137.stdout:0/861: creat d1/d2/dc/d10/d22/db7/f11d x:0 0 0 2024-06-15T01:07:14.203 INFO:tasks.workunit.client.1.smithi142.stdout:9/961: unlink d6/dc/d1b/d6f/d103/df2/d13c/d58/c7d 0 2024-06-15T01:07:14.205 INFO:tasks.workunit.client.0.smithi137.stdout:4/673: dwrite dd/dbf/fd1 [0,4194304] 0 2024-06-15T01:07:14.208 INFO:tasks.workunit.client.0.smithi137.stdout:0/862: dwrite d1/d2/dc/d10/d53/da2/f10c [0,4194304] 0 2024-06-15T01:07:14.211 INFO:tasks.workunit.client.0.smithi137.stdout:4/674: mkdir dd/d15/d21/d9c/d75/df5 0 2024-06-15T01:07:14.212 INFO:tasks.workunit.client.0.smithi137.stdout:0/863: mkdir d1/d7e/d11e 0 2024-06-15T01:07:14.215 INFO:tasks.workunit.client.0.smithi137.stdout:4/675: mkdir dd/d15/d21/d3a/df6 0 2024-06-15T01:07:14.216 INFO:tasks.workunit.client.0.smithi137.stdout:0/864: write d1/d2/dc/d10/d14/d56/d117/d17/f95 [4358663,108217] 0 2024-06-15T01:07:14.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/676: dwrite dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:07:14.223 INFO:tasks.workunit.client.0.smithi137.stdout:0/865: dwrite d1/d2/dc/d10/d14/d56/d117/d17/f95 [0,4194304] 0 2024-06-15T01:07:14.224 INFO:tasks.workunit.client.0.smithi137.stdout:4/677: read - dd/d15/d36/d45/d55/d8d/d1f/d86/fab zero size 2024-06-15T01:07:14.226 INFO:tasks.workunit.client.0.smithi137.stdout:0/866: mkdir d1/d2/dc/d10/d22/db7/d11f 0 2024-06-15T01:07:14.228 INFO:tasks.workunit.client.0.smithi137.stdout:0/867: fdatasync d1/d7e/d87/da0/dc4/dd7/de2/f6b 0 2024-06-15T01:07:14.232 INFO:tasks.workunit.client.0.smithi137.stdout:4/678: rename dd/d15/c35 to dd/cf7 0 2024-06-15T01:07:14.241 INFO:tasks.workunit.client.0.smithi137.stdout:0/868: dread d1/d2/dc/d10/f25 [0,4194304] 0 2024-06-15T01:07:14.245 INFO:tasks.workunit.client.0.smithi137.stdout:0/869: creat d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/f120 x:0 0 0 2024-06-15T01:07:14.247 INFO:tasks.workunit.client.0.smithi137.stdout:0/870: write d1/d7e/d87/da0/dc4/dd7/de2/f110 [725226,72918] 0 2024-06-15T01:07:14.250 INFO:tasks.workunit.client.0.smithi137.stdout:4/679: dwrite f5 [4194304,4194304] 0 2024-06-15T01:07:14.250 INFO:tasks.workunit.client.0.smithi137.stdout:0/871: mknod d1/d2/dc/d10/d14/d56/d99/c121 0 2024-06-15T01:07:14.252 INFO:tasks.workunit.client.0.smithi137.stdout:0/872: chown d1/d2/dc/d10/d14/d56/d99/c121 309744582 1 2024-06-15T01:07:14.252 INFO:tasks.workunit.client.0.smithi137.stdout:4/680: rmdir dd/d15 39 2024-06-15T01:07:14.253 INFO:tasks.workunit.client.0.smithi137.stdout:0/873: write d1/d2/dc/d10/d14/d52/fcc [2764277,80540] 0 2024-06-15T01:07:14.255 INFO:tasks.workunit.client.0.smithi137.stdout:4/681: read - dd/d15/d21/fe9 zero size 2024-06-15T01:07:14.259 INFO:tasks.workunit.client.0.smithi137.stdout:0/874: creat d1/d2/d13/d66/f122 x:0 0 0 2024-06-15T01:07:14.260 INFO:tasks.workunit.client.0.smithi137.stdout:4/682: creat dd/d15/d21/d9c/d75/d99/ff8 x:0 0 0 2024-06-15T01:07:14.261 INFO:tasks.workunit.client.0.smithi137.stdout:0/875: symlink d1/d2/dc/d10/d22/l123 0 2024-06-15T01:07:14.262 INFO:tasks.workunit.client.0.smithi137.stdout:4/683: getdents dd/d15/d21/d9c/d91 0 2024-06-15T01:07:14.265 INFO:tasks.workunit.client.0.smithi137.stdout:4/684: mknod dd/d15/d36/d45/dcb/d83/db2/d72/d81/cf9 0 2024-06-15T01:07:14.266 INFO:tasks.workunit.client.0.smithi137.stdout:0/876: rename d1/d2/dc/d10/d14/d56/d117/d17/d54/ld3 to d1/df6/d11b/l124 0 2024-06-15T01:07:14.268 INFO:tasks.workunit.client.0.smithi137.stdout:4/685: creat dd/dbf/ffa x:0 0 0 2024-06-15T01:07:14.270 INFO:tasks.workunit.client.0.smithi137.stdout:0/877: unlink d1/d2/dc/d10/d14/d56/d117/d17/c4d 0 2024-06-15T01:07:14.271 INFO:tasks.workunit.client.0.smithi137.stdout:4/686: creat dd/d15/d36/d45/d55/d8d/d1f/ffb x:0 0 0 2024-06-15T01:07:14.275 INFO:tasks.workunit.client.0.smithi137.stdout:4/687: getdents dd/d15 0 2024-06-15T01:07:14.277 INFO:tasks.workunit.client.0.smithi137.stdout:0/878: dwrite d1/d2/dc/d10/d14/d56/d117/d1c/f46 [0,4194304] 0 2024-06-15T01:07:14.282 INFO:tasks.workunit.client.0.smithi137.stdout:4/688: rename dd/d15/d21/d9c/d75/ff1 to dd/d15/d21/d3a/df6/ffc 0 2024-06-15T01:07:14.285 INFO:tasks.workunit.client.0.smithi137.stdout:0/879: dwrite d1/d7e/d87/da0/dc4/dd7/de2/f110 [0,4194304] 0 2024-06-15T01:07:14.288 INFO:tasks.workunit.client.0.smithi137.stdout:4/689: dread dd/d15/d36/d45/d55/f82 [0,4194304] 0 2024-06-15T01:07:14.290 INFO:tasks.workunit.client.0.smithi137.stdout:4/690: write dd/d15/d21/d9c/d75/fae [287384,30409] 0 2024-06-15T01:07:14.293 INFO:tasks.workunit.client.0.smithi137.stdout:0/880: dwrite d1/d7e/d87/da0/fcf [0,4194304] 0 2024-06-15T01:07:14.295 INFO:tasks.workunit.client.0.smithi137.stdout:4/691: symlink dd/d15/d36/d45/d55/lfd 0 2024-06-15T01:07:14.300 INFO:tasks.workunit.client.1.smithi142.stdout:9/962: sync 2024-06-15T01:07:14.300 INFO:tasks.workunit.client.0.smithi137.stdout:0/881: creat d1/d7e/f125 x:0 0 0 2024-06-15T01:07:14.305 INFO:tasks.workunit.client.1.smithi142.stdout:9/963: link d6/dc/d4e/d78/fb3 d6/d34/ddd/f15e 0 2024-06-15T01:07:14.306 INFO:tasks.workunit.client.0.smithi137.stdout:4/692: dwrite dd/d15/d21/d3a/fdb [4194304,4194304] 0 2024-06-15T01:07:14.307 INFO:tasks.workunit.client.1.smithi142.stdout:9/964: rmdir d6/dc/d1b/d6f/d103 39 2024-06-15T01:07:14.309 INFO:tasks.workunit.client.0.smithi137.stdout:0/882: dread d1/d2/dc/d10/d53/da2/f10c [0,4194304] 0 2024-06-15T01:07:14.311 INFO:tasks.workunit.client.0.smithi137.stdout:4/693: rmdir dd/d15/d36/d45/d55/d8d/d1f/d86/dd6 39 2024-06-15T01:07:14.313 INFO:tasks.workunit.client.1.smithi142.stdout:9/965: rename d6/dc/d1b/d6f/d103/d8d/d10b/c142 to d6/dc/d1b/d6f/da1/df6/c15f 0 2024-06-15T01:07:14.315 INFO:tasks.workunit.client.1.smithi142.stdout:9/966: chown d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/de7/f106 3 1 2024-06-15T01:07:14.323 INFO:tasks.workunit.client.1.smithi142.stdout:9/967: dwrite d6/dc/d1b/d6f/da1/f104 [0,4194304] 0 2024-06-15T01:07:14.324 INFO:tasks.workunit.client.1.smithi142.stdout:9/968: chown d6/dc/d1b/d6f/d103/df2/d13c 1 1 2024-06-15T01:07:14.325 INFO:tasks.workunit.client.1.smithi142.stdout:9/969: readlink d6/dc/d4e/dae/lc1 0 2024-06-15T01:07:14.325 INFO:tasks.workunit.client.0.smithi137.stdout:4/694: dwrite dd/d15/d36/d45/dcb/d83/db2/f61 [0,4194304] 0 2024-06-15T01:07:14.331 INFO:tasks.workunit.client.1.smithi142.stdout:9/970: rename d6/dc/d1b/d6f/de1/d110/f137 to d6/dc/d1b/f160 0 2024-06-15T01:07:14.332 INFO:tasks.workunit.client.0.smithi137.stdout:4/695: write dd/d15/d21/d3a/f9f [2787364,2352] 0 2024-06-15T01:07:14.332 INFO:tasks.workunit.client.1.smithi142.stdout:9/971: stat d6/dc/d1b/d6f/d103/df2/f111 0 2024-06-15T01:07:14.335 INFO:tasks.workunit.client.0.smithi137.stdout:4/696: symlink dd/d15/d36/d45/lfe 0 2024-06-15T01:07:14.336 INFO:tasks.workunit.client.0.smithi137.stdout:4/697: chown dd/d15/d36/d45/dcb/fc9 811226 1 2024-06-15T01:07:14.337 INFO:tasks.workunit.client.0.smithi137.stdout:4/698: write dd/d15/d36/d45/d55/d8d/fe2 [880695,44394] 0 2024-06-15T01:07:14.338 INFO:tasks.workunit.client.0.smithi137.stdout:4/699: chown dd/d15/d36/d45/d55/d8d/f70 144 1 2024-06-15T01:07:14.341 INFO:tasks.workunit.client.0.smithi137.stdout:4/700: unlink dd/d15/c34 0 2024-06-15T01:07:14.343 INFO:tasks.workunit.client.0.smithi137.stdout:4/701: rmdir dd/d15/d36/d45/dcb/d83/db2/d67 39 2024-06-15T01:07:14.344 INFO:tasks.workunit.client.0.smithi137.stdout:4/702: chown dd/d15/d36/d45/d55/d8d/c2b 250483964 1 2024-06-15T01:07:14.349 INFO:tasks.workunit.client.0.smithi137.stdout:4/703: write dd/d15/d36/d45/d8f/dac/de5/fef [250306,62298] 0 2024-06-15T01:07:14.352 INFO:tasks.workunit.client.1.smithi142.stdout:9/972: dwrite d6/d34/ddd/fa5 [0,4194304] 0 2024-06-15T01:07:14.359 INFO:tasks.workunit.client.1.smithi142.stdout:9/973: fdatasync d6/dc/d1b/d6f/d103/df2/d13c/d58/f64 0 2024-06-15T01:07:14.360 INFO:tasks.workunit.client.0.smithi137.stdout:4/704: dwrite dd/d15/d36/d45/d8f/dac/de5/fef [0,4194304] 0 2024-06-15T01:07:14.360 INFO:tasks.workunit.client.0.smithi137.stdout:4/705: readlink dd/lf 0 2024-06-15T01:07:14.362 INFO:tasks.workunit.client.1.smithi142.stdout:9/974: mknod d6/dc/d1b/d6f/d103/df2/d13c/d58/d74/c161 0 2024-06-15T01:07:14.363 INFO:tasks.workunit.client.0.smithi137.stdout:4/706: dread - dd/d15/d36/d45/dcb/d83/db2/d67/dbe/fe4 zero size 2024-06-15T01:07:14.370 INFO:tasks.workunit.client.1.smithi142.stdout:9/975: creat d6/d34/f162 x:0 0 0 2024-06-15T01:07:14.370 INFO:tasks.workunit.client.0.smithi137.stdout:4/707: creat dd/d15/d21/d3a/d6e/d7b/dee/fff x:0 0 0 2024-06-15T01:07:14.392 INFO:tasks.workunit.client.1.smithi142.stdout:9/976: truncate d6/f6a 1671893 0 2024-06-15T01:07:14.395 INFO:tasks.workunit.client.1.smithi142.stdout:9/977: chown d6/dc/d1b/d6f/d103/df2/d13c/d58/d74/l9b 5 1 2024-06-15T01:07:14.395 INFO:tasks.workunit.client.0.smithi137.stdout:4/708: write dd/d15/d36/d45/f4e [33672,98740] 0 2024-06-15T01:07:14.396 INFO:tasks.workunit.client.0.smithi137.stdout:4/709: readlink dd/d15/d21/d3a/d6e/d7b/le8 0 2024-06-15T01:07:14.397 INFO:tasks.workunit.client.0.smithi137.stdout:4/710: rename dd/d15/d21/d3a/f9f to dd/d15/d36/d45/dcb/d5b/f100 0 2024-06-15T01:07:14.399 INFO:tasks.workunit.client.1.smithi142.stdout:9/978: dwrite d6/dc/d82/f117 [0,4194304] 0 2024-06-15T01:07:14.399 INFO:tasks.workunit.client.0.smithi137.stdout:4/711: write dd/d15/d36/d45/d8f/dac/de5/fef [2734883,123641] 0 2024-06-15T01:07:14.400 INFO:tasks.workunit.client.1.smithi142.stdout:9/979: chown d6/dc/d51/f80 54 1 2024-06-15T01:07:14.402 INFO:tasks.workunit.client.0.smithi137.stdout:4/712: mkdir dd/d15/d21/d9c/d91/d101 0 2024-06-15T01:07:14.403 INFO:tasks.workunit.client.1.smithi142.stdout:9/980: write d6/dc/d4e/d78/f10d [4930686,29794] 0 2024-06-15T01:07:14.404 INFO:tasks.workunit.client.1.smithi142.stdout:9/981: truncate d6/d34/df3/d116/f15c 665757 0 2024-06-15T01:07:14.407 INFO:tasks.workunit.client.0.smithi137.stdout:4/713: dread - dd/d15/d36/d45/d55/d8d/d1f/ff2 zero size 2024-06-15T01:07:14.407 INFO:tasks.workunit.client.1.smithi142.stdout:9/982: write d6/dc/d1b/d6f/d103/f9f [3692021,59547] 0 2024-06-15T01:07:14.408 INFO:tasks.workunit.client.1.smithi142.stdout:9/983: dread - d6/dc/fdf zero size 2024-06-15T01:07:14.413 INFO:tasks.workunit.client.1.smithi142.stdout:9/984: link d6/dc/cf d6/dc/d1b/d6f/de1/d110/c163 0 2024-06-15T01:07:14.414 INFO:tasks.workunit.client.0.smithi137.stdout:4/714: write dd/d15/d36/d45/d55/f82 [1998183,35498] 0 2024-06-15T01:07:14.414 INFO:tasks.workunit.client.1.smithi142.stdout:9/985: dread - d6/f14e zero size 2024-06-15T01:07:14.415 INFO:tasks.workunit.client.0.smithi137.stdout:4/715: truncate dd/d15/d21/d9c/d75/fc2 4761488 0 2024-06-15T01:07:14.415 INFO:tasks.workunit.client.1.smithi142.stdout:9/986: symlink d6/dc/d4e/dae/d13d/l164 0 2024-06-15T01:07:14.417 INFO:tasks.workunit.client.0.smithi137.stdout:4/716: fdatasync dd/d15/d36/d45/d55/d8d/f90 0 2024-06-15T01:07:14.418 INFO:tasks.workunit.client.1.smithi142.stdout:9/987: dread d6/dc/d82/f72 [0,4194304] 0 2024-06-15T01:07:14.418 INFO:tasks.workunit.client.1.smithi142.stdout:9/988: fdatasync d6/d34/df3/d116/f15c 0 2024-06-15T01:07:14.419 INFO:tasks.workunit.client.0.smithi137.stdout:4/717: symlink dd/d15/d36/l102 0 2024-06-15T01:07:14.421 INFO:tasks.workunit.client.0.smithi137.stdout:4/718: truncate dd/d15/d21/d3a/d6e/d7b/faf 842840 0 2024-06-15T01:07:14.425 INFO:tasks.workunit.client.0.smithi137.stdout:4/719: rename dd/d15/d21/d9c/d91/d101 to dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103 0 2024-06-15T01:07:14.426 INFO:tasks.workunit.client.0.smithi137.stdout:4/720: symlink dd/d15/d36/d45/dcb/d83/db2/d72/d81/l104 0 2024-06-15T01:07:14.427 INFO:tasks.workunit.client.1.smithi142.stdout:9/989: dwrite d6/dc/d1b/d6f/d103/df2/d13c/d58/f64 [0,4194304] 0 2024-06-15T01:07:14.428 INFO:tasks.workunit.client.0.smithi137.stdout:4/721: mknod dd/d15/d21/d3a/d6e/d7b/c105 0 2024-06-15T01:07:14.431 INFO:tasks.workunit.client.1.smithi142.stdout:9/990: rename d6/dc/f8e to d6/d34/df3/f165 0 2024-06-15T01:07:14.431 INFO:tasks.workunit.client.1.smithi142.stdout:9/991: readlink d6/d31/l4c 0 2024-06-15T01:07:14.432 INFO:tasks.workunit.client.1.smithi142.stdout:9/992: write d6/f9 [737865,87697] 0 2024-06-15T01:07:14.432 INFO:tasks.workunit.client.0.smithi137.stdout:4/722: getdents dd/d15/d36/d45 0 2024-06-15T01:07:14.435 INFO:tasks.workunit.client.0.smithi137.stdout:4/723: creat dd/d15/d21/d3a/d6e/d7b/dee/f106 x:0 0 0 2024-06-15T01:07:14.436 INFO:tasks.workunit.client.1.smithi142.stdout:9/993: creat d6/dc/f166 x:0 0 0 2024-06-15T01:07:14.440 INFO:tasks.workunit.client.0.smithi137.stdout:4/724: rename f5 to dd/d15/d21/d9c/d91/f107 0 2024-06-15T01:07:14.442 INFO:tasks.workunit.client.1.smithi142.stdout:9/994: write d6/f21 [2034473,97654] 0 2024-06-15T01:07:14.442 INFO:tasks.workunit.client.1.smithi142.stdout:9/995: chown d6/d34/ddd/c89 898883196 1 2024-06-15T01:07:14.443 INFO:tasks.workunit.client.0.smithi137.stdout:4/725: read dd/d15/d21/d3a/d6e/d7b/faf [688368,59771] 0 2024-06-15T01:07:14.444 INFO:tasks.workunit.client.0.smithi137.stdout:4/726: mknod dd/c108 0 2024-06-15T01:07:14.445 INFO:tasks.workunit.client.1.smithi142.stdout:9/996: creat d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/f167 x:0 0 0 2024-06-15T01:07:14.447 INFO:tasks.workunit.client.0.smithi137.stdout:4/727: unlink dd/d15/d36/d45/d55/d8d/d1f/l6d 0 2024-06-15T01:07:14.448 INFO:tasks.workunit.client.0.smithi137.stdout:4/728: stat dd/d15/d36/d45/dcb/d83/db2/d67/f7d 0 2024-06-15T01:07:14.449 INFO:tasks.workunit.client.0.smithi137.stdout:4/729: write dd/d15/d21/d3a/f73 [55046,30345] 0 2024-06-15T01:07:14.451 INFO:tasks.workunit.client.0.smithi137.stdout:4/730: read dd/d15/d36/d45/dcb/d83/db2/d72/d81/fc5 [1127925,65062] 0 2024-06-15T01:07:14.455 INFO:tasks.workunit.client.0.smithi137.stdout:4/731: rename dd/d15/d21/f32 to dd/d15/d21/d9c/d91/f109 0 2024-06-15T01:07:14.457 INFO:tasks.workunit.client.0.smithi137.stdout:4/732: creat dd/d15/d36/d45/dcb/d5b/f10a x:0 0 0 2024-06-15T01:07:14.459 INFO:tasks.workunit.client.0.smithi137.stdout:4/733: mkdir dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d10b 0 2024-06-15T01:07:14.463 INFO:tasks.workunit.client.1.smithi142.stdout:9/997: dread d6/dc/d1b/d6f/d103/df2/d13c/d58/de2/f129 [0,4194304] 0 2024-06-15T01:07:14.465 INFO:tasks.workunit.client.1.smithi142.stdout:9/998: truncate d6/dc/d1b/d6f/d103/df2/d13c/d35/ff1 447566 0 2024-06-15T01:07:14.468 INFO:tasks.workunit.client.0.smithi137.stdout:4/734: dwrite dd/d15/d21/d3a/d6e/d7b/faf [0,4194304] 0 2024-06-15T01:07:14.468 INFO:tasks.workunit.client.1.smithi142.stdout:9/999: rename d6/d34/ddd/c89 to d6/dc/d1b/d6f/d103/df2/d13c/d58/d74/c168 0 2024-06-15T01:07:14.471 INFO:tasks.workunit.client.0.smithi137.stdout:4/735: creat dd/d15/d36/d45/d8f/f10c x:0 0 0 2024-06-15T01:07:14.472 INFO:tasks.workunit.client.1.smithi142.stderr:+ rm -rf -- ./tmp.K3vUaNGYGU 2024-06-15T01:07:14.475 INFO:tasks.workunit.client.0.smithi137.stdout:4/736: creat dd/d15/d21/d9c/d91/f10d x:0 0 0 2024-06-15T01:07:14.477 INFO:tasks.workunit.client.0.smithi137.stdout:4/737: symlink dd/d15/d21/d3a/d6e/d7b/dee/l10e 0 2024-06-15T01:07:14.480 INFO:tasks.workunit.client.0.smithi137.stdout:4/738: dread dd/d15/d36/d45/d55/f82 [0,4194304] 0 2024-06-15T01:07:14.482 INFO:tasks.workunit.client.0.smithi137.stdout:4/739: mknod dd/d15/d36/d45/d8f/dac/de5/c10f 0 2024-06-15T01:07:14.485 INFO:tasks.workunit.client.0.smithi137.stdout:4/740: creat dd/d15/d21/d9c/d75/f110 x:0 0 0 2024-06-15T01:07:14.491 INFO:tasks.workunit.client.0.smithi137.stdout:4/741: write dd/d15/d36/d45/dcb/d83/db2/d67/f6c [1061240,127487] 0 2024-06-15T01:07:14.494 INFO:tasks.workunit.client.0.smithi137.stdout:4/742: creat dd/d15/d36/d45/dcb/d83/db2/dea/f111 x:0 0 0 2024-06-15T01:07:14.498 INFO:tasks.workunit.client.0.smithi137.stdout:4/743: link dd/d15/d30/f6b dd/d15/d36/d45/d55/d8d/f112 0 2024-06-15T01:07:14.501 INFO:tasks.workunit.client.0.smithi137.stdout:4/744: link dd/d15/d36/d45/d55/lfd dd/d15/d36/d45/l113 0 2024-06-15T01:07:14.543 INFO:tasks.workunit.client.0.smithi137.stdout:7/524: sync 2024-06-15T01:07:14.543 INFO:tasks.workunit.client.0.smithi137.stdout:2/485: sync 2024-06-15T01:07:14.543 INFO:tasks.workunit.client.0.smithi137.stdout:1/810: sync 2024-06-15T01:07:14.545 INFO:tasks.workunit.client.0.smithi137.stdout:2/486: chown d0/dc/d22/d23/c89 1721132 1 2024-06-15T01:07:14.545 INFO:tasks.workunit.client.0.smithi137.stdout:1/811: chown d0/d3/d1b/d4c/d59/dd4/l10e 21 1 2024-06-15T01:07:14.545 INFO:tasks.workunit.client.0.smithi137.stdout:7/525: dread - df/d7d/d3b/f93 zero size 2024-06-15T01:07:14.550 INFO:tasks.workunit.client.0.smithi137.stdout:2/487: rename d0/dc/d22/d23 to d0/da/d73/d8d 0 2024-06-15T01:07:14.552 INFO:tasks.workunit.client.0.smithi137.stdout:7/526: mknod df/d7d/d1c/d1d/d79/ca7 0 2024-06-15T01:07:14.561 INFO:tasks.workunit.client.0.smithi137.stdout:2/488: getdents d0/da/d73 0 2024-06-15T01:07:14.562 INFO:tasks.workunit.client.0.smithi137.stdout:7/527: write df/d7d/d1c/d1d/d27/d33/d43/d6c/f37 [1071329,6215] 0 2024-06-15T01:07:14.565 INFO:tasks.workunit.client.0.smithi137.stdout:1/812: dread d0/d2d/f79 [0,4194304] 0 2024-06-15T01:07:14.566 INFO:tasks.workunit.client.0.smithi137.stdout:7/528: symlink df/d13/d2e/d1a/d6a/la8 0 2024-06-15T01:07:14.574 INFO:tasks.workunit.client.0.smithi137.stdout:7/529: rmdir df/d13/d2e/d1a/d3f/d85 0 2024-06-15T01:07:14.575 INFO:tasks.workunit.client.0.smithi137.stdout:2/489: dwrite d0/da/f11 [0,4194304] 0 2024-06-15T01:07:14.576 INFO:tasks.workunit.client.0.smithi137.stdout:2/490: stat d0/da/d73/d8d/f6a 0 2024-06-15T01:07:14.577 INFO:tasks.workunit.client.0.smithi137.stdout:2/491: rmdir d0 39 2024-06-15T01:07:14.581 INFO:tasks.workunit.client.0.smithi137.stdout:7/530: rename df/d7d/c24 to df/ca9 0 2024-06-15T01:07:14.582 INFO:tasks.workunit.client.0.smithi137.stdout:7/531: chown df/d13/l44 6207 1 2024-06-15T01:07:14.585 INFO:tasks.workunit.client.0.smithi137.stdout:1/813: dread d0/d10/d41/d7b/f90 [4194304,4194304] 0 2024-06-15T01:07:14.586 INFO:tasks.workunit.client.0.smithi137.stdout:1/814: write d0/fb0 [1432704,82742] 0 2024-06-15T01:07:14.589 INFO:tasks.workunit.client.0.smithi137.stdout:7/532: write df/d13/d2e/d1a/f2c [1444292,30630] 0 2024-06-15T01:07:14.591 INFO:tasks.workunit.client.0.smithi137.stdout:7/533: fsync df/d7d/d1c/d1d/d27/d33/d43/d6c/f37 0 2024-06-15T01:07:14.595 INFO:tasks.workunit.client.0.smithi137.stdout:1/815: rename d0/d3/d1b to d0/d1f/d60/d99/de3/dea/d10f 0 2024-06-15T01:07:14.602 INFO:tasks.workunit.client.0.smithi137.stdout:2/492: dread d0/f69 [0,4194304] 0 2024-06-15T01:07:14.603 INFO:tasks.workunit.client.0.smithi137.stdout:7/534: link df/d7d/c23 df/caa 0 2024-06-15T01:07:14.605 INFO:tasks.workunit.client.0.smithi137.stdout:1/816: write d0/d1f/d60/d99/de3/dea/d10f/f69 [1167342,74146] 0 2024-06-15T01:07:14.608 INFO:tasks.workunit.client.0.smithi137.stdout:2/493: write d0/dc/d22/f61 [3216523,26631] 0 2024-06-15T01:07:14.609 INFO:tasks.workunit.client.0.smithi137.stdout:1/817: truncate d0/d3/d14/d34/f70 1441770 0 2024-06-15T01:07:14.610 INFO:tasks.workunit.client.0.smithi137.stdout:7/535: getdents df/d13/d3c 0 2024-06-15T01:07:14.612 INFO:tasks.workunit.client.0.smithi137.stdout:7/536: write df/d7d/d1c/d1d/d27/d33/fa5 [585330,61895] 0 2024-06-15T01:07:14.613 INFO:tasks.workunit.client.0.smithi137.stdout:7/537: chown df/d13/d2e/d1a/f2c 29044 1 2024-06-15T01:07:14.614 INFO:tasks.workunit.client.0.smithi137.stdout:1/818: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/f110 x:0 0 0 2024-06-15T01:07:14.617 INFO:tasks.workunit.client.0.smithi137.stdout:7/538: creat df/d13/d2e/d1a/fab x:0 0 0 2024-06-15T01:07:14.618 INFO:tasks.workunit.client.0.smithi137.stdout:2/494: truncate d0/d1c/f66 3221021 0 2024-06-15T01:07:14.620 INFO:tasks.workunit.client.0.smithi137.stdout:1/819: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/f111 x:0 0 0 2024-06-15T01:07:14.621 INFO:tasks.workunit.client.0.smithi137.stdout:7/539: creat df/d7d/d1c/d1d/d27/d33/d59/fac x:0 0 0 2024-06-15T01:07:14.626 INFO:tasks.workunit.client.0.smithi137.stdout:2/495: dwrite d0/da/f11 [0,4194304] 0 2024-06-15T01:07:14.628 INFO:tasks.workunit.client.0.smithi137.stdout:2/496: readlink d0/da/d73/l78 0 2024-06-15T01:07:14.629 INFO:tasks.workunit.client.0.smithi137.stdout:7/540: rename df/d13/d2e/d1a/d6a/la8 to df/d13/d2e/d1a/d3f/lad 0 2024-06-15T01:07:14.631 INFO:tasks.workunit.client.0.smithi137.stdout:1/820: dwrite d0/d3/d47/fd3 [0,4194304] 0 2024-06-15T01:07:14.635 INFO:tasks.workunit.client.0.smithi137.stdout:2/497: creat d0/de/d2b/f8e x:0 0 0 2024-06-15T01:07:14.636 INFO:tasks.workunit.client.0.smithi137.stdout:2/498: readlink d0/de/d10/d3a/l60 0 2024-06-15T01:07:14.641 INFO:tasks.workunit.client.0.smithi137.stdout:7/541: link df/d13/f20 df/d13/d2e/d1a/fae 0 2024-06-15T01:07:14.642 INFO:tasks.workunit.client.0.smithi137.stdout:2/499: fdatasync d0/d1c/f66 0 2024-06-15T01:07:14.644 INFO:tasks.workunit.client.0.smithi137.stdout:2/500: mknod d0/de/d10/c8f 0 2024-06-15T01:07:14.646 INFO:tasks.workunit.client.0.smithi137.stdout:2/501: fsync d0/d1c/f70 0 2024-06-15T01:07:14.660 INFO:tasks.workunit.client.0.smithi137.stdout:1/821: write d0/d1f/d60/d99/de3/dea/d10f/d4c/f5c [1618898,48997] 0 2024-06-15T01:07:14.663 INFO:tasks.workunit.client.0.smithi137.stdout:1/822: symlink d0/d2d/l112 0 2024-06-15T01:07:14.664 INFO:tasks.workunit.client.0.smithi137.stdout:7/542: dread df/d7d/d1c/f1f [0,4194304] 0 2024-06-15T01:07:14.667 INFO:tasks.workunit.client.0.smithi137.stdout:1/823: rename d0/d3/d14/c3f to d0/d3/d47/dd7/c113 0 2024-06-15T01:07:14.668 INFO:tasks.workunit.client.0.smithi137.stdout:7/543: mknod df/d7d/d3b/d5b/caf 0 2024-06-15T01:07:14.669 INFO:tasks.workunit.client.0.smithi137.stdout:1/824: write d0/d10/fb5 [231822,19640] 0 2024-06-15T01:07:14.671 INFO:tasks.workunit.client.0.smithi137.stdout:2/502: dwrite d0/de/d10/f3c [0,4194304] 0 2024-06-15T01:07:14.674 INFO:tasks.workunit.client.0.smithi137.stdout:7/544: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/db0 0 2024-06-15T01:07:14.678 INFO:tasks.workunit.client.0.smithi137.stdout:2/503: mkdir d0/da/d73/d8d/d5c/d90 0 2024-06-15T01:07:14.678 INFO:tasks.workunit.client.0.smithi137.stdout:2/504: chown d0/d42/d49 94 1 2024-06-15T01:07:14.678 INFO:tasks.workunit.client.0.smithi137.stdout:7/545: rmdir df/d7d/d3b/d5b 39 2024-06-15T01:07:14.682 INFO:tasks.workunit.client.0.smithi137.stdout:7/546: creat df/d7d/d1c/d1d/d27/d97/fb1 x:0 0 0 2024-06-15T01:07:14.683 INFO:tasks.workunit.client.0.smithi137.stdout:7/547: mknod df/d7d/d1c/d1d/d79/cb2 0 2024-06-15T01:07:14.686 INFO:tasks.workunit.client.0.smithi137.stdout:7/548: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3 0 2024-06-15T01:07:14.686 INFO:tasks.workunit.client.0.smithi137.stdout:2/505: write d0/f6 [57499,64760] 0 2024-06-15T01:07:14.694 INFO:tasks.workunit.client.0.smithi137.stdout:2/506: dwrite d0/dc/f86 [0,4194304] 0 2024-06-15T01:07:14.700 INFO:tasks.workunit.client.0.smithi137.stdout:7/549: dwrite df/d7d/d3b/d5b/f84 [0,4194304] 0 2024-06-15T01:07:14.701 INFO:tasks.workunit.client.0.smithi137.stdout:7/550: chown df/d7d/d3b/d5b/da6 498 1 2024-06-15T01:07:14.702 INFO:tasks.workunit.client.0.smithi137.stdout:7/551: chown df/d13/d2e/d1a/f26 128853 1 2024-06-15T01:07:14.711 INFO:tasks.workunit.client.0.smithi137.stdout:2/507: dread d0/f1f [0,4194304] 0 2024-06-15T01:07:14.715 INFO:tasks.workunit.client.0.smithi137.stdout:2/508: link d0/dc/d22/f4b d0/dc/f91 0 2024-06-15T01:07:14.716 INFO:tasks.workunit.client.0.smithi137.stdout:2/509: read d0/f8 [1356397,40030] 0 2024-06-15T01:07:14.717 INFO:tasks.workunit.client.0.smithi137.stdout:2/510: read - d0/da/d73/f84 zero size 2024-06-15T01:07:14.718 INFO:tasks.workunit.client.0.smithi137.stdout:2/511: rmdir d0/da/d73/d8d 39 2024-06-15T01:07:14.718 INFO:tasks.workunit.client.0.smithi137.stdout:2/512: stat d0/d1c/f70 0 2024-06-15T01:07:14.720 INFO:tasks.workunit.client.0.smithi137.stdout:2/513: creat d0/d42/f92 x:0 0 0 2024-06-15T01:07:14.720 INFO:tasks.workunit.client.0.smithi137.stdout:2/514: rmdir d0/da 39 2024-06-15T01:07:14.721 INFO:tasks.workunit.client.0.smithi137.stdout:2/515: write d0/da/f11 [373450,51342] 0 2024-06-15T01:07:14.765 INFO:tasks.workunit.client.0.smithi137.stdout:2/516: getdents d0/da 0 2024-06-15T01:07:14.765 INFO:tasks.workunit.client.0.smithi137.stdout:2/517: chown d0/dc/f2c 15 1 2024-06-15T01:07:14.766 INFO:tasks.workunit.client.0.smithi137.stdout:2/518: dread - d0/da/d73/f84 zero size 2024-06-15T01:07:14.770 INFO:tasks.workunit.client.0.smithi137.stdout:2/519: write d0/dc/d2e/f75 [1132258,4488] 0 2024-06-15T01:07:14.771 INFO:tasks.workunit.client.0.smithi137.stdout:2/520: chown d0/d42/d49/c6f 175588 1 2024-06-15T01:07:14.774 INFO:tasks.workunit.client.0.smithi137.stdout:2/521: rename d0/de/d10/f33 to d0/dc/d2e/d72/d56/f93 0 2024-06-15T01:07:14.778 INFO:tasks.workunit.client.0.smithi137.stdout:2/522: fdatasync d0/da/d73/d8d/d5c/f82 0 2024-06-15T01:07:14.778 INFO:tasks.workunit.client.0.smithi137.stdout:2/523: creat d0/dc/f94 x:0 0 0 2024-06-15T01:07:14.780 INFO:tasks.workunit.client.0.smithi137.stdout:2/524: truncate d0/f1f 705628 0 2024-06-15T01:07:14.781 INFO:tasks.workunit.client.0.smithi137.stdout:2/525: write d0/fb [7338303,73305] 0 2024-06-15T01:07:14.783 INFO:tasks.workunit.client.0.smithi137.stdout:2/526: mknod d0/da/d73/d8d/d77/c95 0 2024-06-15T01:07:14.784 INFO:tasks.workunit.client.0.smithi137.stdout:2/527: readlink d0/de/l7b 0 2024-06-15T01:07:14.784 INFO:tasks.workunit.client.0.smithi137.stdout:2/528: fsync d0/f6 0 2024-06-15T01:07:14.786 INFO:tasks.workunit.client.0.smithi137.stdout:2/529: write d0/dc/d2e/d72/f59 [4347744,35488] 0 2024-06-15T01:07:14.798 INFO:tasks.workunit.client.0.smithi137.stdout:2/530: dread d0/de/f52 [0,4194304] 0 2024-06-15T01:07:14.800 INFO:tasks.workunit.client.0.smithi137.stdout:2/531: mkdir d0/dc/d22/d96 0 2024-06-15T01:07:14.808 INFO:tasks.workunit.client.0.smithi137.stdout:2/532: write d0/d42/d49/f62 [224062,40624] 0 2024-06-15T01:07:14.811 INFO:tasks.workunit.client.0.smithi137.stdout:2/533: rename d0/dc/d2e/f75 to d0/da/d73/d8d/d5c/d79/f97 0 2024-06-15T01:07:14.827 INFO:tasks.workunit.client.0.smithi137.stdout:2/534: dread d0/fb [4194304,4194304] 0 2024-06-15T01:07:14.830 INFO:tasks.workunit.client.0.smithi137.stdout:2/535: mknod d0/d42/d49/d57/c98 0 2024-06-15T01:07:14.834 INFO:tasks.workunit.client.0.smithi137.stdout:2/536: truncate d0/dc/d22/f4b 4988800 0 2024-06-15T01:07:14.836 INFO:tasks.workunit.client.0.smithi137.stdout:2/537: mkdir d0/da/d73/d8d/d5c/d99 0 2024-06-15T01:07:14.840 INFO:tasks.workunit.client.0.smithi137.stdout:2/538: rename d0/da/d73/d8d/d5c/f82 to d0/da/d73/f9a 0 2024-06-15T01:07:14.847 INFO:tasks.workunit.client.0.smithi137.stdout:2/539: mknod d0/da/d73/d8d/d5c/d99/c9b 0 2024-06-15T01:07:14.857 INFO:tasks.workunit.client.0.smithi137.stdout:2/540: write d0/da/d73/f9a [603526,42177] 0 2024-06-15T01:07:14.858 INFO:tasks.workunit.client.0.smithi137.stdout:2/541: readlink d0/de/l7b 0 2024-06-15T01:07:14.860 INFO:tasks.workunit.client.0.smithi137.stdout:2/542: creat d0/dc/d2e/d72/f9c x:0 0 0 2024-06-15T01:07:14.861 INFO:tasks.workunit.client.0.smithi137.stdout:2/543: write d0/de/f1a [748446,129860] 0 2024-06-15T01:07:14.862 INFO:tasks.workunit.client.0.smithi137.stdout:2/544: chown d0/dc/f86 1089526 1 2024-06-15T01:07:14.866 INFO:tasks.workunit.client.0.smithi137.stdout:2/545: rename d0/dc/d22/f6c to d0/dc/f9d 0 2024-06-15T01:07:14.869 INFO:tasks.workunit.client.0.smithi137.stdout:2/546: dread d0/f69 [0,4194304] 0 2024-06-15T01:07:14.888 INFO:tasks.workunit.client.0.smithi137.stdout:2/547: write d0/d42/d49/d57/f58 [3320106,49210] 0 2024-06-15T01:07:14.892 INFO:tasks.workunit.client.0.smithi137.stdout:2/548: write d0/dc/f91 [3868903,78005] 0 2024-06-15T01:07:14.894 INFO:tasks.workunit.client.0.smithi137.stdout:2/549: dread - d0/d42/d49/d57/f7e zero size 2024-06-15T01:07:14.899 INFO:tasks.workunit.client.0.smithi137.stdout:2/550: dwrite d0/da/d73/f9a [0,4194304] 0 2024-06-15T01:07:14.904 INFO:tasks.workunit.client.0.smithi137.stdout:2/551: link d0/f69 d0/d42/d49/d57/f9e 0 2024-06-15T01:07:14.906 INFO:tasks.workunit.client.0.smithi137.stdout:2/552: mknod d0/de/c9f 0 2024-06-15T01:07:14.911 INFO:tasks.workunit.client.0.smithi137.stdout:2/553: dwrite d0/d42/f92 [0,4194304] 0 2024-06-15T01:07:14.916 INFO:tasks.workunit.client.0.smithi137.stdout:2/554: rename d0/de/d10 to d0/da/da0 0 2024-06-15T01:07:14.919 INFO:tasks.workunit.client.0.smithi137.stdout:2/555: creat d0/dc/d2e/d72/fa1 x:0 0 0 2024-06-15T01:07:14.921 INFO:tasks.workunit.client.0.smithi137.stdout:6/573: sync 2024-06-15T01:07:14.921 INFO:tasks.workunit.client.0.smithi137.stdout:9/520: sync 2024-06-15T01:07:14.922 INFO:tasks.workunit.client.0.smithi137.stdout:6/574: dread - d0/d1a/d3e/d57/d32/d74/fb2 zero size 2024-06-15T01:07:14.925 INFO:tasks.workunit.client.0.smithi137.stdout:9/521: rename df/d12/d15/f32 to df/d49/d8d/f93 0 2024-06-15T01:07:14.928 INFO:tasks.workunit.client.0.smithi137.stdout:6/575: creat d0/d1a/d3e/d57/d32/d74/fb7 x:0 0 0 2024-06-15T01:07:14.928 INFO:tasks.workunit.client.0.smithi137.stdout:2/556: write d0/de/d2b/f32 [1423272,8557] 0 2024-06-15T01:07:14.929 INFO:tasks.workunit.client.0.smithi137.stdout:9/522: creat df/d12/d29/d4a/f94 x:0 0 0 2024-06-15T01:07:14.930 INFO:tasks.workunit.client.0.smithi137.stdout:2/557: chown d0/f1f 11097428 1 2024-06-15T01:07:14.932 INFO:tasks.workunit.client.0.smithi137.stdout:6/576: dread d0/d5/f83 [0,4194304] 0 2024-06-15T01:07:14.934 INFO:tasks.workunit.client.0.smithi137.stdout:2/558: rename d0/dc/d2e/d72/l8c to d0/da/d73/d8d/d5c/d90/la2 0 2024-06-15T01:07:14.935 INFO:tasks.workunit.client.0.smithi137.stdout:9/523: truncate df/d12/d15/d75/f8a 3523875 0 2024-06-15T01:07:14.937 INFO:tasks.workunit.client.0.smithi137.stdout:9/524: truncate df/f90 368597 0 2024-06-15T01:07:14.939 INFO:tasks.workunit.client.0.smithi137.stdout:2/559: rename d0/da/d73/f84 to d0/da/d73/d8d/d5c/fa3 0 2024-06-15T01:07:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:2/560: rmdir d0/de 39 2024-06-15T01:07:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:2/561: stat d0/dc/c35 0 2024-06-15T01:07:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:9/525: dread f2 [0,4194304] 0 2024-06-15T01:07:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:6/577: write d0/d1a/d3e/d57/d1d/d2e/d62/f76 [1789876,56515] 0 2024-06-15T01:07:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:2/562: rename d0/da/da0/f3c to d0/d1c/fa4 0 2024-06-15T01:07:14.950 INFO:tasks.workunit.client.0.smithi137.stdout:2/563: read d0/f1f [421861,87984] 0 2024-06-15T01:07:14.952 INFO:tasks.workunit.client.0.smithi137.stdout:9/526: mkdir df/d12/d29/d95 0 2024-06-15T01:07:14.953 INFO:tasks.workunit.client.0.smithi137.stdout:6/578: mknod d0/d1a/d68/cb8 0 2024-06-15T01:07:14.956 INFO:tasks.workunit.client.0.smithi137.stdout:6/579: fdatasync d0/d5/d9/f4f 0 2024-06-15T01:07:14.958 INFO:tasks.workunit.client.0.smithi137.stdout:6/580: dread - d0/d1a/d3e/d57/d32/d74/f88 zero size 2024-06-15T01:07:14.961 INFO:tasks.workunit.client.0.smithi137.stdout:9/527: truncate df/f4d 2333351 0 2024-06-15T01:07:14.964 INFO:tasks.workunit.client.0.smithi137.stdout:9/528: getdents df/d12/d1b 0 2024-06-15T01:07:14.966 INFO:tasks.workunit.client.0.smithi137.stdout:9/529: mknod df/d12/d29/d66/c96 0 2024-06-15T01:07:14.968 INFO:tasks.workunit.client.0.smithi137.stdout:9/530: mknod df/c97 0 2024-06-15T01:07:14.968 INFO:tasks.workunit.client.0.smithi137.stdout:9/531: stat df/d12/d29/d4a/f70 0 2024-06-15T01:07:14.969 INFO:tasks.workunit.client.0.smithi137.stdout:9/532: write df/d12/d15/f58 [7405922,88819] 0 2024-06-15T01:07:14.974 INFO:tasks.workunit.client.0.smithi137.stdout:9/533: link ce df/d12/d15/d16/d3c/d51/c98 0 2024-06-15T01:07:14.975 INFO:tasks.workunit.client.0.smithi137.stdout:6/581: dread d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:14.975 INFO:tasks.workunit.client.0.smithi137.stdout:9/534: chown df/c10 62887 1 2024-06-15T01:07:14.980 INFO:tasks.workunit.client.0.smithi137.stdout:8/531: sync 2024-06-15T01:07:14.986 INFO:tasks.workunit.client.0.smithi137.stdout:6/582: write d0/d5/d9/f41 [2025404,106270] 0 2024-06-15T01:07:14.990 INFO:tasks.workunit.client.0.smithi137.stdout:8/532: dwrite d9/de/f83 [0,4194304] 0 2024-06-15T01:07:14.991 INFO:tasks.workunit.client.0.smithi137.stdout:8/533: truncate d9/de/d1e/f75 140844 0 2024-06-15T01:07:15.023 INFO:tasks.workunit.client.0.smithi137.stdout:8/534: dread d9/de/f3c [0,4194304] 0 2024-06-15T01:07:15.024 INFO:tasks.workunit.client.0.smithi137.stdout:8/535: write d9/d2a/f2d [3926798,58877] 0 2024-06-15T01:07:15.026 INFO:tasks.workunit.client.0.smithi137.stdout:8/536: readlink d9/de/l17 0 2024-06-15T01:07:15.026 INFO:tasks.workunit.client.0.smithi137.stdout:6/583: dread d0/f2a [0,4194304] 0 2024-06-15T01:07:15.029 INFO:tasks.workunit.client.0.smithi137.stdout:8/537: creat d9/de/d1e/d35/d5f/fb3 x:0 0 0 2024-06-15T01:07:15.029 INFO:tasks.workunit.client.0.smithi137.stdout:8/538: readlink d9/d22/la0 0 2024-06-15T01:07:15.034 INFO:tasks.workunit.client.0.smithi137.stdout:6/584: dwrite d0/d5/d9/d29/f5b [0,4194304] 0 2024-06-15T01:07:15.036 INFO:tasks.workunit.client.0.smithi137.stdout:6/585: write d0/d1a/d3e/d57/d1d/d2e/fa7 [3785942,126686] 0 2024-06-15T01:07:15.039 INFO:tasks.workunit.client.0.smithi137.stdout:6/586: fdatasync d0/d5/d9/f4f 0 2024-06-15T01:07:15.040 INFO:tasks.workunit.client.0.smithi137.stdout:6/587: dread - d0/d5/d9/d29/d6b/f9d zero size 2024-06-15T01:07:15.115 INFO:tasks.workunit.client.0.smithi137.stdout:6/588: mkdir d0/d5/d9/db9 0 2024-06-15T01:07:15.117 INFO:tasks.workunit.client.0.smithi137.stdout:6/589: rmdir d0/d1a/d3e/d57/d32/d92 39 2024-06-15T01:07:15.122 INFO:tasks.workunit.client.0.smithi137.stdout:6/590: dwrite d0/d1a/d3e/d57/d32/d74/fb7 [0,4194304] 0 2024-06-15T01:07:15.123 INFO:tasks.workunit.client.0.smithi137.stdout:6/591: truncate d0/d5/fb0 341627 0 2024-06-15T01:07:15.128 INFO:tasks.workunit.client.0.smithi137.stdout:6/592: dread d0/d5/d9/f9b [0,4194304] 0 2024-06-15T01:07:15.130 INFO:tasks.workunit.client.0.smithi137.stdout:6/593: fdatasync d0/d1a/d3e/d57/d1d/f1f 0 2024-06-15T01:07:15.133 INFO:tasks.workunit.client.0.smithi137.stdout:6/594: rmdir d0/d5/d46 39 2024-06-15T01:07:15.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/595: fdatasync d0/d1a/d3e/d57/d1d/d2e/fa7 0 2024-06-15T01:07:15.138 INFO:tasks.workunit.client.0.smithi137.stdout:6/596: getdents d0/d5/d9/d3f 0 2024-06-15T01:07:15.140 INFO:tasks.workunit.client.0.smithi137.stdout:6/597: mkdir d0/d1a/d3e/d57/dba 0 2024-06-15T01:07:15.142 INFO:tasks.workunit.client.0.smithi137.stdout:6/598: fsync d0/d5/f7 0 2024-06-15T01:07:15.144 INFO:tasks.workunit.client.0.smithi137.stdout:6/599: creat d0/d5/d46/fbb x:0 0 0 2024-06-15T01:07:15.151 INFO:tasks.workunit.client.0.smithi137.stdout:6/600: dwrite d0/d5/f73 [4194304,4194304] 0 2024-06-15T01:07:15.161 INFO:tasks.workunit.client.0.smithi137.stdout:6/601: dwrite d0/d5/f7c [0,4194304] 0 2024-06-15T01:07:15.166 INFO:tasks.workunit.client.0.smithi137.stdout:6/602: rename d0/d5/d9/f91 to d0/d5/d9/d29/fbc 0 2024-06-15T01:07:15.179 INFO:tasks.workunit.client.0.smithi137.stdout:6/603: write d0/d1a/d3e/d57/d1d/d2e/f38 [1976626,17445] 0 2024-06-15T01:07:15.181 INFO:tasks.workunit.client.0.smithi137.stdout:6/604: rmdir d0/d1a/d3e/d57/d1d/d2e/d62/d84 39 2024-06-15T01:07:15.184 INFO:tasks.workunit.client.0.smithi137.stdout:6/605: creat d0/d1a/d3e/d57/d32/d92/dac/fbd x:0 0 0 2024-06-15T01:07:15.186 INFO:tasks.workunit.client.0.smithi137.stdout:6/606: chown d0/d1a/d3e/d57/d1d/d2e/l79 0 1 2024-06-15T01:07:15.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/607: truncate d0/d5/f2c 2236141 0 2024-06-15T01:07:15.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/608: write d0/d1a/d68/fb5 [632713,61789] 0 2024-06-15T01:07:15.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/609: dread - d0/d5/d9/d29/d6b/fa8 zero size 2024-06-15T01:07:15.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/610: mknod d0/d5/d9/db9/cbe 0 2024-06-15T01:07:15.195 INFO:tasks.workunit.client.0.smithi137.stdout:6/611: dread - d0/d5/fae zero size 2024-06-15T01:07:15.196 INFO:tasks.workunit.client.0.smithi137.stdout:6/612: mkdir d0/d5/d46/dbf 0 2024-06-15T01:07:15.197 INFO:tasks.workunit.client.0.smithi137.stdout:6/613: read d0/d1a/f2f [407047,53170] 0 2024-06-15T01:07:15.383 INFO:tasks.workunit.client.0.smithi137.stdout:5/676: sync 2024-06-15T01:07:15.398 INFO:tasks.workunit.client.0.smithi137.stdout:5/677: dread df/d11/d30/f82 [0,4194304] 0 2024-06-15T01:07:15.399 INFO:tasks.workunit.client.0.smithi137.stdout:5/678: creat df/d11/d14/d3c/d8a/fe2 x:0 0 0 2024-06-15T01:07:15.400 INFO:tasks.workunit.client.0.smithi137.stdout:5/679: readlink df/d11/d30/db1/lb2 0 2024-06-15T01:07:15.405 INFO:tasks.workunit.client.0.smithi137.stdout:5/680: dwrite df/d11/fd9 [0,4194304] 0 2024-06-15T01:07:15.410 INFO:tasks.workunit.client.0.smithi137.stdout:5/681: creat df/d11/d70/fe3 x:0 0 0 2024-06-15T01:07:15.415 INFO:tasks.workunit.client.0.smithi137.stdout:5/682: getdents df/d11/d70/dbf 0 2024-06-15T01:07:15.420 INFO:tasks.workunit.client.0.smithi137.stdout:5/683: creat df/d44/d54/d95/d3b/d4b/fe4 x:0 0 0 2024-06-15T01:07:15.422 INFO:tasks.workunit.client.0.smithi137.stdout:5/684: chown df/d11/l22 483091 1 2024-06-15T01:07:15.427 INFO:tasks.workunit.client.0.smithi137.stdout:5/685: dread df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:15.429 INFO:tasks.workunit.client.0.smithi137.stdout:5/686: fdatasync f3 0 2024-06-15T01:07:15.433 INFO:tasks.workunit.client.0.smithi137.stdout:5/687: rmdir df/d11/d70/dbf/dda 0 2024-06-15T01:07:15.435 INFO:tasks.workunit.client.0.smithi137.stdout:5/688: fsync df/d44/d54/d6b/fb3 0 2024-06-15T01:07:15.438 INFO:tasks.workunit.client.0.smithi137.stdout:5/689: symlink df/d44/le5 0 2024-06-15T01:07:15.439 INFO:tasks.workunit.client.0.smithi137.stdout:5/690: fdatasync fd 0 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:8/539: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:2/564: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:4/745: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:7/552: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:6/614: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:9/535: sync 2024-06-15T01:07:15.442 INFO:tasks.workunit.client.0.smithi137.stdout:0/883: sync 2024-06-15T01:07:15.443 INFO:tasks.workunit.client.0.smithi137.stdout:1/825: sync 2024-06-15T01:07:15.444 INFO:tasks.workunit.client.0.smithi137.stdout:0/884: fsync d1/d2/f47 0 2024-06-15T01:07:15.446 INFO:tasks.workunit.client.0.smithi137.stdout:1/826: stat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59 0 2024-06-15T01:07:15.447 INFO:tasks.workunit.client.0.smithi137.stdout:8/540: write d9/de/f36 [1158986,41313] 0 2024-06-15T01:07:15.447 INFO:tasks.workunit.client.0.smithi137.stdout:0/885: chown d1/d2/dc/d10/d22/db7/dbd/l11a 4745737 1 2024-06-15T01:07:15.448 INFO:tasks.workunit.client.0.smithi137.stdout:6/615: symlink d0/d1a/d3e/d57/d32/d92/lc0 0 2024-06-15T01:07:15.451 INFO:tasks.workunit.client.0.smithi137.stdout:7/553: unlink df/d13/d2e/d1a/c56 0 2024-06-15T01:07:15.456 INFO:tasks.workunit.client.0.smithi137.stdout:5/691: dwrite df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:15.458 INFO:tasks.workunit.client.0.smithi137.stdout:6/616: symlink d0/d5/d9/d29/d64/lc1 0 2024-06-15T01:07:15.462 INFO:tasks.workunit.client.0.smithi137.stdout:0/886: dwrite d1/d7e/fae [4194304,4194304] 0 2024-06-15T01:07:15.467 INFO:tasks.workunit.client.0.smithi137.stdout:7/554: fdatasync df/d7d/d1c/f1f 0 2024-06-15T01:07:15.467 INFO:tasks.workunit.client.0.smithi137.stdout:4/746: getdents dd/d15/d21/d9c 0 2024-06-15T01:07:15.467 INFO:tasks.workunit.client.0.smithi137.stdout:9/536: link df/d13/c3d df/d13/d6f/c99 0 2024-06-15T01:07:15.468 INFO:tasks.workunit.client.0.smithi137.stdout:0/887: mkdir d1/d2/dc/d10/d14/d52/d126 0 2024-06-15T01:07:15.468 INFO:tasks.workunit.client.0.smithi137.stdout:6/617: stat d0/d5/d9/d29/d64/f86 0 2024-06-15T01:07:15.469 INFO:tasks.workunit.client.0.smithi137.stdout:9/537: write df/d1a/f87 [718039,74496] 0 2024-06-15T01:07:15.473 INFO:tasks.workunit.client.0.smithi137.stdout:5/692: rename df/d11/d70/dbf/fcd to df/d76/d7c/d9c/fe6 0 2024-06-15T01:07:15.473 INFO:tasks.workunit.client.0.smithi137.stdout:7/555: dread - df/d7d/d1c/d1d/d27/d33/d43/d6c/f96 zero size 2024-06-15T01:07:15.474 INFO:tasks.workunit.client.0.smithi137.stdout:9/538: chown df/d12/d15/c25 1608530279 1 2024-06-15T01:07:15.475 INFO:tasks.workunit.client.0.smithi137.stdout:0/888: chown d1/d7e/d87/da0/dc4/de4/c119 100 1 2024-06-15T01:07:15.476 INFO:tasks.workunit.client.0.smithi137.stdout:8/541: dwrite d9/de/d1e/d35/f56 [0,4194304] 0 2024-06-15T01:07:15.476 INFO:tasks.workunit.client.0.smithi137.stdout:4/747: creat dd/d15/d21/d3a/d6e/d7b/f114 x:0 0 0 2024-06-15T01:07:15.478 INFO:tasks.workunit.client.0.smithi137.stdout:0/889: readlink d1/d2/dc/d6c/d88/d91/lb3 0 2024-06-15T01:07:15.480 INFO:tasks.workunit.client.0.smithi137.stdout:4/748: chown dd/f16 1783450076 1 2024-06-15T01:07:15.484 INFO:tasks.workunit.client.0.smithi137.stdout:6/618: creat d0/d1a/d3e/d57/d1d/d2e/d62/fc2 x:0 0 0 2024-06-15T01:07:15.484 INFO:tasks.workunit.client.0.smithi137.stdout:7/556: creat df/fb4 x:0 0 0 2024-06-15T01:07:15.491 INFO:tasks.workunit.client.0.smithi137.stdout:8/542: dwrite d9/de/d1e/d67/fa6 [0,4194304] 0 2024-06-15T01:07:15.491 INFO:tasks.workunit.client.0.smithi137.stdout:1/827: dread d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/f8c [0,4194304] 0 2024-06-15T01:07:15.492 INFO:tasks.workunit.client.0.smithi137.stdout:9/539: truncate df/d12/d4e/f5c 1562714 0 2024-06-15T01:07:15.492 INFO:tasks.workunit.client.0.smithi137.stdout:8/543: stat d9/d2a/d2f/d59/f71 0 2024-06-15T01:07:15.494 INFO:tasks.workunit.client.0.smithi137.stdout:5/693: rename df/d11/d70 to df/d44/d54/d95/d3b/d49/d5c/de7 0 2024-06-15T01:07:15.495 INFO:tasks.workunit.client.0.smithi137.stdout:7/557: creat df/d13/fb5 x:0 0 0 2024-06-15T01:07:15.499 INFO:tasks.workunit.client.0.smithi137.stdout:0/890: write d1/d2/d13/d66/f86 [1965911,121131] 0 2024-06-15T01:07:15.501 INFO:tasks.workunit.client.0.smithi137.stdout:0/891: write d1/d2/dc/d10/d14/d56/f82 [2669121,121550] 0 2024-06-15T01:07:15.503 INFO:tasks.workunit.client.0.smithi137.stdout:1/828: symlink d0/d1f/d60/d99/de3/l114 0 2024-06-15T01:07:15.505 INFO:tasks.workunit.client.0.smithi137.stdout:8/544: truncate d9/d22/f25 1436062 0 2024-06-15T01:07:15.506 INFO:tasks.workunit.client.0.smithi137.stdout:5/694: rename df/d11 to df/d11/de8 22 2024-06-15T01:07:15.509 INFO:tasks.workunit.client.0.smithi137.stdout:0/892: dread d1/d2/dc/d10/d14/d56/d117/d17/f95 [0,4194304] 0 2024-06-15T01:07:15.512 INFO:tasks.workunit.client.0.smithi137.stdout:1/829: creat d0/d1f/d60/d99/de3/dea/d10f/f115 x:0 0 0 2024-06-15T01:07:15.513 INFO:tasks.workunit.client.0.smithi137.stdout:7/558: unlink df/d7d/d1c/d1d/d27/la1 0 2024-06-15T01:07:15.514 INFO:tasks.workunit.client.0.smithi137.stdout:9/540: dwrite df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:15.516 INFO:tasks.workunit.client.0.smithi137.stdout:1/830: read - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/ffc zero size 2024-06-15T01:07:15.516 INFO:tasks.workunit.client.0.smithi137.stdout:8/545: creat d9/d2a/d79/d7e/fb4 x:0 0 0 2024-06-15T01:07:15.518 INFO:tasks.workunit.client.0.smithi137.stdout:1/831: write d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/dd4/fe1 [2537285,70324] 0 2024-06-15T01:07:15.519 INFO:tasks.workunit.client.0.smithi137.stdout:4/749: mknod dd/d15/d36/d45/d55/d8d/c115 0 2024-06-15T01:07:15.523 INFO:tasks.workunit.client.0.smithi137.stdout:9/541: fsync f3 0 2024-06-15T01:07:15.526 INFO:tasks.workunit.client.0.smithi137.stdout:6/619: getdents d0/d5/d9/d29/d6b 0 2024-06-15T01:07:15.527 INFO:tasks.workunit.client.0.smithi137.stdout:8/546: fsync d9/f1b 0 2024-06-15T01:07:15.528 INFO:tasks.workunit.client.0.smithi137.stdout:6/620: truncate d0/d1a/d68/f9f 702585 0 2024-06-15T01:07:15.533 INFO:tasks.workunit.client.0.smithi137.stdout:8/547: unlink d9/f1b 0 2024-06-15T01:07:15.537 INFO:tasks.workunit.client.0.smithi137.stdout:0/893: link d1/d2/d55/faf d1/d7e/f127 0 2024-06-15T01:07:15.538 INFO:tasks.workunit.client.0.smithi137.stdout:9/542: mkdir df/d12/d29/d79/d9a 0 2024-06-15T01:07:15.540 INFO:tasks.workunit.client.0.smithi137.stdout:7/559: creat df/d7d/d3b/fb6 x:0 0 0 2024-06-15T01:07:15.542 INFO:tasks.workunit.client.0.smithi137.stdout:6/621: truncate d0/d1a/d3e/d57/d1d/d2e/f8e 354425 0 2024-06-15T01:07:15.544 INFO:tasks.workunit.client.0.smithi137.stdout:0/894: mknod d1/d2/dc/d10/d14/d56/d105/c128 0 2024-06-15T01:07:15.545 INFO:tasks.workunit.client.0.smithi137.stdout:4/750: getdents dd/d15/d21/d3a/d6e 0 2024-06-15T01:07:15.546 INFO:tasks.workunit.client.0.smithi137.stdout:9/543: dwrite df/d12/d15/f5f [0,4194304] 0 2024-06-15T01:07:15.548 INFO:tasks.workunit.client.0.smithi137.stdout:7/560: rmdir df/d13/d2e/d1a/d6a 39 2024-06-15T01:07:15.557 INFO:tasks.workunit.client.0.smithi137.stdout:8/548: write d9/d22/f37 [27692,97845] 0 2024-06-15T01:07:15.557 INFO:tasks.workunit.client.0.smithi137.stdout:8/549: chown d9/d2a/f6f 10512186 1 2024-06-15T01:07:15.558 INFO:tasks.workunit.client.0.smithi137.stdout:6/622: mknod d0/d1a/cc3 0 2024-06-15T01:07:15.559 INFO:tasks.workunit.client.0.smithi137.stdout:7/561: dwrite df/d13/d2e/d1a/f2c [0,4194304] 0 2024-06-15T01:07:15.565 INFO:tasks.workunit.client.0.smithi137.stdout:0/895: truncate d1/d2/d55/f6e 3301674 0 2024-06-15T01:07:15.566 INFO:tasks.workunit.client.0.smithi137.stdout:9/544: write df/f4d [1472206,62435] 0 2024-06-15T01:07:15.567 INFO:tasks.workunit.client.0.smithi137.stdout:7/562: write df/d13/d2e/d1a/d3f/f54 [2213523,121912] 0 2024-06-15T01:07:15.570 INFO:tasks.workunit.client.0.smithi137.stdout:4/751: rename dd/d15/d36/d45/dcb/f85 to dd/d15/d36/d45/dcb/d83/db2/f116 0 2024-06-15T01:07:15.572 INFO:tasks.workunit.client.0.smithi137.stdout:0/896: mknod d1/d2/dc/d10/d22/db7/d11f/c129 0 2024-06-15T01:07:15.572 INFO:tasks.workunit.client.0.smithi137.stdout:7/563: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/db7 0 2024-06-15T01:07:15.574 INFO:tasks.workunit.client.0.smithi137.stdout:8/550: rename d9/l9b to d9/d2a/d79/lb5 0 2024-06-15T01:07:15.575 INFO:tasks.workunit.client.0.smithi137.stdout:7/564: rmdir df/d13/d3c 39 2024-06-15T01:07:15.576 INFO:tasks.workunit.client.0.smithi137.stdout:7/565: fsync df/fb4 0 2024-06-15T01:07:15.578 INFO:tasks.workunit.client.0.smithi137.stdout:0/897: creat d1/d7e/d87/da0/dc4/f12a x:0 0 0 2024-06-15T01:07:15.579 INFO:tasks.workunit.client.0.smithi137.stdout:6/623: truncate d0/d5/f83 104773 0 2024-06-15T01:07:15.579 INFO:tasks.workunit.client.0.smithi137.stdout:0/898: stat d1/d2/dc/d6c/d90 0 2024-06-15T01:07:15.580 INFO:tasks.workunit.client.0.smithi137.stdout:6/624: dread - d0/d5/d9/d29/d6b/f9d zero size 2024-06-15T01:07:15.582 INFO:tasks.workunit.client.0.smithi137.stdout:0/899: write d1/d2/dc/d10/d14/d56/f109 [3474902,31573] 0 2024-06-15T01:07:15.583 INFO:tasks.workunit.client.0.smithi137.stdout:4/752: rename dd/la3 to dd/dbf/l117 0 2024-06-15T01:07:15.584 INFO:tasks.workunit.client.0.smithi137.stdout:0/900: write d1/d2/dc/d10/d22/db7/f11d [342942,121685] 0 2024-06-15T01:07:15.585 INFO:tasks.workunit.client.0.smithi137.stdout:7/566: rmdir df/d7d/d1c/d1d/d27/d33/d43 39 2024-06-15T01:07:15.586 INFO:tasks.workunit.client.0.smithi137.stdout:7/567: write df/fb4 [61132,7932] 0 2024-06-15T01:07:15.587 INFO:tasks.workunit.client.0.smithi137.stdout:0/901: write d1/d7e/d87/da0/dc4/dd7/de2/f6b [3418065,5885] 0 2024-06-15T01:07:15.590 INFO:tasks.workunit.client.0.smithi137.stdout:6/625: symlink d0/d1a/d3e/d87/lc4 0 2024-06-15T01:07:15.591 INFO:tasks.workunit.client.0.smithi137.stdout:9/545: dread df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:15.593 INFO:tasks.workunit.client.0.smithi137.stdout:8/551: rename d9/d2a/d2f/d59/d82/c85 to d9/d22/cb6 0 2024-06-15T01:07:15.598 INFO:tasks.workunit.client.0.smithi137.stdout:0/902: dwrite d1/d2/d13/d66/f3f [0,4194304] 0 2024-06-15T01:07:15.601 INFO:tasks.workunit.client.0.smithi137.stdout:9/546: mkdir df/d49/d4b/d9b 0 2024-06-15T01:07:15.609 INFO:tasks.workunit.client.0.smithi137.stdout:8/552: rename d9/de/d1e/d3f/da5 to d9/de/d1e/d3f/dab/db7 0 2024-06-15T01:07:15.610 INFO:tasks.workunit.client.0.smithi137.stdout:7/568: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fb8 x:0 0 0 2024-06-15T01:07:15.610 INFO:tasks.workunit.client.0.smithi137.stdout:9/547: dwrite df/d12/d15/f26 [0,4194304] 0 2024-06-15T01:07:15.611 INFO:tasks.workunit.client.0.smithi137.stdout:0/903: mkdir d1/df6/d12b 0 2024-06-15T01:07:15.612 INFO:tasks.workunit.client.0.smithi137.stdout:6/626: mknod d0/d1a/d3e/d57/cc5 0 2024-06-15T01:07:15.615 INFO:tasks.workunit.client.0.smithi137.stdout:9/548: mknod df/d12/d1b/d31/d42/c9c 0 2024-06-15T01:07:15.693 INFO:tasks.workunit.client.0.smithi137.stdout:8/553: rename d9/lad to d9/d22/lb8 0 2024-06-15T01:07:15.698 INFO:tasks.workunit.client.0.smithi137.stdout:8/554: mkdir d9/d2a/db9 0 2024-06-15T01:07:15.699 INFO:tasks.workunit.client.0.smithi137.stdout:8/555: truncate f8 5896331 0 2024-06-15T01:07:15.701 INFO:tasks.workunit.client.0.smithi137.stdout:8/556: stat d9/d2a/d79/fa3 0 2024-06-15T01:07:15.701 INFO:tasks.workunit.client.0.smithi137.stdout:9/549: getdents df/d1a 0 2024-06-15T01:07:15.702 INFO:tasks.workunit.client.0.smithi137.stdout:4/753: dwrite dd/d15/d36/d45/d55/d8d/f63 [0,4194304] 0 2024-06-15T01:07:15.703 INFO:tasks.workunit.client.0.smithi137.stdout:8/557: stat d9/de/d1e/d35/d5f/fb3 0 2024-06-15T01:07:15.705 INFO:tasks.workunit.client.0.smithi137.stdout:9/550: mkdir df/d12/d29/d9d 0 2024-06-15T01:07:15.705 INFO:tasks.workunit.client.0.smithi137.stdout:0/904: write d1/d2/dc/d10/d14/d56/f85 [3351783,57982] 0 2024-06-15T01:07:15.705 INFO:tasks.workunit.client.0.smithi137.stdout:9/551: chown df/d1a 1246 1 2024-06-15T01:07:15.707 INFO:tasks.workunit.client.0.smithi137.stdout:0/905: readlink d1/d2/dc/d10/d53/da2/la3 0 2024-06-15T01:07:15.709 INFO:tasks.workunit.client.0.smithi137.stdout:6/627: write d0/d1a/f75 [867954,5831] 0 2024-06-15T01:07:15.712 INFO:tasks.workunit.client.0.smithi137.stdout:9/552: mknod df/d12/d15/d16/d3c/c9e 0 2024-06-15T01:07:15.714 INFO:tasks.workunit.client.0.smithi137.stdout:8/558: fdatasync d9/d22/f25 0 2024-06-15T01:07:15.716 INFO:tasks.workunit.client.0.smithi137.stdout:4/754: unlink dd/d15/d36/d45/d55/d8d/f70 0 2024-06-15T01:07:15.718 INFO:tasks.workunit.client.0.smithi137.stdout:6/628: dread - d0/d5/d9/d29/fbc zero size 2024-06-15T01:07:15.720 INFO:tasks.workunit.client.0.smithi137.stdout:7/569: dwrite df/d7d/d1c/d1d/d27/d33/f9b [0,4194304] 0 2024-06-15T01:07:15.721 INFO:tasks.workunit.client.0.smithi137.stdout:7/570: truncate df/f32 4284991 0 2024-06-15T01:07:15.723 INFO:tasks.workunit.client.0.smithi137.stdout:4/755: truncate dd/f22 3409060 0 2024-06-15T01:07:15.726 INFO:tasks.workunit.client.0.smithi137.stdout:6/629: unlink d0/d1a/d3e/d57/d1d/d2e/d62/f76 0 2024-06-15T01:07:15.731 INFO:tasks.workunit.client.0.smithi137.stdout:7/571: creat df/d7d/d3b/d5b/fb9 x:0 0 0 2024-06-15T01:07:15.733 INFO:tasks.workunit.client.0.smithi137.stdout:4/756: unlink dd/d15/d30/c6f 0 2024-06-15T01:07:15.736 INFO:tasks.workunit.client.0.smithi137.stdout:9/553: truncate df/d12/d15/d75/f8a 1042623 0 2024-06-15T01:07:15.737 INFO:tasks.workunit.client.0.smithi137.stdout:0/906: getdents d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a 0 2024-06-15T01:07:15.738 INFO:tasks.workunit.client.0.smithi137.stdout:7/572: fsync df/d13/d2e/d1a/d3f/d89/f8a 0 2024-06-15T01:07:15.744 INFO:tasks.workunit.client.0.smithi137.stdout:4/757: dwrite dd/f18 [8388608,4194304] 0 2024-06-15T01:07:15.747 INFO:tasks.workunit.client.0.smithi137.stdout:8/559: dread d9/d2a/f97 [0,4194304] 0 2024-06-15T01:07:15.750 INFO:tasks.workunit.client.0.smithi137.stdout:7/573: creat df/d13/d2e/d1a/d8c/fba x:0 0 0 2024-06-15T01:07:15.750 INFO:tasks.workunit.client.0.smithi137.stdout:7/574: dread - df/d7d/d3b/fb6 zero size 2024-06-15T01:07:15.752 INFO:tasks.workunit.client.0.smithi137.stdout:9/554: link df/d12/d29/d66/l68 df/d1a/l9f 0 2024-06-15T01:07:15.756 INFO:tasks.workunit.client.0.smithi137.stdout:4/758: rename dd/d15/c2d to dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d10b/c118 0 2024-06-15T01:07:15.757 INFO:tasks.workunit.client.0.smithi137.stdout:4/759: dread - dd/d15/f98 zero size 2024-06-15T01:07:15.759 INFO:tasks.workunit.client.0.smithi137.stdout:8/560: mknod d9/d2a/d2f/d59/d82/cba 0 2024-06-15T01:07:15.761 INFO:tasks.workunit.client.0.smithi137.stdout:9/555: fdatasync df/f6b 0 2024-06-15T01:07:15.762 INFO:tasks.workunit.client.0.smithi137.stdout:9/556: chown df/d2a/c65 899067 1 2024-06-15T01:07:15.763 INFO:tasks.workunit.client.0.smithi137.stdout:0/907: link d1/d2/dc/d10/d14/d56/d117/f57 d1/df6/d11b/f12c 0 2024-06-15T01:07:15.764 INFO:tasks.workunit.client.0.smithi137.stdout:0/908: chown d1/d2/dc/d10/d14/d56 460118915 1 2024-06-15T01:07:15.764 INFO:tasks.workunit.client.0.smithi137.stdout:4/760: fdatasync dd/d15/d21/d9c/d75/d99/fa9 0 2024-06-15T01:07:15.766 INFO:tasks.workunit.client.0.smithi137.stdout:7/575: creat df/d7d/d3b/d5b/da6/fbb x:0 0 0 2024-06-15T01:07:15.768 INFO:tasks.workunit.client.0.smithi137.stdout:8/561: unlink d9/d22/f25 0 2024-06-15T01:07:15.771 INFO:tasks.workunit.client.0.smithi137.stdout:9/557: rmdir df/d12/d15/d75 39 2024-06-15T01:07:15.773 INFO:tasks.workunit.client.0.smithi137.stdout:9/558: read df/d13/f22 [2206825,89785] 0 2024-06-15T01:07:15.777 INFO:tasks.workunit.client.0.smithi137.stdout:7/576: fsync df/d7d/d1c/f1f 0 2024-06-15T01:07:15.778 INFO:tasks.workunit.client.0.smithi137.stdout:8/562: truncate d9/d2a/f7a 1635745 0 2024-06-15T01:07:15.781 INFO:tasks.workunit.client.0.smithi137.stdout:9/559: rename df/d12/d15/d16/d3c/d51/c7a to df/d12/d15/d16/d3c/d51/d8f/ca0 0 2024-06-15T01:07:15.789 INFO:tasks.workunit.client.0.smithi137.stdout:7/577: dread df/d7d/d1c/d1d/d27/f49 [0,4194304] 0 2024-06-15T01:07:15.790 INFO:tasks.workunit.client.0.smithi137.stdout:4/761: link dd/d15/d36/d45/d8f/dac/de5/fef dd/d15/d21/d9c/d75/df5/f119 0 2024-06-15T01:07:15.791 INFO:tasks.workunit.client.0.smithi137.stdout:9/560: creat df/d12/d29/d79/fa1 x:0 0 0 2024-06-15T01:07:15.791 INFO:tasks.workunit.client.0.smithi137.stdout:8/563: write d9/f1d [3908356,88284] 0 2024-06-15T01:07:15.792 INFO:tasks.workunit.client.0.smithi137.stdout:7/578: rmdir df/d7d/d1c/d1d 39 2024-06-15T01:07:15.796 INFO:tasks.workunit.client.0.smithi137.stdout:8/564: mkdir d9/de/d1e/d3f/dab/db7/dbb 0 2024-06-15T01:07:15.798 INFO:tasks.workunit.client.0.smithi137.stdout:7/579: dread - df/d7d/d1c/d1d/d27/d33/d43/d6c/f96 zero size 2024-06-15T01:07:15.802 INFO:tasks.workunit.client.0.smithi137.stdout:9/561: unlink df/f4d 0 2024-06-15T01:07:15.803 INFO:tasks.workunit.client.0.smithi137.stdout:8/565: creat d9/d2a/d2f/d59/fbc x:0 0 0 2024-06-15T01:07:15.803 INFO:tasks.workunit.client.0.smithi137.stdout:7/580: fdatasync df/d7d/d3b/d5b/f6d 0 2024-06-15T01:07:15.807 INFO:tasks.workunit.client.0.smithi137.stdout:7/581: mkdir df/d7d/d1c/d1d/d79/dbc 0 2024-06-15T01:07:15.808 INFO:tasks.workunit.client.0.smithi137.stdout:8/566: creat d9/de/fbd x:0 0 0 2024-06-15T01:07:15.811 INFO:tasks.workunit.client.0.smithi137.stdout:4/762: truncate dd/d15/d36/d45/d55/d8d/f63 2223408 0 2024-06-15T01:07:15.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:15 smithi142 ceph-mon[37379]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 101 MiB/s rd, 254 MiB/s wr, 529 op/s 2024-06-15T01:07:15.821 INFO:tasks.workunit.client.0.smithi137.stdout:8/567: write d9/d2a/f7a [530560,120905] 0 2024-06-15T01:07:15.823 INFO:tasks.workunit.client.0.smithi137.stdout:7/582: link df/d13/d2e/d1a/l7c df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/lbd 0 2024-06-15T01:07:15.826 INFO:tasks.workunit.client.0.smithi137.stdout:7/583: fsync df/d7d/d3b/f5f 0 2024-06-15T01:07:15.828 INFO:tasks.workunit.client.0.smithi137.stdout:7/584: dread - df/d7d/d3b/d5b/fb9 zero size 2024-06-15T01:07:15.830 INFO:tasks.workunit.client.0.smithi137.stdout:4/763: dread dd/d15/d36/d45/d55/f95 [0,4194304] 0 2024-06-15T01:07:15.834 INFO:tasks.workunit.client.0.smithi137.stdout:8/568: getdents d9/d2a/d79 0 2024-06-15T01:07:15.837 INFO:tasks.workunit.client.0.smithi137.stdout:7/585: dwrite df/d7d/d1c/d1d/d27/d97/fb1 [0,4194304] 0 2024-06-15T01:07:15.839 INFO:tasks.workunit.client.0.smithi137.stdout:7/586: chown df/d13/d2e/d1a/d3f/l53 13321 1 2024-06-15T01:07:15.840 INFO:tasks.workunit.client.0.smithi137.stdout:7/587: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/c91 1393457 1 2024-06-15T01:07:15.841 INFO:tasks.workunit.client.0.smithi137.stdout:4/764: truncate dd/d15/d21/d9c/d75/fae 2391913 0 2024-06-15T01:07:15.843 INFO:tasks.workunit.client.0.smithi137.stdout:8/569: link d9/de/f94 d9/d22/fbe 0 2024-06-15T01:07:15.845 INFO:tasks.workunit.client.0.smithi137.stdout:4/765: unlink dd/d15/d36/d45/d55/d8d/c115 0 2024-06-15T01:07:15.847 INFO:tasks.workunit.client.0.smithi137.stdout:4/766: write dd/d15/d21/d3a/d6e/d7b/f114 [632938,29466] 0 2024-06-15T01:07:15.858 INFO:tasks.workunit.client.0.smithi137.stdout:7/588: dwrite df/d7d/d1c/d1d/d27/d33/f52 [0,4194304] 0 2024-06-15T01:07:15.901 INFO:tasks.workunit.client.0.smithi137.stdout:8/570: getdents d9/de/d1e/db1 0 2024-06-15T01:07:15.906 INFO:tasks.workunit.client.0.smithi137.stdout:4/767: creat dd/d15/d21/d9c/d75/df5/f11a x:0 0 0 2024-06-15T01:07:15.907 INFO:tasks.workunit.client.0.smithi137.stdout:8/571: symlink d9/d2a/d2f/d59/d82/lbf 0 2024-06-15T01:07:15.910 INFO:tasks.workunit.client.0.smithi137.stdout:4/768: symlink dd/d15/d36/d45/dcb/d83/db2/d72/d81/l11b 0 2024-06-15T01:07:15.912 INFO:tasks.workunit.client.0.smithi137.stdout:8/572: creat d9/de/d1e/d3f/d84/fc0 x:0 0 0 2024-06-15T01:07:15.917 INFO:tasks.workunit.client.0.smithi137.stdout:4/769: mknod dd/d15/d36/d45/dcb/d83/db2/d72/c11c 0 2024-06-15T01:07:15.918 INFO:tasks.workunit.client.0.smithi137.stdout:4/770: chown dd/d15/d36/d45/d55/f7c 17 1 2024-06-15T01:07:15.919 INFO:tasks.workunit.client.0.smithi137.stdout:8/573: mknod d9/de/d1e/d67/cc1 0 2024-06-15T01:07:15.919 INFO:tasks.workunit.client.0.smithi137.stdout:4/771: read - dd/d15/d21/f9a zero size 2024-06-15T01:07:15.924 INFO:tasks.workunit.client.0.smithi137.stdout:4/772: creat dd/d15/d36/d45/dcb/d5b/f11d x:0 0 0 2024-06-15T01:07:15.929 INFO:tasks.workunit.client.0.smithi137.stdout:8/574: creat d9/de/fc2 x:0 0 0 2024-06-15T01:07:15.932 INFO:tasks.workunit.client.0.smithi137.stdout:8/575: fdatasync d9/d22/f98 0 2024-06-15T01:07:15.940 INFO:tasks.workunit.client.0.smithi137.stdout:8/576: link d9/de/f3c d9/fc3 0 2024-06-15T01:07:15.940 INFO:tasks.workunit.client.0.smithi137.stdout:8/577: dread - d9/de/fc2 zero size 2024-06-15T01:07:15.946 INFO:tasks.workunit.client.0.smithi137.stdout:8/578: getdents d9/d2a/d2f/d59/d82 0 2024-06-15T01:07:15.948 INFO:tasks.workunit.client.0.smithi137.stdout:4/773: dwrite dd/d15/d30/f46 [0,4194304] 0 2024-06-15T01:07:15.950 INFO:tasks.workunit.client.0.smithi137.stdout:8/579: mknod d9/d2a/d79/d7e/cc4 0 2024-06-15T01:07:15.952 INFO:tasks.workunit.client.0.smithi137.stdout:8/580: truncate d9/d2a/d79/d7e/fb4 101912 0 2024-06-15T01:07:15.954 INFO:tasks.workunit.client.0.smithi137.stdout:8/581: symlink d9/de/d1e/d67/lc5 0 2024-06-15T01:07:15.956 INFO:tasks.workunit.client.0.smithi137.stdout:8/582: rmdir d9/d2a/d79 39 2024-06-15T01:07:15.957 INFO:tasks.workunit.client.0.smithi137.stdout:4/774: getdents dd/d15/d36/d45/d55/d8d/d1f/d86/dd6 0 2024-06-15T01:07:15.958 INFO:tasks.workunit.client.0.smithi137.stdout:8/583: fsync d9/de/d1e/d35/f63 0 2024-06-15T01:07:15.965 INFO:tasks.workunit.client.0.smithi137.stdout:8/584: rename d9/de to d9/d2a/db9/dc6 0 2024-06-15T01:07:15.968 INFO:tasks.workunit.client.0.smithi137.stdout:8/585: symlink d9/d2a/d2f/d59/d82/lc7 0 2024-06-15T01:07:15.970 INFO:tasks.workunit.client.0.smithi137.stdout:8/586: fdatasync d9/d2a/d79/fa3 0 2024-06-15T01:07:15.971 INFO:tasks.workunit.client.0.smithi137.stdout:4/775: dwrite dd/d15/d36/d45/d55/d8d/f1c [0,4194304] 0 2024-06-15T01:07:15.972 INFO:tasks.workunit.client.0.smithi137.stdout:8/587: chown d9/d22/lb8 142502 1 2024-06-15T01:07:15.973 INFO:tasks.workunit.client.0.smithi137.stdout:8/588: write d9/d22/fa8 [380755,73101] 0 2024-06-15T01:07:15.975 INFO:tasks.workunit.client.0.smithi137.stdout:4/776: fsync dd/d15/d36/d45/dcb/f93 0 2024-06-15T01:07:15.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:15 smithi137 ceph-mon[29354]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 101 MiB/s rd, 254 MiB/s wr, 529 op/s 2024-06-15T01:07:15.977 INFO:tasks.workunit.client.0.smithi137.stdout:8/589: mkdir d9/d2a/db9/dc6/d1e/d35/dc8 0 2024-06-15T01:07:15.979 INFO:tasks.workunit.client.0.smithi137.stdout:4/777: mkdir dd/d15/d36/d45/d55/d8d/d1f/d11e 0 2024-06-15T01:07:15.984 INFO:tasks.workunit.client.0.smithi137.stdout:4/778: dread dd/d15/d36/d45/dcb/d83/db2/d72/d81/fc5 [0,4194304] 0 2024-06-15T01:07:15.987 INFO:tasks.workunit.client.0.smithi137.stdout:4/779: mknod dd/d15/d36/d45/d55/c11f 0 2024-06-15T01:07:15.989 INFO:tasks.workunit.client.0.smithi137.stdout:4/780: rename dd/c87 to dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/c120 0 2024-06-15T01:07:15.993 INFO:tasks.workunit.client.0.smithi137.stdout:4/781: link dd/d15/d36/d45/d8f/dac/de5/c10f dd/d15/d21/d3a/d6e/d7b/dee/c121 0 2024-06-15T01:07:16.006 INFO:tasks.workunit.client.0.smithi137.stdout:4/782: dwrite dd/f47 [0,4194304] 0 2024-06-15T01:07:16.007 INFO:tasks.workunit.client.0.smithi137.stdout:4/783: creat dd/d15/d36/d45/d8f/dac/de5/f122 x:0 0 0 2024-06-15T01:07:16.010 INFO:tasks.workunit.client.0.smithi137.stdout:4/784: dread - dd/d15/d30/fd2 zero size 2024-06-15T01:07:16.013 INFO:tasks.workunit.client.0.smithi137.stdout:4/785: truncate dd/d15/d36/d45/d55/d8d/f63 2867850 0 2024-06-15T01:07:16.015 INFO:tasks.workunit.client.0.smithi137.stdout:4/786: unlink dd/f22 0 2024-06-15T01:07:16.021 INFO:tasks.workunit.client.0.smithi137.stdout:4/787: dwrite dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:07:16.027 INFO:tasks.workunit.client.0.smithi137.stdout:4/788: link dd/dbf/ffa dd/d15/d36/d45/d55/d8d/d1f/d86/f123 0 2024-06-15T01:07:16.027 INFO:tasks.workunit.client.0.smithi137.stdout:4/789: chown dd/d15/d30 1546 1 2024-06-15T01:07:16.029 INFO:tasks.workunit.client.0.smithi137.stdout:4/790: chown dd/d15/d36/d45/ca2 159982 1 2024-06-15T01:07:16.048 INFO:tasks.workunit.client.0.smithi137.stdout:4/791: dread dd/d15/d36/d45/fd7 [0,4194304] 0 2024-06-15T01:07:16.062 INFO:tasks.workunit.client.0.smithi137.stdout:4/792: dread dd/d15/d21/d9c/d75/fc2 [0,4194304] 0 2024-06-15T01:07:16.099 INFO:tasks.workunit.client.0.smithi137.stdout:5/695: sync 2024-06-15T01:07:16.099 INFO:tasks.workunit.client.0.smithi137.stdout:2/565: sync 2024-06-15T01:07:16.101 INFO:tasks.workunit.client.0.smithi137.stdout:5/696: mknod df/d11/d14/d6c/ce9 0 2024-06-15T01:07:16.107 INFO:tasks.workunit.client.0.smithi137.stdout:5/697: dwrite df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:16.108 INFO:tasks.workunit.client.0.smithi137.stdout:5/698: chown df/d44/cd5 2 1 2024-06-15T01:07:16.109 INFO:tasks.workunit.client.0.smithi137.stdout:2/566: truncate d0/dc/d22/f4b 208479 0 2024-06-15T01:07:16.112 INFO:tasks.workunit.client.0.smithi137.stdout:5/699: rename df/d7a to df/d44/d54/d95/d35/dea 0 2024-06-15T01:07:16.115 INFO:tasks.workunit.client.0.smithi137.stdout:2/567: rename d0/d42/f5a to d0/dc/d2e/d72/d56/fa5 0 2024-06-15T01:07:16.118 INFO:tasks.workunit.client.0.smithi137.stdout:2/568: unlink d0/dc/d2e/d72/d56/d6e/c80 0 2024-06-15T01:07:16.120 INFO:tasks.workunit.client.0.smithi137.stdout:2/569: rename d0/dc/c35 to d0/de/ca6 0 2024-06-15T01:07:16.120 INFO:tasks.workunit.client.0.smithi137.stdout:2/570: chown d0/dc/f2c 884042 1 2024-06-15T01:07:16.123 INFO:tasks.workunit.client.0.smithi137.stdout:2/571: mknod d0/ca7 0 2024-06-15T01:07:16.132 INFO:tasks.workunit.client.0.smithi137.stdout:2/572: write d0/dc/d22/f4b [266358,19136] 0 2024-06-15T01:07:16.135 INFO:tasks.workunit.client.0.smithi137.stdout:5/700: dwrite df/d11/f31 [0,4194304] 0 2024-06-15T01:07:16.187 INFO:tasks.workunit.client.0.smithi137.stdout:5/701: link df/d44/c74 df/d44/d54/d95/d3b/d49/d5c/de7/ceb 0 2024-06-15T01:07:16.189 INFO:tasks.workunit.client.0.smithi137.stdout:5/702: fdatasync df/d11/d14/d6c/f90 0 2024-06-15T01:07:16.192 INFO:tasks.workunit.client.0.smithi137.stdout:5/703: mkdir df/d44/d54/d95/d35/dea/dec 0 2024-06-15T01:07:16.194 INFO:tasks.workunit.client.0.smithi137.stdout:2/573: dread d0/da/da0/f19 [0,4194304] 0 2024-06-15T01:07:16.195 INFO:tasks.workunit.client.0.smithi137.stdout:2/574: write d0/d42/d49/f62 [1253547,109589] 0 2024-06-15T01:07:16.198 INFO:tasks.workunit.client.0.smithi137.stdout:5/704: dread df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:16.199 INFO:tasks.workunit.client.0.smithi137.stdout:2/575: unlink d0/da/d73/d8d/d5c/f8b 0 2024-06-15T01:07:16.200 INFO:tasks.workunit.client.0.smithi137.stdout:2/576: chown d0/de/d2b/c68 463572536 1 2024-06-15T01:07:16.200 INFO:tasks.workunit.client.0.smithi137.stdout:5/705: symlink df/d44/d54/d95/led 0 2024-06-15T01:07:16.204 INFO:tasks.workunit.client.0.smithi137.stdout:2/577: link d0/f8 d0/dc/d2e/d72/fa8 0 2024-06-15T01:07:16.210 INFO:tasks.workunit.client.0.smithi137.stdout:5/706: link df/d44/d54/d95/d3b/f59 df/d44/d54/d6b/fee 0 2024-06-15T01:07:16.210 INFO:tasks.workunit.client.0.smithi137.stdout:5/707: unlink df/d44/c5b 0 2024-06-15T01:07:16.213 INFO:tasks.workunit.client.0.smithi137.stdout:2/578: write d0/de/f24 [759957,63270] 0 2024-06-15T01:07:16.218 INFO:tasks.workunit.client.0.smithi137.stdout:5/708: dwrite df/d76/d7c/d9c/fe6 [0,4194304] 0 2024-06-15T01:07:16.236 INFO:tasks.workunit.client.0.smithi137.stdout:2/579: dread d0/dc/f3d [0,4194304] 0 2024-06-15T01:07:16.240 INFO:tasks.workunit.client.0.smithi137.stdout:5/709: dwrite df/d44/d54/d95/d3b/f59 [0,4194304] 0 2024-06-15T01:07:16.241 INFO:tasks.workunit.client.0.smithi137.stdout:5/710: fsync df/d11/fd9 0 2024-06-15T01:07:16.242 INFO:tasks.workunit.client.0.smithi137.stdout:2/580: getdents d0/dc 0 2024-06-15T01:07:16.244 INFO:tasks.workunit.client.0.smithi137.stdout:2/581: rename d0/d42/d49/d57 to d0/d42/d49/d57/da9 22 2024-06-15T01:07:16.245 INFO:tasks.workunit.client.0.smithi137.stdout:2/582: read - d0/dc/d2e/d72/f9c zero size 2024-06-15T01:07:16.246 INFO:tasks.workunit.client.0.smithi137.stdout:5/711: creat df/d11/d14/fef x:0 0 0 2024-06-15T01:07:16.249 INFO:tasks.workunit.client.0.smithi137.stdout:5/712: mknod df/d44/d54/d95/d3b/d49/d5c/cf0 0 2024-06-15T01:07:16.250 INFO:tasks.workunit.client.0.smithi137.stdout:5/713: chown df/d44/d54/d6b/fb3 157 1 2024-06-15T01:07:16.251 INFO:tasks.workunit.client.0.smithi137.stdout:5/714: readlink df/d76/d7c/lcb 0 2024-06-15T01:07:16.253 INFO:tasks.workunit.client.0.smithi137.stdout:5/715: fsync df/d44/f4d 0 2024-06-15T01:07:16.254 INFO:tasks.workunit.client.0.smithi137.stdout:5/716: read - df/d76/d8e/fdf zero size 2024-06-15T01:07:16.256 INFO:tasks.workunit.client.0.smithi137.stdout:2/583: dwrite d0/dc/d2e/d72/f67 [0,4194304] 0 2024-06-15T01:07:16.257 INFO:tasks.workunit.client.0.smithi137.stdout:2/584: write d0/d42/d49/f62 [145269,67819] 0 2024-06-15T01:07:16.260 INFO:tasks.workunit.client.0.smithi137.stdout:2/585: creat d0/da/d73/d8d/d5c/d90/faa x:0 0 0 2024-06-15T01:07:16.262 INFO:tasks.workunit.client.0.smithi137.stdout:2/586: rmdir d0/de/d2b/d47 39 2024-06-15T01:07:16.263 INFO:tasks.workunit.client.0.smithi137.stdout:5/717: truncate df/f4a 4618354 0 2024-06-15T01:07:16.264 INFO:tasks.workunit.client.0.smithi137.stdout:2/587: mknod d0/d42/cab 0 2024-06-15T01:07:16.268 INFO:tasks.workunit.client.0.smithi137.stdout:5/718: truncate df/d44/d54/d6b/fb3 134367 0 2024-06-15T01:07:16.276 INFO:tasks.workunit.client.0.smithi137.stdout:5/719: write df/d11/d30/f61 [533234,41782] 0 2024-06-15T01:07:16.278 INFO:tasks.workunit.client.0.smithi137.stdout:2/588: dwrite d0/da/da0/f2a [0,4194304] 0 2024-06-15T01:07:16.280 INFO:tasks.workunit.client.0.smithi137.stdout:5/720: mkdir df/d44/d54/d95/d3b/df1 0 2024-06-15T01:07:16.283 INFO:tasks.workunit.client.0.smithi137.stdout:5/721: symlink df/d76/d7c/d9c/lf2 0 2024-06-15T01:07:16.285 INFO:tasks.workunit.client.0.smithi137.stdout:5/722: dread - df/d76/d7c/d9c/da1/fc5 zero size 2024-06-15T01:07:16.287 INFO:tasks.workunit.client.0.smithi137.stdout:5/723: chown df/d11/d14/f16 13453414 1 2024-06-15T01:07:16.290 INFO:tasks.workunit.client.0.smithi137.stdout:5/724: mknod df/d44/d54/d95/d3b/d4b/cf3 0 2024-06-15T01:07:16.292 INFO:tasks.workunit.client.0.smithi137.stdout:2/589: truncate d0/dc/d2e/d72/f59 3717614 0 2024-06-15T01:07:16.295 INFO:tasks.workunit.client.0.smithi137.stdout:5/725: rename df/l1b to df/d11/d14/lf4 0 2024-06-15T01:07:16.296 INFO:tasks.workunit.client.0.smithi137.stdout:2/590: unlink d0/da/d73/d8d/d5c/d99/c9b 0 2024-06-15T01:07:16.299 INFO:tasks.workunit.client.0.smithi137.stdout:2/591: symlink d0/dc/d2e/d72/d56/lac 0 2024-06-15T01:07:16.299 INFO:tasks.workunit.client.0.smithi137.stdout:2/592: chown d0/f1f 15 1 2024-06-15T01:07:16.301 INFO:tasks.workunit.client.0.smithi137.stdout:2/593: mknod d0/da/d73/d8d/cad 0 2024-06-15T01:07:16.316 INFO:tasks.workunit.client.0.smithi137.stdout:5/726: dread df/f4a [0,4194304] 0 2024-06-15T01:07:16.318 INFO:tasks.workunit.client.0.smithi137.stdout:5/727: symlink df/d11/d30/lf5 0 2024-06-15T01:07:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:5/728: creat df/d11/d30/ff6 x:0 0 0 2024-06-15T01:07:16.328 INFO:tasks.workunit.client.0.smithi137.stdout:5/729: write df/d11/d14/f15 [5206271,88220] 0 2024-06-15T01:07:16.331 INFO:tasks.workunit.client.0.smithi137.stdout:5/730: creat df/d11/ff7 x:0 0 0 2024-06-15T01:07:16.332 INFO:tasks.workunit.client.0.smithi137.stdout:5/731: chown df/d44/d54/d6b 5304058 1 2024-06-15T01:07:16.333 INFO:tasks.workunit.client.0.smithi137.stdout:5/732: chown df/d44/f4e 2860 1 2024-06-15T01:07:16.334 INFO:tasks.workunit.client.0.smithi137.stdout:5/733: creat df/d3d/ff8 x:0 0 0 2024-06-15T01:07:16.336 INFO:tasks.workunit.client.0.smithi137.stdout:5/734: mkdir df/d3d/df9 0 2024-06-15T01:07:16.338 INFO:tasks.workunit.client.0.smithi137.stdout:5/735: mkdir df/d11/dfa 0 2024-06-15T01:07:16.339 INFO:tasks.workunit.client.0.smithi137.stdout:5/736: mkdir df/d11/dfa/dfb 0 2024-06-15T01:07:16.342 INFO:tasks.workunit.client.0.smithi137.stdout:5/737: rename df/d76/d7c/c9e to df/d3d/cfc 0 2024-06-15T01:07:16.345 INFO:tasks.workunit.client.0.smithi137.stdout:5/738: creat df/d44/ffd x:0 0 0 2024-06-15T01:07:16.351 INFO:tasks.workunit.client.0.smithi137.stdout:5/739: dwrite df/d44/d54/d95/d3b/d4b/fc0 [0,4194304] 0 2024-06-15T01:07:16.352 INFO:tasks.workunit.client.0.smithi137.stdout:5/740: chown df/d11/d30/f7b 17963 1 2024-06-15T01:07:16.354 INFO:tasks.workunit.client.0.smithi137.stdout:5/741: mknod df/d44/d54/d95/d35/cfe 0 2024-06-15T01:07:16.360 INFO:tasks.workunit.client.0.smithi137.stdout:5/742: dwrite df/d3d/ff8 [0,4194304] 0 2024-06-15T01:07:16.365 INFO:tasks.workunit.client.0.smithi137.stdout:5/743: rename fd to df/d11/d14/fff 0 2024-06-15T01:07:16.396 INFO:tasks.workunit.client.0.smithi137.stdout:5/744: dread df/d44/d54/d6b/f8c [0,4194304] 0 2024-06-15T01:07:16.398 INFO:tasks.workunit.client.0.smithi137.stdout:5/745: truncate df/d11/d14/f32 3339653 0 2024-06-15T01:07:16.400 INFO:tasks.workunit.client.0.smithi137.stdout:5/746: unlink df/d11/d14/l64 0 2024-06-15T01:07:16.405 INFO:tasks.workunit.client.0.smithi137.stdout:5/747: dwrite df/d11/fd9 [0,4194304] 0 2024-06-15T01:07:16.408 INFO:tasks.workunit.client.0.smithi137.stdout:5/748: rmdir df/d44/d54 39 2024-06-15T01:07:16.410 INFO:tasks.workunit.client.0.smithi137.stdout:5/749: readlink df/d44/d54/d95/d35/l89 0 2024-06-15T01:07:16.414 INFO:tasks.workunit.client.0.smithi137.stdout:5/750: fdatasync df/d44/d54/fd3 0 2024-06-15T01:07:16.417 INFO:tasks.workunit.client.0.smithi137.stdout:5/751: dread df/f12 [4194304,4194304] 0 2024-06-15T01:07:16.420 INFO:tasks.workunit.client.0.smithi137.stdout:5/752: truncate df/d44/d54/d95/d3b/d4b/f63 1605858 0 2024-06-15T01:07:16.421 INFO:tasks.workunit.client.0.smithi137.stdout:5/753: mknod df/d11/d30/db1/c100 0 2024-06-15T01:07:16.422 INFO:tasks.workunit.client.0.smithi137.stdout:5/754: write df/d44/d54/d95/d35/dea/fb5 [2270800,2915] 0 2024-06-15T01:07:16.431 INFO:tasks.workunit.client.0.smithi137.stdout:5/755: truncate df/d3d/f5f 3565405 0 2024-06-15T01:07:16.433 INFO:tasks.workunit.client.0.smithi137.stdout:5/756: truncate df/d44/f4d 275237 0 2024-06-15T01:07:16.434 INFO:tasks.workunit.client.0.smithi137.stdout:5/757: fdatasync df/d76/d7c/d9c/da8/fad 0 2024-06-15T01:07:16.435 INFO:tasks.workunit.client.0.smithi137.stdout:5/758: write df/d76/d7c/d9c/fe6 [964299,28277] 0 2024-06-15T01:07:16.437 INFO:tasks.workunit.client.0.smithi137.stdout:5/759: mkdir df/d76/d7c/d9c/da8/d101 0 2024-06-15T01:07:16.438 INFO:tasks.workunit.client.0.smithi137.stdout:5/760: mkdir df/d76/d7c/d9c/da1/d102 0 2024-06-15T01:07:16.440 INFO:tasks.workunit.client.0.smithi137.stdout:5/761: creat df/d11/d14/d6c/dc7/f103 x:0 0 0 2024-06-15T01:07:16.441 INFO:tasks.workunit.client.0.smithi137.stdout:5/762: fdatasync df/f2b 0 2024-06-15T01:07:16.442 INFO:tasks.workunit.client.0.smithi137.stdout:5/763: rmdir df/d44/d54/d95 39 2024-06-15T01:07:16.504 INFO:tasks.workunit.client.0.smithi137.stdout:5/764: dread df/d44/f4e [0,4194304] 0 2024-06-15T01:07:16.509 INFO:tasks.workunit.client.0.smithi137.stdout:5/765: dwrite df/d11/d30/db1/fd7 [0,4194304] 0 2024-06-15T01:07:16.516 INFO:tasks.workunit.client.0.smithi137.stdout:5/766: rmdir df/d3d/df9 0 2024-06-15T01:07:16.517 INFO:tasks.workunit.client.0.smithi137.stdout:5/767: mkdir df/d11/d30/d104 0 2024-06-15T01:07:16.518 INFO:tasks.workunit.client.0.smithi137.stdout:5/768: chown df/d11/dfa/dfb 12548 1 2024-06-15T01:07:16.527 INFO:tasks.workunit.client.0.smithi137.stdout:5/769: write df/d44/d54/d95/d35/f9b [1406308,114126] 0 2024-06-15T01:07:16.603 INFO:tasks.workunit.client.0.smithi137.stdout:5/770: read df/d11/f13 [1698699,47103] 0 2024-06-15T01:07:16.604 INFO:tasks.workunit.client.0.smithi137.stdout:1/832: sync 2024-06-15T01:07:16.605 INFO:tasks.workunit.client.0.smithi137.stdout:5/771: mkdir df/d44/d54/d80/d105 0 2024-06-15T01:07:16.612 INFO:tasks.workunit.client.0.smithi137.stdout:5/772: dwrite df/d11/d14/d6c/f99 [0,4194304] 0 2024-06-15T01:07:16.618 INFO:tasks.workunit.client.0.smithi137.stdout:5/773: creat df/d76/d7c/d9c/da1/d102/f106 x:0 0 0 2024-06-15T01:07:16.619 INFO:tasks.workunit.client.0.smithi137.stdout:1/833: dwrite d0/d1f/d60/d99/de3/dea/d10f/d4c/de5/fe7 [0,4194304] 0 2024-06-15T01:07:16.621 INFO:tasks.workunit.client.0.smithi137.stdout:5/774: chown df/d76/d7c/d9c/da1 12035166 1 2024-06-15T01:07:16.621 INFO:tasks.workunit.client.0.smithi137.stdout:1/834: fdatasync d0/d1f/d60/d99/de3/dea/d10f/f69 0 2024-06-15T01:07:16.621 INFO:tasks.workunit.client.0.smithi137.stdout:5/775: readlink df/d11/d30/db1/lb2 0 2024-06-15T01:07:16.624 INFO:tasks.workunit.client.0.smithi137.stdout:5/776: symlink df/d11/dfa/l107 0 2024-06-15T01:07:16.630 INFO:tasks.workunit.client.0.smithi137.stdout:5/777: dread df/d11/d14/f16 [0,4194304] 0 2024-06-15T01:07:16.630 INFO:tasks.workunit.client.0.smithi137.stdout:5/778: fdatasync df/d11/ff7 0 2024-06-15T01:07:16.634 INFO:tasks.workunit.client.0.smithi137.stdout:5/779: truncate df/d44/d54/d6b/fb3 1126278 0 2024-06-15T01:07:16.640 INFO:tasks.workunit.client.0.smithi137.stdout:5/780: dwrite df/d44/d54/d95/d3b/d49/fca [0,4194304] 0 2024-06-15T01:07:16.646 INFO:tasks.workunit.client.0.smithi137.stdout:5/781: getdents df/d11 0 2024-06-15T01:07:16.653 INFO:tasks.workunit.client.0.smithi137.stdout:5/782: dwrite df/d44/d54/d95/d3b/f59 [0,4194304] 0 2024-06-15T01:07:16.656 INFO:tasks.workunit.client.0.smithi137.stdout:5/783: fdatasync df/d44/d54/d95/d3b/d49/f96 0 2024-06-15T01:07:16.670 INFO:tasks.workunit.client.0.smithi137.stdout:5/784: write df/d3d/f5f [147767,67944] 0 2024-06-15T01:07:16.673 INFO:tasks.workunit.client.0.smithi137.stdout:5/785: mknod df/d44/d54/d95/d3b/d49/d5c/c108 0 2024-06-15T01:07:16.679 INFO:tasks.workunit.client.0.smithi137.stdout:5/786: dwrite df/d44/d54/d95/d3b/d49/f77 [0,4194304] 0 2024-06-15T01:07:16.682 INFO:tasks.workunit.client.0.smithi137.stdout:5/787: symlink df/l109 0 2024-06-15T01:07:16.689 INFO:tasks.workunit.client.0.smithi137.stdout:5/788: dwrite df/d3d/f5f [0,4194304] 0 2024-06-15T01:07:17.107 INFO:tasks.workunit.client.0.smithi137.stdout:6/630: sync 2024-06-15T01:07:17.107 INFO:tasks.workunit.client.0.smithi137.stdout:0/909: sync 2024-06-15T01:07:17.108 INFO:tasks.workunit.client.0.smithi137.stdout:6/631: write d0/d5/d9/d3f/f67 [4360865,15505] 0 2024-06-15T01:07:17.110 INFO:tasks.workunit.client.0.smithi137.stdout:6/632: dread - d0/d1a/d3e/d57/d32/d74/fb2 zero size 2024-06-15T01:07:17.114 INFO:tasks.workunit.client.0.smithi137.stdout:0/910: rename d1/d7e/d87/da0/dc4/dd7/de2/d8d/dd9/f103 to d1/d2/dc/d10/d22/db7/d11f/f12d 0 2024-06-15T01:07:17.118 INFO:tasks.workunit.client.0.smithi137.stdout:6/633: dwrite d0/d5/f7c [4194304,4194304] 0 2024-06-15T01:07:17.125 INFO:tasks.workunit.client.0.smithi137.stdout:6/634: read d0/d1a/d3e/d57/d32/d74/f78 [250346,50224] 0 2024-06-15T01:07:17.127 INFO:tasks.workunit.client.0.smithi137.stdout:9/562: sync 2024-06-15T01:07:17.127 INFO:tasks.workunit.client.0.smithi137.stdout:8/590: sync 2024-06-15T01:07:17.128 INFO:tasks.workunit.client.0.smithi137.stdout:7/589: sync 2024-06-15T01:07:17.128 INFO:tasks.workunit.client.0.smithi137.stdout:6/635: unlink d0/d1a/d3e/d57/f24 0 2024-06-15T01:07:17.132 INFO:tasks.workunit.client.0.smithi137.stdout:9/563: creat df/d12/d29/d66/fa2 x:0 0 0 2024-06-15T01:07:17.133 INFO:tasks.workunit.client.0.smithi137.stdout:8/591: creat d9/d2a/d79/fc9 x:0 0 0 2024-06-15T01:07:17.134 INFO:tasks.workunit.client.0.smithi137.stdout:6/636: truncate d0/d1a/da0/fa1 435191 0 2024-06-15T01:07:17.136 INFO:tasks.workunit.client.0.smithi137.stdout:9/564: truncate df/d49/d4b/f57 1366236 0 2024-06-15T01:07:17.138 INFO:tasks.workunit.client.0.smithi137.stdout:8/592: read d9/d2a/d2f/f90 [1795176,802] 0 2024-06-15T01:07:17.143 INFO:tasks.workunit.client.0.smithi137.stdout:8/593: creat d9/d2a/db9/fca x:0 0 0 2024-06-15T01:07:17.143 INFO:tasks.workunit.client.0.smithi137.stdout:8/594: truncate d9/d2a/db9/dc6/f36 3602566 0 2024-06-15T01:07:17.148 INFO:tasks.workunit.client.0.smithi137.stdout:8/595: dread d9/d2a/db9/dc6/d1e/d35/f9f [0,4194304] 0 2024-06-15T01:07:17.153 INFO:tasks.workunit.client.0.smithi137.stdout:0/911: rmdir d1/d7e/d87/da0/dc4 39 2024-06-15T01:07:17.154 INFO:tasks.workunit.client.0.smithi137.stdout:8/596: dwrite d9/d22/f37 [0,4194304] 0 2024-06-15T01:07:17.159 INFO:tasks.workunit.client.0.smithi137.stdout:0/912: dread d1/d2/d13/fc0 [0,4194304] 0 2024-06-15T01:07:17.164 INFO:tasks.workunit.client.0.smithi137.stdout:1/835: sync 2024-06-15T01:07:17.164 INFO:tasks.workunit.client.0.smithi137.stdout:2/594: sync 2024-06-15T01:07:17.164 INFO:tasks.workunit.client.0.smithi137.stdout:5/789: sync 2024-06-15T01:07:17.164 INFO:tasks.workunit.client.0.smithi137.stdout:4/793: sync 2024-06-15T01:07:17.164 INFO:tasks.workunit.client.0.smithi137.stdout:7/590: dwrite df/d7d/d3b/f93 [0,4194304] 0 2024-06-15T01:07:17.168 INFO:tasks.workunit.client.0.smithi137.stdout:2/595: stat d0/dc/d2e/d72/c6d 0 2024-06-15T01:07:17.170 INFO:tasks.workunit.client.0.smithi137.stdout:6/637: dwrite d0/f2a [0,4194304] 0 2024-06-15T01:07:17.172 INFO:tasks.workunit.client.0.smithi137.stdout:9/565: dwrite df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:17.175 INFO:tasks.workunit.client.0.smithi137.stdout:4/794: fsync dd/d15/d36/d45/dcb/d83/db2/d67/dbe/ff4 0 2024-06-15T01:07:17.176 INFO:tasks.workunit.client.0.smithi137.stdout:1/836: dread - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/f111 zero size 2024-06-15T01:07:17.176 INFO:tasks.workunit.client.0.smithi137.stdout:0/913: dread d1/d2/dc/d10/d14/d56/d117/d17/f95 [0,4194304] 0 2024-06-15T01:07:17.177 INFO:tasks.workunit.client.0.smithi137.stdout:2/596: symlink d0/da/d73/d8d/d5c/d79/lae 0 2024-06-15T01:07:17.181 INFO:tasks.workunit.client.0.smithi137.stdout:7/591: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/lbe 0 2024-06-15T01:07:17.182 INFO:tasks.workunit.client.0.smithi137.stdout:1/837: chown d0/d10/d41/d7b/c7c 220492188 1 2024-06-15T01:07:17.182 INFO:tasks.workunit.client.0.smithi137.stdout:7/592: chown df/d7d/d3b/c7b 498 1 2024-06-15T01:07:17.184 INFO:tasks.workunit.client.0.smithi137.stdout:4/795: write dd/f28 [3002937,35860] 0 2024-06-15T01:07:17.185 INFO:tasks.workunit.client.0.smithi137.stdout:9/566: rename df/f90 to df/d12/d29/fa3 0 2024-06-15T01:07:17.192 INFO:tasks.workunit.client.0.smithi137.stdout:5/790: dwrite df/d44/d54/d6b/fb3 [0,4194304] 0 2024-06-15T01:07:17.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/638: symlink d0/d1a/d3e/d57/d1d/d2e/lc6 0 2024-06-15T01:07:17.196 INFO:tasks.workunit.client.0.smithi137.stdout:7/593: rename df/c65 to df/d13/d2e/d1a/d6a/d90/cbf 0 2024-06-15T01:07:17.199 INFO:tasks.workunit.client.0.smithi137.stdout:9/567: write df/d12/d15/d75/f7c [1678280,24200] 0 2024-06-15T01:07:17.200 INFO:tasks.workunit.client.0.smithi137.stdout:2/597: dread d0/de/f24 [0,4194304] 0 2024-06-15T01:07:17.203 INFO:tasks.workunit.client.0.smithi137.stdout:9/568: dread - df/d12/d29/d79/fa1 zero size 2024-06-15T01:07:17.204 INFO:tasks.workunit.client.0.smithi137.stdout:9/569: chown df/d49/d4b/f8c 61440 1 2024-06-15T01:07:17.205 INFO:tasks.workunit.client.0.smithi137.stdout:6/639: rmdir d0/d1a 39 2024-06-15T01:07:17.206 INFO:tasks.workunit.client.0.smithi137.stdout:7/594: dread df/d7d/d1c/d1d/d27/d33/f52 [0,4194304] 0 2024-06-15T01:07:17.208 INFO:tasks.workunit.client.0.smithi137.stdout:4/796: link dd/d15/d21/d3a/d6e/d7b/c105 dd/d15/d21/d9c/d91/c124 0 2024-06-15T01:07:17.210 INFO:tasks.workunit.client.0.smithi137.stdout:1/838: link d0/d1f/d76/da1/fa2 d0/d1f/f116 0 2024-06-15T01:07:17.210 INFO:tasks.workunit.client.0.smithi137.stdout:1/839: readlink d0/d10/daf/lb8 0 2024-06-15T01:07:17.211 INFO:tasks.workunit.client.0.smithi137.stdout:2/598: creat d0/da/d73/d8d/d5c/d99/faf x:0 0 0 2024-06-15T01:07:17.213 INFO:tasks.workunit.client.0.smithi137.stdout:9/570: symlink df/la4 0 2024-06-15T01:07:17.216 INFO:tasks.workunit.client.0.smithi137.stdout:6/640: stat d0/d5/d9/c25 0 2024-06-15T01:07:17.218 INFO:tasks.workunit.client.0.smithi137.stdout:0/914: dwrite d1/d2/dc/d10/d14/d56/d117/f30 [0,4194304] 0 2024-06-15T01:07:17.219 INFO:tasks.workunit.client.0.smithi137.stdout:1/840: write d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/fd5 [192396,121628] 0 2024-06-15T01:07:17.219 INFO:tasks.workunit.client.0.smithi137.stdout:7/595: chown df/d13/d3c/f92 1 1 2024-06-15T01:07:17.220 INFO:tasks.workunit.client.0.smithi137.stdout:4/797: mkdir dd/d15/d21/d9c/d75/df5/d125 0 2024-06-15T01:07:17.221 INFO:tasks.workunit.client.0.smithi137.stdout:5/791: dread df/d11/f24 [0,4194304] 0 2024-06-15T01:07:17.221 INFO:tasks.workunit.client.0.smithi137.stdout:1/841: chown d0/d1f/d7d 516509 1 2024-06-15T01:07:17.223 INFO:tasks.workunit.client.0.smithi137.stdout:1/842: chown d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66 1 1 2024-06-15T01:07:17.230 INFO:tasks.workunit.client.0.smithi137.stdout:5/792: fsync df/d44/d54/d95/d35/f81 0 2024-06-15T01:07:17.232 INFO:tasks.workunit.client.0.smithi137.stdout:1/843: dwrite d0/d3/d47/fd3 [0,4194304] 0 2024-06-15T01:07:17.232 INFO:tasks.workunit.client.0.smithi137.stdout:6/641: stat d0/d1a/d3e/d57/d1d/d2e/l31 0 2024-06-15T01:07:17.240 INFO:tasks.workunit.client.0.smithi137.stdout:6/642: write d0/d1a/d68/f9f [383125,108419] 0 2024-06-15T01:07:17.240 INFO:tasks.workunit.client.0.smithi137.stdout:0/915: getdents d1/d2/dc/d10/d14/d56 0 2024-06-15T01:07:17.242 INFO:tasks.workunit.client.0.smithi137.stdout:0/916: chown d1/d2/l49 9106 1 2024-06-15T01:07:17.243 INFO:tasks.workunit.client.0.smithi137.stdout:9/571: dwrite df/d12/d1b/f35 [0,4194304] 0 2024-06-15T01:07:17.245 INFO:tasks.workunit.client.0.smithi137.stdout:1/844: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/d108/f117 x:0 0 0 2024-06-15T01:07:17.246 INFO:tasks.workunit.client.0.smithi137.stdout:4/798: write dd/d15/d36/d45/d55/d8d/f5e [929428,120004] 0 2024-06-15T01:07:17.247 INFO:tasks.workunit.client.0.smithi137.stdout:2/599: dread d0/dc/d22/f61 [0,4194304] 0 2024-06-15T01:07:17.248 INFO:tasks.workunit.client.0.smithi137.stdout:2/600: chown d0/dc/d2e/d72/d56/d6e 4287565 1 2024-06-15T01:07:17.251 INFO:tasks.workunit.client.0.smithi137.stdout:2/601: read d0/de/f1a [3236789,125702] 0 2024-06-15T01:07:17.252 INFO:tasks.workunit.client.0.smithi137.stdout:0/917: dread d1/d2/dc/d10/d14/d52/ff9 [0,4194304] 0 2024-06-15T01:07:17.254 INFO:tasks.workunit.client.0.smithi137.stdout:4/799: creat dd/d15/d21/d3a/d6e/f126 x:0 0 0 2024-06-15T01:07:17.254 INFO:tasks.workunit.client.0.smithi137.stdout:5/793: link df/d11/d14/l25 df/d44/d54/d95/d3b/d49/d5c/de7/l10a 0 2024-06-15T01:07:17.255 INFO:tasks.workunit.client.0.smithi137.stdout:5/794: stat df/d11/c2d 0 2024-06-15T01:07:17.257 INFO:tasks.workunit.client.0.smithi137.stdout:9/572: symlink df/d12/d15/d16/d3c/d51/d8f/la5 0 2024-06-15T01:07:17.259 INFO:tasks.workunit.client.0.smithi137.stdout:9/573: fsync df/d12/d29/d79/fa1 0 2024-06-15T01:07:17.259 INFO:tasks.workunit.client.0.smithi137.stdout:0/918: symlink d1/d2/dc/d10/d22/db7/de3/l12e 0 2024-06-15T01:07:17.264 INFO:tasks.workunit.client.0.smithi137.stdout:4/800: dwrite dd/d15/d36/d45/d55/d8d/f5e [0,4194304] 0 2024-06-15T01:07:17.266 INFO:tasks.workunit.client.0.smithi137.stdout:9/574: rename df/l40 to df/d12/d4e/la6 0 2024-06-15T01:07:17.266 INFO:tasks.workunit.client.0.smithi137.stdout:4/801: stat dd/d15/d36/d45/dcb/d83/db2/d67/dbe/ff4 0 2024-06-15T01:07:17.268 INFO:tasks.workunit.client.0.smithi137.stdout:1/845: dread d0/d2d/f5e [0,4194304] 0 2024-06-15T01:07:17.270 INFO:tasks.workunit.client.0.smithi137.stdout:9/575: mkdir df/d1a/da7 0 2024-06-15T01:07:17.272 INFO:tasks.workunit.client.0.smithi137.stdout:9/576: stat df/d12/d1b/d31/d42/c9c 0 2024-06-15T01:07:17.272 INFO:tasks.workunit.client.0.smithi137.stdout:0/919: getdents d1/d2/d13 0 2024-06-15T01:07:17.276 INFO:tasks.workunit.client.0.smithi137.stdout:5/795: write df/f41 [600400,92079] 0 2024-06-15T01:07:17.279 INFO:tasks.workunit.client.0.smithi137.stdout:1/846: rename d0/d1f/d7d/c85 to d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/df6/d102/c118 0 2024-06-15T01:07:17.282 INFO:tasks.workunit.client.0.smithi137.stdout:0/920: mknod d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/c12f 0 2024-06-15T01:07:17.282 INFO:tasks.workunit.client.0.smithi137.stdout:9/577: dwrite df/d12/d29/d4a/f6c [4194304,4194304] 0 2024-06-15T01:07:17.286 INFO:tasks.workunit.client.0.smithi137.stdout:1/847: creat d0/d1f/d76/da1/f119 x:0 0 0 2024-06-15T01:07:17.287 INFO:tasks.workunit.client.0.smithi137.stdout:5/796: dread df/d3d/ff8 [0,4194304] 0 2024-06-15T01:07:17.289 INFO:tasks.workunit.client.0.smithi137.stdout:2/602: dwrite d0/dc/d2e/d72/d56/f93 [0,4194304] 0 2024-06-15T01:07:17.294 INFO:tasks.workunit.client.0.smithi137.stdout:0/921: creat d1/d2/d13/f130 x:0 0 0 2024-06-15T01:07:17.296 INFO:tasks.workunit.client.0.smithi137.stdout:4/802: dread dd/d15/d36/d45/dcb/d83/db2/d67/f6c [0,4194304] 0 2024-06-15T01:07:17.297 INFO:tasks.workunit.client.0.smithi137.stdout:1/848: rmdir d0/d2d/d3e 39 2024-06-15T01:07:17.301 INFO:tasks.workunit.client.0.smithi137.stdout:5/797: creat df/d44/d54/d80/d105/f10b x:0 0 0 2024-06-15T01:07:17.305 INFO:tasks.workunit.client.0.smithi137.stdout:0/922: mkdir d1/d7e/d87/da0/dc4/de4/d131 0 2024-06-15T01:07:17.306 INFO:tasks.workunit.client.0.smithi137.stdout:2/603: creat d0/d42/d49/fb0 x:0 0 0 2024-06-15T01:07:17.308 INFO:tasks.workunit.client.0.smithi137.stdout:5/798: creat df/d44/d54/d95/d3b/df1/f10c x:0 0 0 2024-06-15T01:07:17.308 INFO:tasks.workunit.client.0.smithi137.stdout:9/578: dread df/d12/f1f [0,4194304] 0 2024-06-15T01:07:17.309 INFO:tasks.workunit.client.0.smithi137.stdout:0/923: truncate d1/d2/d13/f33 780213 0 2024-06-15T01:07:17.316 INFO:tasks.workunit.client.0.smithi137.stdout:2/604: getdents d0/d42/d49/d57 0 2024-06-15T01:07:17.318 INFO:tasks.workunit.client.0.smithi137.stdout:1/849: dwrite d0/d1f/d60/f91 [4194304,4194304] 0 2024-06-15T01:07:17.322 INFO:tasks.workunit.client.0.smithi137.stdout:0/924: rename d1/d2/d13/cbc to d1/d7e/d87/da0/dc4/dd7/de2/d77/c132 0 2024-06-15T01:07:17.325 INFO:tasks.workunit.client.0.smithi137.stdout:2/605: creat d0/da/da0/fb1 x:0 0 0 2024-06-15T01:07:17.326 INFO:tasks.workunit.client.0.smithi137.stdout:9/579: link df/c7d df/d1a/ca8 0 2024-06-15T01:07:17.328 INFO:tasks.workunit.client.0.smithi137.stdout:5/799: dwrite f7 [0,4194304] 0 2024-06-15T01:07:17.328 INFO:tasks.workunit.client.0.smithi137.stdout:9/580: stat df/d2a/c60 0 2024-06-15T01:07:17.331 INFO:tasks.workunit.client.0.smithi137.stdout:0/925: chown d1/d2/dc/d10/d14/d56/d117 81 1 2024-06-15T01:07:17.331 INFO:tasks.workunit.client.0.smithi137.stdout:4/803: dread dd/d15/d36/d45/d55/d8d/f112 [0,4194304] 0 2024-06-15T01:07:17.331 INFO:tasks.workunit.client.0.smithi137.stdout:1/850: rename d0/d3/d47/dd7 to d0/d10/d11a 0 2024-06-15T01:07:17.337 INFO:tasks.workunit.client.0.smithi137.stdout:5/800: mknod df/d44/d54/d95/d35/c10d 0 2024-06-15T01:07:17.342 INFO:tasks.workunit.client.0.smithi137.stdout:9/581: creat df/d12/d15/fa9 x:0 0 0 2024-06-15T01:07:17.343 INFO:tasks.workunit.client.0.smithi137.stdout:0/926: dwrite d1/d2/f47 [8388608,4194304] 0 2024-06-15T01:07:17.346 INFO:tasks.workunit.client.0.smithi137.stdout:0/927: truncate d1/d2/dc/d10/d22/db7/f112 558663 0 2024-06-15T01:07:17.354 INFO:tasks.workunit.client.0.smithi137.stdout:2/606: dwrite d0/da/d73/d8d/f6a [0,4194304] 0 2024-06-15T01:07:17.354 INFO:tasks.workunit.client.0.smithi137.stdout:0/928: dread d1/d2/dc/d10/d22/fc7 [0,4194304] 0 2024-06-15T01:07:17.356 INFO:tasks.workunit.client.0.smithi137.stdout:5/801: rmdir df/d11/d30/d104 0 2024-06-15T01:07:17.359 INFO:tasks.workunit.client.0.smithi137.stdout:1/851: getdents d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/d108 0 2024-06-15T01:07:17.361 INFO:tasks.workunit.client.0.smithi137.stdout:4/804: dwrite dd/d15/d36/d45/d55/d8d/d1f/d86/f8a [0,4194304] 0 2024-06-15T01:07:17.363 INFO:tasks.workunit.client.0.smithi137.stdout:0/929: rmdir d1/d2/dc/d10/d22/db7/d11f 39 2024-06-15T01:07:17.364 INFO:tasks.workunit.client.0.smithi137.stdout:9/582: write df/f6b [846442,64095] 0 2024-06-15T01:07:17.366 INFO:tasks.workunit.client.0.smithi137.stdout:4/805: write dd/d15/d36/d45/fd7 [1885052,61770] 0 2024-06-15T01:07:17.370 INFO:tasks.workunit.client.0.smithi137.stdout:2/607: creat d0/dc/d2e/d72/d8a/fb2 x:0 0 0 2024-06-15T01:07:17.371 INFO:tasks.workunit.client.0.smithi137.stdout:9/583: creat df/d49/faa x:0 0 0 2024-06-15T01:07:17.374 INFO:tasks.workunit.client.0.smithi137.stdout:2/608: write d0/dc/f94 [690784,126254] 0 2024-06-15T01:07:17.375 INFO:tasks.workunit.client.0.smithi137.stdout:9/584: mknod df/d13/d6f/cab 0 2024-06-15T01:07:17.376 INFO:tasks.workunit.client.0.smithi137.stdout:0/930: creat d1/d2/dc/d10/d14/d56/d117/d17/f133 x:0 0 0 2024-06-15T01:07:17.379 INFO:tasks.workunit.client.0.smithi137.stdout:4/806: link dd/d15/d36/d45/dcb/d83/db2/d67/dbe/cd8 dd/d15/d21/d3a/df6/c127 0 2024-06-15T01:07:17.380 INFO:tasks.workunit.client.0.smithi137.stdout:5/802: dwrite df/d44/d54/d95/d3b/f59 [0,4194304] 0 2024-06-15T01:07:17.382 INFO:tasks.workunit.client.0.smithi137.stdout:2/609: creat d0/da/d73/fb3 x:0 0 0 2024-06-15T01:07:17.383 INFO:tasks.workunit.client.0.smithi137.stdout:0/931: symlink d1/d2/dc/d10/d53/da2/l134 0 2024-06-15T01:07:17.384 INFO:tasks.workunit.client.0.smithi137.stdout:5/803: mknod df/d11/d14/d6c/dc7/c10e 0 2024-06-15T01:07:17.387 INFO:tasks.workunit.client.0.smithi137.stdout:4/807: unlink dd/d15/d36/d45/dcb/d83/lc7 0 2024-06-15T01:07:17.388 INFO:tasks.workunit.client.0.smithi137.stdout:2/610: rmdir d0/dc/d22 39 2024-06-15T01:07:17.390 INFO:tasks.workunit.client.0.smithi137.stdout:0/932: creat d1/d2/dc/d10/d22/db7/dbd/f135 x:0 0 0 2024-06-15T01:07:17.392 INFO:tasks.workunit.client.0.smithi137.stdout:5/804: mknod df/d44/d54/d95/d3b/d4b/c10f 0 2024-06-15T01:07:17.393 INFO:tasks.workunit.client.0.smithi137.stdout:4/808: truncate dd/f16 3450267 0 2024-06-15T01:07:17.394 INFO:tasks.workunit.client.0.smithi137.stdout:2/611: symlink d0/de/d2b/lb4 0 2024-06-15T01:07:17.423 INFO:tasks.workunit.client.0.smithi137.stdout:1/852: dwrite d0/d10/d41/d7b/f90 [4194304,4194304] 0 2024-06-15T01:07:17.425 INFO:tasks.workunit.client.0.smithi137.stdout:2/612: truncate d0/da/d73/fb3 1003639 0 2024-06-15T01:07:17.425 INFO:tasks.workunit.client.0.smithi137.stdout:0/933: rename d1/d2/dc/d10/d14/d56/d117/d17/f18 to d1/df6/d11b/f136 0 2024-06-15T01:07:17.427 INFO:tasks.workunit.client.0.smithi137.stdout:4/809: truncate dd/f92 133985 0 2024-06-15T01:07:17.429 INFO:tasks.workunit.client.0.smithi137.stdout:1/853: mkdir d0/d3/deb/d11b 0 2024-06-15T01:07:17.430 INFO:tasks.workunit.client.0.smithi137.stdout:1/854: write d0/d1f/d76/f105 [818536,80582] 0 2024-06-15T01:07:17.432 INFO:tasks.workunit.client.0.smithi137.stdout:0/934: mkdir d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/d137 0 2024-06-15T01:07:17.438 INFO:tasks.workunit.client.0.smithi137.stdout:0/935: read d1/d7e/d87/da0/fcf [676769,51442] 0 2024-06-15T01:07:17.440 INFO:tasks.workunit.client.0.smithi137.stdout:0/936: dread - d1/d2/dc/d10/d22/db7/dbd/f108 zero size 2024-06-15T01:07:17.441 INFO:tasks.workunit.client.0.smithi137.stdout:2/613: rename d0/d42/d49/d57/c98 to d0/dc/d2e/d72/d56/d6e/cb5 0 2024-06-15T01:07:17.441 INFO:tasks.workunit.client.0.smithi137.stdout:2/614: stat d0/d42/d49/l63 0 2024-06-15T01:07:17.445 INFO:tasks.workunit.client.0.smithi137.stdout:1/855: getdents d0/d2d 0 2024-06-15T01:07:17.447 INFO:tasks.workunit.client.0.smithi137.stdout:2/615: creat d0/da/d73/d8d/d5c/d79/fb6 x:0 0 0 2024-06-15T01:07:17.450 INFO:tasks.workunit.client.0.smithi137.stdout:0/937: link d1/d2/f47 d1/d2/d55/f138 0 2024-06-15T01:07:17.453 INFO:tasks.workunit.client.0.smithi137.stdout:5/805: dwrite df/d11/d14/d6c/dc7/fd0 [0,4194304] 0 2024-06-15T01:07:17.454 INFO:tasks.workunit.client.0.smithi137.stdout:2/616: rename d0/ca7 to d0/d42/d49/d57/cb7 0 2024-06-15T01:07:17.455 INFO:tasks.workunit.client.0.smithi137.stdout:5/806: dread - df/d11/d14/d3c/d8a/fe2 zero size 2024-06-15T01:07:17.456 INFO:tasks.workunit.client.0.smithi137.stdout:2/617: readlink d0/de/d2b/l36 0 2024-06-15T01:07:17.457 INFO:tasks.workunit.client.0.smithi137.stdout:5/807: write df/d11/d14/d3c/d8a/fe2 [694692,35127] 0 2024-06-15T01:07:17.466 INFO:tasks.workunit.client.0.smithi137.stdout:5/808: dwrite df/d11/d14/d3c/d8a/fe2 [0,4194304] 0 2024-06-15T01:07:17.468 INFO:tasks.workunit.client.0.smithi137.stdout:5/809: fdatasync df/d11/d30/f61 0 2024-06-15T01:07:17.469 INFO:tasks.workunit.client.0.smithi137.stdout:1/856: write d0/d1f/d7d/fae [733929,48157] 0 2024-06-15T01:07:17.469 INFO:tasks.workunit.client.0.smithi137.stdout:2/618: truncate d0/d1c/f66 1868550 0 2024-06-15T01:07:17.475 INFO:tasks.workunit.client.0.smithi137.stdout:5/810: mknod df/d44/d54/d80/c110 0 2024-06-15T01:07:17.480 INFO:tasks.workunit.client.0.smithi137.stdout:0/938: dwrite d1/df6/d11b/f12c [0,4194304] 0 2024-06-15T01:07:17.480 INFO:tasks.workunit.client.0.smithi137.stdout:5/811: mknod df/d44/d54/d95/d3b/d49/d5c/de7/c111 0 2024-06-15T01:07:17.482 INFO:tasks.workunit.client.0.smithi137.stdout:2/619: write d0/dc/d22/f39 [3145296,71284] 0 2024-06-15T01:07:17.489 INFO:tasks.workunit.client.0.smithi137.stdout:5/812: write f3 [4173965,123823] 0 2024-06-15T01:07:17.495 INFO:tasks.workunit.client.0.smithi137.stdout:2/620: rename d0/da/f7c to d0/de/d2b/d47/fb8 0 2024-06-15T01:07:17.495 INFO:tasks.workunit.client.0.smithi137.stdout:5/813: write df/d11/d14/f15 [2344990,83515] 0 2024-06-15T01:07:17.498 INFO:tasks.workunit.client.0.smithi137.stdout:0/939: dwrite d1/d7e/d87/da0/dc4/dd7/de2/d8d/ffc [4194304,4194304] 0 2024-06-15T01:07:17.503 INFO:tasks.workunit.client.0.smithi137.stdout:0/940: symlink d1/d2/d13/d66/dc3/l139 0 2024-06-15T01:07:17.504 INFO:tasks.workunit.client.0.smithi137.stdout:5/814: mknod df/d44/d54/d95/d3b/c112 0 2024-06-15T01:07:17.504 INFO:tasks.workunit.client.0.smithi137.stdout:0/941: read - d1/d7e/d87/da0/dc4/de4/f118 zero size 2024-06-15T01:07:17.505 INFO:tasks.workunit.client.0.smithi137.stdout:5/815: dread - df/d44/d54/fd3 zero size 2024-06-15T01:07:17.506 INFO:tasks.workunit.client.0.smithi137.stdout:2/621: getdents d0/dc/d2e/d72 0 2024-06-15T01:07:17.507 INFO:tasks.workunit.client.0.smithi137.stdout:1/857: dread d0/d1f/d7d/fae [0,4194304] 0 2024-06-15T01:07:17.508 INFO:tasks.workunit.client.0.smithi137.stdout:0/942: chown d1/d2/dc/d6c/cef 4905 1 2024-06-15T01:07:17.509 INFO:tasks.workunit.client.0.smithi137.stdout:2/622: mkdir d0/db9 0 2024-06-15T01:07:17.510 INFO:tasks.workunit.client.0.smithi137.stdout:2/623: stat d0/dc/d22/f4b 0 2024-06-15T01:07:17.513 INFO:tasks.workunit.client.0.smithi137.stdout:1/858: stat d0/d1f/d60/d99/de3/cd0 0 2024-06-15T01:07:17.515 INFO:tasks.workunit.client.0.smithi137.stdout:5/816: creat df/d44/d54/d95/d3b/f113 x:0 0 0 2024-06-15T01:07:17.516 INFO:tasks.workunit.client.0.smithi137.stdout:5/817: chown df/d44/d54/d80/d105 30 1 2024-06-15T01:07:17.518 INFO:tasks.workunit.client.0.smithi137.stdout:2/624: write d0/f69 [1061755,54353] 0 2024-06-15T01:07:17.520 INFO:tasks.workunit.client.0.smithi137.stdout:5/818: chown df/d11/d14/l26 2022 1 2024-06-15T01:07:17.523 INFO:tasks.workunit.client.0.smithi137.stdout:2/625: symlink d0/dc/d22/lba 0 2024-06-15T01:07:17.523 INFO:tasks.workunit.client.0.smithi137.stdout:5/819: dread - df/d44/d54/d95/d3b/f113 zero size 2024-06-15T01:07:17.524 INFO:tasks.workunit.client.0.smithi137.stdout:0/943: link d1/df6/d11b/l124 d1/d7e/d87/da0/l13a 0 2024-06-15T01:07:17.526 INFO:tasks.workunit.client.0.smithi137.stdout:1/859: dwrite d0/d2d/f62 [0,4194304] 0 2024-06-15T01:07:17.529 INFO:tasks.workunit.client.0.smithi137.stdout:2/626: creat d0/dc/fbb x:0 0 0 2024-06-15T01:07:17.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/860: symlink d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/l11c 0 2024-06-15T01:07:17.535 INFO:tasks.workunit.client.0.smithi137.stdout:1/861: stat d0/d10/l37 0 2024-06-15T01:07:17.535 INFO:tasks.workunit.client.0.smithi137.stdout:5/820: dwrite df/d76/f8f [4194304,4194304] 0 2024-06-15T01:07:17.543 INFO:tasks.workunit.client.0.smithi137.stdout:1/862: getdents d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/df6/d102 0 2024-06-15T01:07:17.544 INFO:tasks.workunit.client.0.smithi137.stdout:0/944: dread d1/d2/dc/d10/d14/d56/d117/f59 [0,4194304] 0 2024-06-15T01:07:17.548 INFO:tasks.workunit.client.0.smithi137.stdout:0/945: symlink d1/d2/dc/d10/d14/d56/d117/d17/l13b 0 2024-06-15T01:07:17.549 INFO:tasks.workunit.client.0.smithi137.stdout:2/627: truncate d0/d42/d49/d57/f9e 616532 0 2024-06-15T01:07:17.553 INFO:tasks.workunit.client.0.smithi137.stdout:0/946: getdents d1/d2/d13/d66/dc3 0 2024-06-15T01:07:17.555 INFO:tasks.workunit.client.0.smithi137.stdout:0/947: symlink d1/d2/dc/d10/d22/l13c 0 2024-06-15T01:07:17.558 INFO:tasks.workunit.client.0.smithi137.stdout:0/948: creat d1/d2/dc/d10/d14/d56/d117/d17/f13d x:0 0 0 2024-06-15T01:07:17.559 INFO:tasks.workunit.client.0.smithi137.stdout:2/628: dread d0/d42/d49/f62 [0,4194304] 0 2024-06-15T01:07:17.561 INFO:tasks.workunit.client.0.smithi137.stdout:0/949: symlink d1/d2/d13/d66/l13e 0 2024-06-15T01:07:17.564 INFO:tasks.workunit.client.0.smithi137.stdout:2/629: getdents d0 0 2024-06-15T01:07:17.564 INFO:tasks.workunit.client.0.smithi137.stdout:0/950: fsync d1/d2/dc/d10/d14/d56/d117/d1c/f1d 0 2024-06-15T01:07:17.567 INFO:tasks.workunit.client.0.smithi137.stdout:0/951: mkdir d1/d7e/dfb/d13f 0 2024-06-15T01:07:17.569 INFO:tasks.workunit.client.0.smithi137.stdout:2/630: rename d0/dc/d22/c37 to d0/de/d2b/cbc 0 2024-06-15T01:07:17.571 INFO:tasks.workunit.client.0.smithi137.stdout:2/631: rename d0/de/d2b/c68 to d0/cbd 0 2024-06-15T01:07:17.582 INFO:tasks.workunit.client.0.smithi137.stdout:0/952: truncate d1/d2/dc/d10/d14/d56/f82 636529 0 2024-06-15T01:07:17.584 INFO:tasks.workunit.client.0.smithi137.stdout:2/632: creat d0/dc/d22/fbe x:0 0 0 2024-06-15T01:07:17.588 INFO:tasks.workunit.client.0.smithi137.stdout:2/633: unlink d0/dc/d2e/d72/d56/c43 0 2024-06-15T01:07:17.589 INFO:tasks.workunit.client.0.smithi137.stdout:2/634: chown d0/dc/d2e/d72/f59 1120063 1 2024-06-15T01:07:17.594 INFO:tasks.workunit.client.0.smithi137.stdout:2/635: dread d0/dc/f94 [0,4194304] 0 2024-06-15T01:07:17.594 INFO:tasks.workunit.client.0.smithi137.stdout:8/597: sync 2024-06-15T01:07:17.595 INFO:tasks.workunit.client.0.smithi137.stdout:8/598: write d9/d22/f3e [4729887,99728] 0 2024-06-15T01:07:17.597 INFO:tasks.workunit.client.0.smithi137.stdout:8/599: rmdir d9/d2a/db9/dc6/d1e/d3f/dab/db7 39 2024-06-15T01:07:17.601 INFO:tasks.workunit.client.0.smithi137.stdout:2/636: dread d0/dc/d22/f4b [0,4194304] 0 2024-06-15T01:07:17.602 INFO:tasks.workunit.client.0.smithi137.stdout:2/637: chown d0/d1c 82 1 2024-06-15T01:07:17.607 INFO:tasks.workunit.client.0.smithi137.stdout:2/638: rename d0/dc/d2e/d72/f67 to d0/d42/d49/d57/fbf 0 2024-06-15T01:07:17.619 INFO:tasks.workunit.client.0.smithi137.stdout:8/600: dwrite d9/d2a/d2f/d59/f96 [0,4194304] 0 2024-06-15T01:07:17.625 INFO:tasks.workunit.client.0.smithi137.stdout:8/601: truncate d9/d22/f98 32392 0 2024-06-15T01:07:17.625 INFO:tasks.workunit.client.0.smithi137.stdout:2/639: dwrite d0/da/d73/d8d/f53 [0,4194304] 0 2024-06-15T01:07:17.628 INFO:tasks.workunit.client.0.smithi137.stdout:8/602: symlink d9/d2a/d2f/lcb 0 2024-06-15T01:07:17.633 INFO:tasks.workunit.client.0.smithi137.stdout:8/603: rename d9/f1d to d9/d2a/d79/d7e/fcc 0 2024-06-15T01:07:17.634 INFO:tasks.workunit.client.0.smithi137.stdout:2/640: getdents d0/de 0 2024-06-15T01:07:17.635 INFO:tasks.workunit.client.0.smithi137.stdout:2/641: readlink d0/de/d2b/lb4 0 2024-06-15T01:07:17.639 INFO:tasks.workunit.client.0.smithi137.stdout:2/642: creat d0/dc/d2e/fc0 x:0 0 0 2024-06-15T01:07:17.641 INFO:tasks.workunit.client.0.smithi137.stdout:2/643: getdents d0/da 0 2024-06-15T01:07:17.644 INFO:tasks.workunit.client.0.smithi137.stdout:2/644: getdents d0/d42 0 2024-06-15T01:07:17.644 INFO:tasks.workunit.client.0.smithi137.stdout:2/645: chown d0/dc/d2e/d72/fa1 0 1 2024-06-15T01:07:17.646 INFO:tasks.workunit.client.0.smithi137.stdout:2/646: rmdir d0/de 39 2024-06-15T01:07:17.649 INFO:tasks.workunit.client.0.smithi137.stdout:2/647: creat d0/da/d73/d8d/d5c/d99/fc1 x:0 0 0 2024-06-15T01:07:17.652 INFO:tasks.workunit.client.0.smithi137.stdout:8/604: write d9/d2a/d2f/fa1 [219717,65880] 0 2024-06-15T01:07:17.654 INFO:tasks.workunit.client.0.smithi137.stdout:2/648: symlink d0/d42/d49/lc2 0 2024-06-15T01:07:17.655 INFO:tasks.workunit.client.0.smithi137.stdout:8/605: chown d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb 189109 1 2024-06-15T01:07:17.655 INFO:tasks.workunit.client.0.smithi137.stdout:2/649: fdatasync d0/da/d73/d8d/d5c/d79/fb6 0 2024-06-15T01:07:17.658 INFO:tasks.workunit.client.0.smithi137.stdout:8/606: symlink d9/d2a/db9/dc6/d1e/db1/lcd 0 2024-06-15T01:07:17.660 INFO:tasks.workunit.client.0.smithi137.stdout:2/650: getdents d0/d1c 0 2024-06-15T01:07:17.661 INFO:tasks.workunit.client.0.smithi137.stdout:2/651: chown d0/da/da0 2083308 1 2024-06-15T01:07:17.663 INFO:tasks.workunit.client.0.smithi137.stdout:8/607: write d9/d2a/db9/dc6/d1e/d67/f6a [1600004,31686] 0 2024-06-15T01:07:17.664 INFO:tasks.workunit.client.0.smithi137.stdout:2/652: dread d0/de/f24 [0,4194304] 0 2024-06-15T01:07:17.671 INFO:tasks.workunit.client.0.smithi137.stdout:8/608: dwrite d9/d2a/db9/dc6/d1e/d67/f6a [0,4194304] 0 2024-06-15T01:07:17.672 INFO:tasks.workunit.client.0.smithi137.stdout:8/609: write f8 [5981501,56790] 0 2024-06-15T01:07:17.675 INFO:tasks.workunit.client.0.smithi137.stdout:8/610: truncate d9/d2a/d79/fa3 840554 0 2024-06-15T01:07:17.678 INFO:tasks.workunit.client.0.smithi137.stdout:8/611: symlink d9/d2a/db9/dc6/d1e/d3f/d84/lce 0 2024-06-15T01:07:17.681 INFO:tasks.workunit.client.0.smithi137.stdout:8/612: truncate d9/d2a/d2f/d59/d82/f70 370136 0 2024-06-15T01:07:17.682 INFO:tasks.workunit.client.0.smithi137.stdout:2/653: dread d0/dc/d2e/d72/d56/f4f [0,4194304] 0 2024-06-15T01:07:17.684 INFO:tasks.workunit.client.0.smithi137.stdout:2/654: creat d0/da/da0/fc3 x:0 0 0 2024-06-15T01:07:17.687 INFO:tasks.workunit.client.0.smithi137.stdout:2/655: creat d0/dc/d2e/d72/d56/d6e/fc4 x:0 0 0 2024-06-15T01:07:17.692 INFO:tasks.workunit.client.0.smithi137.stdout:8/613: write d9/d2a/db9/dc6/d1e/f27 [4656176,85551] 0 2024-06-15T01:07:17.695 INFO:tasks.workunit.client.0.smithi137.stdout:8/614: creat d9/d2a/d2f/d59/fcf x:0 0 0 2024-06-15T01:07:17.695 INFO:tasks.workunit.client.0.smithi137.stdout:2/656: dwrite d0/dc/d2e/d72/d56/f93 [0,4194304] 0 2024-06-15T01:07:17.696 INFO:tasks.workunit.client.0.smithi137.stdout:2/657: read - d0/dc/d2e/d72/d56/d6e/fc4 zero size 2024-06-15T01:07:17.699 INFO:tasks.workunit.client.0.smithi137.stdout:2/658: write d0/da/d73/d8d/d5c/d79/f97 [1519880,4326] 0 2024-06-15T01:07:17.702 INFO:tasks.workunit.client.0.smithi137.stdout:2/659: truncate d0/f5b 284612 0 2024-06-15T01:07:17.704 INFO:tasks.workunit.client.0.smithi137.stdout:8/615: dread d9/d2a/f69 [0,4194304] 0 2024-06-15T01:07:17.705 INFO:tasks.workunit.client.0.smithi137.stdout:2/660: symlink d0/dc/lc5 0 2024-06-15T01:07:17.707 INFO:tasks.workunit.client.0.smithi137.stdout:2/661: write d0/de/d2b/f32 [764842,107090] 0 2024-06-15T01:07:17.707 INFO:tasks.workunit.client.0.smithi137.stdout:8/616: creat d9/d2a/db9/dc6/d1e/db1/fd0 x:0 0 0 2024-06-15T01:07:17.710 INFO:tasks.workunit.client.0.smithi137.stdout:8/617: unlink d9/d2a/d79/fa3 0 2024-06-15T01:07:17.711 INFO:tasks.workunit.client.0.smithi137.stdout:2/662: mknod d0/cc6 0 2024-06-15T01:07:17.714 INFO:tasks.workunit.client.0.smithi137.stdout:8/618: mkdir d9/d2a/d2f/d59/d82/dd1 0 2024-06-15T01:07:17.715 INFO:tasks.workunit.client.0.smithi137.stdout:2/663: mknod d0/da/d73/d8d/d5c/d90/cc7 0 2024-06-15T01:07:17.717 INFO:tasks.workunit.client.0.smithi137.stdout:2/664: read d0/da/d73/d8d/d5c/d79/f97 [194251,127581] 0 2024-06-15T01:07:17.718 INFO:tasks.workunit.client.0.smithi137.stdout:8/619: creat d9/d2a/db9/dc6/d1e/d35/dc8/fd2 x:0 0 0 2024-06-15T01:07:17.719 INFO:tasks.workunit.client.0.smithi137.stdout:2/665: chown d0/de/d2b/l36 380446 1 2024-06-15T01:07:17.721 INFO:tasks.workunit.client.0.smithi137.stdout:8/620: symlink d9/d22/ld3 0 2024-06-15T01:07:17.723 INFO:tasks.workunit.client.0.smithi137.stdout:2/666: mkdir d0/da/d73/d8d/dc8 0 2024-06-15T01:07:17.727 INFO:tasks.workunit.client.0.smithi137.stdout:8/621: creat d9/d2a/db9/dc6/d1e/fd4 x:0 0 0 2024-06-15T01:07:17.728 INFO:tasks.workunit.client.0.smithi137.stdout:2/667: getdents d0/d42/d49/d57 0 2024-06-15T01:07:17.741 INFO:tasks.workunit.client.0.smithi137.stdout:2/668: dwrite d0/d42/d49/d57/f9e [0,4194304] 0 2024-06-15T01:07:17.741 INFO:tasks.workunit.client.0.smithi137.stdout:8/622: dread d9/d22/f33 [0,4194304] 0 2024-06-15T01:07:17.743 INFO:tasks.workunit.client.0.smithi137.stdout:2/669: fsync d0/da/d73/d8d/d5c/d99/fc1 0 2024-06-15T01:07:17.749 INFO:tasks.workunit.client.0.smithi137.stdout:2/670: rename d0/dc/d22/l87 to d0/da/d73/d8d/lc9 0 2024-06-15T01:07:17.749 INFO:tasks.workunit.client.0.smithi137.stdout:8/623: write d9/d22/fbe [799197,127817] 0 2024-06-15T01:07:17.750 INFO:tasks.workunit.client.0.smithi137.stdout:8/624: read - d9/d2a/db9/dc6/d1e/db1/fd0 zero size 2024-06-15T01:07:17.752 INFO:tasks.workunit.client.0.smithi137.stdout:2/671: creat d0/dc/d2e/d72/fca x:0 0 0 2024-06-15T01:07:17.753 INFO:tasks.workunit.client.0.smithi137.stdout:8/625: creat d9/d2a/d2f/d59/d82/fd5 x:0 0 0 2024-06-15T01:07:17.755 INFO:tasks.workunit.client.0.smithi137.stdout:2/672: creat d0/d42/d49/fcb x:0 0 0 2024-06-15T01:07:17.756 INFO:tasks.workunit.client.0.smithi137.stdout:8/626: mknod d9/d2a/db9/cd6 0 2024-06-15T01:07:17.756 INFO:tasks.workunit.client.0.smithi137.stdout:2/673: dread - d0/de/d2b/f8e zero size 2024-06-15T01:07:17.758 INFO:tasks.workunit.client.0.smithi137.stdout:2/674: mknod d0/de/ccc 0 2024-06-15T01:07:17.759 INFO:tasks.workunit.client.0.smithi137.stdout:8/627: symlink d9/d2a/db9/dc6/d1e/ld7 0 2024-06-15T01:07:17.768 INFO:tasks.workunit.client.0.smithi137.stdout:8/628: truncate d9/d2a/db9/dc6/d1e/d67/f6a 3996980 0 2024-06-15T01:07:17.772 INFO:tasks.workunit.client.0.smithi137.stdout:2/675: dread d0/dc/d2e/d72/fa8 [0,4194304] 0 2024-06-15T01:07:17.780 INFO:tasks.workunit.client.0.smithi137.stdout:8/629: dwrite d9/d2a/d79/d7e/f8c [0,4194304] 0 2024-06-15T01:07:17.782 INFO:tasks.workunit.client.0.smithi137.stdout:2/676: dwrite d0/d1c/f66 [0,4194304] 0 2024-06-15T01:07:17.783 INFO:tasks.workunit.client.0.smithi137.stdout:2/677: dread - d0/da/da0/fb1 zero size 2024-06-15T01:07:17.787 INFO:tasks.workunit.client.0.smithi137.stdout:8/630: mkdir d9/d2a/db9/dc6/d1e/d3f/dab/dd8 0 2024-06-15T01:07:17.790 INFO:tasks.workunit.client.0.smithi137.stdout:2/678: link d0/dc/d2e/d72/f9c d0/dc/d2e/d72/d56/fcd 0 2024-06-15T01:07:17.791 INFO:tasks.workunit.client.0.smithi137.stdout:2/679: write d0/da/da0/fc3 [423618,53385] 0 2024-06-15T01:07:17.792 INFO:tasks.workunit.client.0.smithi137.stdout:2/680: read d0/dc/d22/f61 [675615,121640] 0 2024-06-15T01:07:17.809 INFO:tasks.workunit.client.0.smithi137.stdout:2/681: dwrite d0/dc/d2e/d72/d56/fa5 [0,4194304] 0 2024-06-15T01:07:17.810 INFO:tasks.workunit.client.0.smithi137.stdout:8/631: dread d9/d2a/db9/dc6/d1e/d3f/f41 [0,4194304] 0 2024-06-15T01:07:17.816 INFO:tasks.workunit.client.0.smithi137.stdout:2/682: dread d0/dc/d2e/d72/d56/f93 [0,4194304] 0 2024-06-15T01:07:17.816 INFO:tasks.workunit.client.0.smithi137.stdout:8/632: rename d9/d2a/db9/dc6/f83 to d9/d2a/d79/d7e/fd9 0 2024-06-15T01:07:17.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:17 smithi142 ceph-mon[37379]: pgmap v24: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 149 MiB/s rd, 349 MiB/s wr, 761 op/s 2024-06-15T01:07:17.817 INFO:tasks.workunit.client.0.smithi137.stdout:8/633: chown d9/d22/fbe 760 1 2024-06-15T01:07:17.819 INFO:tasks.workunit.client.0.smithi137.stdout:8/634: dread d9/d2a/d79/faa [0,4194304] 0 2024-06-15T01:07:17.820 INFO:tasks.workunit.client.0.smithi137.stdout:2/683: truncate d0/dc/f2c 7254829 0 2024-06-15T01:07:17.821 INFO:tasks.workunit.client.0.smithi137.stdout:8/635: write d9/d2a/d79/d7e/f8c [1556857,103334] 0 2024-06-15T01:07:17.823 INFO:tasks.workunit.client.0.smithi137.stdout:2/684: chown d0/da/d73/d8d/d5c/d90/la2 13515310 1 2024-06-15T01:07:17.828 INFO:tasks.workunit.client.0.smithi137.stdout:2/685: mkdir d0/dce 0 2024-06-15T01:07:17.829 INFO:tasks.workunit.client.0.smithi137.stdout:8/636: rename d9/d2a/d2f/d59/d82/l5e to d9/d2a/db9/dc6/lda 0 2024-06-15T01:07:17.832 INFO:tasks.workunit.client.0.smithi137.stdout:8/637: read d9/d2a/d2f/f72 [177903,57184] 0 2024-06-15T01:07:17.833 INFO:tasks.workunit.client.0.smithi137.stdout:8/638: readlink d9/d2a/db9/dc6/d1e/l88 0 2024-06-15T01:07:17.834 INFO:tasks.workunit.client.0.smithi137.stdout:2/686: creat d0/d42/d49/fcf x:0 0 0 2024-06-15T01:07:17.835 INFO:tasks.workunit.client.0.smithi137.stdout:8/639: mkdir d9/d2a/d2f/d59/ddb 0 2024-06-15T01:07:17.835 INFO:tasks.workunit.client.0.smithi137.stdout:2/687: rmdir d0/de/d2b/d47 39 2024-06-15T01:07:17.842 INFO:tasks.workunit.client.0.smithi137.stdout:2/688: mknod d0/dc/d2e/d72/d56/d6e/cd0 0 2024-06-15T01:07:17.842 INFO:tasks.workunit.client.0.smithi137.stdout:2/689: fdatasync d0/da/d73/d8d/d5c/d90/faa 0 2024-06-15T01:07:17.842 INFO:tasks.workunit.client.0.smithi137.stdout:2/690: chown d0/dc/d2e/d72/l4c 1109590 1 2024-06-15T01:07:17.844 INFO:tasks.workunit.client.0.smithi137.stdout:8/640: write d9/d2a/db9/dc6/d1e/d35/f5b [4569073,70587] 0 2024-06-15T01:07:17.851 INFO:tasks.workunit.client.0.smithi137.stdout:2/691: dwrite d0/dc/d2e/d72/d56/fcd [0,4194304] 0 2024-06-15T01:07:17.852 INFO:tasks.workunit.client.0.smithi137.stdout:8/641: link d9/d2a/db9/dc6/d1e/db1/cb2 d9/d2a/db9/dc6/d1e/cdc 0 2024-06-15T01:07:17.854 INFO:tasks.workunit.client.0.smithi137.stdout:2/692: write d0/da/d73/d8d/d5c/fa3 [991311,59539] 0 2024-06-15T01:07:17.855 INFO:tasks.workunit.client.0.smithi137.stdout:8/642: creat d9/d2a/d2f/d59/ddb/fdd x:0 0 0 2024-06-15T01:07:17.861 INFO:tasks.workunit.client.0.smithi137.stdout:2/693: unlink d0/dc/fbb 0 2024-06-15T01:07:17.881 INFO:tasks.workunit.client.0.smithi137.stdout:8/643: dread d9/d2a/f8a [0,4194304] 0 2024-06-15T01:07:17.884 INFO:tasks.workunit.client.0.smithi137.stdout:2/694: dwrite d0/dc/d2e/d72/d56/f4f [4194304,4194304] 0 2024-06-15T01:07:17.886 INFO:tasks.workunit.client.0.smithi137.stdout:2/695: mkdir d0/dc/d2e/dd1 0 2024-06-15T01:07:17.889 INFO:tasks.workunit.client.0.smithi137.stdout:2/696: creat d0/dc/d22/fd2 x:0 0 0 2024-06-15T01:07:17.889 INFO:tasks.workunit.client.0.smithi137.stdout:8/644: dwrite d9/fae [0,4194304] 0 2024-06-15T01:07:17.892 INFO:tasks.workunit.client.0.smithi137.stdout:8/645: symlink d9/d2a/db9/lde 0 2024-06-15T01:07:17.895 INFO:tasks.workunit.client.0.smithi137.stdout:2/697: rename d0/dc/d2e/dd1 to d0/da/dd3 0 2024-06-15T01:07:17.897 INFO:tasks.workunit.client.0.smithi137.stdout:2/698: getdents d0/d1c 0 2024-06-15T01:07:17.904 INFO:tasks.workunit.client.0.smithi137.stdout:2/699: dwrite d0/dc/d22/fbe [0,4194304] 0 2024-06-15T01:07:17.905 INFO:tasks.workunit.client.0.smithi137.stdout:2/700: dread - d0/dc/d2e/d72/fca zero size 2024-06-15T01:07:17.908 INFO:tasks.workunit.client.0.smithi137.stdout:2/701: read d0/da/da0/fc3 [299910,108386] 0 2024-06-15T01:07:17.909 INFO:tasks.workunit.client.0.smithi137.stdout:2/702: fdatasync d0/da/d73/d8d/f53 0 2024-06-15T01:07:17.916 INFO:tasks.workunit.client.0.smithi137.stdout:2/703: rename d0/da/d73/d8d to d0/da/da0/d3a/dd4 0 2024-06-15T01:07:17.917 INFO:tasks.workunit.client.0.smithi137.stdout:2/704: stat d0/cbd 0 2024-06-15T01:07:17.920 INFO:tasks.workunit.client.0.smithi137.stdout:2/705: mknod d0/de/d2b/cd5 0 2024-06-15T01:07:17.923 INFO:tasks.workunit.client.0.smithi137.stdout:2/706: truncate d0/dc/d2e/d72/f59 2542627 0 2024-06-15T01:07:17.924 INFO:tasks.workunit.client.0.smithi137.stdout:8/646: dread d9/d2a/db9/dc6/d1e/d35/d5f/f74 [0,4194304] 0 2024-06-15T01:07:17.931 INFO:tasks.workunit.client.0.smithi137.stdout:2/707: getdents d0/dc 0 2024-06-15T01:07:17.932 INFO:tasks.workunit.client.0.smithi137.stdout:2/708: mknod d0/d1c/cd6 0 2024-06-15T01:07:17.941 INFO:tasks.workunit.client.0.smithi137.stdout:8/647: dread d9/d2a/f2d [0,4194304] 0 2024-06-15T01:07:17.944 INFO:tasks.workunit.client.0.smithi137.stdout:8/648: mknod d9/d2a/db9/dc6/d1e/d3f/dab/db7/cdf 0 2024-06-15T01:07:17.944 INFO:tasks.workunit.client.0.smithi137.stdout:2/709: write d0/dc/f9d [214892,75134] 0 2024-06-15T01:07:17.946 INFO:tasks.workunit.client.0.smithi137.stdout:8/649: read d9/d2a/db9/dc6/d1e/f7f [38956,101840] 0 2024-06-15T01:07:17.949 INFO:tasks.workunit.client.0.smithi137.stdout:8/650: rename d9/d22/l3b to d9/d2a/db9/dc6/d1e/d3f/dab/db7/le0 0 2024-06-15T01:07:17.950 INFO:tasks.workunit.client.0.smithi137.stdout:2/710: truncate d0/dc/d2e/d72/d56/f93 448502 0 2024-06-15T01:07:17.951 INFO:tasks.workunit.client.0.smithi137.stdout:8/651: write d9/d2a/d79/d7e/fcc [2619539,54083] 0 2024-06-15T01:07:17.952 INFO:tasks.workunit.client.0.smithi137.stdout:2/711: write d0/f69 [3747111,48870] 0 2024-06-15T01:07:17.953 INFO:tasks.workunit.client.0.smithi137.stdout:8/652: write d9/d2a/f7a [2523094,88466] 0 2024-06-15T01:07:17.957 INFO:tasks.workunit.client.0.smithi137.stdout:8/653: mkdir d9/d2a/db9/dc6/de1 0 2024-06-15T01:07:17.959 INFO:tasks.workunit.client.0.smithi137.stdout:2/712: mknod d0/db9/cd7 0 2024-06-15T01:07:17.960 INFO:tasks.workunit.client.0.smithi137.stdout:2/713: stat d0/da/da0/d3a/dd4/d5c/fa3 0 2024-06-15T01:07:17.960 INFO:tasks.workunit.client.0.smithi137.stdout:8/654: symlink d9/d2a/d79/d7e/le2 0 2024-06-15T01:07:17.963 INFO:tasks.workunit.client.0.smithi137.stdout:2/714: unlink d0/da/l26 0 2024-06-15T01:07:17.964 INFO:tasks.workunit.client.0.smithi137.stdout:8/655: mknod d9/d22/ce3 0 2024-06-15T01:07:17.965 INFO:tasks.workunit.client.0.smithi137.stdout:2/715: creat d0/d42/d49/fd8 x:0 0 0 2024-06-15T01:07:17.967 INFO:tasks.workunit.client.0.smithi137.stdout:8/656: dread d9/d2a/d2f/f72 [0,4194304] 0 2024-06-15T01:07:17.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:17 smithi137 ceph-mon[29354]: pgmap v24: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 149 MiB/s rd, 349 MiB/s wr, 761 op/s 2024-06-15T01:07:17.975 INFO:tasks.workunit.client.0.smithi137.stdout:2/716: truncate d0/dc/f86 2117206 0 2024-06-15T01:07:17.980 INFO:tasks.workunit.client.0.smithi137.stdout:8/657: dwrite d9/d2a/f6d [0,4194304] 0 2024-06-15T01:07:17.982 INFO:tasks.workunit.client.0.smithi137.stdout:2/717: fsync d0/de/f1a 0 2024-06-15T01:07:17.984 INFO:tasks.workunit.client.0.smithi137.stdout:8/658: unlink d9/d2a/d2f/fa1 0 2024-06-15T01:07:17.995 INFO:tasks.workunit.client.0.smithi137.stdout:8/659: dwrite d9/d2a/db9/dc6/f13 [4194304,4194304] 0 2024-06-15T01:07:17.995 INFO:tasks.workunit.client.0.smithi137.stdout:2/718: rename d0/c1b to d0/da/da0/d3a/dd4/dc8/cd9 0 2024-06-15T01:07:18.002 INFO:tasks.workunit.client.0.smithi137.stdout:2/719: dwrite d0/da/da0/d3a/dd4/d5c/d99/faf [0,4194304] 0 2024-06-15T01:07:18.005 INFO:tasks.workunit.client.0.smithi137.stdout:8/660: dread d9/d2a/f34 [0,4194304] 0 2024-06-15T01:07:18.006 INFO:tasks.workunit.client.0.smithi137.stdout:2/720: mkdir d0/da/dda 0 2024-06-15T01:07:18.010 INFO:tasks.workunit.client.0.smithi137.stdout:8/661: rename d9/d2a/db9/dc6/d1e/db1 to d9/d2a/db9/dc6/d1e/d3f/d84/de4 0 2024-06-15T01:07:18.011 INFO:tasks.workunit.client.0.smithi137.stdout:2/721: mknod d0/da/dda/cdb 0 2024-06-15T01:07:18.014 INFO:tasks.workunit.client.0.smithi137.stdout:2/722: fsync d0/dc/d2e/fc0 0 2024-06-15T01:07:18.018 INFO:tasks.workunit.client.0.smithi137.stdout:8/662: dwrite d9/d2a/db9/dc6/d1e/d3f/f68 [0,4194304] 0 2024-06-15T01:07:18.018 INFO:tasks.workunit.client.0.smithi137.stdout:2/723: rename d0/cc6 to d0/da/da0/d3a/cdc 0 2024-06-15T01:07:18.020 INFO:tasks.workunit.client.0.smithi137.stdout:2/724: write d0/de/d2b/f8e [868462,29172] 0 2024-06-15T01:07:18.020 INFO:tasks.workunit.client.0.smithi137.stdout:2/725: readlink d0/da/da0/d3a/dd4/d5c/d90/la2 0 2024-06-15T01:07:18.024 INFO:tasks.workunit.client.0.smithi137.stdout:2/726: rename d0/dc/d2e/d72/d56/f4f to d0/dc/d2e/fdd 0 2024-06-15T01:07:18.040 INFO:tasks.workunit.client.0.smithi137.stdout:2/727: truncate d0/dc/d2e/d72/d56/fa5 1041097 0 2024-06-15T01:07:18.042 INFO:tasks.workunit.client.0.smithi137.stdout:2/728: getdents d0/d42 0 2024-06-15T01:07:18.043 INFO:tasks.workunit.client.0.smithi137.stdout:2/729: chown d0/dc/d2e/d72/d56/fcd 1283 1 2024-06-15T01:07:18.310 INFO:tasks.workunit.client.0.smithi137.stdout:7/596: sync 2024-06-15T01:07:18.312 INFO:tasks.workunit.client.0.smithi137.stdout:7/597: mkdir df/d13/d2e/d1a/dc0 0 2024-06-15T01:07:18.314 INFO:tasks.workunit.client.0.smithi137.stdout:7/598: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/c91 1 1 2024-06-15T01:07:18.325 INFO:tasks.workunit.client.0.smithi137.stdout:7/599: dwrite df/d13/f20 [0,4194304] 0 2024-06-15T01:07:18.330 INFO:tasks.workunit.client.0.smithi137.stdout:7/600: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fc1 x:0 0 0 2024-06-15T01:07:18.332 INFO:tasks.workunit.client.0.smithi137.stdout:7/601: write df/d7d/d1c/d1d/d27/d33/fa5 [276190,129807] 0 2024-06-15T01:07:18.333 INFO:tasks.workunit.client.0.smithi137.stdout:7/602: stat df/d13/c99 0 2024-06-15T01:07:18.336 INFO:tasks.workunit.client.0.smithi137.stdout:7/603: rmdir df/d7d/d1c/d1d/d27/d33/d43 39 2024-06-15T01:07:18.341 INFO:tasks.workunit.client.0.smithi137.stdout:7/604: getdents df/d13/d2e 0 2024-06-15T01:07:18.360 INFO:tasks.workunit.client.0.smithi137.stdout:7/605: dread df/d7d/d1c/d1d/d27/d33/d59/f60 [0,4194304] 0 2024-06-15T01:07:18.362 INFO:tasks.workunit.client.0.smithi137.stdout:7/606: symlink df/d13/d2e/d1a/lc2 0 2024-06-15T01:07:18.368 INFO:tasks.workunit.client.0.smithi137.stdout:7/607: dwrite df/d7d/d3b/d5b/f84 [0,4194304] 0 2024-06-15T01:07:18.371 INFO:tasks.workunit.client.0.smithi137.stdout:7/608: creat df/d13/d2e/d1a/dc0/fc3 x:0 0 0 2024-06-15T01:07:18.374 INFO:tasks.workunit.client.0.smithi137.stdout:7/609: mkdir df/d7d/d1c/d1d/dc4 0 2024-06-15T01:07:18.375 INFO:tasks.workunit.client.0.smithi137.stdout:7/610: write df/fb4 [747417,55834] 0 2024-06-15T01:07:18.380 INFO:tasks.workunit.client.0.smithi137.stdout:7/611: mkdir df/d7d/d1c/d1d/d27/d33/d43/dc5 0 2024-06-15T01:07:18.383 INFO:tasks.workunit.client.0.smithi137.stdout:7/612: rename df/d13/d2e/d1a/f80 to df/d7d/d1c/d1d/d27/d33/d59/fc6 0 2024-06-15T01:07:18.385 INFO:tasks.workunit.client.0.smithi137.stdout:7/613: creat df/d13/d2e/d1a/d3f/d89/fc7 x:0 0 0 2024-06-15T01:07:18.388 INFO:tasks.workunit.client.0.smithi137.stdout:7/614: creat df/d13/d2e/d1a/d3f/d89/fc8 x:0 0 0 2024-06-15T01:07:18.388 INFO:tasks.workunit.client.0.smithi137.stdout:7/615: chown df/d13/d2e/d1a/fae 19171704 1 2024-06-15T01:07:18.390 INFO:tasks.workunit.client.0.smithi137.stdout:7/616: creat df/d13/d2e/d1a/d3f/d89/fc9 x:0 0 0 2024-06-15T01:07:18.391 INFO:tasks.workunit.client.0.smithi137.stdout:7/617: dread - df/d13/d2e/d1a/d8c/fba zero size 2024-06-15T01:07:18.393 INFO:tasks.workunit.client.0.smithi137.stdout:7/618: creat df/d13/d2e/d87/fca x:0 0 0 2024-06-15T01:07:18.394 INFO:tasks.workunit.client.0.smithi137.stdout:7/619: chown df/d13/d2e/c1b 53581 1 2024-06-15T01:07:18.395 INFO:tasks.workunit.client.0.smithi137.stdout:7/620: creat df/d7d/d1c/d1d/d79/fcb x:0 0 0 2024-06-15T01:07:18.397 INFO:tasks.workunit.client.0.smithi137.stdout:7/621: mknod df/ccc 0 2024-06-15T01:07:18.398 INFO:tasks.workunit.client.0.smithi137.stdout:7/622: chown df/d13/d2e/d1a/d3f/f54 336303102 1 2024-06-15T01:07:18.404 INFO:tasks.workunit.client.0.smithi137.stdout:7/623: dread - df/d7d/d1c/d1d/d27/d33/d59/fc6 zero size 2024-06-15T01:07:18.406 INFO:tasks.workunit.client.0.smithi137.stdout:7/624: write df/d7d/d1c/d1d/d27/d33/d43/d6c/f37 [655834,47632] 0 2024-06-15T01:07:18.406 INFO:tasks.workunit.client.0.smithi137.stdout:7/625: stat df/d13/d3c 0 2024-06-15T01:07:18.411 INFO:tasks.workunit.client.0.smithi137.stdout:7/626: link df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fc1 df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fcd 0 2024-06-15T01:07:18.412 INFO:tasks.workunit.client.0.smithi137.stdout:7/627: mkdir df/d13/d2e/dce 0 2024-06-15T01:07:18.414 INFO:tasks.workunit.client.0.smithi137.stdout:7/628: mkdir df/d13/d2e/d1a/d6a/d90/dcf 0 2024-06-15T01:07:18.415 INFO:tasks.workunit.client.0.smithi137.stdout:7/629: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fcd 1 1 2024-06-15T01:07:18.417 INFO:tasks.workunit.client.0.smithi137.stdout:7/630: mkdir df/d7d/d1c/d1d/dd0 0 2024-06-15T01:07:18.418 INFO:tasks.workunit.client.0.smithi137.stdout:7/631: write df/d13/d2e/d87/fca [453936,112492] 0 2024-06-15T01:07:18.420 INFO:tasks.workunit.client.0.smithi137.stdout:7/632: unlink df/d13/d2e/f58 0 2024-06-15T01:07:18.422 INFO:tasks.workunit.client.0.smithi137.stdout:7/633: rmdir df/d13/d2e/d1a 39 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:8/663: sync 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:0/953: sync 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:9/585: sync 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:5/821: sync 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:6/643: sync 2024-06-15T01:07:18.423 INFO:tasks.workunit.client.0.smithi137.stdout:2/730: sync 2024-06-15T01:07:18.424 INFO:tasks.workunit.client.0.smithi137.stdout:4/810: sync 2024-06-15T01:07:18.424 INFO:tasks.workunit.client.0.smithi137.stdout:1/863: sync 2024-06-15T01:07:18.425 INFO:tasks.workunit.client.0.smithi137.stdout:7/634: dread - df/d13/d2e/d1a/d3f/d89/fc8 zero size 2024-06-15T01:07:18.426 INFO:tasks.workunit.client.0.smithi137.stdout:2/731: unlink d0/da/da0/f19 0 2024-06-15T01:07:18.427 INFO:tasks.workunit.client.0.smithi137.stdout:0/954: chown d1/d2/dc/d10/d22/l123 103712071 1 2024-06-15T01:07:18.427 INFO:tasks.workunit.client.0.smithi137.stdout:6/644: chown d0/d1a/d3e/d57/d1d/da5/l66 1170 1 2024-06-15T01:07:18.428 INFO:tasks.workunit.client.0.smithi137.stdout:9/586: mknod df/cac 0 2024-06-15T01:07:18.429 INFO:tasks.workunit.client.0.smithi137.stdout:1/864: symlink d0/d10/d1d/l11d 0 2024-06-15T01:07:18.429 INFO:tasks.workunit.client.0.smithi137.stdout:8/664: creat d9/d22/fe5 x:0 0 0 2024-06-15T01:07:18.430 INFO:tasks.workunit.client.0.smithi137.stdout:7/635: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/l4d 211221 1 2024-06-15T01:07:18.431 INFO:tasks.workunit.client.0.smithi137.stdout:8/665: read - d9/d2a/db9/dc6/fbd zero size 2024-06-15T01:07:18.433 INFO:tasks.workunit.client.0.smithi137.stdout:7/636: read df/d13/d2e/d1a/d3f/f54 [1223398,44188] 0 2024-06-15T01:07:18.435 INFO:tasks.workunit.client.0.smithi137.stdout:0/955: readlink d1/d2/dc/d10/d14/d56/d117/d17/d54/l76 0 2024-06-15T01:07:18.444 INFO:tasks.workunit.client.0.smithi137.stdout:0/956: unlink d1/d2/d13/f130 0 2024-06-15T01:07:18.444 INFO:tasks.workunit.client.0.smithi137.stdout:4/811: write dd/d15/d21/d9c/f74 [363626,85362] 0 2024-06-15T01:07:18.444 INFO:tasks.workunit.client.0.smithi137.stdout:5/822: link df/d44/d54/d95/d3b/d4b/c10f df/d11/d30/c114 0 2024-06-15T01:07:18.446 INFO:tasks.workunit.client.0.smithi137.stdout:4/812: dread - dd/d15/d21/d9c/d91/f10d zero size 2024-06-15T01:07:18.447 INFO:tasks.workunit.client.0.smithi137.stdout:6/645: creat d0/d1a/fc7 x:0 0 0 2024-06-15T01:07:18.448 INFO:tasks.workunit.client.0.smithi137.stdout:1/865: write d0/d1f/d76/f80 [3474634,61039] 0 2024-06-15T01:07:18.450 INFO:tasks.workunit.client.0.smithi137.stdout:7/637: mkdir df/d7d/d3b/dd1 0 2024-06-15T01:07:18.451 INFO:tasks.workunit.client.0.smithi137.stdout:5/823: fsync df/d44/f84 0 2024-06-15T01:07:18.452 INFO:tasks.workunit.client.0.smithi137.stdout:4/813: creat dd/d15/d21/d9c/f128 x:0 0 0 2024-06-15T01:07:18.454 INFO:tasks.workunit.client.0.smithi137.stdout:4/814: fdatasync dd/d15/d36/d45/dcb/d5b/f10a 0 2024-06-15T01:07:18.454 INFO:tasks.workunit.client.0.smithi137.stdout:6/646: rename d0/d5/d9/d3f/lb4 to d0/d5/d9/d3f/lc8 0 2024-06-15T01:07:18.458 INFO:tasks.workunit.client.0.smithi137.stdout:9/587: link df/d1a/l9f df/d12/d29/d79/d9a/lad 0 2024-06-15T01:07:18.459 INFO:tasks.workunit.client.0.smithi137.stdout:5/824: dwrite df/d44/d54/d95/d3b/d4b/fe1 [0,4194304] 0 2024-06-15T01:07:18.462 INFO:tasks.workunit.client.0.smithi137.stdout:4/815: mknod dd/d15/d36/c129 0 2024-06-15T01:07:18.463 INFO:tasks.workunit.client.0.smithi137.stdout:8/666: getdents d9/d2a/d79 0 2024-06-15T01:07:18.466 INFO:tasks.workunit.client.0.smithi137.stdout:6/647: unlink d0/d1a/d3e/d57/d1d/d2e/lc6 0 2024-06-15T01:07:18.470 INFO:tasks.workunit.client.0.smithi137.stdout:8/667: mknod d9/d2a/d79/ce6 0 2024-06-15T01:07:18.473 INFO:tasks.workunit.client.0.smithi137.stdout:1/866: dwrite d0/d3/d47/d73/fc7 [0,4194304] 0 2024-06-15T01:07:18.473 INFO:tasks.workunit.client.0.smithi137.stdout:8/668: symlink d9/d2a/le7 0 2024-06-15T01:07:18.475 INFO:tasks.workunit.client.0.smithi137.stdout:6/648: getdents d0/d5/d9/db9 0 2024-06-15T01:07:18.476 INFO:tasks.workunit.client.0.smithi137.stdout:8/669: truncate d9/d22/fe5 304770 0 2024-06-15T01:07:18.480 INFO:tasks.workunit.client.0.smithi137.stdout:1/867: rename d0/d3/d14/d34/fed to d0/d3/d47/d73/f11e 0 2024-06-15T01:07:18.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/670: mkdir d9/de8 0 2024-06-15T01:07:18.490 INFO:tasks.workunit.client.0.smithi137.stdout:1/868: dwrite d0/d10/d41/d7b/f90 [4194304,4194304] 0 2024-06-15T01:07:18.491 INFO:tasks.workunit.client.0.smithi137.stdout:5/825: dwrite df/d44/d54/d95/f6f [0,4194304] 0 2024-06-15T01:07:18.491 INFO:tasks.workunit.client.0.smithi137.stdout:8/671: mkdir d9/de9 0 2024-06-15T01:07:18.494 INFO:tasks.workunit.client.0.smithi137.stdout:9/588: dread df/d49/d8d/f93 [0,4194304] 0 2024-06-15T01:07:18.494 INFO:tasks.workunit.client.0.smithi137.stdout:8/672: stat d9/d2a/f34 0 2024-06-15T01:07:18.495 INFO:tasks.workunit.client.0.smithi137.stdout:5/826: stat df/d11/d14/f15 0 2024-06-15T01:07:18.497 INFO:tasks.workunit.client.0.smithi137.stdout:1/869: read - d0/d3/d89/fbc zero size 2024-06-15T01:07:18.502 INFO:tasks.workunit.client.0.smithi137.stdout:8/673: creat d9/d22/fea x:0 0 0 2024-06-15T01:07:18.502 INFO:tasks.workunit.client.0.smithi137.stdout:1/870: chown d0/d1f/cbe 18846 1 2024-06-15T01:07:18.502 INFO:tasks.workunit.client.0.smithi137.stdout:5/827: chown df/d11/d14/d6c/dc7/fd0 0 1 2024-06-15T01:07:18.505 INFO:tasks.workunit.client.0.smithi137.stdout:6/649: dwrite d0/d1a/d99/fa3 [0,4194304] 0 2024-06-15T01:07:18.507 INFO:tasks.workunit.client.0.smithi137.stdout:5/828: mkdir df/d44/d54/d115 0 2024-06-15T01:07:18.507 INFO:tasks.workunit.client.0.smithi137.stdout:1/871: chown d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/f111 0 1 2024-06-15T01:07:18.508 INFO:tasks.workunit.client.0.smithi137.stdout:9/589: dwrite df/d12/d1b/f35 [0,4194304] 0 2024-06-15T01:07:18.510 INFO:tasks.workunit.client.0.smithi137.stdout:8/674: truncate d9/d2a/d2f/d59/d82/fd5 691232 0 2024-06-15T01:07:18.513 INFO:tasks.workunit.client.0.smithi137.stdout:5/829: symlink df/d44/d54/d80/d105/l116 0 2024-06-15T01:07:18.518 INFO:tasks.workunit.client.0.smithi137.stdout:6/650: rename d0/d1a/d3e/d57/d1d/lab to d0/d1a/d3e/d57/d1d/d2e/lc9 0 2024-06-15T01:07:18.520 INFO:tasks.workunit.client.0.smithi137.stdout:1/872: dread d0/fb0 [0,4194304] 0 2024-06-15T01:07:18.521 INFO:tasks.workunit.client.0.smithi137.stdout:9/590: rename df/d1a/f76 to df/d13/fae 0 2024-06-15T01:07:18.521 INFO:tasks.workunit.client.0.smithi137.stdout:1/873: read - d0/d1f/d67/f94 zero size 2024-06-15T01:07:18.523 INFO:tasks.workunit.client.0.smithi137.stdout:1/874: mkdir d0/d10/d11a/d11f 0 2024-06-15T01:07:18.524 INFO:tasks.workunit.client.0.smithi137.stdout:9/591: mkdir df/d49/d4b/d9b/daf 0 2024-06-15T01:07:18.529 INFO:tasks.workunit.client.0.smithi137.stdout:1/875: rename d0/d1f/d60/d99/de3/dea/d10f/d4c/de5 to d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/df6/d120 0 2024-06-15T01:07:18.531 INFO:tasks.workunit.client.0.smithi137.stdout:1/876: mkdir d0/d3/d14/d34/df0/d121 0 2024-06-15T01:07:18.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/877: mknod d0/d3/d14/d34/df0/c122 0 2024-06-15T01:07:18.534 INFO:tasks.workunit.client.0.smithi137.stdout:5/830: write df/d11/d14/d6c/f90 [172534,95980] 0 2024-06-15T01:07:18.539 INFO:tasks.workunit.client.0.smithi137.stdout:8/675: dwrite d9/d2a/db9/dc6/f3c [0,4194304] 0 2024-06-15T01:07:18.542 INFO:tasks.workunit.client.0.smithi137.stdout:1/878: rename d0/d1f/d67/l6d to d0/d10/d11a/l123 0 2024-06-15T01:07:18.543 INFO:tasks.workunit.client.0.smithi137.stdout:9/592: dwrite df/d12/d15/d16/f92 [0,4194304] 0 2024-06-15T01:07:18.546 INFO:tasks.workunit.client.0.smithi137.stdout:9/593: fdatasync df/d12/d1b/f2f 0 2024-06-15T01:07:18.549 INFO:tasks.workunit.client.0.smithi137.stdout:5/831: mknod df/d44/d54/d95/d35/dea/dec/c117 0 2024-06-15T01:07:18.551 INFO:tasks.workunit.client.0.smithi137.stdout:8/676: creat d9/de9/feb x:0 0 0 2024-06-15T01:07:18.552 INFO:tasks.workunit.client.0.smithi137.stdout:9/594: mknod df/d12/d15/d75/cb0 0 2024-06-15T01:07:18.555 INFO:tasks.workunit.client.0.smithi137.stdout:8/677: creat d9/d55/fec x:0 0 0 2024-06-15T01:07:18.556 INFO:tasks.workunit.client.0.smithi137.stdout:5/832: dread df/d44/f4d [0,4194304] 0 2024-06-15T01:07:18.556 INFO:tasks.workunit.client.0.smithi137.stdout:8/678: read - d9/d2a/db9/dc6/fc2 zero size 2024-06-15T01:07:18.558 INFO:tasks.workunit.client.0.smithi137.stdout:1/879: getdents d0/d10/d41/d7b 0 2024-06-15T01:07:18.559 INFO:tasks.workunit.client.0.smithi137.stdout:8/679: dread d9/d2a/d2f/f72 [0,4194304] 0 2024-06-15T01:07:18.559 INFO:tasks.workunit.client.0.smithi137.stdout:5/833: truncate df/d76/d7c/d9c/da1/d102/f106 842934 0 2024-06-15T01:07:18.561 INFO:tasks.workunit.client.0.smithi137.stdout:5/834: creat df/d11/d30/f118 x:0 0 0 2024-06-15T01:07:18.562 INFO:tasks.workunit.client.0.smithi137.stdout:1/880: symlink d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/l124 0 2024-06-15T01:07:18.566 INFO:tasks.workunit.client.0.smithi137.stdout:9/595: write df/d49/d4b/f4f [2069114,45399] 0 2024-06-15T01:07:18.568 INFO:tasks.workunit.client.0.smithi137.stdout:8/680: dwrite d9/d2a/db9/dc6/d1e/d3f/d84/fc0 [0,4194304] 0 2024-06-15T01:07:18.570 INFO:tasks.workunit.client.0.smithi137.stdout:5/835: getdents df/d44/d54/d95/d35/dea 0 2024-06-15T01:07:18.571 INFO:tasks.workunit.client.0.smithi137.stdout:5/836: dread - df/d76/d8e/fdf zero size 2024-06-15T01:07:18.571 INFO:tasks.workunit.client.0.smithi137.stdout:9/596: fsync df/d12/d1b/d31/d42/f46 0 2024-06-15T01:07:18.574 INFO:tasks.workunit.client.0.smithi137.stdout:8/681: symlink d9/d2a/d2f/d59/d82/dd1/led 0 2024-06-15T01:07:18.576 INFO:tasks.workunit.client.0.smithi137.stdout:5/837: creat df/d76/d7c/d9c/da1/d102/f119 x:0 0 0 2024-06-15T01:07:18.577 INFO:tasks.workunit.client.0.smithi137.stdout:9/597: symlink df/d12/d29/d79/lb1 0 2024-06-15T01:07:18.577 INFO:tasks.workunit.client.0.smithi137.stdout:1/881: dwrite d0/d3/d89/fbc [0,4194304] 0 2024-06-15T01:07:18.578 INFO:tasks.workunit.client.0.smithi137.stdout:5/838: chown df/d44/cd5 343 1 2024-06-15T01:07:18.580 INFO:tasks.workunit.client.0.smithi137.stdout:8/682: mknod d9/d2a/d79/cee 0 2024-06-15T01:07:18.581 INFO:tasks.workunit.client.0.smithi137.stdout:1/882: fdatasync d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/ffc 0 2024-06-15T01:07:18.584 INFO:tasks.workunit.client.0.smithi137.stdout:5/839: rename df/d11/d14/fff to df/d44/d54/d95/d35/dea/dec/f11a 0 2024-06-15T01:07:18.586 INFO:tasks.workunit.client.0.smithi137.stdout:9/598: unlink df/d13/fae 0 2024-06-15T01:07:18.591 INFO:tasks.workunit.client.0.smithi137.stdout:8/683: rename d9/d2a/d79/d7e/f8c to d9/d2a/d79/fef 0 2024-06-15T01:07:18.592 INFO:tasks.workunit.client.0.smithi137.stdout:5/840: dwrite df/d44/d54/d95/f94 [0,4194304] 0 2024-06-15T01:07:18.594 INFO:tasks.workunit.client.0.smithi137.stdout:8/684: chown d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb 0 1 2024-06-15T01:07:18.594 INFO:tasks.workunit.client.0.smithi137.stdout:5/841: write df/d44/d54/d95/d3b/d49/fca [1131216,65430] 0 2024-06-15T01:07:18.596 INFO:tasks.workunit.client.0.smithi137.stdout:9/599: write f5 [2279600,62770] 0 2024-06-15T01:07:18.598 INFO:tasks.workunit.client.0.smithi137.stdout:8/685: stat d9/d2a/db9/dc6/d1e/d35/d5f 0 2024-06-15T01:07:18.605 INFO:tasks.workunit.client.0.smithi137.stdout:9/600: read df/d12/f3e [596077,83004] 0 2024-06-15T01:07:18.606 INFO:tasks.workunit.client.0.smithi137.stdout:1/883: write d0/d1f/d60/da9/fac [2583627,20998] 0 2024-06-15T01:07:18.608 INFO:tasks.workunit.client.0.smithi137.stdout:1/884: stat d0/d1f/d60/d99/de3/cef 0 2024-06-15T01:07:18.612 INFO:tasks.workunit.client.0.smithi137.stdout:1/885: unlink d0/d1f/cbe 0 2024-06-15T01:07:18.614 INFO:tasks.workunit.client.0.smithi137.stdout:8/686: getdents d9/d2a/db9/dc6 0 2024-06-15T01:07:18.615 INFO:tasks.workunit.client.0.smithi137.stdout:5/842: truncate fe 578058 0 2024-06-15T01:07:18.617 INFO:tasks.workunit.client.0.smithi137.stdout:9/601: dread df/d13/f30 [0,4194304] 0 2024-06-15T01:07:18.618 INFO:tasks.workunit.client.0.smithi137.stdout:1/886: rename d0/d1f/d67/ccf to d0/d1f/d76/d109/c125 0 2024-06-15T01:07:18.623 INFO:tasks.workunit.client.0.smithi137.stdout:8/687: dwrite d9/de9/feb [0,4194304] 0 2024-06-15T01:07:18.624 INFO:tasks.workunit.client.0.smithi137.stdout:5/843: rename df/d44/d54/d95/d3b/d4b/dc8 to df/d11/d14/d6c/d11b 0 2024-06-15T01:07:18.626 INFO:tasks.workunit.client.0.smithi137.stdout:1/887: creat d0/d1f/d76/da1/f126 x:0 0 0 2024-06-15T01:07:18.626 INFO:tasks.workunit.client.0.smithi137.stdout:1/888: stat d0/d3/d14/da0 0 2024-06-15T01:07:18.627 INFO:tasks.workunit.client.0.smithi137.stdout:1/889: dread - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/fad zero size 2024-06-15T01:07:18.629 INFO:tasks.workunit.client.0.smithi137.stdout:8/688: symlink d9/d55/lf0 0 2024-06-15T01:07:18.633 INFO:tasks.workunit.client.0.smithi137.stdout:8/689: stat d9/d2a/db9/dc6/d1e/d35/f5b 0 2024-06-15T01:07:18.638 INFO:tasks.workunit.client.0.smithi137.stdout:1/890: dwrite d0/d3/fd6 [4194304,4194304] 0 2024-06-15T01:07:18.639 INFO:tasks.workunit.client.0.smithi137.stdout:8/690: creat d9/d2a/db9/dc6/d1e/d3f/ff1 x:0 0 0 2024-06-15T01:07:18.641 INFO:tasks.workunit.client.0.smithi137.stdout:1/891: fdatasync d0/d1f/d76/f105 0 2024-06-15T01:07:18.642 INFO:tasks.workunit.client.0.smithi137.stdout:9/602: dwrite df/d12/d1b/d31/f67 [0,4194304] 0 2024-06-15T01:07:18.645 INFO:tasks.workunit.client.0.smithi137.stdout:8/691: dread d9/d2a/d2f/d59/d82/fd5 [0,4194304] 0 2024-06-15T01:07:18.646 INFO:tasks.workunit.client.0.smithi137.stdout:9/603: chown df/d2a/c3a 25 1 2024-06-15T01:07:18.653 INFO:tasks.workunit.client.0.smithi137.stdout:9/604: mknod df/d1a/cb2 0 2024-06-15T01:07:18.654 INFO:tasks.workunit.client.0.smithi137.stdout:9/605: chown l6 4836827 1 2024-06-15T01:07:18.656 INFO:tasks.workunit.client.0.smithi137.stdout:5/844: dread df/f41 [0,4194304] 0 2024-06-15T01:07:18.656 INFO:tasks.workunit.client.0.smithi137.stdout:9/606: mkdir df/d12/d29/d95/db3 0 2024-06-15T01:07:18.658 INFO:tasks.workunit.client.0.smithi137.stdout:5/845: write df/d44/d54/d80/d105/f10b [96390,114884] 0 2024-06-15T01:07:18.662 INFO:tasks.workunit.client.0.smithi137.stdout:5/846: rmdir df/d11/d30/db1 39 2024-06-15T01:07:18.662 INFO:tasks.workunit.client.0.smithi137.stdout:5/847: rmdir df/d44/d54/d95/d35/dea/dec 39 2024-06-15T01:07:18.664 INFO:tasks.workunit.client.0.smithi137.stdout:5/848: creat df/d11/f11c x:0 0 0 2024-06-15T01:07:18.676 INFO:tasks.workunit.client.0.smithi137.stdout:5/849: dwrite df/d44/d54/d95/f6f [0,4194304] 0 2024-06-15T01:07:18.678 INFO:tasks.workunit.client.0.smithi137.stdout:8/692: dwrite d9/d2a/d2f/f72 [0,4194304] 0 2024-06-15T01:07:18.679 INFO:tasks.workunit.client.0.smithi137.stdout:1/892: dwrite d0/d1f/d76/fc4 [0,4194304] 0 2024-06-15T01:07:18.686 INFO:tasks.workunit.client.0.smithi137.stdout:9/607: dread df/d12/d1b/d31/d42/f56 [0,4194304] 0 2024-06-15T01:07:18.687 INFO:tasks.workunit.client.0.smithi137.stdout:8/693: symlink d9/d2a/db9/dc6/d1e/lf2 0 2024-06-15T01:07:18.689 INFO:tasks.workunit.client.0.smithi137.stdout:5/850: creat df/d44/d54/d95/d3b/d49/d5c/de7/dbf/f11d x:0 0 0 2024-06-15T01:07:18.692 INFO:tasks.workunit.client.0.smithi137.stdout:9/608: rmdir df/d13 39 2024-06-15T01:07:18.696 INFO:tasks.workunit.client.0.smithi137.stdout:5/851: symlink df/d3d/l11e 0 2024-06-15T01:07:18.697 INFO:tasks.workunit.client.0.smithi137.stdout:9/609: read df/d12/d1b/f2e [1333753,91749] 0 2024-06-15T01:07:18.699 INFO:tasks.workunit.client.0.smithi137.stdout:1/893: dwrite d0/d1f/d60/f55 [0,4194304] 0 2024-06-15T01:07:18.779 INFO:tasks.workunit.client.0.smithi137.stdout:9/610: fdatasync df/d12/d15/d16/f27 0 2024-06-15T01:07:18.780 INFO:tasks.workunit.client.0.smithi137.stdout:1/894: readlink d0/d3/d14/d34/l84 0 2024-06-15T01:07:18.782 INFO:tasks.workunit.client.0.smithi137.stdout:1/895: chown d0/cd 3748 1 2024-06-15T01:07:18.783 INFO:tasks.workunit.client.0.smithi137.stdout:5/852: dread df/f4a [0,4194304] 0 2024-06-15T01:07:18.784 INFO:tasks.workunit.client.0.smithi137.stdout:1/896: chown d0/d3/deb/fec 20829726 1 2024-06-15T01:07:18.786 INFO:tasks.workunit.client.0.smithi137.stdout:9/611: write df/d49/d4b/f57 [1494459,17555] 0 2024-06-15T01:07:18.786 INFO:tasks.workunit.client.0.smithi137.stdout:8/694: dwrite d9/d2a/db9/dc6/f18 [0,4194304] 0 2024-06-15T01:07:18.788 INFO:tasks.workunit.client.0.smithi137.stdout:1/897: creat d0/d3/deb/f127 x:0 0 0 2024-06-15T01:07:18.789 INFO:tasks.workunit.client.0.smithi137.stdout:8/695: chown d9/d2a/db9/dc6/f94 211575740 1 2024-06-15T01:07:18.791 INFO:tasks.workunit.client.0.smithi137.stdout:9/612: creat df/d2a/fb4 x:0 0 0 2024-06-15T01:07:18.792 INFO:tasks.workunit.client.0.smithi137.stdout:5/853: fdatasync df/d44/d54/d95/d35/dea/dec/f11a 0 2024-06-15T01:07:18.792 INFO:tasks.workunit.client.0.smithi137.stdout:1/898: truncate d0/d10/fc1 1248705 0 2024-06-15T01:07:18.796 INFO:tasks.workunit.client.0.smithi137.stdout:1/899: write d0/d1f/d60/d99/de3/dea/d10f/d4c/f8e [5171086,84010] 0 2024-06-15T01:07:18.798 INFO:tasks.workunit.client.0.smithi137.stdout:5/854: write df/d11/d14/f15 [4162285,31399] 0 2024-06-15T01:07:18.799 INFO:tasks.workunit.client.0.smithi137.stdout:9/613: link df/d12/d1b/f2f df/d12/fb5 0 2024-06-15T01:07:18.803 INFO:tasks.workunit.client.0.smithi137.stdout:5/855: creat df/d44/d54/d95/d3b/f11f x:0 0 0 2024-06-15T01:07:18.805 INFO:tasks.workunit.client.0.smithi137.stdout:1/900: dwrite d0/d1f/d60/d99/de3/dea/d10f/f69 [0,4194304] 0 2024-06-15T01:07:18.808 INFO:tasks.workunit.client.0.smithi137.stdout:1/901: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d66/f128 x:0 0 0 2024-06-15T01:07:18.810 INFO:tasks.workunit.client.0.smithi137.stdout:5/856: rename df/d11/f4c to df/d44/d54/d95/d3b/d49/f120 0 2024-06-15T01:07:18.812 INFO:tasks.workunit.client.0.smithi137.stdout:1/902: creat d0/d1f/d60/d99/f129 x:0 0 0 2024-06-15T01:07:18.815 INFO:tasks.workunit.client.0.smithi137.stdout:1/903: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/f12a x:0 0 0 2024-06-15T01:07:18.817 INFO:tasks.workunit.client.0.smithi137.stdout:1/904: getdents d0/d10/d1d/de0 0 2024-06-15T01:07:18.818 INFO:tasks.workunit.client.0.smithi137.stdout:1/905: write d0/d3/f13 [738973,124772] 0 2024-06-15T01:07:18.819 INFO:tasks.workunit.client.0.smithi137.stdout:1/906: chown d0/d2d/d3e/l5b 265185393 1 2024-06-15T01:07:18.821 INFO:tasks.workunit.client.0.smithi137.stdout:1/907: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/f12b x:0 0 0 2024-06-15T01:07:18.823 INFO:tasks.workunit.client.0.smithi137.stdout:1/908: mknod d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/c12c 0 2024-06-15T01:07:18.825 INFO:tasks.workunit.client.0.smithi137.stdout:1/909: mkdir d0/d2d/d12d 0 2024-06-15T01:07:18.827 INFO:tasks.workunit.client.0.smithi137.stdout:1/910: mknod d0/d10/d41/c12e 0 2024-06-15T01:07:18.827 INFO:tasks.workunit.client.0.smithi137.stdout:1/911: readlink d0/d3/d14/l1a 0 2024-06-15T01:07:18.856 INFO:tasks.workunit.client.0.smithi137.stdout:1/912: dread d0/d3/d47/d73/f11e [0,4194304] 0 2024-06-15T01:07:18.857 INFO:tasks.workunit.client.0.smithi137.stdout:8/696: write d9/d22/f98 [295502,130405] 0 2024-06-15T01:07:18.859 INFO:tasks.workunit.client.0.smithi137.stdout:8/697: mkdir d9/d2a/df3 0 2024-06-15T01:07:18.869 INFO:tasks.workunit.client.0.smithi137.stdout:9/614: dwrite df/d12/d15/d16/f27 [0,4194304] 0 2024-06-15T01:07:19.016 INFO:tasks.workunit.client.0.smithi137.stdout:1/913: dwrite d0/d10/d41/d7b/ff5 [0,4194304] 0 2024-06-15T01:07:19.044 INFO:tasks.workunit.client.0.smithi137.stdout:5/857: chown df/d11/d14/cb7 68 1 2024-06-15T01:07:19.046 INFO:tasks.workunit.client.0.smithi137.stdout:9/615: mknod df/d49/d4b/d9b/cb6 0 2024-06-15T01:07:19.051 INFO:tasks.workunit.client.0.smithi137.stdout:9/616: symlink df/d12/d15/d16/d3c/d51/d8f/lb7 0 2024-06-15T01:07:19.054 INFO:tasks.workunit.client.0.smithi137.stdout:8/698: link d9/d2a/db9/dc6/d1e/d3f/d84/fc0 d9/d2a/db9/dc6/d1e/d3f/dab/ff4 0 2024-06-15T01:07:19.055 INFO:tasks.workunit.client.0.smithi137.stdout:5/858: mknod df/d11/d30/c121 0 2024-06-15T01:07:19.056 INFO:tasks.workunit.client.0.smithi137.stdout:8/699: fdatasync d9/d2a/f34 0 2024-06-15T01:07:19.058 INFO:tasks.workunit.client.0.smithi137.stdout:8/700: write d9/d2a/d2f/d59/ddb/fdd [698651,40744] 0 2024-06-15T01:07:19.059 INFO:tasks.workunit.client.0.smithi137.stdout:8/701: readlink d9/d2a/db9/dc6/d1e/l88 0 2024-06-15T01:07:19.060 INFO:tasks.workunit.client.0.smithi137.stdout:8/702: chown d9/d2a/d79/lb5 58 1 2024-06-15T01:07:19.061 INFO:tasks.workunit.client.0.smithi137.stdout:8/703: fsync d9/d2a/d2f/d59/f96 0 2024-06-15T01:07:19.061 INFO:tasks.workunit.client.0.smithi137.stdout:5/859: link df/d44/d54/d95/d3b/d49/d5c/de7/dbf/f11d df/d11/d14/d6c/d11b/f122 0 2024-06-15T01:07:19.063 INFO:tasks.workunit.client.0.smithi137.stdout:8/704: chown d9/d2a/d2f/c81 0 1 2024-06-15T01:07:19.064 INFO:tasks.workunit.client.0.smithi137.stdout:9/617: dread f2 [0,4194304] 0 2024-06-15T01:07:19.065 INFO:tasks.workunit.client.0.smithi137.stdout:9/618: chown df/d12/d1b/d31/d42/c9c 33179715 1 2024-06-15T01:07:19.066 INFO:tasks.workunit.client.0.smithi137.stdout:9/619: fsync df/d12/d1b/f74 0 2024-06-15T01:07:19.067 INFO:tasks.workunit.client.0.smithi137.stdout:8/705: rename d9/d2a/d2f/c6e to d9/d2a/db9/cf5 0 2024-06-15T01:07:19.074 INFO:tasks.workunit.client.0.smithi137.stdout:9/620: dread df/d12/d29/d4a/f5d [0,4194304] 0 2024-06-15T01:07:19.074 INFO:tasks.workunit.client.0.smithi137.stdout:9/621: dread - df/d12/d29/f61 zero size 2024-06-15T01:07:19.075 INFO:tasks.workunit.client.0.smithi137.stdout:8/706: dwrite d9/d2a/db9/dc6/d1e/d3f/f68 [0,4194304] 0 2024-06-15T01:07:19.077 INFO:tasks.workunit.client.0.smithi137.stdout:8/707: chown d9/d2a/db9/dc6/d1e/l88 30592 1 2024-06-15T01:07:19.081 INFO:tasks.workunit.client.0.smithi137.stdout:9/622: dwrite df/d2a/fb4 [0,4194304] 0 2024-06-15T01:07:19.082 INFO:tasks.workunit.client.0.smithi137.stdout:8/708: mknod d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/cf6 0 2024-06-15T01:07:19.084 INFO:tasks.workunit.client.0.smithi137.stdout:9/623: readlink df/d1a/l9f 0 2024-06-15T01:07:19.085 INFO:tasks.workunit.client.0.smithi137.stdout:1/914: truncate d0/d3/d47/d73/fc7 4158399 0 2024-06-15T01:07:19.091 INFO:tasks.workunit.client.0.smithi137.stdout:1/915: fsync d0/d10/d41/f103 0 2024-06-15T01:07:19.093 INFO:tasks.workunit.client.0.smithi137.stdout:8/709: rename d9/d22/fea to d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/ff7 0 2024-06-15T01:07:19.099 INFO:tasks.workunit.client.0.smithi137.stdout:9/624: link df/d1a/l9f df/d12/d15/d75/lb8 0 2024-06-15T01:07:19.101 INFO:tasks.workunit.client.0.smithi137.stdout:8/710: link d9/d55/ca9 d9/d2a/d79/d7e/cf8 0 2024-06-15T01:07:19.107 INFO:tasks.workunit.client.0.smithi137.stdout:8/711: creat d9/d2a/db9/dc6/d1e/d3f/ff9 x:0 0 0 2024-06-15T01:07:19.108 INFO:tasks.workunit.client.0.smithi137.stdout:1/916: dwrite d0/d3/d47/d73/f11e [0,4194304] 0 2024-06-15T01:07:19.108 INFO:tasks.workunit.client.0.smithi137.stdout:5/860: dwrite df/d11/f24 [4194304,4194304] 0 2024-06-15T01:07:19.123 INFO:tasks.workunit.client.0.smithi137.stdout:8/712: rename d9/d2a/db9/dc6/d1e/d35/f63 to d9/d2a/db9/dc6/d1e/d3f/ffa 0 2024-06-15T01:07:19.125 INFO:tasks.workunit.client.0.smithi137.stdout:5/861: mknod df/d11/c123 0 2024-06-15T01:07:19.127 INFO:tasks.workunit.client.0.smithi137.stdout:5/862: read - df/d44/d54/d95/d3b/d49/d5c/de7/fe3 zero size 2024-06-15T01:07:19.128 INFO:tasks.workunit.client.0.smithi137.stdout:8/713: creat d9/d2a/db9/dc6/ffb x:0 0 0 2024-06-15T01:07:19.130 INFO:tasks.workunit.client.0.smithi137.stdout:9/625: dread df/d12/d1b/f2e [0,4194304] 0 2024-06-15T01:07:19.133 INFO:tasks.workunit.client.0.smithi137.stdout:1/917: write d0/d1f/d76/da1/f10a [453525,104533] 0 2024-06-15T01:07:19.139 INFO:tasks.workunit.client.0.smithi137.stdout:5/863: link df/d44/d54/d95/d3b/d49/d5c/de7/c111 df/d44/d54/d95/d3b/d49/d5c/de7/c124 0 2024-06-15T01:07:19.141 INFO:tasks.workunit.client.0.smithi137.stdout:9/626: write df/d12/d4e/f5c [2313695,37514] 0 2024-06-15T01:07:19.144 INFO:tasks.workunit.client.0.smithi137.stdout:8/714: read d9/d2a/db9/dc6/d1e/d35/f56 [2022233,101546] 0 2024-06-15T01:07:19.145 INFO:tasks.workunit.client.0.smithi137.stdout:5/864: creat df/d44/d54/d95/d35/f125 x:0 0 0 2024-06-15T01:07:19.151 INFO:tasks.workunit.client.0.smithi137.stdout:1/918: dread d0/d1f/d60/da9/fac [0,4194304] 0 2024-06-15T01:07:19.153 INFO:tasks.workunit.client.0.smithi137.stdout:1/919: dread - d0/d1f/d67/f94 zero size 2024-06-15T01:07:19.153 INFO:tasks.workunit.client.0.smithi137.stdout:8/715: dwrite d9/d2a/db9/dc6/fac [0,4194304] 0 2024-06-15T01:07:19.159 INFO:tasks.workunit.client.0.smithi137.stdout:1/920: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/f12f x:0 0 0 2024-06-15T01:07:19.159 INFO:tasks.workunit.client.0.smithi137.stdout:5/865: link df/d44/d54/l8d df/d11/dfa/l126 0 2024-06-15T01:07:19.163 INFO:tasks.workunit.client.0.smithi137.stdout:5/866: dread - df/d44/d54/d95/d3b/d4b/fa0 zero size 2024-06-15T01:07:19.163 INFO:tasks.workunit.client.0.smithi137.stdout:9/627: dread df/d12/d1b/d31/d42/f46 [0,4194304] 0 2024-06-15T01:07:19.164 INFO:tasks.workunit.client.0.smithi137.stdout:8/716: dwrite d9/d2a/d2f/d59/fcf [0,4194304] 0 2024-06-15T01:07:19.165 INFO:tasks.workunit.client.0.smithi137.stdout:9/628: readlink df/d12/d1b/l55 0 2024-06-15T01:07:19.170 INFO:tasks.workunit.client.0.smithi137.stdout:9/629: chown df/d13/f71 816390 1 2024-06-15T01:07:19.171 INFO:tasks.workunit.client.0.smithi137.stdout:5/867: creat df/d76/d7c/f127 x:0 0 0 2024-06-15T01:07:19.172 INFO:tasks.workunit.client.0.smithi137.stdout:1/921: creat d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/d104/f130 x:0 0 0 2024-06-15T01:07:19.173 INFO:tasks.workunit.client.0.smithi137.stdout:9/630: symlink df/d12/d29/d79/d9a/lb9 0 2024-06-15T01:07:19.175 INFO:tasks.workunit.client.0.smithi137.stdout:5/868: creat df/d76/d7c/d9c/da1/f128 x:0 0 0 2024-06-15T01:07:19.175 INFO:tasks.workunit.client.0.smithi137.stdout:1/922: mkdir d0/d3/deb/d11b/d131 0 2024-06-15T01:07:19.176 INFO:tasks.workunit.client.0.smithi137.stdout:9/631: creat df/d12/d29/d4a/fba x:0 0 0 2024-06-15T01:07:19.177 INFO:tasks.workunit.client.0.smithi137.stdout:1/923: dread - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/f110 zero size 2024-06-15T01:07:19.179 INFO:tasks.workunit.client.0.smithi137.stdout:1/924: dread - d0/d10/d41/f103 zero size 2024-06-15T01:07:19.179 INFO:tasks.workunit.client.0.smithi137.stdout:9/632: symlink df/d12/d29/d66/lbb 0 2024-06-15T01:07:19.180 INFO:tasks.workunit.client.0.smithi137.stdout:8/717: write d9/d55/f5c [1935666,71337] 0 2024-06-15T01:07:19.181 INFO:tasks.workunit.client.0.smithi137.stdout:5/869: rename df/d11/d30/lf5 to df/d3d/l129 0 2024-06-15T01:07:19.183 INFO:tasks.workunit.client.0.smithi137.stdout:9/633: rename df/d12/d15/d16 to df/d12/d4e/dbc 0 2024-06-15T01:07:19.185 INFO:tasks.workunit.client.0.smithi137.stdout:9/634: chown df/d12/d29/d4a/f5d 24885 1 2024-06-15T01:07:19.186 INFO:tasks.workunit.client.0.smithi137.stdout:1/925: creat d0/d2d/d3e/dff/f132 x:0 0 0 2024-06-15T01:07:19.188 INFO:tasks.workunit.client.0.smithi137.stdout:9/635: symlink df/d12/d29/d4a/lbd 0 2024-06-15T01:07:19.190 INFO:tasks.workunit.client.0.smithi137.stdout:9/636: mkdir df/d2a/d7e/dbe 0 2024-06-15T01:07:19.192 INFO:tasks.workunit.client.0.smithi137.stdout:9/637: chown df/d12/d1b/d31/l91 260533 1 2024-06-15T01:07:19.195 INFO:tasks.workunit.client.0.smithi137.stdout:9/638: truncate df/d12/d1b/d31/d42/f46 1754283 0 2024-06-15T01:07:19.197 INFO:tasks.workunit.client.0.smithi137.stdout:5/870: write df/d11/d14/f32 [3269974,43658] 0 2024-06-15T01:07:19.200 INFO:tasks.workunit.client.0.smithi137.stdout:5/871: mkdir df/d44/d54/d95/d35/d12a 0 2024-06-15T01:07:19.204 INFO:tasks.workunit.client.0.smithi137.stdout:5/872: link df/d11/d14/c1f df/d44/d54/d95/d3b/d49/d5c/de7/c12b 0 2024-06-15T01:07:19.207 INFO:tasks.workunit.client.0.smithi137.stdout:5/873: symlink df/d44/d54/d95/d3b/df1/l12c 0 2024-06-15T01:07:19.207 INFO:tasks.workunit.client.0.smithi137.stdout:1/926: dread d0/d2d/f79 [0,4194304] 0 2024-06-15T01:07:19.211 INFO:tasks.workunit.client.0.smithi137.stdout:5/874: rename df/d44/d54/d80 to df/d76/d7c/d12d 0 2024-06-15T01:07:19.214 INFO:tasks.workunit.client.0.smithi137.stdout:5/875: dread df/d44/d54/d95/f6f [4194304,4194304] 0 2024-06-15T01:07:19.215 INFO:tasks.workunit.client.0.smithi137.stdout:5/876: write df/d11/d14/f43 [1708181,4211] 0 2024-06-15T01:07:19.216 INFO:tasks.workunit.client.0.smithi137.stdout:1/927: rename d0/d1f/c30 to d0/d10/d11a/c133 0 2024-06-15T01:07:19.218 INFO:tasks.workunit.client.0.smithi137.stdout:1/928: readlink d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/df6/d102/lf3 0 2024-06-15T01:07:19.220 INFO:tasks.workunit.client.0.smithi137.stdout:1/929: write d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/ffc [428005,31807] 0 2024-06-15T01:07:19.221 INFO:tasks.workunit.client.0.smithi137.stdout:5/877: mknod df/d11/dfa/c12e 0 2024-06-15T01:07:19.224 INFO:tasks.workunit.client.0.smithi137.stdout:5/878: dread df/d11/d14/d6c/f90 [0,4194304] 0 2024-06-15T01:07:19.225 INFO:tasks.workunit.client.0.smithi137.stdout:1/930: creat d0/d3/d47/d73/f134 x:0 0 0 2024-06-15T01:07:19.228 INFO:tasks.workunit.client.0.smithi137.stdout:5/879: symlink df/d76/d7c/d9c/da8/l12f 0 2024-06-15T01:07:19.233 INFO:tasks.workunit.client.0.smithi137.stdout:5/880: dread df/d44/d54/d95/d3b/d4b/fe1 [0,4194304] 0 2024-06-15T01:07:19.234 INFO:tasks.workunit.client.0.smithi137.stdout:1/931: getdents d0/d3/d14/da0 0 2024-06-15T01:07:19.238 INFO:tasks.workunit.client.0.smithi137.stdout:1/932: creat d0/d3/d47/d73/f135 x:0 0 0 2024-06-15T01:07:19.241 INFO:tasks.workunit.client.0.smithi137.stdout:1/933: write d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/f12f [801992,66601] 0 2024-06-15T01:07:19.244 INFO:tasks.workunit.client.0.smithi137.stdout:1/934: unlink d0/d1f/d7d/fae 0 2024-06-15T01:07:19.244 INFO:tasks.workunit.client.0.smithi137.stdout:5/881: dread df/d44/d54/d95/d3b/d4b/f63 [0,4194304] 0 2024-06-15T01:07:19.249 INFO:tasks.workunit.client.0.smithi137.stdout:5/882: write df/d44/d54/d95/d35/f81 [3248412,125509] 0 2024-06-15T01:07:19.257 INFO:tasks.workunit.client.0.smithi137.stdout:5/883: dwrite df/d44/d54/d95/d3b/f11f [0,4194304] 0 2024-06-15T01:07:19.260 INFO:tasks.workunit.client.0.smithi137.stdout:5/884: fsync df/fb6 0 2024-06-15T01:07:19.261 INFO:tasks.workunit.client.0.smithi137.stdout:5/885: readlink df/d44/d54/d95/d3b/d49/d5c/de7/lc3 0 2024-06-15T01:07:19.263 INFO:tasks.workunit.client.0.smithi137.stdout:5/886: creat df/d11/d14/d3c/f130 x:0 0 0 2024-06-15T01:07:19.267 INFO:tasks.workunit.client.0.smithi137.stdout:1/935: dread d0/d3/d47/f5d [0,4194304] 0 2024-06-15T01:07:19.275 INFO:tasks.workunit.client.0.smithi137.stdout:1/936: dread d0/d10/d11a/fde [0,4194304] 0 2024-06-15T01:07:19.278 INFO:tasks.workunit.client.0.smithi137.stdout:1/937: creat d0/d10/f136 x:0 0 0 2024-06-15T01:07:19.278 INFO:tasks.workunit.client.0.smithi137.stdout:5/887: dread f3 [0,4194304] 0 2024-06-15T01:07:19.281 INFO:tasks.workunit.client.0.smithi137.stdout:1/938: symlink d0/d10/d41/l137 0 2024-06-15T01:07:19.284 INFO:tasks.workunit.client.0.smithi137.stdout:1/939: chown d0/d3/l1c 168 1 2024-06-15T01:07:19.286 INFO:tasks.workunit.client.0.smithi137.stdout:1/940: fdatasync d0/d2d/f5e 0 2024-06-15T01:07:19.288 INFO:tasks.workunit.client.0.smithi137.stdout:1/941: chown d0/d3/d14/d34/f45 233583 1 2024-06-15T01:07:19.290 INFO:tasks.workunit.client.0.smithi137.stdout:1/942: dread - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/f12b zero size 2024-06-15T01:07:19.293 INFO:tasks.workunit.client.0.smithi137.stdout:5/888: dread df/d44/f4e [0,4194304] 0 2024-06-15T01:07:19.294 INFO:tasks.workunit.client.0.smithi137.stdout:5/889: rmdir df 39 2024-06-15T01:07:19.299 INFO:tasks.workunit.client.0.smithi137.stdout:1/943: dwrite d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/df6/d120/fe7 [4194304,4194304] 0 2024-06-15T01:07:19.304 INFO:tasks.workunit.client.0.smithi137.stdout:5/890: dwrite df/d11/d30/ff6 [0,4194304] 0 2024-06-15T01:07:19.306 INFO:tasks.workunit.client.0.smithi137.stdout:1/944: rename d0/d3/d89/c97 to d0/d1f/c138 0 2024-06-15T01:07:19.310 INFO:tasks.workunit.client.0.smithi137.stdout:1/945: mkdir d0/d139 0 2024-06-15T01:07:19.312 INFO:tasks.workunit.client.0.smithi137.stdout:5/891: write df/d44/d54/d95/d3b/d49/d5c/de7/dbf/f11d [229286,99185] 0 2024-06-15T01:07:19.313 INFO:tasks.workunit.client.0.smithi137.stdout:1/946: creat d0/d1f/d76/da1/f13a x:0 0 0 2024-06-15T01:07:19.315 INFO:tasks.workunit.client.0.smithi137.stdout:1/947: fsync d0/f3b 0 2024-06-15T01:07:19.320 INFO:tasks.workunit.client.0.smithi137.stdout:5/892: dwrite df/d44/d54/d95/d3b/d49/fca [0,4194304] 0 2024-06-15T01:07:19.321 INFO:tasks.workunit.client.0.smithi137.stdout:1/948: rename d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/c6c to d0/d139/c13b 0 2024-06-15T01:07:19.323 INFO:tasks.workunit.client.0.smithi137.stdout:5/893: mknod df/d44/d54/d95/d35/dea/dec/c131 0 2024-06-15T01:07:19.328 INFO:tasks.workunit.client.0.smithi137.stdout:5/894: dread df/f41 [0,4194304] 0 2024-06-15T01:07:19.330 INFO:tasks.workunit.client.0.smithi137.stdout:1/949: getdents d0/d139 0 2024-06-15T01:07:19.332 INFO:tasks.workunit.client.0.smithi137.stdout:5/895: write df/d11/d30/f82 [260151,94106] 0 2024-06-15T01:07:19.333 INFO:tasks.workunit.client.0.smithi137.stdout:5/896: chown df/d11/d30 23 1 2024-06-15T01:07:19.335 INFO:tasks.workunit.client.0.smithi137.stdout:5/897: dread - df/d44/d54/d95/d3b/d49/d5c/de7/fe3 zero size 2024-06-15T01:07:19.336 INFO:tasks.workunit.client.0.smithi137.stdout:1/950: rename d0/d3/d14/d34/df0/d121 to d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/d13c 0 2024-06-15T01:07:19.339 INFO:tasks.workunit.client.0.smithi137.stdout:1/951: mknod d0/d3/d14/d34/c13d 0 2024-06-15T01:07:19.341 INFO:tasks.workunit.client.0.smithi137.stdout:5/898: dwrite df/d44/d54/d95/d3b/d49/f96 [0,4194304] 0 2024-06-15T01:07:19.343 INFO:tasks.workunit.client.0.smithi137.stdout:1/952: rmdir d0/d3/deb/d11b/d131 0 2024-06-15T01:07:19.347 INFO:tasks.workunit.client.0.smithi137.stdout:1/953: mknod d0/d10/d41/d7b/c13e 0 2024-06-15T01:07:19.349 INFO:tasks.workunit.client.0.smithi137.stdout:1/954: mkdir d0/d1f/d60/d99/de3/dea/d13f 0 2024-06-15T01:07:19.359 INFO:tasks.workunit.client.0.smithi137.stdout:5/899: dwrite df/d11/d30/db1/fbb [0,4194304] 0 2024-06-15T01:07:19.364 INFO:tasks.workunit.client.0.smithi137.stdout:5/900: getdents df/d76/d7c/d9c/da1/d102 0 2024-06-15T01:07:19.365 INFO:tasks.workunit.client.0.smithi137.stdout:5/901: read - df/d76/d7c/d9c/da1/fc5 zero size 2024-06-15T01:07:19.371 INFO:tasks.workunit.client.0.smithi137.stdout:5/902: dwrite df/d44/d54/d95/d35/f125 [0,4194304] 0 2024-06-15T01:07:19.375 INFO:tasks.workunit.client.0.smithi137.stdout:5/903: unlink df/d44/d54/d95/d3b/d49/fca 0 2024-06-15T01:07:19.419 INFO:tasks.workunit.client.0.smithi137.stdout:2/732: sync 2024-06-15T01:07:19.421 INFO:tasks.workunit.client.0.smithi137.stdout:2/733: fdatasync d0/da/f11 0 2024-06-15T01:07:19.421 INFO:tasks.workunit.client.0.smithi137.stdout:5/904: fsync df/d44/d54/d95/d3b/f11f 0 2024-06-15T01:07:19.423 INFO:tasks.workunit.client.0.smithi137.stdout:5/905: mknod df/d76/c132 0 2024-06-15T01:07:19.424 INFO:tasks.workunit.client.0.smithi137.stdout:2/734: symlink d0/de/d2b/lde 0 2024-06-15T01:07:19.425 INFO:tasks.workunit.client.0.smithi137.stdout:2/735: fdatasync d0/dc/f9d 0 2024-06-15T01:07:19.428 INFO:tasks.workunit.client.0.smithi137.stdout:2/736: mknod d0/da/da0/d3a/dd4/d5c/cdf 0 2024-06-15T01:07:19.430 INFO:tasks.workunit.client.0.smithi137.stdout:5/906: write df/d44/d54/d95/d3b/d4b/fa0 [858125,90883] 0 2024-06-15T01:07:19.432 INFO:tasks.workunit.client.0.smithi137.stdout:5/907: rmdir df/d11 39 2024-06-15T01:07:19.433 INFO:tasks.workunit.client.0.smithi137.stdout:5/908: readlink df/d76/d7c/d9c/da8/l78 0 2024-06-15T01:07:19.434 INFO:tasks.workunit.client.0.smithi137.stdout:2/737: dwrite d0/da/da0/f2a [0,4194304] 0 2024-06-15T01:07:19.435 INFO:tasks.workunit.client.0.smithi137.stdout:2/738: fsync d0/da/d73/f9a 0 2024-06-15T01:07:19.438 INFO:tasks.workunit.client.0.smithi137.stdout:5/909: creat df/d11/d14/d6c/d11b/f133 x:0 0 0 2024-06-15T01:07:19.441 INFO:tasks.workunit.client.0.smithi137.stdout:2/739: truncate d0/da/f11 2518547 0 2024-06-15T01:07:19.442 INFO:tasks.workunit.client.0.smithi137.stdout:5/910: chown df/d44/d54/d95/d3b/d49/d5c/de7/c12b 183343 1 2024-06-15T01:07:19.444 INFO:tasks.workunit.client.0.smithi137.stdout:2/740: write d0/dc/d22/f4b [938407,89803] 0 2024-06-15T01:07:19.445 INFO:tasks.workunit.client.0.smithi137.stdout:2/741: write d0/dc/d2e/d72/d56/d6e/fc4 [273346,72844] 0 2024-06-15T01:07:19.448 INFO:tasks.workunit.client.0.smithi137.stdout:2/742: mknod d0/da/da0/d3a/dd4/d5c/ce0 0 2024-06-15T01:07:19.460 INFO:tasks.workunit.client.0.smithi137.stdout:5/911: dwrite df/d44/d54/d95/d3b/d49/d5c/de7/fab [0,4194304] 0 2024-06-15T01:07:19.469 INFO:tasks.workunit.client.0.smithi137.stdout:5/912: dwrite df/d44/f4e [0,4194304] 0 2024-06-15T01:07:19.477 INFO:tasks.workunit.client.0.smithi137.stdout:5/913: write df/d44/f84 [1394779,113844] 0 2024-06-15T01:07:19.478 INFO:tasks.workunit.client.0.smithi137.stdout:5/914: write df/d11/d14/d6c/f99 [2600054,84534] 0 2024-06-15T01:07:19.483 INFO:tasks.workunit.client.0.smithi137.stdout:5/915: chown df/d76/d7c/d12d/d105/f10b 3 1 2024-06-15T01:07:19.483 INFO:tasks.workunit.client.0.smithi137.stdout:5/916: chown df/d44/d54/d95/d35/c60 1987617 1 2024-06-15T01:07:19.483 INFO:tasks.workunit.client.0.smithi137.stdout:5/917: dread - df/d44/d54/fb8 zero size 2024-06-15T01:07:19.483 INFO:tasks.workunit.client.0.smithi137.stdout:5/918: dread - df/d76/d7c/d9c/da8/fad zero size 2024-06-15T01:07:19.484 INFO:tasks.workunit.client.0.smithi137.stdout:5/919: creat df/d11/d14/d6c/d11b/f134 x:0 0 0 2024-06-15T01:07:19.488 INFO:tasks.workunit.client.0.smithi137.stdout:5/920: dread df/d44/d54/d95/d3b/d49/d5c/de7/fab [0,4194304] 0 2024-06-15T01:07:19.489 INFO:tasks.workunit.client.0.smithi137.stdout:5/921: write df/d11/d30/db1/fbb [3352826,79593] 0 2024-06-15T01:07:19.490 INFO:tasks.workunit.client.0.smithi137.stdout:5/922: chown df/d11/d14/d3c/l88 1334 1 2024-06-15T01:07:19.493 INFO:tasks.workunit.client.0.smithi137.stdout:5/923: creat df/d76/d7c/d9c/da8/d101/f135 x:0 0 0 2024-06-15T01:07:19.494 INFO:tasks.workunit.client.0.smithi137.stdout:5/924: fsync df/f2b 0 2024-06-15T01:07:19.507 INFO:tasks.workunit.client.0.smithi137.stdout:5/925: dread f7 [0,4194304] 0 2024-06-15T01:07:19.508 INFO:tasks.workunit.client.0.smithi137.stdout:5/926: readlink df/d44/d54/laf 0 2024-06-15T01:07:19.515 INFO:tasks.workunit.client.0.smithi137.stdout:5/927: dwrite df/d11/d14/d3c/f3e [0,4194304] 0 2024-06-15T01:07:19.515 INFO:tasks.workunit.client.0.smithi137.stdout:5/928: chown df/d44/c75 2 1 2024-06-15T01:07:19.519 INFO:tasks.workunit.client.0.smithi137.stdout:5/929: write df/d11/d14/d6c/fdb [623743,53577] 0 2024-06-15T01:07:19.525 INFO:tasks.workunit.client.0.smithi137.stdout:5/930: dwrite df/d76/d7c/f127 [0,4194304] 0 2024-06-15T01:07:19.527 INFO:tasks.workunit.client.0.smithi137.stdout:5/931: write df/d11/d14/d6c/d11b/f122 [1330862,129420] 0 2024-06-15T01:07:19.531 INFO:tasks.workunit.client.0.smithi137.stdout:5/932: creat df/d44/d54/d95/d35/dea/dec/f136 x:0 0 0 2024-06-15T01:07:19.535 INFO:tasks.workunit.client.0.smithi137.stdout:5/933: dread df/d11/d30/ff6 [0,4194304] 0 2024-06-15T01:07:19.537 INFO:tasks.workunit.client.0.smithi137.stdout:5/934: creat df/d11/d14/d6c/f137 x:0 0 0 2024-06-15T01:07:19.538 INFO:tasks.workunit.client.0.smithi137.stdout:5/935: creat df/d11/d14/d6c/dc7/f138 x:0 0 0 2024-06-15T01:07:19.544 INFO:tasks.workunit.client.0.smithi137.stdout:5/936: dwrite df/d44/d54/d6b/fee [0,4194304] 0 2024-06-15T01:07:19.551 INFO:tasks.workunit.client.0.smithi137.stdout:5/937: write df/d11/fb9 [449948,84606] 0 2024-06-15T01:07:19.553 INFO:tasks.workunit.client.0.smithi137.stdout:5/938: chown df/d44/d54/d95/d3b/d49/f120 117033 1 2024-06-15T01:07:19.554 INFO:tasks.workunit.client.0.smithi137.stdout:5/939: rmdir df/d11/d30 39 2024-06-15T01:07:19.558 INFO:tasks.workunit.client.0.smithi137.stdout:5/940: write df/d11/d14/f16 [1122141,29450] 0 2024-06-15T01:07:19.559 INFO:tasks.workunit.client.0.smithi137.stdout:5/941: write df/d11/fb9 [1495360,46188] 0 2024-06-15T01:07:19.563 INFO:tasks.workunit.client.0.smithi137.stdout:5/942: rename df/d44/d54/d95/d3b/d49/f77 to df/d76/d7c/d9c/da1/f139 0 2024-06-15T01:07:19.563 INFO:tasks.workunit.client.0.smithi137.stdout:5/943: stat df/d76 0 2024-06-15T01:07:19.567 INFO:tasks.workunit.client.0.smithi137.stdout:5/944: write df/d44/d54/d95/d3b/d49/fd6 [943744,32692] 0 2024-06-15T01:07:19.568 INFO:tasks.workunit.client.0.smithi137.stdout:5/945: dread - df/d44/d54/d95/d3b/f113 zero size 2024-06-15T01:07:19.570 INFO:tasks.workunit.client.0.smithi137.stdout:5/946: unlink df/d44/d54/fd3 0 2024-06-15T01:07:19.571 INFO:tasks.workunit.client.0.smithi137.stdout:5/947: write df/d44/d54/d95/d3b/f113 [389979,47160] 0 2024-06-15T01:07:19.574 INFO:tasks.workunit.client.0.smithi137.stdout:5/948: getdents df/d44 0 2024-06-15T01:07:19.577 INFO:tasks.workunit.client.0.smithi137.stdout:5/949: link df/d44/d54/d95/d35/lbc df/d44/d54/d6b/l13a 0 2024-06-15T01:07:19.578 INFO:tasks.workunit.client.0.smithi137.stdout:5/950: creat df/d11/dfa/f13b x:0 0 0 2024-06-15T01:07:19.579 INFO:tasks.workunit.client.0.smithi137.stdout:5/951: chown df/d11/d14/l1c 0 1 2024-06-15T01:07:19.579 INFO:tasks.workunit.client.0.smithi137.stdout:5/952: rename df to df/d11/d14/d6c/d13c 22 2024-06-15T01:07:19.580 INFO:tasks.workunit.client.0.smithi137.stdout:5/953: chown df/d76/d7c/d9c/da8 3836 1 2024-06-15T01:07:19.581 INFO:tasks.workunit.client.0.smithi137.stdout:5/954: write df/d11/d14/d6c/f137 [178085,93017] 0 2024-06-15T01:07:19.582 INFO:tasks.workunit.client.0.smithi137.stdout:5/955: write df/d44/d54/d95/d3b/f113 [1287767,64809] 0 2024-06-15T01:07:19.584 INFO:tasks.workunit.client.0.smithi137.stdout:5/956: symlink df/d11/d14/l13d 0 2024-06-15T01:07:19.605 INFO:tasks.workunit.client.0.smithi137.stdout:5/957: dread df/d11/d30/f7b [0,4194304] 0 2024-06-15T01:07:19.607 INFO:tasks.workunit.client.0.smithi137.stdout:5/958: creat df/d44/d54/d95/d35/d12a/f13e x:0 0 0 2024-06-15T01:07:19.608 INFO:tasks.workunit.client.0.smithi137.stdout:5/959: rmdir df/d44/d54/d95 39 2024-06-15T01:07:19.609 INFO:tasks.workunit.client.0.smithi137.stdout:5/960: symlink df/d3d/l13f 0 2024-06-15T01:07:19.610 INFO:tasks.workunit.client.0.smithi137.stdout:5/961: readlink df/d11/d14/lcf 0 2024-06-15T01:07:19.610 INFO:tasks.workunit.client.0.smithi137.stdout:5/962: chown df/d11/d14/l13d 2676 1 2024-06-15T01:07:19.612 INFO:tasks.workunit.client.0.smithi137.stdout:5/963: mknod df/d76/d7c/d9c/da1/c140 0 2024-06-15T01:07:19.613 INFO:tasks.workunit.client.0.smithi137.stdout:5/964: dread df/d11/d14/d6c/f90 [0,4194304] 0 2024-06-15T01:07:19.619 INFO:tasks.workunit.client.0.smithi137.stdout:5/965: dwrite df/d11/d14/d6c/d11b/f122 [0,4194304] 0 2024-06-15T01:07:19.623 INFO:tasks.workunit.client.0.smithi137.stdout:5/966: creat df/d11/f141 x:0 0 0 2024-06-15T01:07:19.632 INFO:tasks.workunit.client.0.smithi137.stdout:5/967: truncate df/d44/d54/d95/d3b/d49/f120 2804593 0 2024-06-15T01:07:19.637 INFO:tasks.workunit.client.0.smithi137.stdout:5/968: dwrite f7 [0,4194304] 0 2024-06-15T01:07:19.639 INFO:tasks.workunit.client.0.smithi137.stdout:5/969: write df/d11/dfa/f13b [557938,72284] 0 2024-06-15T01:07:19.642 INFO:tasks.workunit.client.0.smithi137.stdout:5/970: mknod df/d11/d30/c142 0 2024-06-15T01:07:19.644 INFO:tasks.workunit.client.0.smithi137.stdout:5/971: mkdir df/d76/d7c/d9c/da1/d102/d143 0 2024-06-15T01:07:19.740 INFO:tasks.workunit.client.0.smithi137.stdout:5/972: getdents df/d11/d14/d6c/d11b 0 2024-06-15T01:07:19.741 INFO:tasks.workunit.client.0.smithi137.stdout:5/973: write df/d76/d7c/f127 [4837621,35725] 0 2024-06-15T01:07:19.742 INFO:tasks.workunit.client.0.smithi137.stdout:5/974: chown df/d11/d30/ff6 1673767602 1 2024-06-15T01:07:19.744 INFO:tasks.workunit.client.0.smithi137.stdout:5/975: read df/d11/d14/d6c/dc7/fd0 [3721006,49923] 0 2024-06-15T01:07:19.746 INFO:tasks.workunit.client.0.smithi137.stdout:5/976: stat df/d44/d54/l8d 0 2024-06-15T01:07:19.747 INFO:tasks.workunit.client.0.smithi137.stdout:5/977: chown df/d76/d7c/d9c/da8/l78 17891 1 2024-06-15T01:07:19.748 INFO:tasks.workunit.client.0.smithi137.stdout:5/978: chown df/d44/d54/d95/d3b/d49/d5c/de7/dbf/f11d 19 1 2024-06-15T01:07:19.749 INFO:tasks.workunit.client.0.smithi137.stdout:5/979: creat df/d11/d30/db1/f144 x:0 0 0 2024-06-15T01:07:19.751 INFO:tasks.workunit.client.0.smithi137.stdout:5/980: creat df/d76/d7c/f145 x:0 0 0 2024-06-15T01:07:19.752 INFO:tasks.workunit.client.0.smithi137.stdout:5/981: mknod df/d76/d7c/d9c/da8/c146 0 2024-06-15T01:07:19.753 INFO:tasks.workunit.client.0.smithi137.stdout:5/982: chown df/d11/d14/d6c/dc7 3935 1 2024-06-15T01:07:19.758 INFO:tasks.workunit.client.0.smithi137.stdout:5/983: dwrite df/d11/d30/db1/fd7 [0,4194304] 0 2024-06-15T01:07:19.763 INFO:tasks.workunit.client.0.smithi137.stdout:5/984: rename df/fb6 to df/d11/f147 0 2024-06-15T01:07:19.765 INFO:tasks.workunit.client.0.smithi137.stdout:5/985: truncate df/d11/d30/f87 452176 0 2024-06-15T01:07:19.765 INFO:tasks.workunit.client.0.smithi137.stdout:5/986: fsync df/d11/d30/f118 0 2024-06-15T01:07:19.770 INFO:tasks.workunit.client.0.smithi137.stdout:5/987: rename df/d11/d14/d3c to df/d11/d30/db1/d148 0 2024-06-15T01:07:19.771 INFO:tasks.workunit.client.0.smithi137.stdout:5/988: fsync df/d76/d7c/f145 0 2024-06-15T01:07:19.775 INFO:tasks.workunit.client.0.smithi137.stdout:5/989: dread df/d11/d30/db1/d148/f3e [0,4194304] 0 2024-06-15T01:07:19.782 INFO:tasks.workunit.client.0.smithi137.stdout:5/990: truncate df/d11/d14/d6c/d11b/f122 2192182 0 2024-06-15T01:07:19.783 INFO:tasks.workunit.client.0.smithi137.stdout:5/991: rmdir df/d11/dfa 39 2024-06-15T01:07:19.785 INFO:tasks.workunit.client.0.smithi137.stdout:5/992: write df/d44/d54/d6b/f8c [814986,98079] 0 2024-06-15T01:07:19.787 INFO:tasks.workunit.client.0.smithi137.stdout:5/993: readlink df/d11/dfa/l126 0 2024-06-15T01:07:19.790 INFO:tasks.workunit.client.0.smithi137.stdout:5/994: symlink df/d44/d54/d95/d35/l149 0 2024-06-15T01:07:19.791 INFO:tasks.workunit.client.0.smithi137.stdout:5/995: rmdir df/d11/dfa 39 2024-06-15T01:07:19.792 INFO:tasks.workunit.client.0.smithi137.stdout:5/996: dread - df/d11/d30/db1/d148/f130 zero size 2024-06-15T01:07:19.793 INFO:tasks.workunit.client.0.smithi137.stdout:5/997: fdatasync df/d44/d54/d95/d3b/d4b/fe1 0 2024-06-15T01:07:19.795 INFO:tasks.workunit.client.0.smithi137.stdout:5/998: fdatasync df/d44/d54/d95/d35/fbe 0 2024-06-15T01:07:19.804 INFO:tasks.workunit.client.0.smithi137.stdout:5/999: dwrite df/d44/d54/d95/d3b/d49/d5c/de7/dbf/f11d [0,4194304] 0 2024-06-15T01:07:19.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:19 smithi142 ceph-mon[37379]: pgmap v25: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 102 MiB/s rd, 223 MiB/s wr, 507 op/s 2024-06-15T01:07:19.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:19 smithi137 ceph-mon[29354]: pgmap v25: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 102 MiB/s rd, 223 MiB/s wr, 507 op/s 2024-06-15T01:07:19.990 INFO:tasks.workunit.client.0.smithi137.stdout:0/957: sync 2024-06-15T01:07:19.994 INFO:tasks.workunit.client.0.smithi137.stdout:0/958: fdatasync d1/d2/dc/d10/d53/da2/f114 0 2024-06-15T01:07:20.002 INFO:tasks.workunit.client.0.smithi137.stdout:0/959: dread d1/d2/dc/d10/d53/da2/fa5 [0,4194304] 0 2024-06-15T01:07:20.007 INFO:tasks.workunit.client.0.smithi137.stdout:0/960: creat d1/d7e/d87/da0/dc4/f140 x:0 0 0 2024-06-15T01:07:20.016 INFO:tasks.workunit.client.0.smithi137.stdout:0/961: link d1/d2/dc/d10/d53/da2/l134 d1/d2/dc/d10/d14/d56/l141 0 2024-06-15T01:07:20.018 INFO:tasks.workunit.client.0.smithi137.stdout:0/962: symlink d1/d2/l142 0 2024-06-15T01:07:20.020 INFO:tasks.workunit.client.0.smithi137.stdout:0/963: read d1/d2/d13/d66/f86 [285261,5380] 0 2024-06-15T01:07:20.026 INFO:tasks.workunit.client.0.smithi137.stdout:0/964: write d1/d2/dc/fbf [346264,78740] 0 2024-06-15T01:07:20.034 INFO:tasks.workunit.client.0.smithi137.stdout:0/965: dwrite d1/d2/dc/d10/d53/da2/fa5 [0,4194304] 0 2024-06-15T01:07:20.038 INFO:tasks.workunit.client.0.smithi137.stdout:0/966: creat d1/d7e/d87/da0/dc4/dd7/f143 x:0 0 0 2024-06-15T01:07:20.045 INFO:tasks.workunit.client.0.smithi137.stdout:0/967: truncate d1/d2/dc/d10/d14/d56/f85 3552309 0 2024-06-15T01:07:20.047 INFO:tasks.workunit.client.0.smithi137.stdout:0/968: mknod d1/d2/dc/d10/d14/d56/d117/c144 0 2024-06-15T01:07:20.051 INFO:tasks.workunit.client.0.smithi137.stdout:0/969: dread d1/d2/dc/d10/d53/da2/fa5 [0,4194304] 0 2024-06-15T01:07:20.057 INFO:tasks.workunit.client.0.smithi137.stdout:0/970: rename d1/d2/dc/d10/d14/d56/d117/d1c to d1/d2/dc/d10/d14/d56/d117/d17/d54/d145 0 2024-06-15T01:07:20.063 INFO:tasks.workunit.client.0.smithi137.stdout:0/971: write d1/d2/dc/d10/d14/d56/f85 [3996869,8622] 0 2024-06-15T01:07:20.066 INFO:tasks.workunit.client.0.smithi137.stdout:0/972: mknod d1/d2/dc/d10/d22/db7/c146 0 2024-06-15T01:07:20.067 INFO:tasks.workunit.client.0.smithi137.stdout:0/973: readlink d1/d2/d55/l81 0 2024-06-15T01:07:20.067 INFO:tasks.workunit.client.0.smithi137.stdout:4/816: sync 2024-06-15T01:07:20.068 INFO:tasks.workunit.client.0.smithi137.stdout:7/638: sync 2024-06-15T01:07:20.068 INFO:tasks.workunit.client.0.smithi137.stdout:0/974: mknod d1/c147 0 2024-06-15T01:07:20.071 INFO:tasks.workunit.client.0.smithi137.stdout:4/817: read dd/d15/d36/d45/dcb/d83/db2/f80 [17196,120964] 0 2024-06-15T01:07:20.072 INFO:tasks.workunit.client.0.smithi137.stdout:7/639: rmdir df/d13/d2e/dce 0 2024-06-15T01:07:20.073 INFO:tasks.workunit.client.0.smithi137.stdout:4/818: creat dd/d15/f12a x:0 0 0 2024-06-15T01:07:20.075 INFO:tasks.workunit.client.0.smithi137.stdout:4/819: write dd/d15/f12a [430913,122303] 0 2024-06-15T01:07:20.076 INFO:tasks.workunit.client.0.smithi137.stdout:7/640: creat df/d7d/d3b/d5b/da6/fd2 x:0 0 0 2024-06-15T01:07:20.076 INFO:tasks.workunit.client.0.smithi137.stdout:0/975: dwrite d1/d7e/d87/da0/dc4/f140 [0,4194304] 0 2024-06-15T01:07:20.078 INFO:tasks.workunit.client.0.smithi137.stdout:0/976: fdatasync d1/d7e/d87/da0/dc4/f12a 0 2024-06-15T01:07:20.084 INFO:tasks.workunit.client.0.smithi137.stdout:4/820: symlink dd/d15/d36/d45/dcb/d83/db2/d72/l12b 0 2024-06-15T01:07:20.088 INFO:tasks.workunit.client.0.smithi137.stdout:4/821: creat dd/d15/f12c x:0 0 0 2024-06-15T01:07:20.094 INFO:tasks.workunit.client.0.smithi137.stdout:4/822: symlink dd/d15/d21/d3a/df6/l12d 0 2024-06-15T01:07:20.094 INFO:tasks.workunit.client.0.smithi137.stdout:7/641: dwrite df/d7d/d1c/f1f [0,4194304] 0 2024-06-15T01:07:20.095 INFO:tasks.workunit.client.0.smithi137.stdout:4/823: unlink dd/d15/l43 0 2024-06-15T01:07:20.097 INFO:tasks.workunit.client.0.smithi137.stdout:7/642: mknod df/d13/d2e/d1a/d6a/cd3 0 2024-06-15T01:07:20.098 INFO:tasks.workunit.client.0.smithi137.stdout:4/824: chown dd/d15/d36/d45/l113 532099 1 2024-06-15T01:07:20.100 INFO:tasks.workunit.client.0.smithi137.stdout:7/643: write df/d7d/d1c/d1d/d27/d33/f52 [3877260,17748] 0 2024-06-15T01:07:20.100 INFO:tasks.workunit.client.0.smithi137.stdout:4/825: symlink dd/d15/l12e 0 2024-06-15T01:07:20.102 INFO:tasks.workunit.client.0.smithi137.stdout:7/644: dread - df/d13/d2e/d1a/fa4 zero size 2024-06-15T01:07:20.102 INFO:tasks.workunit.client.0.smithi137.stdout:0/977: dread d1/d7e/d87/da0/dc4/dd7/de2/d8d/ffc [0,4194304] 0 2024-06-15T01:07:20.104 INFO:tasks.workunit.client.0.smithi137.stdout:4/826: dread - dd/d15/d36/d45/dcb/d5b/fc3 zero size 2024-06-15T01:07:20.104 INFO:tasks.workunit.client.0.smithi137.stdout:7/645: write df/d7d/d1c/d1d/d27/d33/fa5 [344133,62152] 0 2024-06-15T01:07:20.106 INFO:tasks.workunit.client.0.smithi137.stdout:4/827: write dd/d15/d30/f46 [5183402,43691] 0 2024-06-15T01:07:20.110 INFO:tasks.workunit.client.0.smithi137.stdout:7/646: unlink df/d7d/d1c/d1d/d27/c2a 0 2024-06-15T01:07:20.115 INFO:tasks.workunit.client.0.smithi137.stdout:4/828: creat dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/f12f x:0 0 0 2024-06-15T01:07:20.117 INFO:tasks.workunit.client.0.smithi137.stdout:4/829: mkdir dd/dbf/d130 0 2024-06-15T01:07:20.119 INFO:tasks.workunit.client.0.smithi137.stdout:4/830: chown dd/d15/d36/d45/d8f/dac/de5/f122 29307986 1 2024-06-15T01:07:20.120 INFO:tasks.workunit.client.0.smithi137.stdout:7/647: link df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fcd df/d13/d2e/d1a/d8c/fd4 0 2024-06-15T01:07:20.121 INFO:tasks.workunit.client.0.smithi137.stdout:0/978: truncate d1/d7e/d87/da0/fcf 2464077 0 2024-06-15T01:07:20.123 INFO:tasks.workunit.client.0.smithi137.stdout:4/831: creat dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/f131 x:0 0 0 2024-06-15T01:07:20.132 INFO:tasks.workunit.client.0.smithi137.stdout:7/648: getdents df/d7d/d1c/d1d/d27/d33/d43/dc5 0 2024-06-15T01:07:20.134 INFO:tasks.workunit.client.0.smithi137.stdout:0/979: write d1/d2/dc/d10/d22/db7/de3/f111 [392891,63760] 0 2024-06-15T01:07:20.138 INFO:tasks.workunit.client.0.smithi137.stdout:7/649: creat df/d7d/d1c/d1d/fd5 x:0 0 0 2024-06-15T01:07:20.140 INFO:tasks.workunit.client.0.smithi137.stdout:4/832: dwrite dd/d15/d21/d9c/d75/d99/fbd [0,4194304] 0 2024-06-15T01:07:20.143 INFO:tasks.workunit.client.0.smithi137.stdout:2/743: sync 2024-06-15T01:07:20.143 INFO:tasks.workunit.client.0.smithi137.stdout:9/639: sync 2024-06-15T01:07:20.143 INFO:tasks.workunit.client.0.smithi137.stdout:8/718: sync 2024-06-15T01:07:20.143 INFO:tasks.workunit.client.0.smithi137.stdout:6/651: sync 2024-06-15T01:07:20.143 INFO:tasks.workunit.client.0.smithi137.stdout:1/955: sync 2024-06-15T01:07:20.144 INFO:tasks.workunit.client.0.smithi137.stdout:0/980: truncate d1/d2/dc/d10/d14/d56/d117/d17/d54/d145/f1d 6024279 0 2024-06-15T01:07:20.146 INFO:tasks.workunit.client.0.smithi137.stdout:6/652: chown d0/d5/d9/d3f 4095996 1 2024-06-15T01:07:20.147 INFO:tasks.workunit.client.0.smithi137.stdout:7/650: creat df/d13/d2e/d1a/d6a/fd6 x:0 0 0 2024-06-15T01:07:20.148 INFO:tasks.workunit.client.0.smithi137.stdout:4/833: stat dd/d15/d36/d45/f4e 0 2024-06-15T01:07:20.153 INFO:tasks.workunit.client.0.smithi137.stdout:2/744: symlink d0/dc/d22/le1 0 2024-06-15T01:07:20.154 INFO:tasks.workunit.client.0.smithi137.stdout:9/640: dread df/d12/d4e/dbc/f27 [0,4194304] 0 2024-06-15T01:07:20.156 INFO:tasks.workunit.client.0.smithi137.stdout:9/641: chown df/d12/d1b/d31/d42 15 1 2024-06-15T01:07:20.159 INFO:tasks.workunit.client.0.smithi137.stdout:6/653: creat d0/d1a/d3e/d57/dba/fca x:0 0 0 2024-06-15T01:07:20.160 INFO:tasks.workunit.client.0.smithi137.stdout:9/642: truncate df/d12/d29/d4a/fba 1021617 0 2024-06-15T01:07:20.163 INFO:tasks.workunit.client.0.smithi137.stdout:2/745: mknod d0/dc/ce2 0 2024-06-15T01:07:20.166 INFO:tasks.workunit.client.0.smithi137.stdout:0/981: creat d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/d137/f148 x:0 0 0 2024-06-15T01:07:20.166 INFO:tasks.workunit.client.0.smithi137.stdout:4/834: mkdir dd/d15/d36/d45/d55/d8d/d1f/d86/d132 0 2024-06-15T01:07:20.168 INFO:tasks.workunit.client.0.smithi137.stdout:9/643: mknod df/d2a/d7e/cbf 0 2024-06-15T01:07:20.169 INFO:tasks.workunit.client.0.smithi137.stdout:2/746: stat d0/f8 0 2024-06-15T01:07:20.171 INFO:tasks.workunit.client.0.smithi137.stdout:8/719: getdents d9/d2a/db9/dc6/d1e/d3f 0 2024-06-15T01:07:20.173 INFO:tasks.workunit.client.0.smithi137.stdout:1/956: dread d0/d1f/d60/d99/de3/dea/d10f/d4c/f95 [0,4194304] 0 2024-06-15T01:07:20.175 INFO:tasks.workunit.client.0.smithi137.stdout:7/651: link df/d7d/d1c/d1d/d27/f49 df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fd7 0 2024-06-15T01:07:20.176 INFO:tasks.workunit.client.0.smithi137.stdout:0/982: truncate d1/d2/d13/ffe 442096 0 2024-06-15T01:07:20.177 INFO:tasks.workunit.client.0.smithi137.stdout:9/644: symlink df/d12/d29/d79/lc0 0 2024-06-15T01:07:20.182 INFO:tasks.workunit.client.0.smithi137.stdout:1/957: creat d0/d3/deb/d11b/f140 x:0 0 0 2024-06-15T01:07:20.183 INFO:tasks.workunit.client.0.smithi137.stdout:2/747: dwrite d0/d42/d49/d57/f9e [0,4194304] 0 2024-06-15T01:07:20.186 INFO:tasks.workunit.client.0.smithi137.stdout:6/654: link d0/d5/l50 d0/d1a/d3e/d57/d32/d92/dac/lcb 0 2024-06-15T01:07:20.189 INFO:tasks.workunit.client.0.smithi137.stdout:2/748: write d0/da/da0/d3a/dd4/d5c/d79/fb6 [498992,128366] 0 2024-06-15T01:07:20.194 INFO:tasks.workunit.client.0.smithi137.stdout:9/645: mknod df/d2a/d7e/dbe/cc1 0 2024-06-15T01:07:20.196 INFO:tasks.workunit.client.0.smithi137.stdout:2/749: dread d0/da/d73/fb3 [0,4194304] 0 2024-06-15T01:07:20.198 INFO:tasks.workunit.client.0.smithi137.stdout:6/655: rmdir d0/d5 39 2024-06-15T01:07:20.199 INFO:tasks.workunit.client.0.smithi137.stdout:4/835: dwrite dd/d15/d30/fa7 [0,4194304] 0 2024-06-15T01:07:20.201 INFO:tasks.workunit.client.0.smithi137.stdout:9/646: unlink fd 0 2024-06-15T01:07:20.201 INFO:tasks.workunit.client.0.smithi137.stdout:2/750: write d0/d42/d49/fb0 [956189,11999] 0 2024-06-15T01:07:20.203 INFO:tasks.workunit.client.0.smithi137.stdout:9/647: chown df/d12/d1b/d31 26 1 2024-06-15T01:07:20.204 INFO:tasks.workunit.client.0.smithi137.stdout:1/958: rename d0/d2d/d3e/f96 to d0/d10/d41/d7b/f141 0 2024-06-15T01:07:20.205 INFO:tasks.workunit.client.0.smithi137.stdout:9/648: write df/d12/d4e/dbc/f92 [3081088,20005] 0 2024-06-15T01:07:20.209 INFO:tasks.workunit.client.0.smithi137.stdout:1/959: dread - d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d93/df1/f12b zero size 2024-06-15T01:07:20.213 INFO:tasks.workunit.client.0.smithi137.stdout:2/751: dread d0/da/da0/d3a/dd4/d5c/fa3 [0,4194304] 0 2024-06-15T01:07:20.217 INFO:tasks.workunit.client.0.smithi137.stdout:7/652: write df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fc1 [318271,82536] 0 2024-06-15T01:07:20.217 INFO:tasks.workunit.client.0.smithi137.stdout:8/720: truncate d9/d2a/db9/dc6/f13 354253 0 2024-06-15T01:07:20.218 INFO:tasks.workunit.client.0.smithi137.stdout:8/721: truncate d9/d2a/d2f/d59/fbc 974875 0 2024-06-15T01:07:20.219 INFO:tasks.workunit.client.0.smithi137.stdout:0/983: link d1/d2/dc/d10/d14/d56/d117/d17/d54/d7a/dd0/ld8 d1/d2/dc/d10/d14/d56/d117/d17/d54/d145/l149 0 2024-06-15T01:07:20.222 INFO:tasks.workunit.client.0.smithi137.stdout:1/960: chown d0/lcd 15954 1 2024-06-15T01:07:20.222 INFO:tasks.workunit.client.0.smithi137.stdout:4/836: dread dd/d15/d21/d9c/d75/fc2 [0,4194304] 0 2024-06-15T01:07:20.233 INFO:tasks.workunit.client.0.smithi137.stdout:7/653: write df/d7d/d3b/d5b/f5d [1532855,83956] 0 2024-06-15T01:07:20.303 INFO:tasks.workunit.client.0.smithi137.stdout:2/752: unlink d0/da/da0/c13 0 2024-06-15T01:07:20.303 INFO:tasks.workunit.client.0.smithi137.stdout:7/654: write df/d13/d2e/d1a/d3f/d89/fc8 [328373,105550] 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:6/656: getdents d0/d1a/d3e/d57/d32/d92/dac 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:1/961: symlink d0/d3/deb/l142 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:4/837: symlink dd/d15/d30/l133 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:1/962: write d0/d3/deb/f127 [852696,9670] 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:9/649: getdents df/d49/d4b/d9b 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:8/722: dwrite d9/d2a/db9/dc6/d1e/d3f/ffa [0,4194304] 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:0/984: dwrite d1/d2/dc/ddc/fee [4194304,4194304] 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:8/723: truncate d9/d2a/db9/dc6/d1e/d35/dc8/fd2 473141 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:6/657: creat d0/d1a/d3e/d57/d1d/d2e/d70/fcc x:0 0 0 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:0/985: rmdir d1/d2/dc/d10/d14/d56/d117/d17/d54 39 2024-06-15T01:07:20.304 INFO:tasks.workunit.client.0.smithi137.stdout:9/650: write df/d12/d15/fa9 [144279,65621] 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:2/753: mknod d0/dc/d2e/d72/d8a/ce3 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:8/724: creat d9/d2a/db9/dc6/d1e/d3f/d84/ffc x:0 0 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:7/655: dread df/d13/d2e/d1a/f40 [0,4194304] 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:4/838: rename dd/d15/d36/d45/d55/f62 to dd/d15/d36/d45/d55/f134 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:1/963: creat d0/d1f/d60/f143 x:0 0 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:6/658: truncate d0/d1a/d3e/d57/d32/d74/f88 789691 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:0/986: chown d1/d2/dc/d10/d14/d56/d117/f50 535763092 1 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:9/651: rmdir df/d13/d6f 39 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:0/987: chown d1/d2/dc/d10/d22/l7c 35780851 1 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:8/725: dwrite d9/d2a/d2f/d59/fcf [4194304,4194304] 0 2024-06-15T01:07:20.305 INFO:tasks.workunit.client.0.smithi137.stdout:9/652: read df/d1a/f87 [313781,66743] 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:7/656: rename df/d13/d2e/d1a/d3f/f54 to df/d7d/d1c/d1d/d79/fd8 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:8/726: mkdir d9/d2a/db9/dc6/d1e/d35/dfd 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:9/653: creat df/d49/d72/fc2 x:0 0 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:2/754: creat d0/da/da0/fe4 x:0 0 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:0/988: symlink d1/d7e/d87/da0/l14a 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:1/964: dwrite d0/d3/f22 [0,4194304] 0 2024-06-15T01:07:20.306 INFO:tasks.workunit.client.0.smithi137.stdout:0/989: read d1/d2/f47 [1320677,8186] 0 2024-06-15T01:07:20.307 INFO:tasks.workunit.client.0.smithi137.stdout:6/659: creat d0/d5/d9/d29/fcd x:0 0 0 2024-06-15T01:07:20.308 INFO:tasks.workunit.client.0.smithi137.stdout:4/839: getdents dd/d15/d21/d3a/d6e 0 2024-06-15T01:07:20.310 INFO:tasks.workunit.client.0.smithi137.stdout:9/654: dwrite df/d12/d4e/f5c [0,4194304] 0 2024-06-15T01:07:20.313 INFO:tasks.workunit.client.0.smithi137.stdout:7/657: dread df/d7d/d1c/d1d/d27/d33/f9b [0,4194304] 0 2024-06-15T01:07:20.314 INFO:tasks.workunit.client.0.smithi137.stdout:7/658: truncate df/fb4 1132458 0 2024-06-15T01:07:20.319 INFO:tasks.workunit.client.0.smithi137.stdout:2/755: read d0/dc/d2e/d72/f59 [1748264,46969] 0 2024-06-15T01:07:20.319 INFO:tasks.workunit.client.0.smithi137.stdout:7/659: fsync df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fb8 0 2024-06-15T01:07:20.320 INFO:tasks.workunit.client.0.smithi137.stdout:0/990: mknod d1/d2/dc/d10/d14/d56/d117/d17/d54/c14b 0 2024-06-15T01:07:20.320 INFO:tasks.workunit.client.0.smithi137.stdout:8/727: write d9/d2a/db9/dc6/d1e/f44 [827576,125967] 0 2024-06-15T01:07:20.320 INFO:tasks.workunit.client.0.smithi137.stdout:9/655: truncate df/d12/d29/f44 3579611 0 2024-06-15T01:07:20.320 INFO:tasks.workunit.client.0.smithi137.stdout:0/991: stat d1/d2/dc/d6c/f10a 0 2024-06-15T01:07:20.321 INFO:tasks.workunit.client.0.smithi137.stdout:1/965: getdents d0/d10/d41 0 2024-06-15T01:07:20.324 INFO:tasks.workunit.client.0.smithi137.stdout:4/840: write dd/d15/d21/d3a/d6e/d7b/faf [146407,5532] 0 2024-06-15T01:07:20.327 INFO:tasks.workunit.client.0.smithi137.stdout:8/728: creat d9/d22/ffe x:0 0 0 2024-06-15T01:07:20.334 INFO:tasks.workunit.client.0.smithi137.stdout:1/966: creat d0/d3/d14/da0/f144 x:0 0 0 2024-06-15T01:07:20.334 INFO:tasks.workunit.client.0.smithi137.stdout:4/841: dread dd/d15/d21/d3a/d6e/d7b/f114 [0,4194304] 0 2024-06-15T01:07:20.335 INFO:tasks.workunit.client.0.smithi137.stdout:8/729: mknod d9/d2a/db9/dc6/d1e/d35/dc8/cff 0 2024-06-15T01:07:20.336 INFO:tasks.workunit.client.0.smithi137.stdout:8/730: truncate d9/d2a/db9/dc6/d1e/d35/dc8/fd2 1394077 0 2024-06-15T01:07:20.337 INFO:tasks.workunit.client.0.smithi137.stdout:7/660: creat df/d7d/d1c/fd9 x:0 0 0 2024-06-15T01:07:20.338 INFO:tasks.workunit.client.0.smithi137.stdout:1/967: symlink d0/d3/deb/l145 0 2024-06-15T01:07:20.339 INFO:tasks.workunit.client.0.smithi137.stdout:6/660: link d0/d5/d46/c85 d0/d1a/d3e/d57/d1d/cce 0 2024-06-15T01:07:20.345 INFO:tasks.workunit.client.0.smithi137.stdout:4/842: dread - dd/d15/d21/fe9 zero size 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:2/756: getdents d0/da/da0 0 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:9/656: write df/d12/d15/f63 [1358272,128511] 0 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:2/757: write d0/da/da0/d3a/dd4/d5c/d99/faf [5084741,130704] 0 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:7/661: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/fda x:0 0 0 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:0/992: dread d1/d2/dc/d6c/d90/fe1 [0,4194304] 0 2024-06-15T01:07:20.347 INFO:tasks.workunit.client.0.smithi137.stdout:1/968: creat d0/d1f/d60/d99/f146 x:0 0 0 2024-06-15T01:07:20.350 INFO:tasks.workunit.client.0.smithi137.stdout:1/969: fdatasync d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/f12f 0 2024-06-15T01:07:20.399 INFO:tasks.workunit.client.0.smithi137.stdout:4/843: creat dd/dbf/d130/f135 x:0 0 0 2024-06-15T01:07:20.400 INFO:tasks.workunit.client.0.smithi137.stdout:8/731: creat d9/d2a/db9/dc6/d1e/d3f/dab/f100 x:0 0 0 2024-06-15T01:07:20.400 INFO:tasks.workunit.client.0.smithi137.stdout:2/758: mkdir d0/de/d2b/de5 0 2024-06-15T01:07:20.401 INFO:tasks.workunit.client.0.smithi137.stdout:7/662: truncate df/d13/d3c/f6e 2820410 0 2024-06-15T01:07:20.402 INFO:tasks.workunit.client.0.smithi137.stdout:1/970: mknod d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/d13c/c147 0 2024-06-15T01:07:20.403 INFO:tasks.workunit.client.0.smithi137.stdout:2/759: readlink d0/dc/d2e/d72/d56/l7d 0 2024-06-15T01:07:20.403 INFO:tasks.workunit.client.0.smithi137.stdout:8/732: rename d9/d2a/db9 to d9/d2a/db9/dc6/d101 22 2024-06-15T01:07:20.403 INFO:tasks.workunit.client.0.smithi137.stdout:1/971: stat d0/d3/d47/f92 0 2024-06-15T01:07:20.404 INFO:tasks.workunit.client.0.smithi137.stdout:7/663: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a 47 1 2024-06-15T01:07:20.405 INFO:tasks.workunit.client.0.smithi137.stdout:4/844: truncate dd/d15/d30/f6b 4331587 0 2024-06-15T01:07:20.406 INFO:tasks.workunit.client.0.smithi137.stdout:4/845: dread - dd/d15/f98 zero size 2024-06-15T01:07:20.408 INFO:tasks.workunit.client.0.smithi137.stdout:4/846: stat dd/d15/d36/d45/d55/d8d/d1f/d86/d132 0 2024-06-15T01:07:20.409 INFO:tasks.workunit.client.0.smithi137.stdout:1/972: symlink d0/d3/l148 0 2024-06-15T01:07:20.410 INFO:tasks.workunit.client.0.smithi137.stdout:6/661: dread d0/d1a/d68/f6a [0,4194304] 0 2024-06-15T01:07:20.411 INFO:tasks.workunit.client.0.smithi137.stdout:7/664: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb x:0 0 0 2024-06-15T01:07:20.412 INFO:tasks.workunit.client.0.smithi137.stdout:8/733: dread d9/d2a/db9/dc6/d1e/f44 [0,4194304] 0 2024-06-15T01:07:20.412 INFO:tasks.workunit.client.0.smithi137.stdout:4/847: fsync dd/f17 0 2024-06-15T01:07:20.414 INFO:tasks.workunit.client.0.smithi137.stdout:6/662: truncate d0/fa4 541295 0 2024-06-15T01:07:20.417 INFO:tasks.workunit.client.0.smithi137.stdout:7/665: mknod df/d7d/d1c/d1d/d79/cdc 0 2024-06-15T01:07:20.419 INFO:tasks.workunit.client.0.smithi137.stdout:8/734: mkdir d9/d2a/db9/dc6/d1e/d35/d102 0 2024-06-15T01:07:20.421 INFO:tasks.workunit.client.0.smithi137.stdout:2/760: link d0/f1 d0/da/dd3/fe6 0 2024-06-15T01:07:20.423 INFO:tasks.workunit.client.0.smithi137.stdout:2/761: write d0/da/da0/d3a/dd4/d5c/d90/faa [327872,68291] 0 2024-06-15T01:07:20.426 INFO:tasks.workunit.client.0.smithi137.stdout:8/735: mkdir d9/d2a/db9/dc6/d1e/d35/dc8/d103 0 2024-06-15T01:07:20.427 INFO:tasks.workunit.client.0.smithi137.stdout:6/663: mkdir d0/d1a/d99/dcf 0 2024-06-15T01:07:20.428 INFO:tasks.workunit.client.0.smithi137.stdout:7/666: creat df/d7d/d1c/d1d/d27/d33/d43/dc5/fdd x:0 0 0 2024-06-15T01:07:20.429 INFO:tasks.workunit.client.0.smithi137.stdout:2/762: symlink d0/da/da0/d3a/dd4/d5c/d79/le7 0 2024-06-15T01:07:20.433 INFO:tasks.workunit.client.0.smithi137.stdout:2/763: chown d0/dc/c14 597789 1 2024-06-15T01:07:20.434 INFO:tasks.workunit.client.0.smithi137.stdout:2/764: fdatasync d0/da/da0/fe4 0 2024-06-15T01:07:20.434 INFO:tasks.workunit.client.0.smithi137.stdout:4/848: getdents dd/d15/d21/d3a/df6 0 2024-06-15T01:07:20.437 INFO:tasks.workunit.client.0.smithi137.stdout:7/667: dread df/d13/d3c/f5e [0,4194304] 0 2024-06-15T01:07:20.441 INFO:tasks.workunit.client.0.smithi137.stdout:2/765: mkdir d0/dc/d22/de8 0 2024-06-15T01:07:20.442 INFO:tasks.workunit.client.0.smithi137.stdout:8/736: dread d9/d2a/db9/dc6/d1e/d35/f5b [0,4194304] 0 2024-06-15T01:07:20.442 INFO:tasks.workunit.client.0.smithi137.stdout:4/849: mknod dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d10b/c136 0 2024-06-15T01:07:20.443 INFO:tasks.workunit.client.0.smithi137.stdout:4/850: chown dd/d15/d30/f41 1160876 1 2024-06-15T01:07:20.447 INFO:tasks.workunit.client.0.smithi137.stdout:8/737: rename d9/d2a/db9/dc6/d1e/d35/l3a to d9/d55/l104 0 2024-06-15T01:07:20.449 INFO:tasks.workunit.client.0.smithi137.stdout:6/664: dwrite d0/d1a/f35 [0,4194304] 0 2024-06-15T01:07:20.453 INFO:tasks.workunit.client.0.smithi137.stdout:4/851: fdatasync dd/d15/d36/d45/d55/d8d/f39 0 2024-06-15T01:07:20.454 INFO:tasks.workunit.client.0.smithi137.stdout:8/738: symlink d9/d2a/d2f/d59/ddb/l105 0 2024-06-15T01:07:20.455 INFO:tasks.workunit.client.0.smithi137.stdout:2/766: write d0/d1c/f70 [890440,33446] 0 2024-06-15T01:07:20.457 INFO:tasks.workunit.client.0.smithi137.stdout:7/668: dwrite df/d13/d2e/f4f [0,4194304] 0 2024-06-15T01:07:20.460 INFO:tasks.workunit.client.0.smithi137.stdout:8/739: chown d9/d2a/db9/dc6/d1e/d3f/dab/db7/fa7 6818610 1 2024-06-15T01:07:20.461 INFO:tasks.workunit.client.0.smithi137.stdout:2/767: unlink d0/c88 0 2024-06-15T01:07:20.463 INFO:tasks.workunit.client.0.smithi137.stdout:2/768: write d0/dc/d2e/d72/fa1 [606909,95466] 0 2024-06-15T01:07:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:2/769: readlink d0/da/da0/d3a/dd4/d5c/d79/lae 0 2024-06-15T01:07:20.520 INFO:tasks.workunit.client.0.smithi137.stdout:8/740: rename d9/d2a/d2f/d59 to d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106 0 2024-06-15T01:07:20.520 INFO:tasks.workunit.client.0.smithi137.stdout:6/665: dwrite d0/d1a/d3e/d57/d1d/f45 [0,4194304] 0 2024-06-15T01:07:20.520 INFO:tasks.workunit.client.0.smithi137.stdout:4/852: creat dd/d15/d36/d45/d55/f137 x:0 0 0 2024-06-15T01:07:20.521 INFO:tasks.workunit.client.0.smithi137.stdout:8/741: write d9/de9/feb [1802098,30079] 0 2024-06-15T01:07:20.535 INFO:tasks.workunit.client.0.smithi137.stdout:6/666: fsync d0/d1a/d68/f82 0 2024-06-15T01:07:20.535 INFO:tasks.workunit.client.0.smithi137.stdout:7/669: dread df/d7d/d3b/d5b/f84 [0,4194304] 0 2024-06-15T01:07:20.536 INFO:tasks.workunit.client.0.smithi137.stdout:8/742: chown d9/d2a/db9/dc6/d1e/cdc 29974474 1 2024-06-15T01:07:20.536 INFO:tasks.workunit.client.0.smithi137.stdout:4/853: dread - dd/d15/d36/d45/dcb/d83/db2/d67/f7d zero size 2024-06-15T01:07:20.542 INFO:tasks.workunit.client.0.smithi137.stdout:2/770: write d0/da/da0/d3a/dd4/f76 [1379035,88768] 0 2024-06-15T01:07:20.544 INFO:tasks.workunit.client.0.smithi137.stdout:8/743: symlink d9/d2a/db9/dc6/d1e/l107 0 2024-06-15T01:07:20.546 INFO:tasks.workunit.client.0.smithi137.stdout:6/667: dwrite d0/d1a/d3e/d57/d32/d92/dac/fbd [0,4194304] 0 2024-06-15T01:07:20.551 INFO:tasks.workunit.client.0.smithi137.stdout:6/668: chown d0/d5/d9/d29/f4d 21786951 1 2024-06-15T01:07:20.551 INFO:tasks.workunit.client.0.smithi137.stdout:2/771: truncate d0/f1f 118749 0 2024-06-15T01:07:20.553 INFO:tasks.workunit.client.0.smithi137.stdout:8/744: write d9/d2a/d2f/f90 [1562292,110308] 0 2024-06-15T01:07:20.554 INFO:tasks.workunit.client.0.smithi137.stdout:7/670: dwrite df/d7d/d1c/f9c [0,4194304] 0 2024-06-15T01:07:20.558 INFO:tasks.workunit.client.0.smithi137.stdout:4/854: dwrite dd/f17 [0,4194304] 0 2024-06-15T01:07:20.560 INFO:tasks.workunit.client.0.smithi137.stdout:7/671: write df/d13/d2e/d1a/f2c [2039672,109947] 0 2024-06-15T01:07:20.562 INFO:tasks.workunit.client.0.smithi137.stdout:2/772: mkdir d0/dc/d2e/d72/d56/de9 0 2024-06-15T01:07:20.566 INFO:tasks.workunit.client.0.smithi137.stdout:8/745: dread d9/d2a/db9/dc6/d1e/d35/f5b [0,4194304] 0 2024-06-15T01:07:20.566 INFO:tasks.workunit.client.0.smithi137.stdout:4/855: truncate dd/d15/d21/f9a 603763 0 2024-06-15T01:07:20.570 INFO:tasks.workunit.client.0.smithi137.stdout:6/669: getdents d0/d1a 0 2024-06-15T01:07:20.573 INFO:tasks.workunit.client.0.smithi137.stdout:6/670: readlink d0/d1a/d3e/l59 0 2024-06-15T01:07:20.574 INFO:tasks.workunit.client.0.smithi137.stdout:2/773: mknod d0/d42/d49/cea 0 2024-06-15T01:07:20.575 INFO:tasks.workunit.client.0.smithi137.stdout:7/672: dwrite df/d7d/d1c/d1d/d27/d33/d43/dc5/fdd [0,4194304] 0 2024-06-15T01:07:20.577 INFO:tasks.workunit.client.0.smithi137.stdout:7/673: chown df/d13/d2e/d1a/c25 989 1 2024-06-15T01:07:20.580 INFO:tasks.workunit.client.0.smithi137.stdout:2/774: read d0/d1c/f66 [1273710,52384] 0 2024-06-15T01:07:20.584 INFO:tasks.workunit.client.0.smithi137.stdout:4/856: rename dd/d15/d30/f6b to dd/d15/d36/d45/dcb/f138 0 2024-06-15T01:07:20.584 INFO:tasks.workunit.client.0.smithi137.stdout:6/671: dread d0/d5/d9/d3f/f4a [0,4194304] 0 2024-06-15T01:07:20.589 INFO:tasks.workunit.client.0.smithi137.stdout:7/674: rename df/d13/d2e/d1a/d3f to df/d13/d2e/d1a/d6a/dde 0 2024-06-15T01:07:20.592 INFO:tasks.workunit.client.0.smithi137.stdout:8/746: dread d9/d2a/f7a [0,4194304] 0 2024-06-15T01:07:20.598 INFO:tasks.workunit.client.0.smithi137.stdout:2/775: rename d0/d42/d49 to d0/dc/d22/deb 0 2024-06-15T01:07:20.599 INFO:tasks.workunit.client.0.smithi137.stdout:7/675: readlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/lbd 0 2024-06-15T01:07:20.600 INFO:tasks.workunit.client.0.smithi137.stdout:8/747: dwrite d9/d2a/db9/dc6/d1e/d67/fa6 [0,4194304] 0 2024-06-15T01:07:20.603 INFO:tasks.workunit.client.0.smithi137.stdout:8/748: write d9/d22/f37 [3486927,77032] 0 2024-06-15T01:07:20.605 INFO:tasks.workunit.client.0.smithi137.stdout:2/776: mkdir d0/da/da0/d3a/dd4/dec 0 2024-06-15T01:07:20.607 INFO:tasks.workunit.client.0.smithi137.stdout:2/777: chown d0/dc/d22/deb/c6f 65031 1 2024-06-15T01:07:20.609 INFO:tasks.workunit.client.0.smithi137.stdout:8/749: mknod d9/d2a/db9/dc6/d1e/d3f/d84/de4/c108 0 2024-06-15T01:07:20.612 INFO:tasks.workunit.client.0.smithi137.stdout:8/750: write d9/d22/f3e [2662402,1654] 0 2024-06-15T01:07:20.614 INFO:tasks.workunit.client.0.smithi137.stdout:2/778: symlink d0/dc/d22/deb/led 0 2024-06-15T01:07:20.617 INFO:tasks.workunit.client.0.smithi137.stdout:7/676: dread df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fd7 [0,4194304] 0 2024-06-15T01:07:20.618 INFO:tasks.workunit.client.0.smithi137.stdout:4/857: dwrite dd/d15/f42 [0,4194304] 0 2024-06-15T01:07:20.620 INFO:tasks.workunit.client.0.smithi137.stdout:8/751: readlink d9/d2a/db9/dc6/d1e/d3f/d84/lce 0 2024-06-15T01:07:20.620 INFO:tasks.workunit.client.0.smithi137.stdout:2/779: creat d0/d42/fee x:0 0 0 2024-06-15T01:07:20.622 INFO:tasks.workunit.client.0.smithi137.stdout:6/672: dwrite d0/d5/f90 [0,4194304] 0 2024-06-15T01:07:20.627 INFO:tasks.workunit.client.0.smithi137.stdout:7/677: mkdir df/d13/d2e/d1a/d6a/d90/ddf 0 2024-06-15T01:07:20.628 INFO:tasks.workunit.client.0.smithi137.stdout:4/858: symlink dd/d15/d21/d9c/d75/df5/l139 0 2024-06-15T01:07:20.631 INFO:tasks.workunit.client.0.smithi137.stdout:6/673: symlink d0/d1a/d3e/d57/d32/d92/ld0 0 2024-06-15T01:07:20.633 INFO:tasks.workunit.client.0.smithi137.stdout:2/780: mkdir d0/dc/d22/deb/def 0 2024-06-15T01:07:20.633 INFO:tasks.workunit.client.0.smithi137.stdout:6/674: creat d0/d1a/d3e/d57/d32/d74/fd1 x:0 0 0 2024-06-15T01:07:20.635 INFO:tasks.workunit.client.0.smithi137.stdout:2/781: write d0/dc/d22/f4b [1684588,108547] 0 2024-06-15T01:07:20.637 INFO:tasks.workunit.client.0.smithi137.stdout:7/678: mkdir df/d7d/d1c/d1d/d27/d33/d43/de0 0 2024-06-15T01:07:20.641 INFO:tasks.workunit.client.0.smithi137.stdout:4/859: getdents dd 0 2024-06-15T01:07:20.643 INFO:tasks.workunit.client.0.smithi137.stdout:7/679: chown df/d13/d2e/c75 1966 1 2024-06-15T01:07:20.647 INFO:tasks.workunit.client.0.smithi137.stdout:7/680: mkdir df/d13/d2e/de1 0 2024-06-15T01:07:20.647 INFO:tasks.workunit.client.0.smithi137.stdout:6/675: dwrite d0/d5/fae [0,4194304] 0 2024-06-15T01:07:20.652 INFO:tasks.workunit.client.0.smithi137.stdout:6/676: read - d0/d5/d46/f9c zero size 2024-06-15T01:07:20.653 INFO:tasks.workunit.client.0.smithi137.stdout:6/677: readlink d0/d1a/d3e/d57/d1d/d2e/l79 0 2024-06-15T01:07:20.655 INFO:tasks.workunit.client.0.smithi137.stdout:6/678: write d0/d1a/d3e/d57/d32/d74/fb7 [1496865,112713] 0 2024-06-15T01:07:20.658 INFO:tasks.workunit.client.0.smithi137.stdout:2/782: write d0/de/f1a [1068794,6223] 0 2024-06-15T01:07:20.661 INFO:tasks.workunit.client.0.smithi137.stdout:2/783: read d0/de/d2b/f8e [342931,121670] 0 2024-06-15T01:07:20.661 INFO:tasks.workunit.client.0.smithi137.stdout:6/679: rmdir d0/d5/d9/d29/d81 0 2024-06-15T01:07:20.664 INFO:tasks.workunit.client.0.smithi137.stdout:7/681: write df/d7d/d3b/d5b/f6d [3680808,7424] 0 2024-06-15T01:07:20.665 INFO:tasks.workunit.client.0.smithi137.stdout:8/752: dread d9/fc3 [0,4194304] 0 2024-06-15T01:07:20.668 INFO:tasks.workunit.client.0.smithi137.stdout:2/784: creat d0/da/da0/d3a/dd4/d5c/d79/ff0 x:0 0 0 2024-06-15T01:07:20.669 INFO:tasks.workunit.client.0.smithi137.stdout:4/860: dwrite dd/d15/d36/d45/dcb/f93 [0,4194304] 0 2024-06-15T01:07:20.670 INFO:tasks.workunit.client.0.smithi137.stdout:7/682: creat df/d13/d2e/d1a/d8c/fe2 x:0 0 0 2024-06-15T01:07:20.673 INFO:tasks.workunit.client.0.smithi137.stdout:2/785: mkdir d0/de/d2b/de5/df1 0 2024-06-15T01:07:20.676 INFO:tasks.workunit.client.0.smithi137.stdout:7/683: creat df/d13/d2e/d1a/d6a/dde/fe3 x:0 0 0 2024-06-15T01:07:20.678 INFO:tasks.workunit.client.0.smithi137.stdout:4/861: creat dd/d15/d21/d9c/d75/df5/d125/f13a x:0 0 0 2024-06-15T01:07:20.680 INFO:tasks.workunit.client.0.smithi137.stdout:8/753: getdents d9/d22 0 2024-06-15T01:07:20.680 INFO:tasks.workunit.client.0.smithi137.stdout:6/680: dwrite d0/d1a/d3e/d57/d32/d74/f61 [0,4194304] 0 2024-06-15T01:07:20.683 INFO:tasks.workunit.client.0.smithi137.stdout:4/862: dread - dd/d15/d21/d9c/d75/fc1 zero size 2024-06-15T01:07:20.683 INFO:tasks.workunit.client.0.smithi137.stdout:6/681: chown d0/d1a/d68/c95 11 1 2024-06-15T01:07:20.683 INFO:tasks.workunit.client.0.smithi137.stdout:2/786: truncate d0/da/dd3/fe6 1591699 0 2024-06-15T01:07:20.684 INFO:tasks.workunit.client.0.smithi137.stdout:2/787: fdatasync d0/d42/f92 0 2024-06-15T01:07:20.684 INFO:tasks.workunit.client.0.smithi137.stdout:7/684: mkdir df/d13/d2e/d1a/de4 0 2024-06-15T01:07:20.687 INFO:tasks.workunit.client.0.smithi137.stdout:8/754: unlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/l73 0 2024-06-15T01:07:20.689 INFO:tasks.workunit.client.0.smithi137.stdout:4/863: fdatasync dd/d15/d36/d45/d55/d8d/d1f/f56 0 2024-06-15T01:07:20.695 INFO:tasks.workunit.client.0.smithi137.stdout:8/755: symlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/l109 0 2024-06-15T01:07:20.696 INFO:tasks.workunit.client.0.smithi137.stdout:2/788: dread d0/dc/d22/deb/d57/f9e [0,4194304] 0 2024-06-15T01:07:20.696 INFO:tasks.workunit.client.0.smithi137.stdout:7/685: dwrite df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fcd [0,4194304] 0 2024-06-15T01:07:20.698 INFO:tasks.workunit.client.0.smithi137.stdout:6/682: getdents d0/d1a/da0 0 2024-06-15T01:07:20.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:20 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:20.822 INFO:tasks.workunit.client.0.smithi137.stdout:8/756: chown d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/dd1/led 17076 1 2024-06-15T01:07:20.823 INFO:tasks.workunit.client.0.smithi137.stdout:4/864: rename dd/d15/d36/d45/d8f/dac/de5 to dd/d15/d21/d13b 0 2024-06-15T01:07:20.823 INFO:tasks.workunit.client.0.smithi137.stdout:2/789: mknod d0/da/da0/d3a/dd4/d5c/d90/cf2 0 2024-06-15T01:07:20.824 INFO:tasks.workunit.client.0.smithi137.stdout:7/686: write df/d13/d3c/f5e [1228583,780] 0 2024-06-15T01:07:20.825 INFO:tasks.workunit.client.0.smithi137.stdout:6/683: link d0/d1a/fc7 d0/d1a/d99/fd2 0 2024-06-15T01:07:20.827 INFO:tasks.workunit.client.0.smithi137.stdout:2/790: mknod d0/dc/d22/deb/d57/cf3 0 2024-06-15T01:07:20.829 INFO:tasks.workunit.client.0.smithi137.stdout:6/684: creat d0/d1a/d3e/d87/fd3 x:0 0 0 2024-06-15T01:07:20.865 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:20 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:20.907 INFO:tasks.workunit.client.0.smithi137.stdout:7/687: creat df/d7d/d1c/d1d/d27/d33/fe5 x:0 0 0 2024-06-15T01:07:20.910 INFO:tasks.workunit.client.0.smithi137.stdout:8/757: dwrite d9/d2a/f7a [0,4194304] 0 2024-06-15T01:07:20.910 INFO:tasks.workunit.client.0.smithi137.stdout:6/685: dread d0/d5/d9/d3f/f4a [0,4194304] 0 2024-06-15T01:07:20.914 INFO:tasks.workunit.client.0.smithi137.stdout:4/865: dread dd/d15/d36/d45/d55/d8d/f112 [0,4194304] 0 2024-06-15T01:07:20.926 INFO:tasks.workunit.client.0.smithi137.stdout:6/686: write d0/d1a/d68/fb5 [350371,122707] 0 2024-06-15T01:07:20.928 INFO:tasks.workunit.client.0.smithi137.stdout:4/866: mkdir dd/d15/d21/d9c/d91/d13c 0 2024-06-15T01:07:20.929 INFO:tasks.workunit.client.0.smithi137.stdout:2/791: getdents d0/dc/d2e 0 2024-06-15T01:07:20.929 INFO:tasks.workunit.client.0.smithi137.stdout:4/867: fsync dd/d15/d21/d13b/f122 0 2024-06-15T01:07:20.930 INFO:tasks.workunit.client.0.smithi137.stdout:6/687: mknod d0/cd4 0 2024-06-15T01:07:20.932 INFO:tasks.workunit.client.0.smithi137.stdout:2/792: fsync d0/da/d73/fb3 0 2024-06-15T01:07:20.932 INFO:tasks.workunit.client.0.smithi137.stdout:6/688: write d0/d1a/d3e/d57/d32/d74/fd1 [410353,427] 0 2024-06-15T01:07:20.933 INFO:tasks.workunit.client.0.smithi137.stdout:4/868: symlink dd/dbf/d130/l13d 0 2024-06-15T01:07:20.935 INFO:tasks.workunit.client.0.smithi137.stdout:6/689: write d0/d5/d9/d29/d6b/f9d [4252635,17845] 0 2024-06-15T01:07:20.942 INFO:tasks.workunit.client.0.smithi137.stdout:2/793: dwrite d0/dc/d22/deb/fcb [0,4194304] 0 2024-06-15T01:07:20.945 INFO:tasks.workunit.client.0.smithi137.stdout:2/794: fsync d0/da/da0/d3a/dd4/d5c/d79/ff0 0 2024-06-15T01:07:20.948 INFO:tasks.workunit.client.0.smithi137.stdout:2/795: truncate d0/dc/d22/deb/fb0 1156454 0 2024-06-15T01:07:20.950 INFO:tasks.workunit.client.0.smithi137.stdout:4/869: dread f8 [0,4194304] 0 2024-06-15T01:07:20.954 INFO:tasks.workunit.client.0.smithi137.stdout:6/690: dread d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:20.959 INFO:tasks.workunit.client.0.smithi137.stdout:6/691: truncate d0/d1a/da0/fa1 300832 0 2024-06-15T01:07:20.959 INFO:tasks.workunit.client.0.smithi137.stdout:8/758: write d9/d2a/db9/dc6/d1e/f7f [240050,110864] 0 2024-06-15T01:07:20.963 INFO:tasks.workunit.client.0.smithi137.stdout:2/796: dread d0/dc/d22/f4b [0,4194304] 0 2024-06-15T01:07:20.965 INFO:tasks.workunit.client.0.smithi137.stdout:6/692: creat d0/d1a/d3e/d57/db6/fd5 x:0 0 0 2024-06-15T01:07:20.967 INFO:tasks.workunit.client.0.smithi137.stdout:2/797: write d0/f6 [3737802,111662] 0 2024-06-15T01:07:20.969 INFO:tasks.workunit.client.0.smithi137.stdout:4/870: write dd/d15/d36/d45/d55/d8d/f39 [3164836,52030] 0 2024-06-15T01:07:20.972 INFO:tasks.workunit.client.0.smithi137.stdout:6/693: dread d0/d5/f90 [0,4194304] 0 2024-06-15T01:07:20.974 INFO:tasks.workunit.client.0.smithi137.stdout:4/871: creat dd/d15/d21/d3a/d6e/d7b/d96/f13e x:0 0 0 2024-06-15T01:07:20.978 INFO:tasks.workunit.client.0.smithi137.stdout:8/759: dwrite d9/d2a/f8a [0,4194304] 0 2024-06-15T01:07:20.978 INFO:tasks.workunit.client.0.smithi137.stdout:6/694: creat d0/d1a/d3e/d57/d32/d74/fd6 x:0 0 0 2024-06-15T01:07:20.980 INFO:tasks.workunit.client.0.smithi137.stdout:8/760: readlink d9/d2a/db9/dc6/d1e/l49 0 2024-06-15T01:07:20.981 INFO:tasks.workunit.client.0.smithi137.stdout:2/798: truncate d0/f6 3243371 0 2024-06-15T01:07:20.983 INFO:tasks.workunit.client.0.smithi137.stdout:6/695: mkdir d0/d1a/d3e/d87/dd7 0 2024-06-15T01:07:20.983 INFO:tasks.workunit.client.0.smithi137.stdout:6/696: chown d0/l49 3 1 2024-06-15T01:07:20.985 INFO:tasks.workunit.client.0.smithi137.stdout:4/872: rename dd/d15/d36/d45/d55/d8d/c88 to dd/d15/d36/d45/d8f/c13f 0 2024-06-15T01:07:20.990 INFO:tasks.workunit.client.0.smithi137.stdout:6/697: creat d0/d5/d9/d29/fd8 x:0 0 0 2024-06-15T01:07:20.991 INFO:tasks.workunit.client.0.smithi137.stdout:2/799: dwrite d0/dc/d2e/fc0 [0,4194304] 0 2024-06-15T01:07:20.992 INFO:tasks.workunit.client.0.smithi137.stdout:6/698: dread - d0/d5/d46/f9c zero size 2024-06-15T01:07:20.993 INFO:tasks.workunit.client.0.smithi137.stdout:4/873: mkdir dd/d15/d36/d45/dcb/d140 0 2024-06-15T01:07:20.994 INFO:tasks.workunit.client.0.smithi137.stdout:2/800: mknod d0/dc/d2e/d72/cf4 0 2024-06-15T01:07:20.995 INFO:tasks.workunit.client.0.smithi137.stdout:6/699: truncate d0/d5/f73 2440908 0 2024-06-15T01:07:20.997 INFO:tasks.workunit.client.0.smithi137.stdout:2/801: mknod d0/da/da0/d3a/dd4/d77/cf5 0 2024-06-15T01:07:21.005 INFO:tasks.workunit.client.0.smithi137.stdout:8/761: dwrite d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/f71 [0,4194304] 0 2024-06-15T01:07:21.011 INFO:tasks.workunit.client.0.smithi137.stdout:8/762: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/c10a 0 2024-06-15T01:07:21.012 INFO:tasks.workunit.client.0.smithi137.stdout:8/763: read d9/d2a/f7a [83081,51021] 0 2024-06-15T01:07:21.017 INFO:tasks.workunit.client.0.smithi137.stdout:8/764: dread d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/fbc [0,4194304] 0 2024-06-15T01:07:21.018 INFO:tasks.workunit.client.0.smithi137.stdout:6/700: dread d0/d5/f7 [0,4194304] 0 2024-06-15T01:07:21.022 INFO:tasks.workunit.client.0.smithi137.stdout:6/701: chown d0/d1a/d3e/d57/d1d/d2e/f8d 762019947 1 2024-06-15T01:07:21.022 INFO:tasks.workunit.client.0.smithi137.stdout:2/802: dread d0/d1c/fa4 [0,4194304] 0 2024-06-15T01:07:21.025 INFO:tasks.workunit.client.0.smithi137.stdout:4/874: dread dd/dbf/fd1 [0,4194304] 0 2024-06-15T01:07:21.027 INFO:tasks.workunit.client.0.smithi137.stdout:2/803: unlink d0/da/da0/fb1 0 2024-06-15T01:07:21.028 INFO:tasks.workunit.client.0.smithi137.stdout:6/702: dread d0/d5/f83 [0,4194304] 0 2024-06-15T01:07:21.029 INFO:tasks.workunit.client.0.smithi137.stdout:8/765: link d9/d2a/f2d d9/d2a/db9/dc6/d1e/f10b 0 2024-06-15T01:07:21.031 INFO:tasks.workunit.client.0.smithi137.stdout:4/875: mkdir dd/d15/d21/d9c/d75/d99/d141 0 2024-06-15T01:07:21.032 INFO:tasks.workunit.client.0.smithi137.stdout:2/804: chown d0/c28 23 1 2024-06-15T01:07:21.034 INFO:tasks.workunit.client.0.smithi137.stdout:6/703: creat d0/d1a/d3e/d57/d1d/da5/fd9 x:0 0 0 2024-06-15T01:07:21.034 INFO:tasks.workunit.client.0.smithi137.stdout:2/805: read - d0/da/da0/d3a/dd4/d5c/d79/ff0 zero size 2024-06-15T01:07:21.037 INFO:tasks.workunit.client.0.smithi137.stdout:6/704: fdatasync d0/d5/f83 0 2024-06-15T01:07:21.038 INFO:tasks.workunit.client.0.smithi137.stdout:2/806: mknod d0/dc/d2e/d72/d8a/cf6 0 2024-06-15T01:07:21.040 INFO:tasks.workunit.client.0.smithi137.stdout:2/807: dread - d0/da/da0/d3a/dd4/d5c/d99/fc1 zero size 2024-06-15T01:07:21.041 INFO:tasks.workunit.client.0.smithi137.stdout:2/808: chown d0/de/d2b/l36 118351413 1 2024-06-15T01:07:21.043 INFO:tasks.workunit.client.0.smithi137.stdout:6/705: symlink d0/d1a/d3e/d57/lda 0 2024-06-15T01:07:21.045 INFO:tasks.workunit.client.0.smithi137.stdout:2/809: creat d0/da/da0/d3a/dd4/d5c/ff7 x:0 0 0 2024-06-15T01:07:21.047 INFO:tasks.workunit.client.0.smithi137.stdout:4/876: write dd/d15/d36/d45/dcb/d83/db2/dea/f111 [625768,108092] 0 2024-06-15T01:07:21.048 INFO:tasks.workunit.client.0.smithi137.stdout:2/810: fdatasync d0/f69 0 2024-06-15T01:07:21.049 INFO:tasks.workunit.client.0.smithi137.stdout:6/706: creat d0/fdb x:0 0 0 2024-06-15T01:07:21.050 INFO:tasks.workunit.client.0.smithi137.stdout:4/877: chown dd/d15/f42 6831 1 2024-06-15T01:07:21.052 INFO:tasks.workunit.client.0.smithi137.stdout:8/766: dwrite d9/d2a/d2f/f4b [0,4194304] 0 2024-06-15T01:07:21.080 INFO:tasks.workunit.client.0.smithi137.stdout:6/707: rmdir d0/d1a/d3e/d57/d1d 39 2024-06-15T01:07:21.092 INFO:tasks.workunit.client.0.smithi137.stdout:8/767: dread d9/d2a/db9/dc6/d1e/d67/f6a [0,4194304] 0 2024-06-15T01:07:21.134 INFO:tasks.workunit.client.0.smithi137.stdout:4/878: mkdir dd/d15/d21/d3a/d6e/d7b/d96/d142 0 2024-06-15T01:07:21.137 INFO:tasks.workunit.client.0.smithi137.stdout:8/768: rmdir d9/d2a/db9/dc6/d1e/d3f/d84/de4 39 2024-06-15T01:07:21.142 INFO:tasks.workunit.client.0.smithi137.stdout:2/811: dwrite d0/f1f [0,4194304] 0 2024-06-15T01:07:21.143 INFO:tasks.workunit.client.0.smithi137.stdout:2/812: read d0/dc/d22/fbe [1761877,103425] 0 2024-06-15T01:07:21.154 INFO:tasks.workunit.client.0.smithi137.stdout:4/879: symlink dd/d15/d21/d3a/d6e/d7b/dee/l143 0 2024-06-15T01:07:21.155 INFO:tasks.workunit.client.0.smithi137.stdout:4/880: stat dd/d15/d30/f41 0 2024-06-15T01:07:21.155 INFO:tasks.workunit.client.0.smithi137.stdout:6/708: readlink d0/d1a/d3e/d57/d1d/d2e/l7f 0 2024-06-15T01:07:21.160 INFO:tasks.workunit.client.0.smithi137.stdout:2/813: rename d0/de/d2b/cd5 to d0/de/d2b/de5/df1/cf8 0 2024-06-15T01:07:21.162 INFO:tasks.workunit.client.0.smithi137.stdout:6/709: creat d0/d5/d9/db9/fdc x:0 0 0 2024-06-15T01:07:21.163 INFO:tasks.workunit.client.0.smithi137.stdout:4/881: mknod dd/d15/d36/d45/dcb/d83/db2/d72/c144 0 2024-06-15T01:07:21.164 INFO:tasks.workunit.client.0.smithi137.stdout:6/710: write d0/d1a/d3e/d57/d1d/f45 [338227,7412] 0 2024-06-15T01:07:21.164 INFO:tasks.workunit.client.0.smithi137.stdout:8/769: write d9/d2a/f7a [2278225,19156] 0 2024-06-15T01:07:21.166 INFO:tasks.workunit.client.0.smithi137.stdout:6/711: readlink d0/d1a/d3e/d57/d32/d92/lc0 0 2024-06-15T01:07:21.169 INFO:tasks.workunit.client.0.smithi137.stdout:8/770: creat d9/d2a/db9/dc6/d1e/d35/f10c x:0 0 0 2024-06-15T01:07:21.170 INFO:tasks.workunit.client.0.smithi137.stdout:6/712: mkdir d0/d1a/d68/ddd 0 2024-06-15T01:07:21.173 INFO:tasks.workunit.client.0.smithi137.stdout:4/882: creat dd/d15/d36/d45/dcb/d83/db2/f145 x:0 0 0 2024-06-15T01:07:21.174 INFO:tasks.workunit.client.0.smithi137.stdout:8/771: unlink d9/d2a/db9/dc6/d1e/d35/d5f/f74 0 2024-06-15T01:07:21.176 INFO:tasks.workunit.client.0.smithi137.stdout:2/814: rename d0/dc/f2c to d0/da/d73/ff9 0 2024-06-15T01:07:21.177 INFO:tasks.workunit.client.0.smithi137.stdout:6/713: symlink d0/d1a/d3e/d57/d1d/d2e/d70/lde 0 2024-06-15T01:07:21.177 INFO:tasks.workunit.client.0.smithi137.stdout:2/815: write d0/d42/f92 [30848,126503] 0 2024-06-15T01:07:21.179 INFO:tasks.workunit.client.0.smithi137.stdout:8/772: symlink d9/d2a/db9/dc6/d1e/d35/l10d 0 2024-06-15T01:07:21.180 INFO:tasks.workunit.client.0.smithi137.stdout:8/773: write d9/d55/f5c [5038161,585] 0 2024-06-15T01:07:21.182 INFO:tasks.workunit.client.0.smithi137.stdout:6/714: symlink d0/d5/d46/ldf 0 2024-06-15T01:07:21.183 INFO:tasks.workunit.client.0.smithi137.stdout:2/816: dread d0/dc/d22/deb/d57/f9e [0,4194304] 0 2024-06-15T01:07:21.184 INFO:tasks.workunit.client.0.smithi137.stdout:8/774: write d9/d2a/db9/dc6/d1e/d3f/d84/de4/fd0 [731347,57650] 0 2024-06-15T01:07:21.186 INFO:tasks.workunit.client.0.smithi137.stdout:2/817: dread - d0/da/da0/d3a/dd4/d5c/d99/fc1 zero size 2024-06-15T01:07:21.188 INFO:tasks.workunit.client.0.smithi137.stdout:6/715: rename d0/d5/d46/dbf to d0/d1a/d3e/d87/de0 0 2024-06-15T01:07:21.190 INFO:tasks.workunit.client.0.smithi137.stdout:2/818: readlink d0/dc/lc5 0 2024-06-15T01:07:21.191 INFO:tasks.workunit.client.0.smithi137.stdout:6/716: symlink d0/d1a/le1 0 2024-06-15T01:07:21.196 INFO:tasks.workunit.client.0.smithi137.stdout:6/717: unlink d0/d1a/d3e/d57/d1d/d2e/d62/f9e 0 2024-06-15T01:07:21.200 INFO:tasks.workunit.client.0.smithi137.stdout:8/775: getdents d9/d2a/db9/dc6/d1e/d35 0 2024-06-15T01:07:21.203 INFO:tasks.workunit.client.0.smithi137.stdout:6/718: truncate d0/d5/d9/d29/d64/f86 90306 0 2024-06-15T01:07:21.205 INFO:tasks.workunit.client.0.smithi137.stdout:4/883: dwrite dd/d15/d21/d3a/d6e/f7e [0,4194304] 0 2024-06-15T01:07:21.205 INFO:tasks.workunit.client.0.smithi137.stdout:2/819: rmdir d0/dce 0 2024-06-15T01:07:21.211 INFO:tasks.workunit.client.0.smithi137.stdout:8/776: dwrite d9/d22/f98 [0,4194304] 0 2024-06-15T01:07:21.212 INFO:tasks.workunit.client.0.smithi137.stdout:6/719: write d0/d1a/f26 [1433428,48052] 0 2024-06-15T01:07:21.215 INFO:tasks.workunit.client.0.smithi137.stdout:2/820: rename d0/da/da0/d3a/dd4/d5c/ff7 to d0/da/dda/ffa 0 2024-06-15T01:07:21.216 INFO:tasks.workunit.client.0.smithi137.stdout:4/884: truncate dd/d15/d21/d3a/d6e/d7b/dee/f106 277515 0 2024-06-15T01:07:21.220 INFO:tasks.workunit.client.0.smithi137.stdout:6/720: mknod d0/d1a/d3e/d57/ce2 0 2024-06-15T01:07:21.220 INFO:tasks.workunit.client.0.smithi137.stdout:8/777: mknod d9/d2a/db9/dc6/d1e/d35/dc8/d103/c10e 0 2024-06-15T01:07:21.221 INFO:tasks.workunit.client.0.smithi137.stdout:2/821: mknod d0/db9/cfb 0 2024-06-15T01:07:21.221 INFO:tasks.workunit.client.0.smithi137.stdout:6/721: dread - d0/d1a/d3e/d57/d32/d74/fb2 zero size 2024-06-15T01:07:21.223 INFO:tasks.workunit.client.0.smithi137.stdout:4/885: rename dd/d15/d36/d45/d55/d8d/d1f/d86/fab to dd/d15/d21/d9c/d75/df5/f146 0 2024-06-15T01:07:21.224 INFO:tasks.workunit.client.0.smithi137.stdout:8/778: symlink d9/d2a/d79/d7e/l10f 0 2024-06-15T01:07:21.225 INFO:tasks.workunit.client.0.smithi137.stdout:2/822: rmdir d0/de 39 2024-06-15T01:07:21.230 INFO:tasks.workunit.client.0.smithi137.stdout:6/722: dwrite d0/d1a/d68/f9f [0,4194304] 0 2024-06-15T01:07:21.230 INFO:tasks.workunit.client.0.smithi137.stdout:2/823: creat d0/da/da0/d3a/dd4/d5c/d79/ffc x:0 0 0 2024-06-15T01:07:21.238 INFO:tasks.workunit.client.0.smithi137.stdout:4/886: dwrite dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/f131 [0,4194304] 0 2024-06-15T01:07:21.242 INFO:tasks.workunit.client.0.smithi137.stdout:6/723: rename d0/d1a/d3e/d57/d32/d74/fb2 to d0/d1a/d3e/d57/d1d/da5/fe3 0 2024-06-15T01:07:21.243 INFO:tasks.workunit.client.0.smithi137.stdout:6/724: stat d0/d1a/f98 0 2024-06-15T01:07:21.246 INFO:tasks.workunit.client.0.smithi137.stdout:4/887: read dd/d15/d36/d45/f4b [2042400,19281] 0 2024-06-15T01:07:21.246 INFO:tasks.workunit.client.0.smithi137.stdout:6/725: mknod d0/d1a/d99/ce4 0 2024-06-15T01:07:21.252 INFO:tasks.workunit.client.0.smithi137.stdout:6/726: truncate d0/d5/f7 7935963 0 2024-06-15T01:07:21.253 INFO:tasks.workunit.client.0.smithi137.stdout:6/727: dread - d0/d1a/fc7 zero size 2024-06-15T01:07:21.253 INFO:tasks.workunit.client.0.smithi137.stdout:2/824: dread d0/dc/d2e/d72/f9c [0,4194304] 0 2024-06-15T01:07:21.256 INFO:tasks.workunit.client.0.smithi137.stdout:4/888: rmdir dd/d15/d36/d45/d55/d8d 39 2024-06-15T01:07:21.257 INFO:tasks.workunit.client.0.smithi137.stdout:8/779: dread d9/d22/f37 [0,4194304] 0 2024-06-15T01:07:21.260 INFO:tasks.workunit.client.0.smithi137.stdout:8/780: write d9/d2a/db9/dc6/d1e/d35/dc8/fd2 [1577508,85353] 0 2024-06-15T01:07:21.264 INFO:tasks.workunit.client.0.smithi137.stdout:8/781: truncate d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/ff7 990509 0 2024-06-15T01:07:21.266 INFO:tasks.workunit.client.0.smithi137.stdout:8/782: truncate d9/d2a/db9/dc6/d1e/d3f/ff1 693923 0 2024-06-15T01:07:21.269 INFO:tasks.workunit.client.0.smithi137.stdout:2/825: getdents d0/da/da0/d3a/dd4/d5c/d99 0 2024-06-15T01:07:21.270 INFO:tasks.workunit.client.0.smithi137.stdout:8/783: mknod d9/d2a/d2f/c110 0 2024-06-15T01:07:21.271 INFO:tasks.workunit.client.0.smithi137.stdout:6/728: dread d0/d1a/d3e/d57/f80 [0,4194304] 0 2024-06-15T01:07:21.272 INFO:tasks.workunit.client.0.smithi137.stdout:4/889: write dd/d15/d36/d45/d55/d8d/d1f/ffb [238438,94030] 0 2024-06-15T01:07:21.280 INFO:tasks.workunit.client.0.smithi137.stdout:6/729: truncate d0/d1a/d3e/d57/d32/d74/f78 3945701 0 2024-06-15T01:07:21.281 INFO:tasks.workunit.client.0.smithi137.stdout:2/826: dwrite d0/dc/d22/deb/d57/f9e [0,4194304] 0 2024-06-15T01:07:21.281 INFO:tasks.workunit.client.0.smithi137.stdout:8/784: write d9/d2a/db9/dc6/d1e/d35/d5f/fb3 [560345,27309] 0 2024-06-15T01:07:21.284 INFO:tasks.workunit.client.0.smithi137.stdout:4/890: dwrite dd/f47 [0,4194304] 0 2024-06-15T01:07:21.289 INFO:tasks.workunit.client.0.smithi137.stdout:2/827: mkdir d0/da/da0/d3a/dd4/d5c/dfd 0 2024-06-15T01:07:21.290 INFO:tasks.workunit.client.0.smithi137.stdout:4/891: mknod dd/d15/d21/d9c/d91/c147 0 2024-06-15T01:07:21.292 INFO:tasks.workunit.client.0.smithi137.stdout:4/892: chown dd/d15/d36/d45/f4b 0 1 2024-06-15T01:07:21.292 INFO:tasks.workunit.client.0.smithi137.stdout:8/785: write d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/ff7 [1501995,122845] 0 2024-06-15T01:07:21.308 INFO:tasks.workunit.client.0.smithi137.stdout:4/893: dwrite dd/d15/d21/d9c/d75/df5/f119 [0,4194304] 0 2024-06-15T01:07:21.309 INFO:tasks.workunit.client.0.smithi137.stdout:4/894: chown dd/d15/d36/d45/f4e 16105 1 2024-06-15T01:07:21.311 INFO:tasks.workunit.client.0.smithi137.stdout:6/730: dread d0/d1a/d3e/d57/d1d/f40 [0,4194304] 0 2024-06-15T01:07:21.314 INFO:tasks.workunit.client.0.smithi137.stdout:6/731: symlink d0/d5/d9/d3f/le5 0 2024-06-15T01:07:21.315 INFO:tasks.workunit.client.0.smithi137.stdout:4/895: dread dd/dbf/fd1 [0,4194304] 0 2024-06-15T01:07:21.326 INFO:tasks.workunit.client.0.smithi137.stdout:6/732: write d0/d5/d9/f9b [1000712,39687] 0 2024-06-15T01:07:21.328 INFO:tasks.workunit.client.0.smithi137.stdout:6/733: getdents d0/d5/d9/d29/d6b 0 2024-06-15T01:07:21.330 INFO:tasks.workunit.client.0.smithi137.stdout:4/896: dwrite dd/d15/d36/d45/d8f/f10c [0,4194304] 0 2024-06-15T01:07:21.343 INFO:tasks.workunit.client.0.smithi137.stdout:6/734: dwrite d0/fa4 [0,4194304] 0 2024-06-15T01:07:21.343 INFO:tasks.workunit.client.0.smithi137.stdout:4/897: dwrite fa [4194304,4194304] 0 2024-06-15T01:07:21.349 INFO:tasks.workunit.client.0.smithi137.stdout:6/735: read d0/d5/f2c [551783,21540] 0 2024-06-15T01:07:21.351 INFO:tasks.workunit.client.0.smithi137.stdout:6/736: mkdir d0/d1a/d3e/d57/dba/de6 0 2024-06-15T01:07:21.351 INFO:tasks.workunit.client.0.smithi137.stdout:4/898: write dd/d15/d36/d45/d55/d8d/f65 [429305,29480] 0 2024-06-15T01:07:21.354 INFO:tasks.workunit.client.0.smithi137.stdout:6/737: write d0/d5/d9/d3f/f5f [2317946,49442] 0 2024-06-15T01:07:21.354 INFO:tasks.workunit.client.0.smithi137.stdout:4/899: chown dd/d15/d36/d45/dcb/f138 63 1 2024-06-15T01:07:21.356 INFO:tasks.workunit.client.0.smithi137.stdout:6/738: read - d0/d1a/d3e/d57/d1d/d2e/d70/fcc zero size 2024-06-15T01:07:21.357 INFO:tasks.workunit.client.0.smithi137.stdout:6/739: readlink d0/d1a/d3e/d57/lda 0 2024-06-15T01:07:21.357 INFO:tasks.workunit.client.0.smithi137.stdout:4/900: symlink dd/d15/d21/d3a/df6/l148 0 2024-06-15T01:07:21.358 INFO:tasks.workunit.client.0.smithi137.stdout:6/740: dread - d0/d5/d46/f9c zero size 2024-06-15T01:07:21.361 INFO:tasks.workunit.client.0.smithi137.stdout:4/901: symlink dd/d15/d21/d9c/d75/d99/d141/l149 0 2024-06-15T01:07:21.362 INFO:tasks.workunit.client.0.smithi137.stdout:6/741: creat d0/d1a/d3e/d87/dd7/fe7 x:0 0 0 2024-06-15T01:07:21.365 INFO:tasks.workunit.client.0.smithi137.stdout:6/742: mknod d0/d1a/da0/ce8 0 2024-06-15T01:07:21.366 INFO:tasks.workunit.client.0.smithi137.stdout:4/902: creat dd/d15/d21/d3a/d6e/d7b/d96/d142/f14a x:0 0 0 2024-06-15T01:07:21.366 INFO:tasks.workunit.client.0.smithi137.stdout:6/743: write d0/d1a/f26 [655373,126274] 0 2024-06-15T01:07:21.371 INFO:tasks.workunit.client.0.smithi137.stdout:4/903: getdents dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103 0 2024-06-15T01:07:21.373 INFO:tasks.workunit.client.0.smithi137.stdout:4/904: mknod dd/d15/d21/d9c/d91/c14b 0 2024-06-15T01:07:21.376 INFO:tasks.workunit.client.0.smithi137.stdout:4/905: mknod dd/d15/d36/d45/c14c 0 2024-06-15T01:07:21.378 INFO:tasks.workunit.client.0.smithi137.stdout:4/906: creat dd/d15/d36/f14d x:0 0 0 2024-06-15T01:07:21.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/744: write d0/d1a/d3e/d57/d1d/f40 [4317394,11943] 0 2024-06-15T01:07:21.382 INFO:tasks.workunit.client.0.smithi137.stdout:4/907: mkdir dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d10b/d14e 0 2024-06-15T01:07:21.383 INFO:tasks.workunit.client.0.smithi137.stdout:6/745: rename d0/d5/d9/d3f/f4a to d0/d1a/d3e/d87/de0/fe9 0 2024-06-15T01:07:21.511 INFO:tasks.workunit.client.0.smithi137.stdout:4/908: creat dd/d15/d21/d3a/d6e/d7b/d96/d142/f14f x:0 0 0 2024-06-15T01:07:21.511 INFO:tasks.workunit.client.0.smithi137.stdout:6/746: symlink d0/d1a/d3e/d87/dd7/lea 0 2024-06-15T01:07:21.513 INFO:tasks.workunit.client.0.smithi137.stdout:6/747: rmdir d0/d1a/d3e/d57/d1d 39 2024-06-15T01:07:21.515 INFO:tasks.workunit.client.0.smithi137.stdout:6/748: write d0/d5/d9/f9b [1097710,36472] 0 2024-06-15T01:07:21.545 INFO:tasks.workunit.client.0.smithi137.stdout:4/909: dwrite dd/d15/d36/d45/dcb/d5b/fc3 [0,4194304] 0 2024-06-15T01:07:21.547 INFO:tasks.workunit.client.0.smithi137.stdout:4/910: dread - dd/d15/d36/d45/dcb/d5b/fda zero size 2024-06-15T01:07:21.550 INFO:tasks.workunit.client.0.smithi137.stdout:4/911: mknod dd/d15/d36/d45/dcb/d83/db2/d72/d81/c150 0 2024-06-15T01:07:21.552 INFO:tasks.workunit.client.0.smithi137.stdout:4/912: creat dd/d15/d21/d13b/f151 x:0 0 0 2024-06-15T01:07:21.556 INFO:tasks.workunit.client.0.smithi137.stdout:4/913: getdents dd/d15/d36 0 2024-06-15T01:07:21.557 INFO:tasks.workunit.client.0.smithi137.stdout:4/914: fsync dd/d15/d36/f14d 0 2024-06-15T01:07:21.560 INFO:tasks.workunit.client.0.smithi137.stdout:4/915: mknod dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d10b/c152 0 2024-06-15T01:07:21.561 INFO:tasks.workunit.client.0.smithi137.stdout:4/916: chown dd/d15/d36/d45/dcb/d5b/fda 51904719 1 2024-06-15T01:07:21.567 INFO:tasks.workunit.client.0.smithi137.stdout:4/917: dwrite dd/f12 [0,4194304] 0 2024-06-15T01:07:21.579 INFO:tasks.workunit.client.0.smithi137.stdout:4/918: dwrite dd/d15/d36/d45/dcb/d83/db2/f116 [0,4194304] 0 2024-06-15T01:07:21.602 INFO:tasks.workunit.client.0.smithi137.stdout:4/919: dread dd/d15/d21/d9c/d75/fae [0,4194304] 0 2024-06-15T01:07:21.603 INFO:tasks.workunit.client.0.smithi137.stdout:4/920: fdatasync dd/d15/d21/f29 0 2024-06-15T01:07:21.604 INFO:tasks.workunit.client.0.smithi137.stdout:4/921: readlink dd/d15/d36/d45/dcb/d83/db2/l9e 0 2024-06-15T01:07:21.607 INFO:tasks.workunit.client.0.smithi137.stdout:4/922: link dd/d15/d21/d3a/f73 dd/d15/d21/d13b/f153 0 2024-06-15T01:07:21.609 INFO:tasks.workunit.client.0.smithi137.stdout:4/923: symlink dd/d15/d21/d3a/d6e/d7b/d96/d142/l154 0 2024-06-15T01:07:21.610 INFO:tasks.workunit.client.0.smithi137.stdout:4/924: chown dd/d15/d21/d9c/d91/f10d 41383025 1 2024-06-15T01:07:21.616 INFO:tasks.workunit.client.0.smithi137.stdout:4/925: dwrite dd/d15/d21/d3a/d6e/d7b/d96/d142/f14a [0,4194304] 0 2024-06-15T01:07:21.621 INFO:tasks.workunit.client.0.smithi137.stdout:4/926: getdents dd/d15/d21/d9c 0 2024-06-15T01:07:21.630 INFO:tasks.workunit.client.0.smithi137.stdout:4/927: dwrite dd/d15/d21/d9c/d75/f110 [0,4194304] 0 2024-06-15T01:07:21.633 INFO:tasks.workunit.client.0.smithi137.stdout:4/928: mknod dd/d15/d21/d3a/d6e/c155 0 2024-06-15T01:07:21.639 INFO:tasks.workunit.client.0.smithi137.stdout:4/929: write dd/d15/d36/d45/d55/fd4 [87713,89653] 0 2024-06-15T01:07:21.642 INFO:tasks.workunit.client.0.smithi137.stdout:4/930: symlink dd/d15/d21/d3a/d6e/d7b/d96/l156 0 2024-06-15T01:07:21.643 INFO:tasks.workunit.client.0.smithi137.stdout:4/931: chown dd/d15/d36/d45/d55/d8d/d1f/d86 797565849 1 2024-06-15T01:07:21.645 INFO:tasks.workunit.client.0.smithi137.stdout:4/932: creat dd/d15/d21/d3a/d6e/d7b/dee/f157 x:0 0 0 2024-06-15T01:07:21.649 INFO:tasks.workunit.client.0.smithi137.stdout:4/933: getdents dd/d15/d36/d45/d55/d8d 0 2024-06-15T01:07:21.651 INFO:tasks.workunit.client.0.smithi137.stdout:4/934: creat dd/d15/d36/d45/d8f/dac/f158 x:0 0 0 2024-06-15T01:07:21.652 INFO:tasks.workunit.client.0.smithi137.stdout:4/935: mkdir dd/d15/d30/d159 0 2024-06-15T01:07:21.654 INFO:tasks.workunit.client.0.smithi137.stdout:4/936: mkdir dd/d15/d36/d45/dcb/d83/db2/dea/d15a 0 2024-06-15T01:07:21.655 INFO:tasks.workunit.client.0.smithi137.stdout:4/937: fsync dd/d15/d36/f14d 0 2024-06-15T01:07:21.657 INFO:tasks.workunit.client.0.smithi137.stdout:4/938: mknod dd/d15/d21/d3a/df6/c15b 0 2024-06-15T01:07:21.657 INFO:tasks.workunit.client.0.smithi137.stdout:4/939: read dd/dbf/fd1 [3754372,99389] 0 2024-06-15T01:07:21.658 INFO:tasks.workunit.client.0.smithi137.stdout:4/940: dread - dd/d15/d36/d45/dcb/d83/db2/d67/dbe/ff4 zero size 2024-06-15T01:07:21.660 INFO:tasks.workunit.client.0.smithi137.stdout:4/941: symlink dd/d15/d36/d45/d55/d8d/d1f/d11e/l15c 0 2024-06-15T01:07:21.662 INFO:tasks.workunit.client.0.smithi137.stdout:4/942: mkdir dd/d15/d21/d3a/d6e/d7b/d96/d15d 0 2024-06-15T01:07:21.663 INFO:tasks.workunit.client.0.smithi137.stdout:4/943: fdatasync dd/d15/d21/d9c/f128 0 2024-06-15T01:07:21.664 INFO:tasks.workunit.client.0.smithi137.stdout:4/944: stat dd/d15/d21/d9c/d75/d99/fa9 0 2024-06-15T01:07:21.665 INFO:tasks.workunit.client.0.smithi137.stdout:4/945: chown dd/d15/d21/d3a/d6e/d7b/faf 1 1 2024-06-15T01:07:21.674 INFO:tasks.workunit.client.0.smithi137.stdout:4/946: dwrite dd/fe [0,4194304] 0 2024-06-15T01:07:21.705 INFO:tasks.workunit.client.0.smithi137.stdout:4/947: dread dd/d15/d36/d45/d55/d8d/f5e [0,4194304] 0 2024-06-15T01:07:21.708 INFO:tasks.workunit.client.0.smithi137.stdout:4/948: rename dd/d15/d21/d9c/d75/df5/d125 to dd/d15/d36/d45/dcb/d83/d15e 0 2024-06-15T01:07:21.709 INFO:tasks.workunit.client.0.smithi137.stdout:4/949: write dd/f12 [342380,101748] 0 2024-06-15T01:07:21.712 INFO:tasks.workunit.client.0.smithi137.stdout:4/950: mknod dd/d15/d36/d45/d8f/dac/c15f 0 2024-06-15T01:07:21.713 INFO:tasks.workunit.client.0.smithi137.stdout:4/951: fsync dd/d15/d36/d45/d55/d8d/f63 0 2024-06-15T01:07:21.716 INFO:tasks.workunit.client.0.smithi137.stdout:4/952: truncate dd/d15/d36/d45/fb1 561046 0 2024-06-15T01:07:21.717 INFO:tasks.workunit.client.0.smithi137.stdout:4/953: chown dd/d15/d21/fe9 12103 1 2024-06-15T01:07:21.721 INFO:tasks.workunit.client.0.smithi137.stdout:4/954: write dd/d15/d21/fe9 [908375,99854] 0 2024-06-15T01:07:21.725 INFO:tasks.workunit.client.0.smithi137.stdout:4/955: write dd/d15/d36/d45/d55/d8d/d1f/fb6 [2315784,105424] 0 2024-06-15T01:07:21.730 INFO:tasks.workunit.client.0.smithi137.stdout:4/956: link dd/f3b dd/d15/d21/d9c/d91/f160 0 2024-06-15T01:07:21.733 INFO:tasks.workunit.client.0.smithi137.stdout:4/957: rename dd/d15/d36/d45/d55/d8d/f90 to dd/d15/f161 0 2024-06-15T01:07:21.735 INFO:tasks.workunit.client.0.smithi137.stdout:4/958: creat dd/d15/d21/d9c/d75/d99/d141/f162 x:0 0 0 2024-06-15T01:07:21.736 INFO:tasks.workunit.client.0.smithi137.stdout:4/959: chown dd/d15/d30/f46 9215 1 2024-06-15T01:07:21.738 INFO:tasks.workunit.client.0.smithi137.stdout:4/960: getdents dd/d15/d36 0 2024-06-15T01:07:21.741 INFO:tasks.workunit.client.0.smithi137.stdout:4/961: rename dd/d15/d21/d9c/d75/d99/l9d to dd/d15/d21/d3a/d6e/l163 0 2024-06-15T01:07:21.741 INFO:tasks.workunit.client.0.smithi137.stdout:4/962: fsync dd/f28 0 2024-06-15T01:07:21.748 INFO:tasks.workunit.client.0.smithi137.stdout:4/963: dread - dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/fec zero size 2024-06-15T01:07:21.751 INFO:tasks.workunit.client.0.smithi137.stdout:4/964: truncate dd/d15/d21/d9c/d75/fc1 21599 0 2024-06-15T01:07:21.760 INFO:tasks.workunit.client.0.smithi137.stdout:4/965: dwrite dd/d15/d36/d45/d55/f134 [0,4194304] 0 2024-06-15T01:07:21.761 INFO:tasks.workunit.client.0.smithi137.stdout:4/966: chown dd/f16 390328906 1 2024-06-15T01:07:21.762 INFO:tasks.workunit.client.0.smithi137.stdout:4/967: stat dd/d15/d36/d45/d55/d8d/d1f/d86/f8a 0 2024-06-15T01:07:21.763 INFO:tasks.workunit.client.0.smithi137.stdout:4/968: stat dd/d15/d21/d9c/d75/df5/f11a 0 2024-06-15T01:07:21.775 INFO:tasks.workunit.client.0.smithi137.stdout:4/969: truncate dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/f131 4024560 0 2024-06-15T01:07:21.777 INFO:tasks.workunit.client.0.smithi137.stdout:4/970: truncate dd/d15/d36/d45/d55/d8d/f65 1041397 0 2024-06-15T01:07:21.780 INFO:tasks.workunit.client.0.smithi137.stdout:4/971: read dd/d15/d36/d45/f4e [11008,129530] 0 2024-06-15T01:07:21.781 INFO:tasks.workunit.client.0.smithi137.stdout:4/972: fsync dd/d15/d36/f14d 0 2024-06-15T01:07:21.783 INFO:tasks.workunit.client.0.smithi137.stdout:4/973: read dd/d15/d21/d9c/d75/fc1 [17198,115092] 0 2024-06-15T01:07:21.783 INFO:tasks.workunit.client.0.smithi137.stdout:4/974: fdatasync dd/d15/d36/d45/fd7 0 2024-06-15T01:07:21.785 INFO:tasks.workunit.client.0.smithi137.stdout:4/975: read dd/d15/d30/f41 [3707287,56699] 0 2024-06-15T01:07:21.787 INFO:tasks.workunit.client.0.smithi137.stdout:4/976: mknod dd/d15/d21/d3a/d6e/c164 0 2024-06-15T01:07:21.789 INFO:tasks.workunit.client.0.smithi137.stdout:4/977: mkdir dd/d15/d36/d45/dcb/d83/db2/d67/dbe/d165 0 2024-06-15T01:07:21.790 INFO:tasks.workunit.client.0.smithi137.stdout:4/978: stat dd/d15/d21/d3a/laa 0 2024-06-15T01:07:21.791 INFO:tasks.workunit.client.0.smithi137.stdout:4/979: stat dd/d15/d21/d3a/d6e/d7b/c105 0 2024-06-15T01:07:21.793 INFO:tasks.workunit.client.0.smithi137.stdout:4/980: symlink dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/l166 0 2024-06-15T01:07:21.793 INFO:tasks.workunit.client.0.smithi137.stdout:4/981: stat dd/d15/d36/d45/d55/fd4 0 2024-06-15T01:07:21.794 INFO:tasks.workunit.client.0.smithi137.stdout:4/982: chown dd/d15/d21/d9c/f74 31169423 1 2024-06-15T01:07:21.797 INFO:tasks.workunit.client.0.smithi137.stdout:4/983: dread fa [4194304,4194304] 0 2024-06-15T01:07:21.803 INFO:tasks.workunit.client.0.smithi137.stdout:4/984: dwrite dd/d15/d21/d9c/d75/d99/d141/f162 [0,4194304] 0 2024-06-15T01:07:21.806 INFO:tasks.workunit.client.0.smithi137.stdout:4/985: symlink dd/d15/d21/d9c/d75/df5/l167 0 2024-06-15T01:07:21.812 INFO:tasks.workunit.client.0.smithi137.stdout:4/986: dread dd/d15/d30/f46 [4194304,4194304] 0 2024-06-15T01:07:21.815 INFO:tasks.workunit.client.0.smithi137.stdout:4/987: creat dd/d15/d36/d45/dcb/d83/db2/d67/dbe/f168 x:0 0 0 2024-06-15T01:07:21.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:21 smithi142 ceph-mon[37379]: pgmap v26: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 102 MiB/s rd, 223 MiB/s wr, 507 op/s 2024-06-15T01:07:21.817 INFO:tasks.workunit.client.0.smithi137.stdout:4/988: creat dd/d15/d36/d45/d55/d8d/d1f/d86/dd6/d103/f169 x:0 0 0 2024-06-15T01:07:21.830 INFO:tasks.workunit.client.0.smithi137.stdout:4/989: dread dd/d15/d21/f29 [0,4194304] 0 2024-06-15T01:07:21.832 INFO:tasks.workunit.client.0.smithi137.stdout:4/990: mknod dd/d15/d21/d3a/d6e/d7b/d96/d15d/c16a 0 2024-06-15T01:07:21.836 INFO:tasks.workunit.client.0.smithi137.stdout:4/991: rename dd/d15/d36/d45/dcb/d83/db2/d72 to dd/d15/d21/d9c/d16b 0 2024-06-15T01:07:21.843 INFO:tasks.workunit.client.0.smithi137.stdout:4/992: write dd/d15/d36/d45/d55/d8d/d1f/d86/f123 [470018,93977] 0 2024-06-15T01:07:21.845 INFO:tasks.workunit.client.0.smithi137.stdout:4/993: symlink dd/d15/d21/l16c 0 2024-06-15T01:07:21.847 INFO:tasks.workunit.client.0.smithi137.stdout:4/994: fsync dd/dbf/fd1 0 2024-06-15T01:07:21.849 INFO:tasks.workunit.client.0.smithi137.stdout:4/995: mkdir dd/d15/d21/d3a/d6e/d16d 0 2024-06-15T01:07:21.851 INFO:tasks.workunit.client.0.smithi137.stdout:4/996: mknod dd/d15/c16e 0 2024-06-15T01:07:21.851 INFO:tasks.workunit.client.0.smithi137.stdout:4/997: chown dd/d15/d36/d45/f4b 0 1 2024-06-15T01:07:21.854 INFO:tasks.workunit.client.0.smithi137.stdout:4/998: getdents dd/d15/d36/d45/dcb 0 2024-06-15T01:07:21.856 INFO:tasks.workunit.client.0.smithi137.stdout:4/999: creat dd/d15/d21/d3a/d6e/d7b/d96/d142/f16f x:0 0 0 2024-06-15T01:07:21.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:21 smithi137 ceph-mon[29354]: pgmap v26: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 102 MiB/s rd, 223 MiB/s wr, 507 op/s 2024-06-15T01:07:22.175 INFO:tasks.workunit.client.0.smithi137.stdout:7/688: sync 2024-06-15T01:07:22.175 INFO:tasks.workunit.client.0.smithi137.stdout:1/973: sync 2024-06-15T01:07:22.175 INFO:tasks.workunit.client.0.smithi137.stdout:0/993: sync 2024-06-15T01:07:22.175 INFO:tasks.workunit.client.0.smithi137.stdout:9/657: sync 2024-06-15T01:07:22.177 INFO:tasks.workunit.client.0.smithi137.stdout:0/994: write d1/d2/dc/d10/d22/db7/f112 [440873,43073] 0 2024-06-15T01:07:22.180 INFO:tasks.workunit.client.0.smithi137.stdout:9/658: symlink df/d12/lc3 0 2024-06-15T01:07:22.182 INFO:tasks.workunit.client.0.smithi137.stdout:7/689: dread df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fcd [0,4194304] 0 2024-06-15T01:07:22.183 INFO:tasks.workunit.client.0.smithi137.stdout:7/690: stat df/d13/d2e/d1a/d6a 0 2024-06-15T01:07:22.189 INFO:tasks.workunit.client.0.smithi137.stdout:0/995: creat d1/d2/dc/d10/dd4/f14c x:0 0 0 2024-06-15T01:07:22.193 INFO:tasks.workunit.client.0.smithi137.stdout:9/659: rename df/d49/d72/f82 to df/d12/fc4 0 2024-06-15T01:07:22.194 INFO:tasks.workunit.client.0.smithi137.stdout:0/996: truncate d1/d2/dc/d10/d14/d56/d117/d17/d54/d145/f46 2449137 0 2024-06-15T01:07:22.197 INFO:tasks.workunit.client.0.smithi137.stdout:6/749: sync 2024-06-15T01:07:22.197 INFO:tasks.workunit.client.0.smithi137.stdout:8/786: sync 2024-06-15T01:07:22.198 INFO:tasks.workunit.client.0.smithi137.stdout:2/828: sync 2024-06-15T01:07:22.199 INFO:tasks.workunit.client.0.smithi137.stdout:0/997: dread d1/d2/d13/ffe [0,4194304] 0 2024-06-15T01:07:22.200 INFO:tasks.workunit.client.0.smithi137.stdout:8/787: dread - d9/d2a/db9/dc6/d1e/d3f/dab/f100 zero size 2024-06-15T01:07:22.201 INFO:tasks.workunit.client.0.smithi137.stdout:6/750: creat d0/d5/d9/d3f/feb x:0 0 0 2024-06-15T01:07:22.203 INFO:tasks.workunit.client.0.smithi137.stdout:2/829: mkdir d0/da/d73/dfe 0 2024-06-15T01:07:22.205 INFO:tasks.workunit.client.0.smithi137.stdout:8/788: chown d9/d2a/db9/dc6/de1 3 1 2024-06-15T01:07:22.207 INFO:tasks.workunit.client.0.smithi137.stdout:1/974: dwrite d0/d3/f2f [0,4194304] 0 2024-06-15T01:07:22.209 INFO:tasks.workunit.client.0.smithi137.stdout:7/691: write df/d7d/d1c/d1d/d27/d97/fb1 [2805180,43049] 0 2024-06-15T01:07:22.211 INFO:tasks.workunit.client.0.smithi137.stdout:8/789: write d9/d22/f37 [4490380,31837] 0 2024-06-15T01:07:22.213 INFO:tasks.workunit.client.0.smithi137.stdout:8/790: stat d9/d2a/d79/d7e 0 2024-06-15T01:07:22.213 INFO:tasks.workunit.client.0.smithi137.stdout:2/830: mkdir d0/dc/dff 0 2024-06-15T01:07:22.215 INFO:tasks.workunit.client.0.smithi137.stdout:1/975: fsync d0/d10/fc1 0 2024-06-15T01:07:22.217 INFO:tasks.workunit.client.0.smithi137.stdout:7/692: rmdir df/d13/d2e/d1a/d6a/dde 39 2024-06-15T01:07:22.217 INFO:tasks.workunit.client.0.smithi137.stdout:0/998: dread d1/f9 [0,4194304] 0 2024-06-15T01:07:22.220 INFO:tasks.workunit.client.0.smithi137.stdout:2/831: chown d0/fb 254504519 1 2024-06-15T01:07:22.224 INFO:tasks.workunit.client.0.smithi137.stdout:9/660: dwrite df/d12/fb5 [0,4194304] 0 2024-06-15T01:07:22.226 INFO:tasks.workunit.client.0.smithi137.stdout:8/791: symlink d9/de8/l111 0 2024-06-15T01:07:22.227 INFO:tasks.workunit.client.0.smithi137.stdout:8/792: chown d9/d2a/d79/fc9 7831965 1 2024-06-15T01:07:22.228 INFO:tasks.workunit.client.0.smithi137.stdout:8/793: chown d9/d2a/db9/dc6/ffb 2 1 2024-06-15T01:07:22.228 INFO:tasks.workunit.client.0.smithi137.stdout:0/999: rmdir d1/d7e/d87/da0/dc4 39 2024-06-15T01:07:22.231 INFO:tasks.workunit.client.0.smithi137.stdout:7/693: unlink df/d13/d2e/fa2 0 2024-06-15T01:07:22.233 INFO:tasks.workunit.client.0.smithi137.stdout:7/694: write df/d13/d2e/d1a/d8c/fe2 [948822,112677] 0 2024-06-15T01:07:22.233 INFO:tasks.workunit.client.0.smithi137.stdout:6/751: truncate d0/d1a/d3e/d57/d1d/da5/f8b 2433260 0 2024-06-15T01:07:22.235 INFO:tasks.workunit.client.0.smithi137.stdout:9/661: creat df/d12/d29/d79/fc5 x:0 0 0 2024-06-15T01:07:22.237 INFO:tasks.workunit.client.0.smithi137.stdout:8/794: mknod d9/d2a/db9/dc6/d1e/d35/c112 0 2024-06-15T01:07:22.242 INFO:tasks.workunit.client.0.smithi137.stdout:7/695: fdatasync df/d7d/d3b/fb6 0 2024-06-15T01:07:22.242 INFO:tasks.workunit.client.0.smithi137.stdout:9/662: mkdir df/d12/d4e/dc6 0 2024-06-15T01:07:22.243 INFO:tasks.workunit.client.0.smithi137.stdout:8/795: fsync d9/d2a/db9/dc6/fc2 0 2024-06-15T01:07:22.246 INFO:tasks.workunit.client.0.smithi137.stdout:1/976: rename d0/d2d/l48 to d0/d1f/d60/l149 0 2024-06-15T01:07:22.250 INFO:tasks.workunit.client.0.smithi137.stdout:9/663: creat df/d12/d29/d79/fc7 x:0 0 0 2024-06-15T01:07:22.252 INFO:tasks.workunit.client.0.smithi137.stdout:7/696: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/de6 0 2024-06-15T01:07:22.253 INFO:tasks.workunit.client.0.smithi137.stdout:2/832: truncate d0/da/da0/d3a/dd4/f6a 3429370 0 2024-06-15T01:07:22.256 INFO:tasks.workunit.client.0.smithi137.stdout:6/752: write d0/d5/d9/d29/d6b/fa8 [56788,7863] 0 2024-06-15T01:07:22.259 INFO:tasks.workunit.client.0.smithi137.stdout:9/664: creat df/d12/d4e/dbc/d3c/d51/fc8 x:0 0 0 2024-06-15T01:07:22.261 INFO:tasks.workunit.client.0.smithi137.stdout:2/833: dwrite d0/dc/d2e/d72/d8a/fb2 [0,4194304] 0 2024-06-15T01:07:22.264 INFO:tasks.workunit.client.0.smithi137.stdout:6/753: creat d0/d1a/d3e/d57/d32/d74/fec x:0 0 0 2024-06-15T01:07:22.265 INFO:tasks.workunit.client.0.smithi137.stdout:2/834: truncate d0/da/da0/d3a/dd4/d5c/d79/ff0 279839 0 2024-06-15T01:07:22.266 INFO:tasks.workunit.client.0.smithi137.stdout:6/754: chown d0/d1a/d3e/d57/d1d/d2e/d70 408014 1 2024-06-15T01:07:22.267 INFO:tasks.workunit.client.0.smithi137.stdout:9/665: stat df/d12/d29/d79/d9a/lad 0 2024-06-15T01:07:22.271 INFO:tasks.workunit.client.0.smithi137.stdout:7/697: link df/d7d/d1c/d1d/d27/d97/fb1 df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/db0/fe7 0 2024-06-15T01:07:22.272 INFO:tasks.workunit.client.0.smithi137.stdout:1/977: write d0/d1f/d60/d99/de3/dea/d10f/fe6 [1021828,106217] 0 2024-06-15T01:07:22.273 INFO:tasks.workunit.client.0.smithi137.stdout:2/835: symlink d0/dc/d2e/l100 0 2024-06-15T01:07:22.274 INFO:tasks.workunit.client.0.smithi137.stdout:8/796: truncate f6 1918347 0 2024-06-15T01:07:22.276 INFO:tasks.workunit.client.0.smithi137.stdout:8/797: read - d9/d2a/db9/dc6/d1e/d35/f10c zero size 2024-06-15T01:07:22.276 INFO:tasks.workunit.client.0.smithi137.stdout:6/755: creat d0/d1a/d3e/d57/d1d/d2e/d70/fed x:0 0 0 2024-06-15T01:07:22.278 INFO:tasks.workunit.client.0.smithi137.stdout:1/978: mkdir d0/d2d/d3e/dff/d14a 0 2024-06-15T01:07:22.279 INFO:tasks.workunit.client.0.smithi137.stdout:1/979: chown d0/d1f/d60/d99/de3/cee 548466 1 2024-06-15T01:07:22.280 INFO:tasks.workunit.client.0.smithi137.stdout:1/980: stat d0/d1f/d60/da9/fac 0 2024-06-15T01:07:22.281 INFO:tasks.workunit.client.0.smithi137.stdout:7/698: rename df/d7d/d1c/d1d/fd5 to df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fe8 0 2024-06-15T01:07:22.287 INFO:tasks.workunit.client.0.smithi137.stdout:7/699: write df/d13/d2e/d1a/d6a/dde/d89/fc8 [254825,25904] 0 2024-06-15T01:07:22.291 INFO:tasks.workunit.client.0.smithi137.stdout:8/798: creat d9/d2a/db9/dc6/d1e/f113 x:0 0 0 2024-06-15T01:07:22.295 INFO:tasks.workunit.client.0.smithi137.stdout:9/666: write df/d12/d29/f61 [964634,91117] 0 2024-06-15T01:07:22.295 INFO:tasks.workunit.client.0.smithi137.stdout:2/836: creat d0/de/d2b/f101 x:0 0 0 2024-06-15T01:07:22.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/756: link d0/d1a/d3e/d87/dd7/lea d0/lee 0 2024-06-15T01:07:22.382 INFO:tasks.workunit.client.0.smithi137.stdout:1/981: truncate d0/d10/d41/d7b/f90 5017889 0 2024-06-15T01:07:22.383 INFO:tasks.workunit.client.0.smithi137.stdout:2/837: mknod d0/da/da0/c102 0 2024-06-15T01:07:22.387 INFO:tasks.workunit.client.0.smithi137.stdout:6/757: creat d0/d1a/d99/fef x:0 0 0 2024-06-15T01:07:22.388 INFO:tasks.workunit.client.0.smithi137.stdout:9/667: dwrite df/d12/d1b/f35 [4194304,4194304] 0 2024-06-15T01:07:22.389 INFO:tasks.workunit.client.0.smithi137.stdout:6/758: dread d0/d5/f83 [0,4194304] 0 2024-06-15T01:07:22.394 INFO:tasks.workunit.client.0.smithi137.stdout:9/668: dread df/d12/d1b/f2f [4194304,4194304] 0 2024-06-15T01:07:22.396 INFO:tasks.workunit.client.0.smithi137.stdout:6/759: truncate d0/d5/d9/f4f 3561802 0 2024-06-15T01:07:22.398 INFO:tasks.workunit.client.0.smithi137.stdout:6/760: chown d0/d1a/d3e/d87/de0 1302573254 1 2024-06-15T01:07:22.399 INFO:tasks.workunit.client.0.smithi137.stdout:1/982: write d0/d1f/d67/f94 [94057,34178] 0 2024-06-15T01:07:22.401 INFO:tasks.workunit.client.0.smithi137.stdout:9/669: fsync df/d13/d6f/f7b 0 2024-06-15T01:07:22.401 INFO:tasks.workunit.client.0.smithi137.stdout:7/700: dwrite df/d13/d2e/f51 [0,4194304] 0 2024-06-15T01:07:22.403 INFO:tasks.workunit.client.0.smithi137.stdout:7/701: fsync df/d7d/d1c/d1d/d79/fcb 0 2024-06-15T01:07:22.404 INFO:tasks.workunit.client.0.smithi137.stdout:1/983: mkdir d0/d10/daf/d14b 0 2024-06-15T01:07:22.409 INFO:tasks.workunit.client.0.smithi137.stdout:1/984: rename d0/d1f/d60/c17 to d0/d1f/d76/d109/c14c 0 2024-06-15T01:07:22.410 INFO:tasks.workunit.client.0.smithi137.stdout:1/985: dread - d0/d3/d47/d73/f134 zero size 2024-06-15T01:07:22.412 INFO:tasks.workunit.client.0.smithi137.stdout:7/702: mkdir df/d7d/de9 0 2024-06-15T01:07:22.415 INFO:tasks.workunit.client.0.smithi137.stdout:7/703: creat df/d7d/d3b/d5b/da6/fea x:0 0 0 2024-06-15T01:07:22.422 INFO:tasks.workunit.client.0.smithi137.stdout:1/986: dwrite d0/d1f/d60/da9/fac [0,4194304] 0 2024-06-15T01:07:22.423 INFO:tasks.workunit.client.0.smithi137.stdout:7/704: mknod df/d7d/d1c/ceb 0 2024-06-15T01:07:22.424 INFO:tasks.workunit.client.0.smithi137.stdout:9/670: dwrite df/d2a/f34 [0,4194304] 0 2024-06-15T01:07:22.430 INFO:tasks.workunit.client.0.smithi137.stdout:7/705: symlink df/d7d/d1c/d1d/d27/d33/d43/dc5/lec 0 2024-06-15T01:07:22.431 INFO:tasks.workunit.client.0.smithi137.stdout:1/987: symlink d0/d10/d1d/de0/l14d 0 2024-06-15T01:07:22.433 INFO:tasks.workunit.client.0.smithi137.stdout:1/988: fdatasync d0/d1f/f4d 0 2024-06-15T01:07:22.436 INFO:tasks.workunit.client.0.smithi137.stdout:7/706: creat df/d13/d2e/d1a/d6a/d90/ddf/fed x:0 0 0 2024-06-15T01:07:22.439 INFO:tasks.workunit.client.0.smithi137.stdout:9/671: truncate df/d12/d15/d75/f8a 1905876 0 2024-06-15T01:07:22.442 INFO:tasks.workunit.client.0.smithi137.stdout:9/672: symlink df/d13/d6f/lc9 0 2024-06-15T01:07:22.443 INFO:tasks.workunit.client.0.smithi137.stdout:1/989: dwrite d0/d3/d14/da0/f144 [0,4194304] 0 2024-06-15T01:07:22.444 INFO:tasks.workunit.client.0.smithi137.stdout:7/707: creat df/d13/d2e/d62/fee x:0 0 0 2024-06-15T01:07:22.444 INFO:tasks.workunit.client.0.smithi137.stdout:6/761: dread d0/d5/d9/f4f [0,4194304] 0 2024-06-15T01:07:22.450 INFO:tasks.workunit.client.0.smithi137.stdout:7/708: rename df/c42 to df/d7d/d1c/d1d/d27/d33/d43/de0/cef 0 2024-06-15T01:07:22.452 INFO:tasks.workunit.client.0.smithi137.stdout:7/709: chown df/d13/d2e/d1a/d6a/dde/d89 1761348341 1 2024-06-15T01:07:22.456 INFO:tasks.workunit.client.0.smithi137.stdout:9/673: link df/d12/d15/f73 df/d12/d4e/dbc/d3c/d51/d8f/fca 0 2024-06-15T01:07:22.457 INFO:tasks.workunit.client.0.smithi137.stdout:6/762: getdents d0/d1a/d3e/d57/d32 0 2024-06-15T01:07:22.459 INFO:tasks.workunit.client.0.smithi137.stdout:9/674: rmdir df/d49/d4b 39 2024-06-15T01:07:22.463 INFO:tasks.workunit.client.0.smithi137.stdout:7/710: creat df/d7d/d1c/d1d/d27/d33/d43/ff0 x:0 0 0 2024-06-15T01:07:22.465 INFO:tasks.workunit.client.0.smithi137.stdout:1/990: dwrite d0/d1f/d60/f74 [4194304,4194304] 0 2024-06-15T01:07:22.466 INFO:tasks.workunit.client.0.smithi137.stdout:9/675: write df/d49/d4b/f57 [282895,83638] 0 2024-06-15T01:07:22.467 INFO:tasks.workunit.client.0.smithi137.stdout:6/763: fsync d0/d5/f12 0 2024-06-15T01:07:22.471 INFO:tasks.workunit.client.0.smithi137.stdout:7/711: mknod df/d7d/d3b/d5b/cf1 0 2024-06-15T01:07:22.476 INFO:tasks.workunit.client.0.smithi137.stdout:6/764: dread d0/d5/fb0 [0,4194304] 0 2024-06-15T01:07:22.479 INFO:tasks.workunit.client.0.smithi137.stdout:1/991: dwrite d0/d3/d14/da0/f144 [4194304,4194304] 0 2024-06-15T01:07:22.480 INFO:tasks.workunit.client.0.smithi137.stdout:1/992: fsync d0/d1f/d76/fc4 0 2024-06-15T01:07:22.484 INFO:tasks.workunit.client.0.smithi137.stdout:9/676: write df/d12/d4e/dbc/d3c/d51/d8f/fca [257773,123425] 0 2024-06-15T01:07:22.484 INFO:tasks.workunit.client.0.smithi137.stdout:6/765: creat d0/d1a/d3e/d87/de0/ff0 x:0 0 0 2024-06-15T01:07:22.486 INFO:tasks.workunit.client.0.smithi137.stdout:9/677: chown l6 242 1 2024-06-15T01:07:22.491 INFO:tasks.workunit.client.0.smithi137.stdout:7/712: dwrite df/d7d/d3b/f5f [0,4194304] 0 2024-06-15T01:07:22.492 INFO:tasks.workunit.client.0.smithi137.stdout:9/678: mkdir df/d13/dcb 0 2024-06-15T01:07:22.493 INFO:tasks.workunit.client.0.smithi137.stdout:6/766: dread d0/d1a/d99/fa3 [0,4194304] 0 2024-06-15T01:07:22.497 INFO:tasks.workunit.client.0.smithi137.stdout:7/713: write df/d13/d2e/d1a/d8c/fba [315578,9637] 0 2024-06-15T01:07:22.497 INFO:tasks.workunit.client.0.smithi137.stdout:9/679: creat df/d12/d4e/dc6/fcc x:0 0 0 2024-06-15T01:07:22.497 INFO:tasks.workunit.client.0.smithi137.stdout:6/767: dread - d0/d5/d9/d3f/feb zero size 2024-06-15T01:07:22.499 INFO:tasks.workunit.client.0.smithi137.stdout:1/993: truncate d0/d1f/d60/d99/de3/dea/d10f/d4c/f8e 833572 0 2024-06-15T01:07:22.499 INFO:tasks.workunit.client.0.smithi137.stdout:6/768: readlink d0/d5/d9/d3f/le5 0 2024-06-15T01:07:22.500 INFO:tasks.workunit.client.0.smithi137.stdout:9/680: readlink df/d12/d4e/dbc/d3c/l81 0 2024-06-15T01:07:22.502 INFO:tasks.workunit.client.0.smithi137.stdout:9/681: unlink df/d13/f30 0 2024-06-15T01:07:22.503 INFO:tasks.workunit.client.0.smithi137.stdout:6/769: creat d0/d1a/d99/ff1 x:0 0 0 2024-06-15T01:07:22.503 INFO:tasks.workunit.client.0.smithi137.stdout:6/770: chown d0/d1a/d3e/d57/d1d/da5 1259288 1 2024-06-15T01:07:22.505 INFO:tasks.workunit.client.0.smithi137.stdout:7/714: rename df/d13/d2e/d1a/d8c/fe2 to df/d7d/d1c/d1d/d27/d33/d43/de0/ff2 0 2024-06-15T01:07:22.509 INFO:tasks.workunit.client.0.smithi137.stdout:7/715: fdatasync df/d13/d2e/d1a/f26 0 2024-06-15T01:07:22.512 INFO:tasks.workunit.client.0.smithi137.stdout:7/716: stat df/d7d/d1c/c94 0 2024-06-15T01:07:22.513 INFO:tasks.workunit.client.0.smithi137.stdout:7/717: chown df/d7d/d3b/c7b 55841861 1 2024-06-15T01:07:22.514 INFO:tasks.workunit.client.0.smithi137.stdout:7/718: truncate df/f32 4919037 0 2024-06-15T01:07:22.515 INFO:tasks.workunit.client.0.smithi137.stdout:6/771: getdents d0/d1a/d3e/d87/de0 0 2024-06-15T01:07:22.521 INFO:tasks.workunit.client.0.smithi137.stdout:7/719: rename df/d7d/d1c/d1d/d27/d33/d59 to df/d7d/d1c/d1d/d79/dbc/df3 0 2024-06-15T01:07:22.523 INFO:tasks.workunit.client.0.smithi137.stdout:6/772: fdatasync d0/d5/f6d 0 2024-06-15T01:07:22.530 INFO:tasks.workunit.client.0.smithi137.stdout:6/773: write d0/d5/d9/f4f [2176168,85839] 0 2024-06-15T01:07:22.532 INFO:tasks.workunit.client.0.smithi137.stdout:7/720: write df/d13/d2e/d1a/d6a/dde/d89/f8a [1024365,75918] 0 2024-06-15T01:07:22.534 INFO:tasks.workunit.client.0.smithi137.stdout:1/994: dwrite d0/d1f/d60/d99/de3/dea/d10f/d4c/d59/d64/f6e [0,4194304] 0 2024-06-15T01:07:22.534 INFO:tasks.workunit.client.0.smithi137.stdout:6/774: creat d0/d1a/d3e/d57/d32/d92/ff2 x:0 0 0 2024-06-15T01:07:22.537 INFO:tasks.workunit.client.0.smithi137.stdout:9/682: dwrite df/d12/d4e/dbc/f92 [0,4194304] 0 2024-06-15T01:07:22.542 INFO:tasks.workunit.client.0.smithi137.stdout:9/683: symlink df/d49/d4b/d9b/lcd 0 2024-06-15T01:07:22.542 INFO:tasks.workunit.client.0.smithi137.stdout:1/995: creat d0/d1f/d60/d99/de3/dea/f14e x:0 0 0 2024-06-15T01:07:22.543 INFO:tasks.workunit.client.0.smithi137.stdout:7/721: creat df/d7d/d1c/d1d/dc4/ff4 x:0 0 0 2024-06-15T01:07:22.549 INFO:tasks.workunit.client.0.smithi137.stdout:7/722: mkdir df/d13/df5 0 2024-06-15T01:07:22.551 INFO:tasks.workunit.client.0.smithi137.stdout:6/775: dread d0/d1a/f75 [0,4194304] 0 2024-06-15T01:07:22.552 INFO:tasks.workunit.client.0.smithi137.stdout:9/684: dwrite df/d12/d15/d75/f8a [0,4194304] 0 2024-06-15T01:07:22.553 INFO:tasks.workunit.client.0.smithi137.stdout:7/723: dread - df/d13/d2e/d1a/d6a/d90/ddf/fed zero size 2024-06-15T01:07:22.553 INFO:tasks.workunit.client.0.smithi137.stdout:1/996: link d0/d3/d47/d73/fb9 d0/d3/d14/f14f 0 2024-06-15T01:07:22.554 INFO:tasks.workunit.client.0.smithi137.stdout:7/724: fdatasync df/d7d/d3b/f5f 0 2024-06-15T01:07:22.558 INFO:tasks.workunit.client.0.smithi137.stdout:7/725: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/lbe 301685 1 2024-06-15T01:07:22.560 INFO:tasks.workunit.client.0.smithi137.stdout:6/776: creat d0/ff3 x:0 0 0 2024-06-15T01:07:22.561 INFO:tasks.workunit.client.0.smithi137.stdout:9/685: dread df/d12/d29/d4a/f70 [0,4194304] 0 2024-06-15T01:07:22.563 INFO:tasks.workunit.client.0.smithi137.stdout:1/997: link d0/d10/d41/d7b/c7c d0/d139/c150 0 2024-06-15T01:07:22.563 INFO:tasks.workunit.client.0.smithi137.stdout:7/726: read df/d7d/d1c/d1d/d79/fd8 [2768950,51876] 0 2024-06-15T01:07:22.565 INFO:tasks.workunit.client.0.smithi137.stdout:7/727: dread - df/d7d/d3b/d5b/da6/fd2 zero size 2024-06-15T01:07:22.566 INFO:tasks.workunit.client.0.smithi137.stdout:9/686: mkdir df/d12/d29/d66/dce 0 2024-06-15T01:07:22.570 INFO:tasks.workunit.client.0.smithi137.stdout:9/687: mknod df/d12/d4e/dbc/d3c/d51/ccf 0 2024-06-15T01:07:22.571 INFO:tasks.workunit.client.0.smithi137.stdout:6/777: link d0/d5/d9/d29/la9 d0/d5/d9/d29/d6b/lf4 0 2024-06-15T01:07:22.573 INFO:tasks.workunit.client.0.smithi137.stdout:7/728: dwrite df/fb4 [0,4194304] 0 2024-06-15T01:07:22.574 INFO:tasks.workunit.client.0.smithi137.stdout:7/729: chown df/d7d/d1c/d1d/d79 3011 1 2024-06-15T01:07:22.577 INFO:tasks.workunit.client.0.smithi137.stdout:9/688: mkdir df/d13/dcb/dd0 0 2024-06-15T01:07:22.579 INFO:tasks.workunit.client.0.smithi137.stdout:6/778: rename d0/d5/d9/d29/fd8 to d0/d1a/ff5 0 2024-06-15T01:07:22.580 INFO:tasks.workunit.client.0.smithi137.stdout:9/689: truncate df/d49/d8d/f93 3978679 0 2024-06-15T01:07:22.580 INFO:tasks.workunit.client.0.smithi137.stdout:6/779: write d0/d5/d9/d29/fcd [492505,72349] 0 2024-06-15T01:07:22.587 INFO:tasks.workunit.client.0.smithi137.stdout:6/780: rename d0/d1a/d3e/d57/d32/d74/fb7 to d0/d5/d9/d29/d6b/ff6 0 2024-06-15T01:07:22.592 INFO:tasks.workunit.client.0.smithi137.stdout:1/998: dread d0/d3/f22 [0,4194304] 0 2024-06-15T01:07:22.593 INFO:tasks.workunit.client.0.smithi137.stdout:1/999: chown d0/d3/d14/d34 10 1 2024-06-15T01:07:22.600 INFO:tasks.workunit.client.0.smithi137.stdout:9/690: write df/d13/f22 [797755,54501] 0 2024-06-15T01:07:22.605 INFO:tasks.workunit.client.0.smithi137.stdout:9/691: creat df/d12/d4e/dbc/fd1 x:0 0 0 2024-06-15T01:07:22.605 INFO:tasks.workunit.client.0.smithi137.stdout:6/781: write d0/d1a/d3e/d57/d1d/d2e/f8e [947356,122105] 0 2024-06-15T01:07:22.607 INFO:tasks.workunit.client.0.smithi137.stdout:9/692: chown df/d12/d4e/dbc/d3c/d51/d8f/lb7 182 1 2024-06-15T01:07:22.607 INFO:tasks.workunit.client.0.smithi137.stdout:6/782: readlink d0/d1a/d3e/d57/d1d/d2e/l31 0 2024-06-15T01:07:22.608 INFO:tasks.workunit.client.0.smithi137.stdout:9/693: write df/d2a/f34 [2215641,54670] 0 2024-06-15T01:07:22.613 INFO:tasks.workunit.client.0.smithi137.stdout:9/694: link df/d13/f22 df/d12/d1b/d31/fd2 0 2024-06-15T01:07:22.615 INFO:tasks.workunit.client.0.smithi137.stdout:6/783: dwrite d0/d1a/d3e/d57/d32/d74/f61 [4194304,4194304] 0 2024-06-15T01:07:22.615 INFO:tasks.workunit.client.0.smithi137.stdout:9/695: write df/d49/d4b/f4f [2265080,48364] 0 2024-06-15T01:07:22.620 INFO:tasks.workunit.client.0.smithi137.stdout:6/784: mkdir d0/d1a/d68/ddd/df7 0 2024-06-15T01:07:22.620 INFO:tasks.workunit.client.0.smithi137.stdout:9/696: symlink df/d49/d4b/d9b/daf/ld3 0 2024-06-15T01:07:22.632 INFO:tasks.workunit.client.0.smithi137.stdout:6/785: write d0/d1a/f75 [460550,60887] 0 2024-06-15T01:07:22.633 INFO:tasks.workunit.client.0.smithi137.stdout:6/786: readlink d0/d1a/d3e/d87/lc4 0 2024-06-15T01:07:22.635 INFO:tasks.workunit.client.0.smithi137.stdout:6/787: symlink d0/d5/d9/d29/d64/lf8 0 2024-06-15T01:07:22.636 INFO:tasks.workunit.client.0.smithi137.stdout:9/697: dwrite df/f28 [0,4194304] 0 2024-06-15T01:07:22.638 INFO:tasks.workunit.client.0.smithi137.stdout:9/698: chown df/d12/d15 6 1 2024-06-15T01:07:22.642 INFO:tasks.workunit.client.0.smithi137.stdout:6/788: rmdir d0/d1a/d3e/d57 39 2024-06-15T01:07:22.654 INFO:tasks.workunit.client.0.smithi137.stdout:9/699: mkdir df/d13/dcb/dd4 0 2024-06-15T01:07:22.657 INFO:tasks.workunit.client.0.smithi137.stdout:9/700: symlink df/d12/d29/d66/dce/ld5 0 2024-06-15T01:07:22.663 INFO:tasks.workunit.client.0.smithi137.stdout:6/789: dread d0/d1a/f26 [0,4194304] 0 2024-06-15T01:07:22.710 INFO:tasks.workunit.client.0.smithi137.stdout:9/701: dwrite df/d12/d29/d66/f86 [0,4194304] 0 2024-06-15T01:07:22.710 INFO:tasks.workunit.client.0.smithi137.stdout:6/790: creat d0/d5/d9/db9/ff9 x:0 0 0 2024-06-15T01:07:22.713 INFO:tasks.workunit.client.0.smithi137.stdout:9/702: unlink df/d49/c88 0 2024-06-15T01:07:22.714 INFO:tasks.workunit.client.0.smithi137.stdout:9/703: write df/d49/d72/fc2 [238102,17687] 0 2024-06-15T01:07:22.716 INFO:tasks.workunit.client.0.smithi137.stdout:6/791: dread d0/d1a/d68/f9f [0,4194304] 0 2024-06-15T01:07:22.717 INFO:tasks.workunit.client.0.smithi137.stdout:9/704: symlink df/d12/d4e/dc6/ld6 0 2024-06-15T01:07:22.718 INFO:tasks.workunit.client.0.smithi137.stdout:9/705: fdatasync df/d12/d1b/d31/f67 0 2024-06-15T01:07:22.723 INFO:tasks.workunit.client.0.smithi137.stdout:6/792: rename d0/d5/f90 to d0/d1a/d3e/d57/db6/ffa 0 2024-06-15T01:07:22.725 INFO:tasks.workunit.client.0.smithi137.stdout:9/706: link df/d12/d29/d4a/f7f df/d49/d4b/d9b/fd7 0 2024-06-15T01:07:22.726 INFO:tasks.workunit.client.0.smithi137.stdout:9/707: write df/d12/d15/f73 [287243,7638] 0 2024-06-15T01:07:22.726 INFO:tasks.workunit.client.0.smithi137.stdout:6/793: truncate d0/d5/d9/fe 3499049 0 2024-06-15T01:07:22.727 INFO:tasks.workunit.client.0.smithi137.stdout:6/794: chown d0/d1a/d3e/d57/d1d/d2e/f8d 966858053 1 2024-06-15T01:07:22.733 INFO:tasks.workunit.client.0.smithi137.stdout:6/795: rename d0/d5/d9/d29/d6b/ff6 to d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f/ffb 0 2024-06-15T01:07:22.734 INFO:tasks.workunit.client.0.smithi137.stdout:6/796: fsync d0/d1a/f26 0 2024-06-15T01:07:22.735 INFO:tasks.workunit.client.0.smithi137.stdout:6/797: truncate d0/d1a/d3e/d57/d1d/d2e/d70/fcc 456807 0 2024-06-15T01:07:22.741 INFO:tasks.workunit.client.0.smithi137.stdout:6/798: dwrite d0/d1a/d3e/d57/d1d/d2e/d70/fcc [0,4194304] 0 2024-06-15T01:07:22.745 INFO:tasks.workunit.client.0.smithi137.stdout:6/799: read d0/d1a/d3e/d57/d32/d92/dac/fbd [3019382,115383] 0 2024-06-15T01:07:22.748 INFO:tasks.workunit.client.0.smithi137.stdout:6/800: chown d0/d1a/d3e/d57/d32/d74/c8a 5 1 2024-06-15T01:07:22.750 INFO:tasks.workunit.client.0.smithi137.stdout:6/801: creat d0/d5/d9/d3f/ffc x:0 0 0 2024-06-15T01:07:22.751 INFO:tasks.workunit.client.0.smithi137.stdout:6/802: stat d0/d5/d9/db9/ff9 0 2024-06-15T01:07:22.754 INFO:tasks.workunit.client.0.smithi137.stdout:6/803: truncate d0/d1a/d3e/d57/d1d/d2e/f38 1549454 0 2024-06-15T01:07:22.756 INFO:tasks.workunit.client.0.smithi137.stdout:6/804: creat d0/d5/d9/db9/ffd x:0 0 0 2024-06-15T01:07:22.767 INFO:tasks.workunit.client.0.smithi137.stdout:6/805: dwrite d0/d1a/d3e/d57/d32/d74/f88 [0,4194304] 0 2024-06-15T01:07:22.776 INFO:tasks.workunit.client.0.smithi137.stdout:6/806: dwrite d0/d5/d9/db9/fdc [0,4194304] 0 2024-06-15T01:07:22.790 INFO:tasks.workunit.client.0.smithi137.stdout:6/807: dwrite d0/d1a/d3e/d57/d32/d92/dac/fb3 [0,4194304] 0 2024-06-15T01:07:22.792 INFO:tasks.workunit.client.0.smithi137.stdout:6/808: creat d0/d5/d46/ffe x:0 0 0 2024-06-15T01:07:22.795 INFO:tasks.workunit.client.0.smithi137.stdout:6/809: rmdir d0/d1a/d3e/d57/d32/d74 39 2024-06-15T01:07:22.798 INFO:tasks.workunit.client.0.smithi137.stdout:6/810: symlink d0/d1a/d3e/d87/dd7/lff 0 2024-06-15T01:07:22.800 INFO:tasks.workunit.client.0.smithi137.stdout:6/811: mknod d0/d5/d9/d3f/c100 0 2024-06-15T01:07:22.801 INFO:tasks.workunit.client.0.smithi137.stdout:6/812: write d0/d5/fae [2946285,122548] 0 2024-06-15T01:07:22.805 INFO:tasks.workunit.client.0.smithi137.stdout:6/813: mkdir d0/d1a/d3e/d57/d32/d92/d101 0 2024-06-15T01:07:22.809 INFO:tasks.workunit.client.0.smithi137.stdout:6/814: getdents d0/d1a/d3e/d57/db6 0 2024-06-15T01:07:22.816 INFO:tasks.workunit.client.0.smithi137.stdout:6/815: dwrite d0/d5/d9/d3f/f7d [0,4194304] 0 2024-06-15T01:07:22.821 INFO:tasks.workunit.client.0.smithi137.stdout:6/816: write d0/d5/d9/d29/f4d [2398435,66289] 0 2024-06-15T01:07:22.824 INFO:tasks.workunit.client.0.smithi137.stdout:6/817: mkdir d0/d1a/d3e/d57/d32/d92/dac/d102 0 2024-06-15T01:07:22.829 INFO:tasks.workunit.client.0.smithi137.stdout:6/818: dread d0/d5/d9/f9b [0,4194304] 0 2024-06-15T01:07:22.839 INFO:tasks.workunit.client.0.smithi137.stdout:6/819: dwrite d0/d5/f7 [0,4194304] 0 2024-06-15T01:07:22.841 INFO:tasks.workunit.client.0.smithi137.stdout:6/820: fdatasync d0/d1a/f2f 0 2024-06-15T01:07:22.844 INFO:tasks.workunit.client.0.smithi137.stdout:6/821: creat d0/d1a/d68/f103 x:0 0 0 2024-06-15T01:07:22.847 INFO:tasks.workunit.client.0.smithi137.stdout:6/822: dread d0/d5/d9/d29/d6b/fa8 [0,4194304] 0 2024-06-15T01:07:22.848 INFO:tasks.workunit.client.0.smithi137.stdout:6/823: chown d0/d5/d9/d29/d64/lc1 193 1 2024-06-15T01:07:22.854 INFO:tasks.workunit.client.0.smithi137.stdout:6/824: truncate d0/d5/f7 6362215 0 2024-06-15T01:07:22.857 INFO:tasks.workunit.client.0.smithi137.stdout:6/825: truncate d0/d1a/d3e/d57/d1d/da5/f8b 262076 0 2024-06-15T01:07:22.863 INFO:tasks.workunit.client.0.smithi137.stdout:6/826: dwrite d0/d1a/d3e/d57/d1d/f45 [0,4194304] 0 2024-06-15T01:07:22.868 INFO:tasks.workunit.client.0.smithi137.stdout:6/827: truncate d0/d1a/d3e/d57/d1d/d2e/f38 2287552 0 2024-06-15T01:07:22.872 INFO:tasks.workunit.client.0.smithi137.stdout:6/828: unlink d0/ca 0 2024-06-15T01:07:22.874 INFO:tasks.workunit.client.0.smithi137.stdout:6/829: mknod d0/d5/d9/c104 0 2024-06-15T01:07:22.881 INFO:tasks.workunit.client.0.smithi137.stdout:6/830: dwrite d0/d1a/d3e/d57/d1d/da5/fe3 [0,4194304] 0 2024-06-15T01:07:23.364 INFO:tasks.workunit.client.0.smithi137.stdout:8/799: sync 2024-06-15T01:07:23.364 INFO:tasks.workunit.client.0.smithi137.stdout:9/708: sync 2024-06-15T01:07:23.364 INFO:tasks.workunit.client.0.smithi137.stdout:2/838: sync 2024-06-15T01:07:23.364 INFO:tasks.workunit.client.0.smithi137.stdout:6/831: sync 2024-06-15T01:07:23.364 INFO:tasks.workunit.client.0.smithi137.stdout:7/730: sync 2024-06-15T01:07:23.367 INFO:tasks.workunit.client.0.smithi137.stdout:2/839: mknod d0/dc/d2e/d72/d56/c103 0 2024-06-15T01:07:23.368 INFO:tasks.workunit.client.0.smithi137.stdout:9/709: write df/f69 [860700,28226] 0 2024-06-15T01:07:23.368 INFO:tasks.workunit.client.0.smithi137.stdout:2/840: fsync d0/da/da0/d3a/dd4/d5c/d99/fc1 0 2024-06-15T01:07:23.369 INFO:tasks.workunit.client.0.smithi137.stdout:9/710: fsync df/d12/d1b/d31/d42/f56 0 2024-06-15T01:07:23.369 INFO:tasks.workunit.client.0.smithi137.stdout:8/800: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/dd1/c114 0 2024-06-15T01:07:23.370 INFO:tasks.workunit.client.0.smithi137.stdout:6/832: creat d0/d1a/d68/ddd/df7/f105 x:0 0 0 2024-06-15T01:07:23.372 INFO:tasks.workunit.client.0.smithi137.stdout:8/801: write d9/d2a/db9/dc6/d1e/d67/fa6 [954612,74658] 0 2024-06-15T01:07:23.373 INFO:tasks.workunit.client.0.smithi137.stdout:7/731: rename df/d7d/d1c/d1d/d27/d33/d43/c4b to df/d7d/d1c/cf6 0 2024-06-15T01:07:23.376 INFO:tasks.workunit.client.0.smithi137.stdout:2/841: dread d0/da/da0/d3a/dd4/d5c/d79/fb6 [0,4194304] 0 2024-06-15T01:07:23.377 INFO:tasks.workunit.client.0.smithi137.stdout:6/833: creat d0/d1a/d68/ddd/f106 x:0 0 0 2024-06-15T01:07:23.379 INFO:tasks.workunit.client.0.smithi137.stdout:7/732: creat df/d7d/d1c/d1d/d79/dbc/ff7 x:0 0 0 2024-06-15T01:07:23.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/834: dread d0/d5/f83 [0,4194304] 0 2024-06-15T01:07:23.383 INFO:tasks.workunit.client.0.smithi137.stdout:7/733: symlink df/d7d/lf8 0 2024-06-15T01:07:23.386 INFO:tasks.workunit.client.0.smithi137.stdout:7/734: fsync df/f78 0 2024-06-15T01:07:23.390 INFO:tasks.workunit.client.0.smithi137.stdout:9/711: dread df/d49/d4b/d9b/fd7 [0,4194304] 0 2024-06-15T01:07:23.390 INFO:tasks.workunit.client.0.smithi137.stdout:9/712: chown df/l78 71984470 1 2024-06-15T01:07:23.390 INFO:tasks.workunit.client.0.smithi137.stdout:6/835: write d0/fa6 [3664267,50963] 0 2024-06-15T01:07:23.393 INFO:tasks.workunit.client.0.smithi137.stdout:9/713: creat df/d12/d1b/d31/d42/fd8 x:0 0 0 2024-06-15T01:07:23.393 INFO:tasks.workunit.client.0.smithi137.stdout:6/836: readlink d0/d5/d9/d29/la9 0 2024-06-15T01:07:23.394 INFO:tasks.workunit.client.0.smithi137.stdout:2/842: dread d0/da/d73/ff9 [0,4194304] 0 2024-06-15T01:07:23.395 INFO:tasks.workunit.client.0.smithi137.stdout:9/714: truncate df/d12/d1b/d31/f67 5222018 0 2024-06-15T01:07:23.397 INFO:tasks.workunit.client.0.smithi137.stdout:2/843: mkdir d0/dc/d2e/d72/d56/d6e/d104 0 2024-06-15T01:07:23.398 INFO:tasks.workunit.client.0.smithi137.stdout:9/715: dread df/d12/d29/d4a/f5d [0,4194304] 0 2024-06-15T01:07:23.400 INFO:tasks.workunit.client.0.smithi137.stdout:6/837: write d0/d5/d9/d29/f9a [356408,45252] 0 2024-06-15T01:07:23.405 INFO:tasks.workunit.client.0.smithi137.stdout:2/844: dwrite d0/dc/d2e/fc0 [0,4194304] 0 2024-06-15T01:07:23.408 INFO:tasks.workunit.client.0.smithi137.stdout:6/838: truncate d0/d5/fae 3298469 0 2024-06-15T01:07:23.409 INFO:tasks.workunit.client.0.smithi137.stdout:9/716: rename df/d12/d15/c25 to df/d13/cd9 0 2024-06-15T01:07:23.411 INFO:tasks.workunit.client.0.smithi137.stdout:6/839: unlink d0/d1a/d3e/f77 0 2024-06-15T01:07:23.413 INFO:tasks.workunit.client.0.smithi137.stdout:9/717: mknod df/d12/d29/d95/db3/cda 0 2024-06-15T01:07:23.416 INFO:tasks.workunit.client.0.smithi137.stdout:6/840: creat d0/d1a/d3e/f107 x:0 0 0 2024-06-15T01:07:23.421 INFO:tasks.workunit.client.0.smithi137.stdout:9/718: link df/d12/d29/d79/fc5 df/d13/fdb 0 2024-06-15T01:07:23.428 INFO:tasks.workunit.client.0.smithi137.stdout:2/845: dread d0/dc/f91 [0,4194304] 0 2024-06-15T01:07:23.429 INFO:tasks.workunit.client.0.smithi137.stdout:2/846: chown d0/dc/d2e/d72/d56/de9 39 1 2024-06-15T01:07:23.432 INFO:tasks.workunit.client.0.smithi137.stdout:2/847: creat d0/da/da0/f105 x:0 0 0 2024-06-15T01:07:23.436 INFO:tasks.workunit.client.0.smithi137.stdout:6/841: dwrite d0/d1a/fc7 [0,4194304] 0 2024-06-15T01:07:23.437 INFO:tasks.workunit.client.0.smithi137.stdout:6/842: dread - d0/d5/d9/d3f/feb zero size 2024-06-15T01:07:23.445 INFO:tasks.workunit.client.0.smithi137.stdout:2/848: truncate d0/da/da0/d3a/dd4/d5c/d90/faa 383321 0 2024-06-15T01:07:23.451 INFO:tasks.workunit.client.0.smithi137.stdout:6/843: dwrite d0/d1a/d3e/d57/d1d/d2e/d62/fc2 [0,4194304] 0 2024-06-15T01:07:23.452 INFO:tasks.workunit.client.0.smithi137.stdout:2/849: link d0/dc/d2e/d72/d56/f93 d0/de/d2b/f106 0 2024-06-15T01:07:23.459 INFO:tasks.workunit.client.0.smithi137.stdout:2/850: dwrite d0/d42/fee [0,4194304] 0 2024-06-15T01:07:23.481 INFO:tasks.workunit.client.0.smithi137.stdout:2/851: dread d0/dc/d22/deb/d57/f9e [0,4194304] 0 2024-06-15T01:07:23.483 INFO:tasks.workunit.client.0.smithi137.stdout:2/852: symlink d0/da/da0/d3a/dd4/l107 0 2024-06-15T01:07:23.484 INFO:tasks.workunit.client.0.smithi137.stdout:2/853: stat d0/da/da0/d3a/dd4/d5c/d99/faf 0 2024-06-15T01:07:23.485 INFO:tasks.workunit.client.0.smithi137.stdout:2/854: symlink d0/dc/d22/deb/def/l108 0 2024-06-15T01:07:23.489 INFO:tasks.workunit.client.0.smithi137.stdout:2/855: link d0/da/da0/d3a/cdc d0/dc/d2e/d72/d56/c109 0 2024-06-15T01:07:23.490 INFO:tasks.workunit.client.0.smithi137.stdout:2/856: chown d0/da/da0/d3a/dd4/f53 2 1 2024-06-15T01:07:23.495 INFO:tasks.workunit.client.0.smithi137.stdout:2/857: dread d0/da/da0/d3a/dd4/d5c/fa3 [0,4194304] 0 2024-06-15T01:07:23.497 INFO:tasks.workunit.client.0.smithi137.stdout:2/858: write d0/dc/d2e/fdd [8753215,117285] 0 2024-06-15T01:07:23.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:23 smithi137 ceph-mon[29354]: pgmap v27: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 120 MiB/s rd, 269 MiB/s wr, 656 op/s 2024-06-15T01:07:23.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:23 smithi142 ceph-mon[37379]: pgmap v27: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 120 MiB/s rd, 269 MiB/s wr, 656 op/s 2024-06-15T01:07:23.849 INFO:tasks.workunit.client.0.smithi137.stdout:8/802: sync 2024-06-15T01:07:23.850 INFO:tasks.workunit.client.0.smithi137.stdout:8/803: write d9/d2a/f97 [3250762,21549] 0 2024-06-15T01:07:23.852 INFO:tasks.workunit.client.0.smithi137.stdout:8/804: mknod d9/d2a/db9/dc6/d1e/d35/dfd/c115 0 2024-06-15T01:07:23.853 INFO:tasks.workunit.client.0.smithi137.stdout:8/805: fdatasync d9/d2a/db9/dc6/d1e/d3f/dab/f100 0 2024-06-15T01:07:23.854 INFO:tasks.workunit.client.0.smithi137.stdout:8/806: readlink d9/d2a/db9/lde 0 2024-06-15T01:07:23.855 INFO:tasks.workunit.client.0.smithi137.stdout:8/807: mkdir d9/d2a/db9/dc6/d1e/d35/d116 0 2024-06-15T01:07:23.861 INFO:tasks.workunit.client.0.smithi137.stdout:8/808: link d9/d2a/db9/dc6/d1e/d3f/dab/ff4 d9/d2a/db9/dc6/d1e/d3f/dab/f117 0 2024-06-15T01:07:23.861 INFO:tasks.workunit.client.0.smithi137.stdout:8/809: chown d9/d2a/db9/dc6/d1e/d67/fa6 8006 1 2024-06-15T01:07:23.863 INFO:tasks.workunit.client.0.smithi137.stdout:8/810: creat d9/d2a/d79/d7e/f118 x:0 0 0 2024-06-15T01:07:23.876 INFO:tasks.workunit.client.0.smithi137.stdout:8/811: dread d9/d2a/db9/dc6/d1e/d67/fa6 [0,4194304] 0 2024-06-15T01:07:23.878 INFO:tasks.workunit.client.0.smithi137.stdout:8/812: mknod d9/d2a/db9/dc6/d1e/d35/dfd/c119 0 2024-06-15T01:07:23.891 INFO:tasks.workunit.client.0.smithi137.stdout:8/813: dread d9/d2a/db9/dc6/d1e/f10b [0,4194304] 0 2024-06-15T01:07:23.895 INFO:tasks.workunit.client.0.smithi137.stdout:8/814: write d9/d2a/db9/dc6/d1e/f75 [336263,84702] 0 2024-06-15T01:07:23.898 INFO:tasks.workunit.client.0.smithi137.stdout:8/815: truncate d9/d2a/db9/dc6/d1e/d3f/d84/de4/fd0 1153995 0 2024-06-15T01:07:23.899 INFO:tasks.workunit.client.0.smithi137.stdout:8/816: write d9/d2a/db9/dc6/f18 [2811680,73089] 0 2024-06-15T01:07:23.900 INFO:tasks.workunit.client.0.smithi137.stdout:8/817: chown d9/d2a/d79/cee 46 1 2024-06-15T01:07:23.903 INFO:tasks.workunit.client.0.smithi137.stdout:8/818: rename d9/l76 to d9/l11a 0 2024-06-15T01:07:23.905 INFO:tasks.workunit.client.0.smithi137.stdout:8/819: mkdir d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d11b 0 2024-06-15T01:07:23.910 INFO:tasks.workunit.client.0.smithi137.stdout:8/820: write d9/d2a/db9/dc6/d1e/d3f/dab/ff4 [4673018,76639] 0 2024-06-15T01:07:23.910 INFO:tasks.workunit.client.0.smithi137.stdout:8/821: chown d9/d2a/d79/d7e/le2 18944568 1 2024-06-15T01:07:23.913 INFO:tasks.workunit.client.0.smithi137.stdout:8/822: creat d9/d2a/db9/dc6/de1/f11c x:0 0 0 2024-06-15T01:07:23.917 INFO:tasks.workunit.client.0.smithi137.stdout:7/735: sync 2024-06-15T01:07:23.917 INFO:tasks.workunit.client.0.smithi137.stdout:9/719: sync 2024-06-15T01:07:23.917 INFO:tasks.workunit.client.0.smithi137.stdout:6/844: sync 2024-06-15T01:07:23.918 INFO:tasks.workunit.client.0.smithi137.stdout:8/823: creat d9/d2a/db9/dc6/d1e/d35/d102/f11d x:0 0 0 2024-06-15T01:07:23.919 INFO:tasks.workunit.client.0.smithi137.stdout:6/845: chown d0/d1a/d3e/d57/d1d/d2e/f8d 59969 1 2024-06-15T01:07:23.948 INFO:tasks.workunit.client.0.smithi137.stdout:8/824: symlink d9/d2a/l11e 0 2024-06-15T01:07:23.949 INFO:tasks.workunit.client.0.smithi137.stdout:7/736: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/db7/lf9 0 2024-06-15T01:07:23.956 INFO:tasks.workunit.client.0.smithi137.stdout:7/737: creat df/d13/d2e/de1/ffa x:0 0 0 2024-06-15T01:07:23.963 INFO:tasks.workunit.client.0.smithi137.stdout:7/738: dwrite df/d7d/d1c/d1d/d79/fcb [0,4194304] 0 2024-06-15T01:07:23.969 INFO:tasks.workunit.client.0.smithi137.stdout:7/739: rename df/ccc to df/cfb 0 2024-06-15T01:07:23.971 INFO:tasks.workunit.client.0.smithi137.stdout:8/825: truncate d9/d55/f5c 1868916 0 2024-06-15T01:07:23.974 INFO:tasks.workunit.client.0.smithi137.stdout:9/720: dwrite df/d13/fdb [0,4194304] 0 2024-06-15T01:07:23.976 INFO:tasks.workunit.client.0.smithi137.stdout:9/721: stat df/l78 0 2024-06-15T01:07:23.976 INFO:tasks.workunit.client.0.smithi137.stdout:8/826: readlink d9/d55/l104 0 2024-06-15T01:07:23.980 INFO:tasks.workunit.client.0.smithi137.stdout:6/846: dwrite d0/d5/f2c [0,4194304] 0 2024-06-15T01:07:23.980 INFO:tasks.workunit.client.0.smithi137.stdout:9/722: rmdir df/d12 39 2024-06-15T01:07:23.983 INFO:tasks.workunit.client.0.smithi137.stdout:7/740: rename df/d13/d2e/d1a/d8c/fd4 to df/d7d/d1c/d1d/d79/ffc 0 2024-06-15T01:07:23.984 INFO:tasks.workunit.client.0.smithi137.stdout:8/827: creat d9/de8/f11f x:0 0 0 2024-06-15T01:07:23.986 INFO:tasks.workunit.client.0.smithi137.stdout:2/859: sync 2024-06-15T01:07:23.990 INFO:tasks.workunit.client.0.smithi137.stdout:6/847: mkdir d0/d5/d108 0 2024-06-15T01:07:23.991 INFO:tasks.workunit.client.0.smithi137.stdout:9/723: fsync df/d12/d1b/f2f 0 2024-06-15T01:07:23.992 INFO:tasks.workunit.client.0.smithi137.stdout:6/848: dread - d0/d1a/d3e/d57/d1d/d2e/d70/fed zero size 2024-06-15T01:07:23.994 INFO:tasks.workunit.client.0.smithi137.stdout:7/741: symlink df/d7d/d3b/dd1/lfd 0 2024-06-15T01:07:23.996 INFO:tasks.workunit.client.0.smithi137.stdout:9/724: mknod df/d49/d4b/d9b/daf/cdc 0 2024-06-15T01:07:23.997 INFO:tasks.workunit.client.0.smithi137.stdout:9/725: stat df/d12/lc3 0 2024-06-15T01:07:23.997 INFO:tasks.workunit.client.0.smithi137.stdout:7/742: fsync df/d7d/d3b/d5b/da6/fd2 0 2024-06-15T01:07:23.998 INFO:tasks.workunit.client.0.smithi137.stdout:2/860: link d0/da/dda/ffa d0/dc/d22/deb/def/f10a 0 2024-06-15T01:07:24.001 INFO:tasks.workunit.client.0.smithi137.stdout:6/849: getdents d0/d1a/d3e/d57/db6 0 2024-06-15T01:07:24.005 INFO:tasks.workunit.client.0.smithi137.stdout:8/828: dwrite d9/d2a/db9/dc6/d1e/f44 [0,4194304] 0 2024-06-15T01:07:24.006 INFO:tasks.workunit.client.0.smithi137.stdout:2/861: mknod d0/dc/d2e/d72/d8a/c10b 0 2024-06-15T01:07:24.009 INFO:tasks.workunit.client.0.smithi137.stdout:6/850: mkdir d0/d1a/d3e/d87/de0/d109 0 2024-06-15T01:07:24.010 INFO:tasks.workunit.client.0.smithi137.stdout:7/743: mkdir df/d13/d2e/d1a/d6a/dfe 0 2024-06-15T01:07:24.011 INFO:tasks.workunit.client.0.smithi137.stdout:6/851: chown d0/d1a/d3e/d57/ce2 1297 1 2024-06-15T01:07:24.013 INFO:tasks.workunit.client.0.smithi137.stdout:6/852: dread - d0/d1a/d99/fef zero size 2024-06-15T01:07:24.014 INFO:tasks.workunit.client.0.smithi137.stdout:8/829: rename d9/d2a/db9/dc6/d1e/f10b to d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/f120 0 2024-06-15T01:07:24.021 INFO:tasks.workunit.client.0.smithi137.stdout:6/853: symlink d0/d5/d9/d29/d64/l10a 0 2024-06-15T01:07:24.024 INFO:tasks.workunit.client.0.smithi137.stdout:9/726: dread df/d12/d1b/d31/d42/f46 [0,4194304] 0 2024-06-15T01:07:24.031 INFO:tasks.workunit.client.0.smithi137.stdout:7/744: creat df/d7d/d1c/d1d/d27/d33/d43/fff x:0 0 0 2024-06-15T01:07:24.034 INFO:tasks.workunit.client.0.smithi137.stdout:8/830: symlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d11b/l121 0 2024-06-15T01:07:24.035 INFO:tasks.workunit.client.0.smithi137.stdout:8/831: fdatasync d9/d22/f37 0 2024-06-15T01:07:24.036 INFO:tasks.workunit.client.0.smithi137.stdout:8/832: dread - d9/d2a/db9/dc6/d1e/f113 zero size 2024-06-15T01:07:24.038 INFO:tasks.workunit.client.0.smithi137.stdout:9/727: fdatasync f2 0 2024-06-15T01:07:24.042 INFO:tasks.workunit.client.0.smithi137.stdout:2/862: getdents d0/dc/d2e/d72/d56 0 2024-06-15T01:07:24.042 INFO:tasks.workunit.client.0.smithi137.stdout:6/854: write d0/d1a/d3e/d57/d1d/d2e/f8d [671315,25644] 0 2024-06-15T01:07:24.044 INFO:tasks.workunit.client.0.smithi137.stdout:6/855: write d0/d1a/d99/fef [439915,63997] 0 2024-06-15T01:07:24.049 INFO:tasks.workunit.client.0.smithi137.stdout:7/745: mknod df/d7d/de9/c100 0 2024-06-15T01:07:24.050 INFO:tasks.workunit.client.0.smithi137.stdout:2/863: mkdir d0/da/da0/d3a/dd4/d10c 0 2024-06-15T01:07:24.051 INFO:tasks.workunit.client.0.smithi137.stdout:7/746: fdatasync df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb 0 2024-06-15T01:07:24.052 INFO:tasks.workunit.client.0.smithi137.stdout:9/728: creat df/d1a/da7/fdd x:0 0 0 2024-06-15T01:07:24.055 INFO:tasks.workunit.client.0.smithi137.stdout:8/833: write d9/d2a/db9/dc6/d1e/f27 [3883452,96382] 0 2024-06-15T01:07:24.059 INFO:tasks.workunit.client.0.smithi137.stdout:2/864: rename d0/da/da0/d3a/dd4/d10c to d0/da/da0/d3a/d10d 0 2024-06-15T01:07:24.060 INFO:tasks.workunit.client.0.smithi137.stdout:9/729: dread df/f89 [0,4194304] 0 2024-06-15T01:07:24.063 INFO:tasks.workunit.client.0.smithi137.stdout:7/747: rmdir df/d7d/d1c/d1d/d79 39 2024-06-15T01:07:24.068 INFO:tasks.workunit.client.0.smithi137.stdout:9/730: dwrite df/d49/d4b/f57 [0,4194304] 0 2024-06-15T01:07:24.070 INFO:tasks.workunit.client.0.smithi137.stdout:2/865: creat d0/da/da0/d3a/dd4/d5c/dfd/f10e x:0 0 0 2024-06-15T01:07:24.082 INFO:tasks.workunit.client.0.smithi137.stdout:9/731: dwrite df/d12/d29/fa3 [0,4194304] 0 2024-06-15T01:07:24.083 INFO:tasks.workunit.client.0.smithi137.stdout:2/866: creat d0/dc/f10f x:0 0 0 2024-06-15T01:07:24.083 INFO:tasks.workunit.client.0.smithi137.stdout:7/748: write df/d7d/d1c/d1d/d27/d97/fb1 [1931823,15121] 0 2024-06-15T01:07:24.086 INFO:tasks.workunit.client.0.smithi137.stdout:7/749: truncate df/d13/d2e/d62/fee 959050 0 2024-06-15T01:07:24.089 INFO:tasks.workunit.client.0.smithi137.stdout:7/750: chown df/d13/d2e/d1a/de4 2 1 2024-06-15T01:07:24.091 INFO:tasks.workunit.client.0.smithi137.stdout:8/834: dread d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/f96 [0,4194304] 0 2024-06-15T01:07:24.093 INFO:tasks.workunit.client.0.smithi137.stdout:6/856: dread d0/d5/d9/f63 [4194304,4194304] 0 2024-06-15T01:07:24.095 INFO:tasks.workunit.client.0.smithi137.stdout:9/732: creat df/d12/d4e/dbc/fde x:0 0 0 2024-06-15T01:07:24.099 INFO:tasks.workunit.client.0.smithi137.stdout:8/835: write d9/d2a/db9/dc6/f3c [1365706,18708] 0 2024-06-15T01:07:24.168 INFO:tasks.workunit.client.0.smithi137.stdout:2/867: write d0/f8 [4043200,125827] 0 2024-06-15T01:07:24.170 INFO:tasks.workunit.client.0.smithi137.stdout:9/733: write df/f6b [1918200,45082] 0 2024-06-15T01:07:24.174 INFO:tasks.workunit.client.0.smithi137.stdout:2/868: link d0/da/da0/f105 d0/dc/dff/f110 0 2024-06-15T01:07:24.174 INFO:tasks.workunit.client.0.smithi137.stdout:8/836: write d9/d2a/db9/dc6/f7b [564381,18589] 0 2024-06-15T01:07:24.176 INFO:tasks.workunit.client.0.smithi137.stdout:9/734: rename df/d49/d4b/d9b/daf/ld3 to df/d13/d6f/ldf 0 2024-06-15T01:07:24.177 INFO:tasks.workunit.client.0.smithi137.stdout:6/857: dwrite d0/d5/d9/d29/d64/f86 [0,4194304] 0 2024-06-15T01:07:24.181 INFO:tasks.workunit.client.0.smithi137.stdout:8/837: truncate d9/d2a/db9/dc6/d1e/d35/d5f/fb3 1362446 0 2024-06-15T01:07:24.181 INFO:tasks.workunit.client.0.smithi137.stdout:2/869: creat d0/da/da0/d3a/dd4/d5c/d79/f111 x:0 0 0 2024-06-15T01:07:24.184 INFO:tasks.workunit.client.0.smithi137.stdout:9/735: rename df/f69 to df/d12/d29/fe0 0 2024-06-15T01:07:24.186 INFO:tasks.workunit.client.0.smithi137.stdout:6/858: symlink d0/d1a/d3e/d57/dba/l10b 0 2024-06-15T01:07:24.188 INFO:tasks.workunit.client.0.smithi137.stdout:6/859: write d0/d1a/d3e/d57/f5e [4138722,18389] 0 2024-06-15T01:07:24.188 INFO:tasks.workunit.client.0.smithi137.stdout:2/870: creat d0/dc/d2e/d72/f112 x:0 0 0 2024-06-15T01:07:24.192 INFO:tasks.workunit.client.0.smithi137.stdout:8/838: creat d9/d2a/db9/dc6/d1e/d35/d116/f122 x:0 0 0 2024-06-15T01:07:24.193 INFO:tasks.workunit.client.0.smithi137.stdout:9/736: dwrite df/d12/d29/fa3 [0,4194304] 0 2024-06-15T01:07:24.194 INFO:tasks.workunit.client.0.smithi137.stdout:6/860: write d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f/ffb [781166,71230] 0 2024-06-15T01:07:24.199 INFO:tasks.workunit.client.0.smithi137.stdout:2/871: dwrite d0/da/da0/f2a [0,4194304] 0 2024-06-15T01:07:24.201 INFO:tasks.workunit.client.0.smithi137.stdout:2/872: stat d0/f1f 0 2024-06-15T01:07:24.203 INFO:tasks.workunit.client.0.smithi137.stdout:6/861: mkdir d0/d1a/d3e/d57/d1d/da5/d10c 0 2024-06-15T01:07:24.210 INFO:tasks.workunit.client.0.smithi137.stdout:9/737: dwrite df/d12/d29/d66/f86 [0,4194304] 0 2024-06-15T01:07:24.211 INFO:tasks.workunit.client.0.smithi137.stdout:2/873: link d0/dc/d2e/d72/d56/fa5 d0/de/d2b/de5/f113 0 2024-06-15T01:07:24.212 INFO:tasks.workunit.client.0.smithi137.stdout:8/839: dwrite d9/d2a/f69 [0,4194304] 0 2024-06-15T01:07:24.217 INFO:tasks.workunit.client.0.smithi137.stdout:2/874: rename d0/dc/d2e/l5d to d0/dc/d22/de8/l114 0 2024-06-15T01:07:24.218 INFO:tasks.workunit.client.0.smithi137.stdout:2/875: fsync d0/d1c/f70 0 2024-06-15T01:07:24.220 INFO:tasks.workunit.client.0.smithi137.stdout:8/840: getdents d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/ddb 0 2024-06-15T01:07:24.221 INFO:tasks.workunit.client.0.smithi137.stdout:2/876: creat d0/da/da0/d3a/dd4/d5c/d79/f115 x:0 0 0 2024-06-15T01:07:24.222 INFO:tasks.workunit.client.0.smithi137.stdout:8/841: dread - d9/d2a/db9/dc6/d1e/d67/f95 zero size 2024-06-15T01:07:24.224 INFO:tasks.workunit.client.0.smithi137.stdout:2/877: symlink d0/db9/l116 0 2024-06-15T01:07:24.227 INFO:tasks.workunit.client.0.smithi137.stdout:2/878: fdatasync d0/de/d2b/de5/f113 0 2024-06-15T01:07:24.228 INFO:tasks.workunit.client.0.smithi137.stdout:2/879: read d0/dc/d2e/fc0 [162553,66653] 0 2024-06-15T01:07:24.231 INFO:tasks.workunit.client.0.smithi137.stdout:2/880: creat d0/dc/d2e/d72/d56/d6e/f117 x:0 0 0 2024-06-15T01:07:24.242 INFO:tasks.workunit.client.0.smithi137.stdout:6/862: write d0/d5/d46/f9c [182484,122310] 0 2024-06-15T01:07:24.243 INFO:tasks.workunit.client.0.smithi137.stdout:9/738: write df/d49/faa [409228,110582] 0 2024-06-15T01:07:24.246 INFO:tasks.workunit.client.0.smithi137.stdout:6/863: symlink d0/d1a/d68/ddd/l10d 0 2024-06-15T01:07:24.248 INFO:tasks.workunit.client.0.smithi137.stdout:8/842: write d9/d55/f5c [1062862,93553] 0 2024-06-15T01:07:24.250 INFO:tasks.workunit.client.0.smithi137.stdout:8/843: chown d9/d2a/d79/d7e/cc4 431447710 1 2024-06-15T01:07:24.251 INFO:tasks.workunit.client.0.smithi137.stdout:9/739: link df/d12/d1b/d31/d42/l85 df/le1 0 2024-06-15T01:07:24.253 INFO:tasks.workunit.client.0.smithi137.stdout:8/844: creat d9/d2a/db9/dc6/d1e/d3f/dab/dd8/f123 x:0 0 0 2024-06-15T01:07:24.254 INFO:tasks.workunit.client.0.smithi137.stdout:9/740: mknod df/d2a/d7e/ce2 0 2024-06-15T01:07:24.255 INFO:tasks.workunit.client.0.smithi137.stdout:2/881: write d0/da/f18 [41984,120569] 0 2024-06-15T01:07:24.255 INFO:tasks.workunit.client.0.smithi137.stdout:6/864: write d0/f39 [4394761,16744] 0 2024-06-15T01:07:24.257 INFO:tasks.workunit.client.0.smithi137.stdout:9/741: unlink df/d2a/f4c 0 2024-06-15T01:07:24.259 INFO:tasks.workunit.client.0.smithi137.stdout:6/865: mknod d0/d1a/d3e/d57/db6/c10e 0 2024-06-15T01:07:24.260 INFO:tasks.workunit.client.0.smithi137.stdout:8/845: truncate d9/d2a/db9/dc6/d1e/d3f/dab/db7/fa7 2951657 0 2024-06-15T01:07:24.375 INFO:tasks.workunit.client.0.smithi137.stdout:2/882: link d0/da/da0/d3a/dd4/lc9 d0/da/da0/d3a/dd4/d5c/d79/l118 0 2024-06-15T01:07:24.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/866: creat d0/d1a/d3e/d57/d1d/d2e/d62/f10f x:0 0 0 2024-06-15T01:07:24.381 INFO:tasks.workunit.client.0.smithi137.stdout:9/742: dwrite df/d12/d29/d4a/f94 [0,4194304] 0 2024-06-15T01:07:24.381 INFO:tasks.workunit.client.0.smithi137.stdout:8/846: rename d9/d2a/db9/dc6/lda to d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/ddb/l124 0 2024-06-15T01:07:24.386 INFO:tasks.workunit.client.0.smithi137.stdout:9/743: mknod df/d12/d29/d66/dce/ce3 0 2024-06-15T01:07:24.388 INFO:tasks.workunit.client.0.smithi137.stdout:6/867: symlink d0/d1a/d3e/d57/d1d/d2e/l110 0 2024-06-15T01:07:24.392 INFO:tasks.workunit.client.0.smithi137.stdout:2/883: dwrite d0/dc/d2e/d72/d56/fcd [0,4194304] 0 2024-06-15T01:07:24.394 INFO:tasks.workunit.client.0.smithi137.stdout:2/884: stat d0/da/da0/d3a 0 2024-06-15T01:07:24.395 INFO:tasks.workunit.client.0.smithi137.stdout:6/868: write d0/d1a/d3e/d57/d32/d74/fd6 [169424,32587] 0 2024-06-15T01:07:24.395 INFO:tasks.workunit.client.0.smithi137.stdout:8/847: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/c125 0 2024-06-15T01:07:24.402 INFO:tasks.workunit.client.0.smithi137.stdout:2/885: mknod d0/dc/d22/deb/c119 0 2024-06-15T01:07:24.404 INFO:tasks.workunit.client.0.smithi137.stdout:6/869: getdents d0/d5/d46 0 2024-06-15T01:07:24.406 INFO:tasks.workunit.client.0.smithi137.stdout:6/870: chown d0/d5/d9/d29/d6b/c96 7933688 1 2024-06-15T01:07:24.406 INFO:tasks.workunit.client.0.smithi137.stdout:2/886: getdents d0/dc/d22/d96 0 2024-06-15T01:07:24.410 INFO:tasks.workunit.client.0.smithi137.stdout:6/871: write d0/d1a/d3e/f107 [109863,5144] 0 2024-06-15T01:07:24.411 INFO:tasks.workunit.client.0.smithi137.stdout:2/887: creat d0/da/da0/d3a/dd4/d5c/f11a x:0 0 0 2024-06-15T01:07:24.412 INFO:tasks.workunit.client.0.smithi137.stdout:6/872: rmdir d0/d1a/d99 39 2024-06-15T01:07:24.415 INFO:tasks.workunit.client.0.smithi137.stdout:8/848: dwrite d9/d2a/d2f/f72 [4194304,4194304] 0 2024-06-15T01:07:24.418 INFO:tasks.workunit.client.0.smithi137.stdout:8/849: chown d9/d2a/f8a 0 1 2024-06-15T01:07:24.418 INFO:tasks.workunit.client.0.smithi137.stdout:9/744: dread df/d12/d15/f5f [0,4194304] 0 2024-06-15T01:07:24.419 INFO:tasks.workunit.client.0.smithi137.stdout:8/850: read - d9/d2a/db9/dc6/d1e/d35/d102/f11d zero size 2024-06-15T01:07:24.421 INFO:tasks.workunit.client.0.smithi137.stdout:2/888: rename d0/dc/d22/d96 to d0/da/da0/d3a/dd4/d77/d11b 0 2024-06-15T01:07:24.422 INFO:tasks.workunit.client.0.smithi137.stdout:9/745: truncate df/d12/d29/d4a/f70 1112016 0 2024-06-15T01:07:24.424 INFO:tasks.workunit.client.0.smithi137.stdout:8/851: mknod d9/d2a/c126 0 2024-06-15T01:07:24.430 INFO:tasks.workunit.client.0.smithi137.stdout:8/852: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/c127 0 2024-06-15T01:07:24.432 INFO:tasks.workunit.client.0.smithi137.stdout:6/873: dread d0/f2a [0,4194304] 0 2024-06-15T01:07:24.434 INFO:tasks.workunit.client.0.smithi137.stdout:9/746: dwrite df/d12/d1b/f35 [0,4194304] 0 2024-06-15T01:07:24.438 INFO:tasks.workunit.client.0.smithi137.stdout:8/853: symlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/l128 0 2024-06-15T01:07:24.440 INFO:tasks.workunit.client.0.smithi137.stdout:2/889: dread d0/d1c/fa4 [0,4194304] 0 2024-06-15T01:07:24.442 INFO:tasks.workunit.client.0.smithi137.stdout:8/854: mknod d9/c129 0 2024-06-15T01:07:24.444 INFO:tasks.workunit.client.0.smithi137.stdout:6/874: write d0/d5/d9/fb1 [3675666,107866] 0 2024-06-15T01:07:24.445 INFO:tasks.workunit.client.0.smithi137.stdout:6/875: read - d0/d1a/d3e/d57/d32/d92/ff2 zero size 2024-06-15T01:07:24.446 INFO:tasks.workunit.client.0.smithi137.stdout:9/747: write df/d13/f71 [30809,36798] 0 2024-06-15T01:07:24.449 INFO:tasks.workunit.client.0.smithi137.stdout:8/855: creat d9/d2a/f12a x:0 0 0 2024-06-15T01:07:24.452 INFO:tasks.workunit.client.0.smithi137.stdout:9/748: creat df/fe4 x:0 0 0 2024-06-15T01:07:24.453 INFO:tasks.workunit.client.0.smithi137.stdout:6/876: dread d0/d1a/d3e/d57/f80 [0,4194304] 0 2024-06-15T01:07:24.455 INFO:tasks.workunit.client.0.smithi137.stdout:9/749: readlink df/d12/d29/d66/lbb 0 2024-06-15T01:07:24.457 INFO:tasks.workunit.client.0.smithi137.stdout:6/877: mknod d0/d5/d9/d29/d6b/c111 0 2024-06-15T01:07:24.460 INFO:tasks.workunit.client.0.smithi137.stdout:9/750: fsync df/d12/d15/f58 0 2024-06-15T01:07:24.460 INFO:tasks.workunit.client.0.smithi137.stdout:8/856: dwrite d9/d2a/db9/dc6/d1e/d3f/dab/dd8/f123 [0,4194304] 0 2024-06-15T01:07:24.462 INFO:tasks.workunit.client.0.smithi137.stdout:2/890: dwrite d0/da/d73/fb3 [0,4194304] 0 2024-06-15T01:07:24.465 INFO:tasks.workunit.client.0.smithi137.stdout:8/857: readlink d9/la 0 2024-06-15T01:07:24.465 INFO:tasks.workunit.client.0.smithi137.stdout:9/751: chown df/d12/d15/d75/lb8 6232 1 2024-06-15T01:07:24.470 INFO:tasks.workunit.client.0.smithi137.stdout:8/858: symlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/l12b 0 2024-06-15T01:07:24.472 INFO:tasks.workunit.client.0.smithi137.stdout:2/891: creat d0/dc/d2e/d72/d56/d6e/d104/f11c x:0 0 0 2024-06-15T01:07:24.473 INFO:tasks.workunit.client.0.smithi137.stdout:9/752: creat df/d13/dcb/dd4/fe5 x:0 0 0 2024-06-15T01:07:24.475 INFO:tasks.workunit.client.0.smithi137.stdout:8/859: dread d9/d2a/db9/dc6/d1e/f75 [0,4194304] 0 2024-06-15T01:07:24.476 INFO:tasks.workunit.client.0.smithi137.stdout:6/878: rmdir d0/d1a/d3e/d57/d32/d92/dac/d102 0 2024-06-15T01:07:24.480 INFO:tasks.workunit.client.0.smithi137.stdout:9/753: creat df/d49/d4b/d9b/daf/fe6 x:0 0 0 2024-06-15T01:07:24.480 INFO:tasks.workunit.client.0.smithi137.stdout:6/879: dread d0/d5/d9/d29/fcd [0,4194304] 0 2024-06-15T01:07:24.480 INFO:tasks.workunit.client.0.smithi137.stdout:8/860: mkdir d9/d2a/d79/d7e/d12c 0 2024-06-15T01:07:24.483 INFO:tasks.workunit.client.0.smithi137.stdout:8/861: chown d9/d2a/d2f/c65 10143 1 2024-06-15T01:07:24.485 INFO:tasks.workunit.client.0.smithi137.stdout:8/862: read d9/d2a/db9/dc6/d1e/d67/fa6 [2566032,36498] 0 2024-06-15T01:07:24.487 INFO:tasks.workunit.client.0.smithi137.stdout:8/863: write d9/d2a/db9/dc6/f3c [3587656,10535] 0 2024-06-15T01:07:24.496 INFO:tasks.workunit.client.0.smithi137.stdout:9/754: truncate df/d12/d15/f63 279478 0 2024-06-15T01:07:24.496 INFO:tasks.workunit.client.0.smithi137.stdout:8/864: dwrite d9/d2a/db9/dc6/d1e/d35/f56 [0,4194304] 0 2024-06-15T01:07:24.501 INFO:tasks.workunit.client.0.smithi137.stdout:2/892: dwrite d0/f6 [0,4194304] 0 2024-06-15T01:07:24.502 INFO:tasks.workunit.client.0.smithi137.stdout:9/755: truncate df/d12/d1b/f2f 4588230 0 2024-06-15T01:07:24.504 INFO:tasks.workunit.client.0.smithi137.stdout:2/893: write d0/d1c/f70 [1947683,95331] 0 2024-06-15T01:07:24.505 INFO:tasks.workunit.client.0.smithi137.stdout:6/880: dread d0/d1a/d3e/d57/d1d/d2e/fa7 [0,4194304] 0 2024-06-15T01:07:24.507 INFO:tasks.workunit.client.0.smithi137.stdout:7/751: sync 2024-06-15T01:07:24.509 INFO:tasks.workunit.client.0.smithi137.stdout:8/865: rename d9/d2a/db9/dc6/d1e/d3f/dab/f117 to d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/f12d 0 2024-06-15T01:07:24.512 INFO:tasks.workunit.client.0.smithi137.stdout:9/756: creat df/d12/d4e/dc6/fe7 x:0 0 0 2024-06-15T01:07:24.514 INFO:tasks.workunit.client.0.smithi137.stdout:2/894: mkdir d0/da/da0/d3a/dd4/d5c/d11d 0 2024-06-15T01:07:24.515 INFO:tasks.workunit.client.0.smithi137.stdout:6/881: symlink d0/d1a/d3e/d57/d32/d74/l112 0 2024-06-15T01:07:24.516 INFO:tasks.workunit.client.0.smithi137.stdout:6/882: chown d0/d1a/d3e/d57 15989899 1 2024-06-15T01:07:24.518 INFO:tasks.workunit.client.0.smithi137.stdout:2/895: rename d0/db9/l116 to d0/de/l11e 0 2024-06-15T01:07:24.520 INFO:tasks.workunit.client.0.smithi137.stdout:8/866: mkdir d9/d2a/db9/dc6/d1e/d12e 0 2024-06-15T01:07:24.522 INFO:tasks.workunit.client.0.smithi137.stdout:9/757: link df/d12/d1b/f74 df/d12/d29/d79/d9a/fe8 0 2024-06-15T01:07:24.524 INFO:tasks.workunit.client.0.smithi137.stdout:9/758: truncate df/d49/d4b/d9b/daf/fe6 909994 0 2024-06-15T01:07:24.525 INFO:tasks.workunit.client.0.smithi137.stdout:2/896: mkdir d0/da/da0/d3a/dd4/d11f 0 2024-06-15T01:07:24.528 INFO:tasks.workunit.client.0.smithi137.stdout:6/883: mkdir d0/d5/d108/d113 0 2024-06-15T01:07:24.531 INFO:tasks.workunit.client.0.smithi137.stdout:2/897: symlink d0/dc/d2e/d72/d56/de9/l120 0 2024-06-15T01:07:24.532 INFO:tasks.workunit.client.0.smithi137.stdout:8/867: link d9/d2a/d79/f7d d9/d2a/db9/dc6/d1e/d35/dc8/f12f 0 2024-06-15T01:07:24.534 INFO:tasks.workunit.client.0.smithi137.stdout:2/898: readlink d0/da/da0/d3a/l71 0 2024-06-15T01:07:24.536 INFO:tasks.workunit.client.0.smithi137.stdout:6/884: creat d0/d1a/d3e/d57/d1d/f114 x:0 0 0 2024-06-15T01:07:24.536 INFO:tasks.workunit.client.0.smithi137.stdout:7/752: dread df/d13/d2e/f51 [0,4194304] 0 2024-06-15T01:07:24.538 INFO:tasks.workunit.client.0.smithi137.stdout:2/899: creat d0/da/da0/d3a/dd4/d5c/d99/f121 x:0 0 0 2024-06-15T01:07:24.541 INFO:tasks.workunit.client.0.smithi137.stdout:6/885: unlink d0/d1a/d3e/d87/dd7/fe7 0 2024-06-15T01:07:24.542 INFO:tasks.workunit.client.0.smithi137.stdout:8/868: dread d9/d2a/db9/dc6/f13 [0,4194304] 0 2024-06-15T01:07:24.543 INFO:tasks.workunit.client.0.smithi137.stdout:6/886: chown d0/d1a/d68 13 1 2024-06-15T01:07:24.545 INFO:tasks.workunit.client.0.smithi137.stdout:2/900: fsync d0/da/da0/d3a/dd4/d5c/d79/f97 0 2024-06-15T01:07:24.545 INFO:tasks.workunit.client.0.smithi137.stdout:7/753: dwrite df/d7d/d1c/d1d/dc4/ff4 [0,4194304] 0 2024-06-15T01:07:24.547 INFO:tasks.workunit.client.0.smithi137.stdout:6/887: chown d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f/ffb 257675403 1 2024-06-15T01:07:24.547 INFO:tasks.workunit.client.0.smithi137.stdout:7/754: chown df/d13/f20 2279947 1 2024-06-15T01:07:24.549 INFO:tasks.workunit.client.0.smithi137.stdout:9/759: getdents df/d12/d29/d79/d9a 0 2024-06-15T01:07:24.550 INFO:tasks.workunit.client.0.smithi137.stdout:6/888: chown d0/d1a/d3e/d57/dba/fca 0 1 2024-06-15T01:07:24.550 INFO:tasks.workunit.client.0.smithi137.stdout:8/869: dread d9/d2a/db9/dc6/d1e/d3f/ff1 [0,4194304] 0 2024-06-15T01:07:24.559 INFO:tasks.workunit.client.0.smithi137.stdout:6/889: mknod d0/d1a/d3e/d87/c115 0 2024-06-15T01:07:24.562 INFO:tasks.workunit.client.0.smithi137.stdout:7/755: link df/d7d/d3b/d5b/caf df/d13/d2e/d1a/d6a/d90/c101 0 2024-06-15T01:07:24.566 INFO:tasks.workunit.client.0.smithi137.stdout:2/901: dread d0/dc/d22/fbe [0,4194304] 0 2024-06-15T01:07:24.567 INFO:tasks.workunit.client.0.smithi137.stdout:9/760: write df/d12/d15/f58 [36051,10959] 0 2024-06-15T01:07:24.569 INFO:tasks.workunit.client.0.smithi137.stdout:8/870: dwrite d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/f120 [4194304,4194304] 0 2024-06-15T01:07:24.572 INFO:tasks.workunit.client.0.smithi137.stdout:2/902: mkdir d0/de/d2b/de5/df1/d122 0 2024-06-15T01:07:24.581 INFO:tasks.workunit.client.0.smithi137.stdout:2/903: dwrite d0/da/da0/d3a/dd4/d5c/f11a [0,4194304] 0 2024-06-15T01:07:24.583 INFO:tasks.workunit.client.0.smithi137.stdout:6/890: dread d0/d5/f73 [0,4194304] 0 2024-06-15T01:07:24.586 INFO:tasks.workunit.client.0.smithi137.stdout:2/904: unlink d0/de/d2b/cbc 0 2024-06-15T01:07:24.589 INFO:tasks.workunit.client.0.smithi137.stdout:6/891: symlink d0/d1a/d3e/d57/d32/d92/d101/l116 0 2024-06-15T01:07:24.591 INFO:tasks.workunit.client.0.smithi137.stdout:9/761: dwrite df/d12/d29/d4a/f5d [0,4194304] 0 2024-06-15T01:07:24.599 INFO:tasks.workunit.client.0.smithi137.stdout:6/892: creat d0/d1a/d99/f117 x:0 0 0 2024-06-15T01:07:24.599 INFO:tasks.workunit.client.0.smithi137.stdout:7/756: dread df/d7d/d1c/d1d/d27/d33/d43/d6c/f37 [0,4194304] 0 2024-06-15T01:07:24.603 INFO:tasks.workunit.client.0.smithi137.stdout:8/871: dwrite d9/d2a/db9/dc6/fbd [0,4194304] 0 2024-06-15T01:07:24.607 INFO:tasks.workunit.client.0.smithi137.stdout:2/905: write d0/da/f11 [1973637,39741] 0 2024-06-15T01:07:24.614 INFO:tasks.workunit.client.0.smithi137.stdout:2/906: symlink d0/dc/d2e/d72/d8a/l123 0 2024-06-15T01:07:24.615 INFO:tasks.workunit.client.0.smithi137.stdout:2/907: dread - d0/da/da0/fe4 zero size 2024-06-15T01:07:24.615 INFO:tasks.workunit.client.0.smithi137.stdout:8/872: link d9/c4c d9/de9/c130 0 2024-06-15T01:07:24.615 INFO:tasks.workunit.client.0.smithi137.stdout:2/908: dread - d0/da/dda/ffa zero size 2024-06-15T01:07:24.618 INFO:tasks.workunit.client.0.smithi137.stdout:8/873: creat d9/d2a/db9/dc6/d1e/d35/d102/f131 x:0 0 0 2024-06-15T01:07:24.625 INFO:tasks.workunit.client.0.smithi137.stdout:8/874: write d9/d22/fa8 [1030192,112448] 0 2024-06-15T01:07:24.625 INFO:tasks.workunit.client.0.smithi137.stdout:2/909: write d0/da/da0/d3a/dd4/f6a [439146,68886] 0 2024-06-15T01:07:24.627 INFO:tasks.workunit.client.0.smithi137.stdout:7/757: dread df/d7d/d1c/f9c [0,4194304] 0 2024-06-15T01:07:24.627 INFO:tasks.workunit.client.0.smithi137.stdout:9/762: dread df/d12/d1b/f74 [0,4194304] 0 2024-06-15T01:07:24.629 INFO:tasks.workunit.client.0.smithi137.stdout:7/758: write df/d7d/d3b/d5b/f6d [2183804,43417] 0 2024-06-15T01:07:24.630 INFO:tasks.workunit.client.0.smithi137.stdout:8/875: truncate d9/d2a/db9/dc6/de1/f11c 90478 0 2024-06-15T01:07:24.630 INFO:tasks.workunit.client.0.smithi137.stdout:2/910: symlink d0/dc/d2e/d72/d56/de9/l124 0 2024-06-15T01:07:24.632 INFO:tasks.workunit.client.0.smithi137.stdout:7/759: mkdir df/d7d/d1c/d1d/d27/d33/d43/dc5/d102 0 2024-06-15T01:07:24.634 INFO:tasks.workunit.client.0.smithi137.stdout:2/911: mkdir d0/da/da0/d3a/dd4/d11f/d125 0 2024-06-15T01:07:24.635 INFO:tasks.workunit.client.0.smithi137.stdout:8/876: truncate d9/d2a/db9/dc6/d1e/d67/f6a 4314314 0 2024-06-15T01:07:24.636 INFO:tasks.workunit.client.0.smithi137.stdout:7/760: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/l103 0 2024-06-15T01:07:24.638 INFO:tasks.workunit.client.0.smithi137.stdout:2/912: unlink d0/da/da0/d3a/dd4/c89 0 2024-06-15T01:07:24.640 INFO:tasks.workunit.client.0.smithi137.stdout:2/913: truncate d0/da/dda/ffa 1009369 0 2024-06-15T01:07:24.644 INFO:tasks.workunit.client.0.smithi137.stdout:2/914: link d0/dc/d2e/d72/d56/d6e/fc4 d0/da/da0/f126 0 2024-06-15T01:07:24.646 INFO:tasks.workunit.client.0.smithi137.stdout:2/915: creat d0/de/d2b/de5/f127 x:0 0 0 2024-06-15T01:07:24.651 INFO:tasks.workunit.client.0.smithi137.stdout:2/916: dread d0/f6 [0,4194304] 0 2024-06-15T01:07:24.653 INFO:tasks.workunit.client.0.smithi137.stdout:2/917: stat d0/dc/d22/deb/d57/f9e 0 2024-06-15T01:07:24.653 INFO:tasks.workunit.client.0.smithi137.stdout:8/877: dwrite d9/d2a/db9/dc6/d1e/f42 [0,4194304] 0 2024-06-15T01:07:24.657 INFO:tasks.workunit.client.0.smithi137.stdout:2/918: mknod d0/da/da0/d3a/dd4/d11f/d125/c128 0 2024-06-15T01:07:24.658 INFO:tasks.workunit.client.0.smithi137.stdout:8/878: truncate f6 1726242 0 2024-06-15T01:07:24.660 INFO:tasks.workunit.client.0.smithi137.stdout:2/919: read d0/da/f11 [1921856,128291] 0 2024-06-15T01:07:24.667 INFO:tasks.workunit.client.0.smithi137.stdout:2/920: dread d0/dc/d2e/d72/fa1 [0,4194304] 0 2024-06-15T01:07:24.673 INFO:tasks.workunit.client.0.smithi137.stdout:2/921: link d0/l30 d0/dc/d2e/d72/d56/d6e/l129 0 2024-06-15T01:07:24.673 INFO:tasks.workunit.client.0.smithi137.stdout:8/879: write f8 [1150310,26266] 0 2024-06-15T01:07:24.675 INFO:tasks.workunit.client.0.smithi137.stdout:8/880: chown d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106 3641257 1 2024-06-15T01:07:24.675 INFO:tasks.workunit.client.0.smithi137.stdout:2/922: chown d0/dc/d22/deb/def/f10a 1369 1 2024-06-15T01:07:24.676 INFO:tasks.workunit.client.0.smithi137.stdout:2/923: fdatasync d0/f69 0 2024-06-15T01:07:24.676 INFO:tasks.workunit.client.0.smithi137.stdout:2/924: stat d0/db9 0 2024-06-15T01:07:24.677 INFO:tasks.workunit.client.0.smithi137.stdout:2/925: stat d0/dc/d22/le1 0 2024-06-15T01:07:24.680 INFO:tasks.workunit.client.0.smithi137.stdout:2/926: rename d0/da/da0/fe4 to d0/dc/dff/f12a 0 2024-06-15T01:07:24.683 INFO:tasks.workunit.client.0.smithi137.stdout:8/881: dread d9/d55/f5c [0,4194304] 0 2024-06-15T01:07:24.684 INFO:tasks.workunit.client.0.smithi137.stdout:2/927: rename d0/da/dd3/fe6 to d0/da/da0/d3a/dd4/d77/f12b 0 2024-06-15T01:07:24.685 INFO:tasks.workunit.client.0.smithi137.stdout:8/882: truncate d9/d2a/db9/dc6/d1e/d3f/f41 1292347 0 2024-06-15T01:07:24.686 INFO:tasks.workunit.client.0.smithi137.stdout:2/928: write d0/da/da0/d3a/dd4/d5c/d99/fc1 [124864,50190] 0 2024-06-15T01:07:24.687 INFO:tasks.workunit.client.0.smithi137.stdout:8/883: creat d9/d2a/f132 x:0 0 0 2024-06-15T01:07:24.688 INFO:tasks.workunit.client.0.smithi137.stdout:8/884: dread - d9/d2a/db9/dc6/d1e/d3f/d84/f86 zero size 2024-06-15T01:07:24.689 INFO:tasks.workunit.client.0.smithi137.stdout:2/929: dread d0/da/d73/f85 [0,4194304] 0 2024-06-15T01:07:24.690 INFO:tasks.workunit.client.0.smithi137.stdout:2/930: chown d0/dc/d22/deb/fcb 110744 1 2024-06-15T01:07:24.695 INFO:tasks.workunit.client.0.smithi137.stdout:8/885: dwrite d9/d2a/db9/dc6/fc2 [0,4194304] 0 2024-06-15T01:07:24.701 INFO:tasks.workunit.client.0.smithi137.stdout:8/886: dwrite d9/d2a/d2f/f72 [0,4194304] 0 2024-06-15T01:07:24.702 INFO:tasks.workunit.client.0.smithi137.stdout:8/887: stat d9/d2a/db9/dc6/d1e/d35/d102 0 2024-06-15T01:07:24.708 INFO:tasks.workunit.client.0.smithi137.stdout:8/888: symlink d9/d2a/d79/d7e/d12c/l133 0 2024-06-15T01:07:24.709 INFO:tasks.workunit.client.0.smithi137.stdout:8/889: chown d9/d2a/db9/dc6/d1e/f7f 19593 1 2024-06-15T01:07:24.715 INFO:tasks.workunit.client.0.smithi137.stdout:8/890: rmdir d9/d2a/db9/dc6/d1e 39 2024-06-15T01:07:24.719 INFO:tasks.workunit.client.0.smithi137.stdout:8/891: creat d9/d2a/db9/dc6/d1e/f134 x:0 0 0 2024-06-15T01:07:24.722 INFO:tasks.workunit.client.0.smithi137.stdout:8/892: creat d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/f135 x:0 0 0 2024-06-15T01:07:24.723 INFO:tasks.workunit.client.0.smithi137.stdout:8/893: readlink d9/d2a/db9/dc6/d1e/l1f 0 2024-06-15T01:07:24.729 INFO:tasks.workunit.client.0.smithi137.stdout:8/894: dwrite d9/d2a/db9/dc6/f18 [0,4194304] 0 2024-06-15T01:07:24.749 INFO:tasks.workunit.client.0.smithi137.stdout:8/895: dread d9/d2a/db9/dc6/d1e/d67/f6a [0,4194304] 0 2024-06-15T01:07:24.750 INFO:tasks.workunit.client.0.smithi137.stdout:8/896: stat d9/c1a 0 2024-06-15T01:07:24.764 INFO:tasks.workunit.client.0.smithi137.stdout:8/897: dread d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/f71 [0,4194304] 0 2024-06-15T01:07:24.769 INFO:tasks.workunit.client.0.smithi137.stdout:8/898: dread d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/ddb/fdd [0,4194304] 0 2024-06-15T01:07:24.775 INFO:tasks.workunit.client.0.smithi137.stdout:8/899: truncate d9/d2a/db9/dc6/d1e/d3f/dab/ff4 1473702 0 2024-06-15T01:07:24.776 INFO:tasks.workunit.client.0.smithi137.stdout:8/900: chown d9/d2a/d79/d7e/le2 32923437 1 2024-06-15T01:07:24.781 INFO:tasks.workunit.client.0.smithi137.stdout:8/901: write d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/fbc [1644193,47850] 0 2024-06-15T01:07:24.784 INFO:tasks.workunit.client.0.smithi137.stdout:8/902: symlink d9/de9/l136 0 2024-06-15T01:07:24.785 INFO:tasks.workunit.client.0.smithi137.stdout:8/903: mknod d9/d2a/db9/dc6/d1e/d3f/c137 0 2024-06-15T01:07:24.787 INFO:tasks.workunit.client.0.smithi137.stdout:8/904: mkdir d9/d2a/db9/dc6/d1e/d138 0 2024-06-15T01:07:24.788 INFO:tasks.workunit.client.0.smithi137.stdout:8/905: write d9/d2a/d79/d7e/f118 [426632,86122] 0 2024-06-15T01:07:24.793 INFO:tasks.workunit.client.0.smithi137.stdout:8/906: rename d9/d2a/f7a to d9/d2a/db9/dc6/d1e/f139 0 2024-06-15T01:07:24.796 INFO:tasks.workunit.client.0.smithi137.stdout:8/907: creat d9/d2a/db9/dc6/de1/f13a x:0 0 0 2024-06-15T01:07:24.807 INFO:tasks.workunit.client.0.smithi137.stdout:8/908: dwrite d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/fd5 [0,4194304] 0 2024-06-15T01:07:24.811 INFO:tasks.workunit.client.0.smithi137.stdout:8/909: rename d9/d2a/d2f/c62 to d9/d2a/db9/dc6/d1e/d67/c13b 0 2024-06-15T01:07:24.811 INFO:tasks.workunit.client.0.smithi137.stdout:8/910: chown d9/d22/f98 484 1 2024-06-15T01:07:24.812 INFO:tasks.workunit.client.0.smithi137.stdout:8/911: write d9/d2a/f8a [3690296,21676] 0 2024-06-15T01:07:24.813 INFO:tasks.workunit.client.0.smithi137.stdout:8/912: stat d9/d2a/le7 0 2024-06-15T01:07:24.818 INFO:tasks.workunit.client.0.smithi137.stdout:8/913: write d9/d2a/db9/dc6/d1e/f27 [3612964,87255] 0 2024-06-15T01:07:24.818 INFO:tasks.workunit.client.0.smithi137.stdout:8/914: mkdir d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/d13c 0 2024-06-15T01:07:24.820 INFO:tasks.workunit.client.0.smithi137.stdout:8/915: write d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/fcf [1786438,10041] 0 2024-06-15T01:07:24.822 INFO:tasks.workunit.client.0.smithi137.stdout:8/916: write d9/d2a/db9/dc6/f36 [1762444,48826] 0 2024-06-15T01:07:24.824 INFO:tasks.workunit.client.0.smithi137.stdout:8/917: write d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/f120 [1497700,40550] 0 2024-06-15T01:07:25.322 INFO:tasks.workunit.client.0.smithi137.stdout:6/893: sync 2024-06-15T01:07:25.323 INFO:tasks.workunit.client.0.smithi137.stdout:6/894: truncate d0/d1a/da0/fa1 1368561 0 2024-06-15T01:07:25.324 INFO:tasks.workunit.client.0.smithi137.stdout:6/895: dread - d0/d1a/d3e/d57/d1d/d2e/d62/f10f zero size 2024-06-15T01:07:25.326 INFO:tasks.workunit.client.0.smithi137.stdout:6/896: truncate d0/d1a/f2f 798928 0 2024-06-15T01:07:25.329 INFO:tasks.workunit.client.0.smithi137.stdout:6/897: creat d0/d5/d9/d29/f118 x:0 0 0 2024-06-15T01:07:25.332 INFO:tasks.workunit.client.0.smithi137.stdout:6/898: rename d0/d1a/d3e/d57/d1d/da5/c5c to d0/d1a/d3e/c119 0 2024-06-15T01:07:25.334 INFO:tasks.workunit.client.0.smithi137.stdout:6/899: symlink d0/d1a/d3e/d87/l11a 0 2024-06-15T01:07:25.336 INFO:tasks.workunit.client.0.smithi137.stdout:6/900: mkdir d0/d1a/d3e/d87/d11b 0 2024-06-15T01:07:25.337 INFO:tasks.workunit.client.0.smithi137.stdout:6/901: readlink d0/d1a/d3e/d57/d1d/d2e/l79 0 2024-06-15T01:07:25.357 INFO:tasks.workunit.client.0.smithi137.stdout:6/902: getdents d0/d5/d9/d29 0 2024-06-15T01:07:25.358 INFO:tasks.workunit.client.0.smithi137.stdout:6/903: rmdir d0/d1a/d3e/d57 39 2024-06-15T01:07:25.361 INFO:tasks.workunit.client.0.smithi137.stdout:6/904: creat d0/d1a/d3e/d57/d32/d92/dac/f11c x:0 0 0 2024-06-15T01:07:25.362 INFO:tasks.workunit.client.0.smithi137.stdout:6/905: readlink d0/d1a/d3e/d57/d1d/l6c 0 2024-06-15T01:07:25.364 INFO:tasks.workunit.client.0.smithi137.stdout:6/906: creat d0/d1a/d3e/f11d x:0 0 0 2024-06-15T01:07:25.365 INFO:tasks.workunit.client.0.smithi137.stdout:6/907: truncate d0/d1a/d3e/f107 666382 0 2024-06-15T01:07:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:6/908: dread d0/d5/f7 [0,4194304] 0 2024-06-15T01:07:25.378 INFO:tasks.workunit.client.0.smithi137.stdout:6/909: rmdir d0/d1a/d3e/d57/d32 39 2024-06-15T01:07:25.379 INFO:tasks.workunit.client.0.smithi137.stdout:6/910: write d0/ff3 [925630,36708] 0 2024-06-15T01:07:25.382 INFO:tasks.workunit.client.0.smithi137.stdout:6/911: write d0/d1a/d3e/d57/d32/d74/f61 [1623215,130628] 0 2024-06-15T01:07:25.384 INFO:tasks.workunit.client.0.smithi137.stdout:6/912: symlink d0/d1a/d68/ddd/l11e 0 2024-06-15T01:07:25.386 INFO:tasks.workunit.client.0.smithi137.stdout:6/913: symlink d0/d1a/d3e/d57/db6/l11f 0 2024-06-15T01:07:25.387 INFO:tasks.workunit.client.0.smithi137.stdout:6/914: fdatasync d0/d1a/d99/fd2 0 2024-06-15T01:07:25.387 INFO:tasks.workunit.client.0.smithi137.stdout:6/915: chown d0/d5/f73 23763 1 2024-06-15T01:07:25.392 INFO:tasks.workunit.client.0.smithi137.stdout:6/916: link d0/d5/d9/d29/f9a d0/d1a/d3e/d57/d32/d74/f120 0 2024-06-15T01:07:25.393 INFO:tasks.workunit.client.0.smithi137.stdout:6/917: dread d0/d5/d9/d29/fcd [0,4194304] 0 2024-06-15T01:07:25.394 INFO:tasks.workunit.client.0.smithi137.stdout:6/918: write d0/d1a/f75 [208276,127186] 0 2024-06-15T01:07:25.407 INFO:tasks.workunit.client.0.smithi137.stdout:6/919: dread d0/d5/fae [0,4194304] 0 2024-06-15T01:07:25.408 INFO:tasks.workunit.client.0.smithi137.stdout:6/920: mknod d0/d1a/d68/c121 0 2024-06-15T01:07:25.413 INFO:tasks.workunit.client.0.smithi137.stdout:6/921: write d0/d1a/f58 [4010057,121269] 0 2024-06-15T01:07:25.413 INFO:tasks.workunit.client.0.smithi137.stdout:8/918: sync 2024-06-15T01:07:25.414 INFO:tasks.workunit.client.0.smithi137.stdout:7/761: sync 2024-06-15T01:07:25.414 INFO:tasks.workunit.client.0.smithi137.stdout:9/763: sync 2024-06-15T01:07:25.414 INFO:tasks.workunit.client.0.smithi137.stdout:2/931: sync 2024-06-15T01:07:25.416 INFO:tasks.workunit.client.0.smithi137.stdout:9/764: creat df/d49/d8d/fe9 x:0 0 0 2024-06-15T01:07:25.417 INFO:tasks.workunit.client.0.smithi137.stdout:8/919: mknod d9/d2a/db9/dc6/d1e/d67/c13d 0 2024-06-15T01:07:25.419 INFO:tasks.workunit.client.0.smithi137.stdout:7/762: fsync df/d7d/d1c/d1d/d27/f48 0 2024-06-15T01:07:25.421 INFO:tasks.workunit.client.0.smithi137.stdout:9/765: creat df/d12/d29/d95/db3/fea x:0 0 0 2024-06-15T01:07:25.422 INFO:tasks.workunit.client.0.smithi137.stdout:9/766: write df/fe4 [433257,57772] 0 2024-06-15T01:07:25.431 INFO:tasks.workunit.client.0.smithi137.stdout:7/763: fdatasync df/d7d/d1c/d1d/d27/d33/d43/d6c/f96 0 2024-06-15T01:07:25.431 INFO:tasks.workunit.client.0.smithi137.stdout:9/767: mkdir df/d12/d4e/dc6/deb 0 2024-06-15T01:07:25.431 INFO:tasks.workunit.client.0.smithi137.stdout:6/922: truncate d0/d1a/da0/fa1 1054240 0 2024-06-15T01:07:25.432 INFO:tasks.workunit.client.0.smithi137.stdout:9/768: fdatasync df/d1a/da7/fdd 0 2024-06-15T01:07:25.432 INFO:tasks.workunit.client.0.smithi137.stdout:8/920: dwrite d9/d55/fec [0,4194304] 0 2024-06-15T01:07:25.432 INFO:tasks.workunit.client.0.smithi137.stdout:2/932: getdents d0/db9 0 2024-06-15T01:07:25.437 INFO:tasks.workunit.client.0.smithi137.stdout:6/923: mknod d0/c122 0 2024-06-15T01:07:25.440 INFO:tasks.workunit.client.0.smithi137.stdout:6/924: read d0/d1a/d3e/d57/d1d/d2e/d70/fcc [3632294,54318] 0 2024-06-15T01:07:25.443 INFO:tasks.workunit.client.0.smithi137.stdout:8/921: symlink d9/d2a/db9/dc6/d1e/d35/d116/l13e 0 2024-06-15T01:07:25.448 INFO:tasks.workunit.client.0.smithi137.stdout:7/764: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/de6/f104 x:0 0 0 2024-06-15T01:07:25.449 INFO:tasks.workunit.client.0.smithi137.stdout:8/922: unlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/fd5 0 2024-06-15T01:07:25.451 INFO:tasks.workunit.client.0.smithi137.stdout:8/923: fdatasync d9/d2a/db9/dc6/f36 0 2024-06-15T01:07:25.452 INFO:tasks.workunit.client.0.smithi137.stdout:2/933: write d0/dc/f3b [4078756,17868] 0 2024-06-15T01:07:25.453 INFO:tasks.workunit.client.0.smithi137.stdout:6/925: rename d0/d1a/d3e/d57/d1d/d2e/l110 to d0/d1a/d68/ddd/df7/l123 0 2024-06-15T01:07:25.455 INFO:tasks.workunit.client.0.smithi137.stdout:6/926: read d0/d5/d9/d29/f9a [304229,10876] 0 2024-06-15T01:07:25.457 INFO:tasks.workunit.client.0.smithi137.stdout:2/934: mknod d0/dc/d22/de8/c12c 0 2024-06-15T01:07:25.459 INFO:tasks.workunit.client.0.smithi137.stdout:8/924: chown d9/d2a/db9/dc6/fc2 23475150 1 2024-06-15T01:07:25.460 INFO:tasks.workunit.client.0.smithi137.stdout:9/769: dread df/d12/d1b/d31/f33 [0,4194304] 0 2024-06-15T01:07:25.461 INFO:tasks.workunit.client.0.smithi137.stdout:7/765: rename df/d13/d2e/d1a/d6a/d90/cbf to df/d7d/d1c/d1d/d27/d33/d43/dc5/c105 0 2024-06-15T01:07:25.462 INFO:tasks.workunit.client.0.smithi137.stdout:2/935: symlink d0/da/l12d 0 2024-06-15T01:07:25.464 INFO:tasks.workunit.client.0.smithi137.stdout:7/766: write df/d7d/d1c/fd9 [685774,11325] 0 2024-06-15T01:07:25.469 INFO:tasks.workunit.client.0.smithi137.stdout:2/936: creat d0/dc/d2e/d72/f12e x:0 0 0 2024-06-15T01:07:25.469 INFO:tasks.workunit.client.0.smithi137.stdout:6/927: dread d0/d5/d9/d3f/f5f [0,4194304] 0 2024-06-15T01:07:25.476 INFO:tasks.workunit.client.0.smithi137.stdout:2/937: rmdir d0/dc/d2e/d72/d56/d6e/d104 39 2024-06-15T01:07:25.476 INFO:tasks.workunit.client.0.smithi137.stdout:2/938: stat d0/dc/d2e/d72/cf4 0 2024-06-15T01:07:25.477 INFO:tasks.workunit.client.0.smithi137.stdout:2/939: dread - d0/de/d2b/de5/f127 zero size 2024-06-15T01:07:25.478 INFO:tasks.workunit.client.0.smithi137.stdout:6/928: truncate d0/d1a/d3e/d57/d1d/d2e/d70/fcc 1578293 0 2024-06-15T01:07:25.480 INFO:tasks.workunit.client.0.smithi137.stdout:6/929: chown d0/fa4 24506089 1 2024-06-15T01:07:25.481 INFO:tasks.workunit.client.0.smithi137.stdout:8/925: getdents d9/d2a/db9/dc6/d1e/d3f/dab/dd8 0 2024-06-15T01:07:25.482 INFO:tasks.workunit.client.0.smithi137.stdout:7/767: write df/fb4 [4317487,43315] 0 2024-06-15T01:07:25.483 INFO:tasks.workunit.client.0.smithi137.stdout:9/770: truncate df/d12/d29/d4a/f5d 4071371 0 2024-06-15T01:07:25.486 INFO:tasks.workunit.client.0.smithi137.stdout:2/940: dwrite d0/da/d73/fb3 [0,4194304] 0 2024-06-15T01:07:25.490 INFO:tasks.workunit.client.0.smithi137.stdout:8/926: dread d9/d2a/db9/dc6/d1e/f42 [0,4194304] 0 2024-06-15T01:07:25.491 INFO:tasks.workunit.client.0.smithi137.stdout:2/941: mkdir d0/da/da0/d3a/dd4/d5c/d99/d12f 0 2024-06-15T01:07:25.491 INFO:tasks.workunit.client.0.smithi137.stdout:6/930: truncate d0/d1a/f65 2243636 0 2024-06-15T01:07:25.494 INFO:tasks.workunit.client.0.smithi137.stdout:7/768: creat df/d7d/d1c/d1d/dd0/f106 x:0 0 0 2024-06-15T01:07:25.496 INFO:tasks.workunit.client.0.smithi137.stdout:9/771: truncate df/d12/d15/f63 22424 0 2024-06-15T01:07:25.499 INFO:tasks.workunit.client.0.smithi137.stdout:2/942: mknod d0/dc/d22/de8/c130 0 2024-06-15T01:07:25.499 INFO:tasks.workunit.client.0.smithi137.stdout:2/943: stat d0/dc/d22/f4b 0 2024-06-15T01:07:25.501 INFO:tasks.workunit.client.0.smithi137.stdout:6/931: chown d0/d5/d46/c85 11575960 1 2024-06-15T01:07:25.503 INFO:tasks.workunit.client.0.smithi137.stdout:7/769: rmdir df/d7d 39 2024-06-15T01:07:25.506 INFO:tasks.workunit.client.0.smithi137.stdout:8/927: truncate d9/d2a/db9/dc6/f94 507460 0 2024-06-15T01:07:25.507 INFO:tasks.workunit.client.0.smithi137.stdout:6/932: truncate d0/d5/f83 682561 0 2024-06-15T01:07:25.509 INFO:tasks.workunit.client.0.smithi137.stdout:6/933: read d0/d1a/d3e/d57/d1d/d2e/fa7 [277427,105328] 0 2024-06-15T01:07:25.511 INFO:tasks.workunit.client.0.smithi137.stdout:7/770: rename df/d13/d2e/d1a/d8c to df/d13/d2e/de1/d107 0 2024-06-15T01:07:25.515 INFO:tasks.workunit.client.0.smithi137.stdout:6/934: mknod d0/d1a/d68/ddd/df7/c124 0 2024-06-15T01:07:25.517 INFO:tasks.workunit.client.0.smithi137.stdout:9/772: write df/d49/d4b/f8c [279374,91337] 0 2024-06-15T01:07:25.519 INFO:tasks.workunit.client.0.smithi137.stdout:6/935: stat d0/d1a/d68/ddd/df7/f105 0 2024-06-15T01:07:25.522 INFO:tasks.workunit.client.0.smithi137.stdout:8/928: dwrite d9/d2a/d79/fef [0,4194304] 0 2024-06-15T01:07:25.522 INFO:tasks.workunit.client.0.smithi137.stdout:9/773: rmdir df/d12/d29/d79/d9a 39 2024-06-15T01:07:25.525 INFO:tasks.workunit.client.0.smithi137.stdout:2/944: dread d0/f1f [0,4194304] 0 2024-06-15T01:07:25.526 INFO:tasks.workunit.client.0.smithi137.stdout:8/929: truncate d9/d2a/db9/dc6/d1e/d35/f10c 403125 0 2024-06-15T01:07:25.527 INFO:tasks.workunit.client.0.smithi137.stdout:9/774: fdatasync df/d1a/f87 0 2024-06-15T01:07:25.530 INFO:tasks.workunit.client.0.smithi137.stdout:7/771: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/db0/f108 x:0 0 0 2024-06-15T01:07:25.532 INFO:tasks.workunit.client.0.smithi137.stdout:9/775: read df/d12/d1b/d31/f67 [3523585,55118] 0 2024-06-15T01:07:25.536 INFO:tasks.workunit.client.0.smithi137.stdout:6/936: getdents d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f 0 2024-06-15T01:07:25.536 INFO:tasks.workunit.client.0.smithi137.stdout:8/930: dwrite d9/d2a/db9/dc6/f18 [0,4194304] 0 2024-06-15T01:07:25.537 INFO:tasks.workunit.client.0.smithi137.stdout:6/937: fsync d0/fa4 0 2024-06-15T01:07:25.537 INFO:tasks.workunit.client.0.smithi137.stdout:9/776: creat df/d12/d4e/dc6/fec x:0 0 0 2024-06-15T01:07:25.539 INFO:tasks.workunit.client.0.smithi137.stdout:9/777: write df/d12/d15/fa9 [196287,45505] 0 2024-06-15T01:07:25.540 INFO:tasks.workunit.client.0.smithi137.stdout:7/772: rename df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/db0/fe7 to df/d13/df5/f109 0 2024-06-15T01:07:25.547 INFO:tasks.workunit.client.0.smithi137.stdout:8/931: chown d9/d2a/db9/dc6/c9e 1038405605 1 2024-06-15T01:07:25.549 INFO:tasks.workunit.client.0.smithi137.stdout:7/773: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/l10a 0 2024-06-15T01:07:25.550 INFO:tasks.workunit.client.0.smithi137.stdout:6/938: getdents d0/d1a/d3e/d87/d11b 0 2024-06-15T01:07:25.550 INFO:tasks.workunit.client.0.smithi137.stdout:2/945: dwrite d0/dc/d2e/d72/fca [0,4194304] 0 2024-06-15T01:07:25.553 INFO:tasks.workunit.client.0.smithi137.stdout:9/778: mknod df/d12/d4e/dc6/deb/ced 0 2024-06-15T01:07:25.555 INFO:tasks.workunit.client.0.smithi137.stdout:8/932: rmdir d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/ddb 39 2024-06-15T01:07:25.556 INFO:tasks.workunit.client.0.smithi137.stdout:2/946: chown d0/da/da0/d3a/dd4/d5c/d90 29744 1 2024-06-15T01:07:25.558 INFO:tasks.workunit.client.0.smithi137.stdout:9/779: symlink df/d12/d4e/dbc/d3c/lee 0 2024-06-15T01:07:25.559 INFO:tasks.workunit.client.0.smithi137.stdout:2/947: fdatasync d0/dc/d22/deb/fcf 0 2024-06-15T01:07:25.561 INFO:tasks.workunit.client.0.smithi137.stdout:8/933: creat d9/d2a/db9/dc6/d1e/d35/d102/f13f x:0 0 0 2024-06-15T01:07:25.562 INFO:tasks.workunit.client.0.smithi137.stdout:6/939: truncate d0/d1a/d3e/d57/d32/d74/f120 973172 0 2024-06-15T01:07:25.564 INFO:tasks.workunit.client.0.smithi137.stdout:2/948: truncate d0/fb 296892 0 2024-06-15T01:07:25.566 INFO:tasks.workunit.client.0.smithi137.stdout:6/940: fsync d0/d5/f7 0 2024-06-15T01:07:25.568 INFO:tasks.workunit.client.0.smithi137.stdout:2/949: symlink d0/de/d2b/de5/df1/l131 0 2024-06-15T01:07:25.571 INFO:tasks.workunit.client.0.smithi137.stdout:8/934: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/d13c/c140 0 2024-06-15T01:07:25.574 INFO:tasks.workunit.client.0.smithi137.stdout:6/941: unlink d0/d1a/d68/ddd/df7/f105 0 2024-06-15T01:07:25.575 INFO:tasks.workunit.client.0.smithi137.stdout:6/942: chown d0/d5/d108 800 1 2024-06-15T01:07:25.575 INFO:tasks.workunit.client.0.smithi137.stdout:9/780: getdents df/d12/d4e/dbc 0 2024-06-15T01:07:25.577 INFO:tasks.workunit.client.0.smithi137.stdout:6/943: chown d0/d5/fae 1398088 1 2024-06-15T01:07:25.578 INFO:tasks.workunit.client.0.smithi137.stdout:6/944: write d0/d5/d46/f9c [1298160,10091] 0 2024-06-15T01:07:25.579 INFO:tasks.workunit.client.0.smithi137.stdout:9/781: dread df/d49/faa [0,4194304] 0 2024-06-15T01:07:25.580 INFO:tasks.workunit.client.0.smithi137.stdout:8/935: mknod d9/d2a/db9/dc6/d1e/d3f/dab/dd8/c141 0 2024-06-15T01:07:25.583 INFO:tasks.workunit.client.0.smithi137.stdout:2/950: fsync d0/de/d2b/f106 0 2024-06-15T01:07:25.583 INFO:tasks.workunit.client.0.smithi137.stdout:7/774: dread df/fb4 [0,4194304] 0 2024-06-15T01:07:25.584 INFO:tasks.workunit.client.0.smithi137.stdout:2/951: stat d0/dc/d2e/d72/c6d 0 2024-06-15T01:07:25.584 INFO:tasks.workunit.client.0.smithi137.stdout:7/775: chown df/d13/d2e/d1a/f40 49979319 1 2024-06-15T01:07:25.586 INFO:tasks.workunit.client.0.smithi137.stdout:9/782: rmdir df/d12/d29/d66 39 2024-06-15T01:07:25.586 INFO:tasks.workunit.client.0.smithi137.stdout:2/952: read d0/dc/d22/deb/d57/fbf [6527108,83109] 0 2024-06-15T01:07:25.587 INFO:tasks.workunit.client.0.smithi137.stdout:9/783: chown df/d2a/fb4 587 1 2024-06-15T01:07:25.593 INFO:tasks.workunit.client.0.smithi137.stdout:7/776: mkdir df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b 0 2024-06-15T01:07:25.596 INFO:tasks.workunit.client.0.smithi137.stdout:6/945: symlink d0/d1a/d3e/d87/de0/d109/l125 0 2024-06-15T01:07:25.599 INFO:tasks.workunit.client.0.smithi137.stdout:6/946: chown d0/d1a/d3e/d57/d1d/d2e/d62/d84/d8f/ffb 1996882961 1 2024-06-15T01:07:25.600 INFO:tasks.workunit.client.0.smithi137.stdout:7/777: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/d10c 0 2024-06-15T01:07:25.601 INFO:tasks.workunit.client.0.smithi137.stdout:2/953: truncate d0/f1 154782 0 2024-06-15T01:07:25.602 INFO:tasks.workunit.client.0.smithi137.stdout:9/784: symlink df/d12/d29/d9d/lef 0 2024-06-15T01:07:25.602 INFO:tasks.workunit.client.0.smithi137.stdout:7/778: readlink df/d7d/d1c/d1d/d27/d33/d43/d6c/l10a 0 2024-06-15T01:07:25.603 INFO:tasks.workunit.client.0.smithi137.stdout:8/936: creat d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/f142 x:0 0 0 2024-06-15T01:07:25.604 INFO:tasks.workunit.client.0.smithi137.stdout:6/947: symlink d0/d5/d9/db9/l126 0 2024-06-15T01:07:25.605 INFO:tasks.workunit.client.0.smithi137.stdout:2/954: symlink d0/dc/d22/de8/l132 0 2024-06-15T01:07:25.609 INFO:tasks.workunit.client.0.smithi137.stdout:7/779: unlink df/d13/c15 0 2024-06-15T01:07:25.612 INFO:tasks.workunit.client.0.smithi137.stdout:2/955: creat d0/de/f133 x:0 0 0 2024-06-15T01:07:25.612 INFO:tasks.workunit.client.0.smithi137.stdout:9/785: dwrite df/d12/d15/d75/f8a [0,4194304] 0 2024-06-15T01:07:25.616 INFO:tasks.workunit.client.0.smithi137.stdout:2/956: stat d0/da/da0/d3a/dd4/d5c/d11d 0 2024-06-15T01:07:25.617 INFO:tasks.workunit.client.0.smithi137.stdout:6/948: dwrite d0/d1a/d3e/d57/d1d/d2e/d62/fc2 [0,4194304] 0 2024-06-15T01:07:25.622 INFO:tasks.workunit.client.0.smithi137.stdout:2/957: dread - d0/da/da0/d3a/dd4/d5c/dfd/f10e zero size 2024-06-15T01:07:25.622 INFO:tasks.workunit.client.0.smithi137.stdout:7/780: mknod df/d7d/d1c/d1d/d79/dbc/c10d 0 2024-06-15T01:07:25.624 INFO:tasks.workunit.client.0.smithi137.stdout:9/786: dread df/d12/d29/d4a/f94 [0,4194304] 0 2024-06-15T01:07:25.627 INFO:tasks.workunit.client.0.smithi137.stdout:2/958: creat d0/dc/d2e/f134 x:0 0 0 2024-06-15T01:07:25.629 INFO:tasks.workunit.client.0.smithi137.stdout:8/937: write d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/f12d [790531,618] 0 2024-06-15T01:07:25.630 INFO:tasks.workunit.client.0.smithi137.stdout:6/949: read d0/d5/d9/f41 [172759,78172] 0 2024-06-15T01:07:25.631 INFO:tasks.workunit.client.0.smithi137.stdout:7/781: unlink df/d7d/d3b/d5b/da6/fd2 0 2024-06-15T01:07:25.634 INFO:tasks.workunit.client.0.smithi137.stdout:9/787: dwrite df/d12/d29/f61 [0,4194304] 0 2024-06-15T01:07:25.636 INFO:tasks.workunit.client.0.smithi137.stdout:6/950: read - d0/d1a/d3e/d57/d1d/da5/fd9 zero size 2024-06-15T01:07:25.638 INFO:tasks.workunit.client.0.smithi137.stdout:8/938: mknod d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/c143 0 2024-06-15T01:07:25.644 INFO:tasks.workunit.client.0.smithi137.stdout:7/782: link df/d7d/d3b/f5f df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/f10e 0 2024-06-15T01:07:25.647 INFO:tasks.workunit.client.0.smithi137.stdout:7/783: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fe8 1 1 2024-06-15T01:07:25.647 INFO:tasks.workunit.client.0.smithi137.stdout:9/788: rename df/d12/d4e/dc6/ld6 to df/d12/d29/d79/lf0 0 2024-06-15T01:07:25.648 INFO:tasks.workunit.client.0.smithi137.stdout:8/939: getdents d9/d22 0 2024-06-15T01:07:25.650 INFO:tasks.workunit.client.0.smithi137.stdout:9/789: chown ce 126370 1 2024-06-15T01:07:25.653 INFO:tasks.workunit.client.0.smithi137.stdout:6/951: write d0/d1a/d3e/d57/d32/d74/f78 [4195343,38915] 0 2024-06-15T01:07:25.656 INFO:tasks.workunit.client.0.smithi137.stdout:9/790: mknod df/d13/dcb/dd4/cf1 0 2024-06-15T01:07:25.657 INFO:tasks.workunit.client.0.smithi137.stdout:2/959: dwrite d0/dc/d22/deb/f62 [0,4194304] 0 2024-06-15T01:07:25.657 INFO:tasks.workunit.client.0.smithi137.stdout:6/952: read d0/d5/fae [1122400,121326] 0 2024-06-15T01:07:25.658 INFO:tasks.workunit.client.0.smithi137.stdout:7/784: rename df/d13/d2e/d1a/f40 to df/d7d/d1c/f10f 0 2024-06-15T01:07:25.660 INFO:tasks.workunit.client.0.smithi137.stdout:6/953: dread - d0/d1a/d3e/d57/d1d/f114 zero size 2024-06-15T01:07:25.661 INFO:tasks.workunit.client.0.smithi137.stdout:9/791: truncate df/d12/d4e/dbc/f92 4611842 0 2024-06-15T01:07:25.663 INFO:tasks.workunit.client.0.smithi137.stdout:2/960: symlink d0/da/l135 0 2024-06-15T01:07:25.664 INFO:tasks.workunit.client.0.smithi137.stdout:6/954: creat d0/d1a/da0/f127 x:0 0 0 2024-06-15T01:07:25.667 INFO:tasks.workunit.client.0.smithi137.stdout:7/785: rename df/d7d/d1c/d1d/d27/d33/d43/dc5/d102 to df/d13/d2e/d1a/d6a/d90/ddf/d110 0 2024-06-15T01:07:25.668 INFO:tasks.workunit.client.0.smithi137.stdout:6/955: write d0/d5/d9/d3f/f67 [3302097,48289] 0 2024-06-15T01:07:25.670 INFO:tasks.workunit.client.0.smithi137.stdout:9/792: creat df/d12/d15/d75/ff2 x:0 0 0 2024-06-15T01:07:25.670 INFO:tasks.workunit.client.0.smithi137.stdout:8/940: dwrite d9/d2a/d79/f7d [0,4194304] 0 2024-06-15T01:07:25.672 INFO:tasks.workunit.client.0.smithi137.stdout:6/956: fsync d0/d5/d46/ffe 0 2024-06-15T01:07:25.677 INFO:tasks.workunit.client.0.smithi137.stdout:8/941: stat d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/l109 0 2024-06-15T01:07:25.679 INFO:tasks.workunit.client.0.smithi137.stdout:7/786: dwrite df/d7d/d1c/fd9 [0,4194304] 0 2024-06-15T01:07:25.679 INFO:tasks.workunit.client.0.smithi137.stdout:9/793: chown df/f6b 20 1 2024-06-15T01:07:25.680 INFO:tasks.workunit.client.0.smithi137.stdout:6/957: rmdir d0/d1a/d3e/d57/d32/d74 39 2024-06-15T01:07:25.680 INFO:tasks.workunit.client.0.smithi137.stdout:8/942: fsync d9/d2a/d2f/f72 0 2024-06-15T01:07:25.682 INFO:tasks.workunit.client.0.smithi137.stdout:7/787: read - df/d13/d2e/d1a/fa4 zero size 2024-06-15T01:07:25.685 INFO:tasks.workunit.client.0.smithi137.stdout:9/794: read df/d12/d29/d4a/f94 [4177390,62560] 0 2024-06-15T01:07:25.688 INFO:tasks.workunit.client.0.smithi137.stdout:8/943: symlink d9/d2a/db9/dc6/d1e/d12e/l144 0 2024-06-15T01:07:25.689 INFO:tasks.workunit.client.0.smithi137.stdout:9/795: dread - df/d12/d4e/dbc/fd1 zero size 2024-06-15T01:07:25.689 INFO:tasks.workunit.client.0.smithi137.stdout:7/788: creat df/d13/d2e/f111 x:0 0 0 2024-06-15T01:07:25.690 INFO:tasks.workunit.client.0.smithi137.stdout:6/958: getdents d0/d5/d9/db9 0 2024-06-15T01:07:25.692 INFO:tasks.workunit.client.0.smithi137.stdout:7/789: dread - df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb zero size 2024-06-15T01:07:25.693 INFO:tasks.workunit.client.0.smithi137.stdout:6/959: mknod d0/d1a/d99/c128 0 2024-06-15T01:07:25.694 INFO:tasks.workunit.client.0.smithi137.stdout:9/796: truncate df/d12/d15/f63 51753 0 2024-06-15T01:07:25.694 INFO:tasks.workunit.client.0.smithi137.stdout:6/960: readlink d0/d1a/d3e/d57/dba/l10b 0 2024-06-15T01:07:25.694 INFO:tasks.workunit.client.0.smithi137.stdout:8/944: getdents d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/d13c 0 2024-06-15T01:07:25.695 INFO:tasks.workunit.client.0.smithi137.stdout:9/797: chown df/c97 3 1 2024-06-15T01:07:25.696 INFO:tasks.workunit.client.0.smithi137.stdout:7/790: write df/d13/d2e/d1a/fa4 [725336,88369] 0 2024-06-15T01:07:25.697 INFO:tasks.workunit.client.0.smithi137.stdout:6/961: read - d0/d1a/d3e/d57/d1d/da5/fd9 zero size 2024-06-15T01:07:25.701 INFO:tasks.workunit.client.0.smithi137.stdout:7/791: symlink df/d7d/d3b/dd1/l112 0 2024-06-15T01:07:25.702 INFO:tasks.workunit.client.0.smithi137.stdout:7/792: chown df/d7d/d1c/d1d/d27/f49 123 1 2024-06-15T01:07:25.703 INFO:tasks.workunit.client.0.smithi137.stdout:6/962: creat d0/d5/d108/f129 x:0 0 0 2024-06-15T01:07:25.703 INFO:tasks.workunit.client.0.smithi137.stdout:8/945: dwrite d9/de9/feb [0,4194304] 0 2024-06-15T01:07:25.705 INFO:tasks.workunit.client.0.smithi137.stdout:9/798: write df/d1a/f87 [998416,63792] 0 2024-06-15T01:07:25.705 INFO:tasks.workunit.client.0.smithi137.stdout:7/793: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fd7 63744673 1 2024-06-15T01:07:25.710 INFO:tasks.workunit.client.0.smithi137.stdout:8/946: write d9/d2a/db9/dc6/d1e/f75 [438339,48725] 0 2024-06-15T01:07:25.714 INFO:tasks.workunit.client.0.smithi137.stdout:7/794: dwrite df/d7d/d3b/d5b/f5d [0,4194304] 0 2024-06-15T01:07:25.720 INFO:tasks.workunit.client.0.smithi137.stdout:7/795: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/d10c/f113 x:0 0 0 2024-06-15T01:07:25.723 INFO:tasks.workunit.client.0.smithi137.stdout:7/796: truncate df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/de6/f104 202692 0 2024-06-15T01:07:25.725 INFO:tasks.workunit.client.0.smithi137.stdout:8/947: truncate d9/d2a/db9/dc6/d1e/d3f/f68 1857173 0 2024-06-15T01:07:25.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:25 smithi137 ceph-mon[29354]: pgmap v28: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 66 MiB/s rd, 141 MiB/s wr, 380 op/s 2024-06-15T01:07:25.736 INFO:tasks.workunit.client.0.smithi137.stdout:7/797: truncate df/d7d/d3b/d5b/f84 752439 0 2024-06-15T01:07:25.740 INFO:tasks.workunit.client.0.smithi137.stdout:9/799: dread df/d12/d15/f26 [4194304,4194304] 0 2024-06-15T01:07:25.741 INFO:tasks.workunit.client.0.smithi137.stdout:7/798: getdents df/d7d/d3b/d5b 0 2024-06-15T01:07:25.742 INFO:tasks.workunit.client.0.smithi137.stdout:7/799: fsync df/d13/d2e/d1a/fa4 0 2024-06-15T01:07:25.744 INFO:tasks.workunit.client.0.smithi137.stdout:7/800: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/lbd 12929 1 2024-06-15T01:07:25.745 INFO:tasks.workunit.client.0.smithi137.stdout:9/800: getdents df/d49/d72 0 2024-06-15T01:07:25.746 INFO:tasks.workunit.client.0.smithi137.stdout:9/801: chown df/d13/dcb/dd4 19944 1 2024-06-15T01:07:25.746 INFO:tasks.workunit.client.0.smithi137.stdout:8/948: dread d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/f12d [0,4194304] 0 2024-06-15T01:07:25.747 INFO:tasks.workunit.client.0.smithi137.stdout:7/801: unlink df/d13/d2e/d1a/fa4 0 2024-06-15T01:07:25.748 INFO:tasks.workunit.client.0.smithi137.stdout:8/949: chown d9/d2a/db9/dc6/d1e/d67/c13d 18545 1 2024-06-15T01:07:25.751 INFO:tasks.workunit.client.0.smithi137.stdout:7/802: chown df/d7d/d1c/d1d/d27/c83 1752 1 2024-06-15T01:07:25.754 INFO:tasks.workunit.client.0.smithi137.stdout:8/950: mkdir d9/d2a/db9/dc6/d1e/d3f/d84/d145 0 2024-06-15T01:07:25.755 INFO:tasks.workunit.client.0.smithi137.stdout:7/803: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d114 0 2024-06-15T01:07:25.758 INFO:tasks.workunit.client.0.smithi137.stdout:7/804: symlink df/d7d/d1c/d1d/d79/dbc/df3/l115 0 2024-06-15T01:07:25.761 INFO:tasks.workunit.client.0.smithi137.stdout:7/805: truncate df/d13/d2e/d1a/d6a/fd6 1015080 0 2024-06-15T01:07:25.761 INFO:tasks.workunit.client.0.smithi137.stdout:8/951: write d9/d2a/db9/dc6/d1e/d3f/ff9 [405889,32511] 0 2024-06-15T01:07:25.766 INFO:tasks.workunit.client.0.smithi137.stdout:9/802: dread f5 [0,4194304] 0 2024-06-15T01:07:25.769 INFO:tasks.workunit.client.0.smithi137.stdout:9/803: readlink df/d13/d6f/ldf 0 2024-06-15T01:07:25.771 INFO:tasks.workunit.client.0.smithi137.stdout:8/952: dwrite d9/d2a/db9/dc6/d1e/d35/d102/f13f [0,4194304] 0 2024-06-15T01:07:25.773 INFO:tasks.workunit.client.0.smithi137.stdout:8/953: read d9/d2a/db9/dc6/d1e/f27 [1393781,59627] 0 2024-06-15T01:07:25.776 INFO:tasks.workunit.client.0.smithi137.stdout:8/954: dread - d9/d2a/db9/dc6/ffb zero size 2024-06-15T01:07:25.781 INFO:tasks.workunit.client.0.smithi137.stdout:8/955: creat d9/d2a/db9/dc6/d1e/d3f/d84/f146 x:0 0 0 2024-06-15T01:07:25.785 INFO:tasks.workunit.client.0.smithi137.stdout:8/956: dread d9/d2a/f2d [4194304,4194304] 0 2024-06-15T01:07:25.787 INFO:tasks.workunit.client.0.smithi137.stdout:9/804: dread df/d12/d1b/d31/fd2 [0,4194304] 0 2024-06-15T01:07:25.787 INFO:tasks.workunit.client.0.smithi137.stdout:8/957: rmdir d9/d2a/d79/d7e/d12c 39 2024-06-15T01:07:25.788 INFO:tasks.workunit.client.0.smithi137.stdout:7/806: dread df/d13/d2e/d1a/fae [0,4194304] 0 2024-06-15T01:07:25.789 INFO:tasks.workunit.client.0.smithi137.stdout:8/958: chown d9/d22/fe5 392531 1 2024-06-15T01:07:25.789 INFO:tasks.workunit.client.0.smithi137.stdout:8/959: readlink d9/d2a/l11e 0 2024-06-15T01:07:25.790 INFO:tasks.workunit.client.0.smithi137.stdout:9/805: mkdir df/d12/d1b/df3 0 2024-06-15T01:07:25.791 INFO:tasks.workunit.client.0.smithi137.stdout:9/806: dread - df/d13/dcb/dd4/fe5 zero size 2024-06-15T01:07:25.792 INFO:tasks.workunit.client.0.smithi137.stdout:7/807: mkdir df/d13/d2e/d1a/d6a/d116 0 2024-06-15T01:07:25.794 INFO:tasks.workunit.client.0.smithi137.stdout:8/960: mkdir d9/d2a/db9/dc6/d1e/d35/dfd/d147 0 2024-06-15T01:07:25.794 INFO:tasks.workunit.client.0.smithi137.stdout:9/807: mknod df/d12/d29/d9d/cf4 0 2024-06-15T01:07:25.794 INFO:tasks.workunit.client.0.smithi137.stdout:8/961: chown d9/d2a/db9/dc6/d1e/l107 2934 1 2024-06-15T01:07:25.797 INFO:tasks.workunit.client.0.smithi137.stdout:9/808: creat df/d49/ff5 x:0 0 0 2024-06-15T01:07:25.798 INFO:tasks.workunit.client.0.smithi137.stdout:9/809: write df/d12/d4e/dbc/f92 [2838416,56681] 0 2024-06-15T01:07:25.805 INFO:tasks.workunit.client.0.smithi137.stdout:8/962: write d9/d2a/db9/dc6/d1e/fd4 [118179,14507] 0 2024-06-15T01:07:25.807 INFO:tasks.workunit.client.0.smithi137.stdout:9/810: dwrite df/d12/d4e/dbc/f92 [0,4194304] 0 2024-06-15T01:07:25.808 INFO:tasks.workunit.client.0.smithi137.stdout:8/963: rmdir d9/d2a/db9/dc6/d1e/d67 39 2024-06-15T01:07:25.808 INFO:tasks.workunit.client.0.smithi137.stdout:7/808: dread df/d13/d2e/d1a/f68 [0,4194304] 0 2024-06-15T01:07:25.810 INFO:tasks.workunit.client.0.smithi137.stdout:8/964: write f6 [1723980,111930] 0 2024-06-15T01:07:25.811 INFO:tasks.workunit.client.0.smithi137.stdout:7/809: rmdir df/d13/d2e/de1/d107 39 2024-06-15T01:07:25.814 INFO:tasks.workunit.client.0.smithi137.stdout:9/811: getdents df/d2a/d7e 0 2024-06-15T01:07:25.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:25 smithi142 ceph-mon[37379]: pgmap v28: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 66 MiB/s rd, 141 MiB/s wr, 380 op/s 2024-06-15T01:07:25.818 INFO:tasks.workunit.client.0.smithi137.stdout:8/965: link d9/d2a/c126 d9/d2a/db9/dc6/d1e/d3f/d84/c148 0 2024-06-15T01:07:25.824 INFO:tasks.workunit.client.0.smithi137.stdout:7/810: dwrite df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fe8 [0,4194304] 0 2024-06-15T01:07:25.982 INFO:tasks.workunit.client.0.smithi137.stdout:7/811: dwrite df/d13/d2e/d1a/d6a/dde/d89/fc8 [0,4194304] 0 2024-06-15T01:07:25.983 INFO:tasks.workunit.client.0.smithi137.stdout:8/966: dwrite d9/d2a/db9/fca [0,4194304] 0 2024-06-15T01:07:25.991 INFO:tasks.workunit.client.0.smithi137.stdout:8/967: dwrite d9/d2a/d79/fef [0,4194304] 0 2024-06-15T01:07:26.033 INFO:tasks.workunit.client.0.smithi137.stdout:8/968: chown d9/d2a/db9/dc6/f94 4428091 1 2024-06-15T01:07:26.034 INFO:tasks.workunit.client.0.smithi137.stdout:7/812: symlink df/d7d/d1c/d1d/d27/l117 0 2024-06-15T01:07:26.034 INFO:tasks.workunit.client.0.smithi137.stdout:8/969: write d9/d2a/d2f/f72 [7789072,44566] 0 2024-06-15T01:07:26.038 INFO:tasks.workunit.client.0.smithi137.stdout:7/813: read df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/fd7 [565152,18701] 0 2024-06-15T01:07:26.040 INFO:tasks.workunit.client.0.smithi137.stdout:8/970: dread d9/d2a/db9/fca [0,4194304] 0 2024-06-15T01:07:26.046 INFO:tasks.workunit.client.0.smithi137.stdout:7/814: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/c118 0 2024-06-15T01:07:26.072 INFO:tasks.workunit.client.0.smithi137.stdout:8/971: dread d9/d2a/db9/dc6/f7b [0,4194304] 0 2024-06-15T01:07:26.075 INFO:tasks.workunit.client.0.smithi137.stdout:8/972: fsync d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/d82/f12d 0 2024-06-15T01:07:26.079 INFO:tasks.workunit.client.0.smithi137.stdout:8/973: mkdir d9/d2a/db9/dc6/d1e/d3f/dab/db7/d149 0 2024-06-15T01:07:26.082 INFO:tasks.workunit.client.0.smithi137.stdout:8/974: creat d9/d2a/db9/dc6/d1e/d3f/d84/d145/f14a x:0 0 0 2024-06-15T01:07:26.084 INFO:tasks.workunit.client.0.smithi137.stdout:8/975: readlink d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/l8b 0 2024-06-15T01:07:26.086 INFO:tasks.workunit.client.0.smithi137.stdout:8/976: stat d9/d2a/db9/dc6/c14 0 2024-06-15T01:07:26.088 INFO:tasks.workunit.client.0.smithi137.stdout:8/977: symlink d9/d2a/db9/dc6/d1e/d35/d5f/l14b 0 2024-06-15T01:07:26.091 INFO:tasks.workunit.client.0.smithi137.stdout:8/978: truncate d9/d2a/db9/dc6/d1e/d35/d116/f122 254840 0 2024-06-15T01:07:26.092 INFO:tasks.workunit.client.0.smithi137.stdout:8/979: readlink d9/d2a/d79/d7e/l9a 0 2024-06-15T01:07:26.098 INFO:tasks.workunit.client.0.smithi137.stdout:8/980: dread d9/d2a/f6f [0,4194304] 0 2024-06-15T01:07:26.100 INFO:tasks.workunit.client.0.smithi137.stdout:8/981: write d9/d2a/db9/dc6/d1e/d35/d116/f122 [1242985,98261] 0 2024-06-15T01:07:26.102 INFO:tasks.workunit.client.0.smithi137.stdout:8/982: write d9/d2a/db9/dc6/d1e/f42 [2753122,103393] 0 2024-06-15T01:07:26.108 INFO:tasks.workunit.client.0.smithi137.stdout:8/983: dread d9/d2a/db9/dc6/d1e/d35/dc8/f12f [0,4194304] 0 2024-06-15T01:07:26.111 INFO:tasks.workunit.client.0.smithi137.stdout:8/984: truncate d9/d2a/db9/dc6/f94 1514691 0 2024-06-15T01:07:26.112 INFO:tasks.workunit.client.0.smithi137.stdout:8/985: stat d9/d22/ld3 0 2024-06-15T01:07:26.115 INFO:tasks.workunit.client.0.smithi137.stdout:8/986: mknod d9/d2a/db9/dc6/d1e/d35/dfd/c14c 0 2024-06-15T01:07:26.115 INFO:tasks.workunit.client.0.smithi137.stdout:8/987: stat c3 0 2024-06-15T01:07:26.116 INFO:tasks.workunit.client.0.smithi137.stdout:8/988: chown d9/d2a/f6f 2977032 1 2024-06-15T01:07:26.736 INFO:tasks.workunit.client.0.smithi137.stdout:9/812: sync 2024-06-15T01:07:26.737 INFO:tasks.workunit.client.0.smithi137.stdout:2/961: sync 2024-06-15T01:07:26.737 INFO:tasks.workunit.client.0.smithi137.stdout:6/963: sync 2024-06-15T01:07:26.739 INFO:tasks.workunit.client.0.smithi137.stdout:2/962: read d0/da/da0/d3a/dd4/d5c/fa3 [404661,95504] 0 2024-06-15T01:07:26.743 INFO:tasks.workunit.client.0.smithi137.stdout:6/964: getdents d0 0 2024-06-15T01:07:26.745 INFO:tasks.workunit.client.0.smithi137.stdout:9/813: truncate df/f6b 9457 0 2024-06-15T01:07:26.746 INFO:tasks.workunit.client.0.smithi137.stdout:2/963: dread d0/de/d2b/f8e [0,4194304] 0 2024-06-15T01:07:26.747 INFO:tasks.workunit.client.0.smithi137.stdout:6/965: rmdir d0/d1a/d3e/d57/d32/d92/dac 39 2024-06-15T01:07:26.748 INFO:tasks.workunit.client.0.smithi137.stdout:6/966: dread - d0/d1a/d68/ddd/f106 zero size 2024-06-15T01:07:26.752 INFO:tasks.workunit.client.0.smithi137.stdout:2/964: rename d0/dc/f91 to d0/da/da0/d3a/dd4/d77/f136 0 2024-06-15T01:07:26.754 INFO:tasks.workunit.client.0.smithi137.stdout:6/967: fsync d0/d1a/d3e/d57/d32/d74/fd6 0 2024-06-15T01:07:26.754 INFO:tasks.workunit.client.0.smithi137.stdout:9/814: write df/d12/d1b/d31/d42/f46 [602921,26707] 0 2024-06-15T01:07:26.756 INFO:tasks.workunit.client.0.smithi137.stdout:9/815: read df/d12/d15/f5f [3363413,95589] 0 2024-06-15T01:07:26.758 INFO:tasks.workunit.client.0.smithi137.stdout:2/965: symlink d0/da/da0/d3a/dd4/d5c/d99/d12f/l137 0 2024-06-15T01:07:26.759 INFO:tasks.workunit.client.0.smithi137.stdout:6/968: mkdir d0/d1a/d3e/d57/d32/d12a 0 2024-06-15T01:07:26.760 INFO:tasks.workunit.client.0.smithi137.stdout:6/969: stat d0/d1a/d3e/d57/dba 0 2024-06-15T01:07:26.761 INFO:tasks.workunit.client.0.smithi137.stdout:2/966: mkdir d0/da/d138 0 2024-06-15T01:07:26.762 INFO:tasks.workunit.client.0.smithi137.stdout:9/816: creat df/ff6 x:0 0 0 2024-06-15T01:07:26.763 INFO:tasks.workunit.client.0.smithi137.stdout:6/970: mkdir d0/d5/d9/d3f/d12b 0 2024-06-15T01:07:26.765 INFO:tasks.workunit.client.0.smithi137.stdout:9/817: write df/d12/d15/f52 [4479576,29401] 0 2024-06-15T01:07:26.766 INFO:tasks.workunit.client.0.smithi137.stdout:2/967: creat d0/da/d73/dfe/f139 x:0 0 0 2024-06-15T01:07:26.767 INFO:tasks.workunit.client.0.smithi137.stdout:9/818: chown df/d12/d4e/dc6/fcc 24014 1 2024-06-15T01:07:26.767 INFO:tasks.workunit.client.0.smithi137.stdout:6/971: read d0/d1a/d3e/d57/d1d/da5/f8b [51296,84734] 0 2024-06-15T01:07:26.772 INFO:tasks.workunit.client.0.smithi137.stdout:6/972: rename d0/d1a/d3e/d57/d1d/f45 to d0/d5/d108/f12c 0 2024-06-15T01:07:26.774 INFO:tasks.workunit.client.0.smithi137.stdout:2/968: dwrite d0/da/da0/d3a/dd4/d5c/d79/f111 [0,4194304] 0 2024-06-15T01:07:26.778 INFO:tasks.workunit.client.0.smithi137.stdout:9/819: getdents df/d12/d29/d4a 0 2024-06-15T01:07:26.779 INFO:tasks.workunit.client.0.smithi137.stdout:2/969: creat d0/da/da0/d3a/dd4/d77/f13a x:0 0 0 2024-06-15T01:07:26.784 INFO:tasks.workunit.client.0.smithi137.stdout:9/820: dread df/d12/d15/f63 [0,4194304] 0 2024-06-15T01:07:26.789 INFO:tasks.workunit.client.0.smithi137.stdout:9/821: truncate df/d12/fb5 4313180 0 2024-06-15T01:07:26.792 INFO:tasks.workunit.client.0.smithi137.stdout:9/822: mknod df/d2a/d7e/dbe/cf7 0 2024-06-15T01:07:26.793 INFO:tasks.workunit.client.0.smithi137.stdout:2/970: write d0/f1 [439060,71072] 0 2024-06-15T01:07:26.797 INFO:tasks.workunit.client.0.smithi137.stdout:9/823: creat df/d12/d29/d66/dce/ff8 x:0 0 0 2024-06-15T01:07:26.798 INFO:tasks.workunit.client.0.smithi137.stdout:2/971: fsync d0/dc/d2e/d72/d56/fa5 0 2024-06-15T01:07:26.799 INFO:tasks.workunit.client.0.smithi137.stdout:6/973: dwrite d0/d5/d9/d29/fbc [0,4194304] 0 2024-06-15T01:07:26.800 INFO:tasks.workunit.client.0.smithi137.stdout:2/972: chown d0/dc/dff 3657 1 2024-06-15T01:07:26.801 INFO:tasks.workunit.client.0.smithi137.stdout:9/824: rmdir df/d13/d6f 39 2024-06-15T01:07:26.802 INFO:tasks.workunit.client.0.smithi137.stdout:6/974: fdatasync d0/d1a/d3e/d57/d1d/fa2 0 2024-06-15T01:07:26.805 INFO:tasks.workunit.client.0.smithi137.stdout:2/973: mkdir d0/da/da0/d3a/dd4/d5c/d99/d12f/d13b 0 2024-06-15T01:07:26.806 INFO:tasks.workunit.client.0.smithi137.stdout:2/974: stat d0/da/da0/d3a/dd4/d77/f136 0 2024-06-15T01:07:26.808 INFO:tasks.workunit.client.0.smithi137.stdout:9/825: fsync f2 0 2024-06-15T01:07:26.810 INFO:tasks.workunit.client.0.smithi137.stdout:2/975: mkdir d0/dc/d2e/d72/d56/de9/d13c 0 2024-06-15T01:07:26.813 INFO:tasks.workunit.client.0.smithi137.stdout:6/975: creat d0/d1a/d3e/d57/dba/de6/f12d x:0 0 0 2024-06-15T01:07:26.814 INFO:tasks.workunit.client.0.smithi137.stdout:9/826: creat df/d12/d4e/dbc/d3c/d51/ff9 x:0 0 0 2024-06-15T01:07:26.820 INFO:tasks.workunit.client.0.smithi137.stdout:6/976: creat d0/d1a/d3e/d87/d11b/f12e x:0 0 0 2024-06-15T01:07:26.821 INFO:tasks.workunit.client.0.smithi137.stdout:9/827: dwrite df/fe4 [0,4194304] 0 2024-06-15T01:07:26.822 INFO:tasks.workunit.client.0.smithi137.stdout:6/977: dread - d0/d1a/d3e/d57/d32/d92/ff2 zero size 2024-06-15T01:07:26.823 INFO:tasks.workunit.client.0.smithi137.stdout:2/976: write d0/da/d73/ff9 [2469034,110884] 0 2024-06-15T01:07:26.824 INFO:tasks.workunit.client.0.smithi137.stdout:2/977: chown d0/de/d2b/de5 0 1 2024-06-15T01:07:26.825 INFO:tasks.workunit.client.0.smithi137.stdout:6/978: mkdir d0/d1a/d3e/d57/d1d/da5/d12f 0 2024-06-15T01:07:26.825 INFO:tasks.workunit.client.0.smithi137.stdout:7/815: sync 2024-06-15T01:07:26.825 INFO:tasks.workunit.client.0.smithi137.stdout:8/989: sync 2024-06-15T01:07:26.826 INFO:tasks.workunit.client.0.smithi137.stdout:9/828: mknod df/d49/cfa 0 2024-06-15T01:07:26.827 INFO:tasks.workunit.client.0.smithi137.stdout:2/978: mknod d0/db9/c13d 0 2024-06-15T01:07:26.827 INFO:tasks.workunit.client.0.smithi137.stdout:9/829: chown df/d12/d29/d4a 161372 1 2024-06-15T01:07:26.830 INFO:tasks.workunit.client.0.smithi137.stdout:7/816: creat df/d7d/d1c/d1d/dc4/f119 x:0 0 0 2024-06-15T01:07:26.830 INFO:tasks.workunit.client.0.smithi137.stdout:2/979: rmdir d0/da/d73 39 2024-06-15T01:07:26.837 INFO:tasks.workunit.client.0.smithi137.stdout:6/979: rename d0/d1a/d3e/d87/dd7/lea to d0/d1a/d3e/d57/d32/d92/dac/l130 0 2024-06-15T01:07:26.839 INFO:tasks.workunit.client.0.smithi137.stdout:7/817: creat df/d7d/d1c/d1d/d27/d33/f11a x:0 0 0 2024-06-15T01:07:26.842 INFO:tasks.workunit.client.0.smithi137.stdout:8/990: getdents d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb 0 2024-06-15T01:07:26.843 INFO:tasks.workunit.client.0.smithi137.stdout:6/980: chown d0/d1a/d3e/d57/d32/laa 14117066 1 2024-06-15T01:07:26.844 INFO:tasks.workunit.client.0.smithi137.stdout:2/980: getdents d0/da/da0/d3a/dd4/d77 0 2024-06-15T01:07:26.844 INFO:tasks.workunit.client.0.smithi137.stdout:6/981: stat d0/d1a/d99/dcf 0 2024-06-15T01:07:26.845 INFO:tasks.workunit.client.0.smithi137.stdout:7/818: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/de6/c11b 0 2024-06-15T01:07:26.846 INFO:tasks.workunit.client.0.smithi137.stdout:7/819: readlink df/d7d/d1c/d1d/d27/l117 0 2024-06-15T01:07:26.848 INFO:tasks.workunit.client.0.smithi137.stdout:7/820: fdatasync df/d13/d2e/d1a/d6a/d90/ddf/fed 0 2024-06-15T01:07:26.848 INFO:tasks.workunit.client.0.smithi137.stdout:8/991: rename d9/d2a/d2f/lcb to d9/d2a/db9/dc6/d1e/d35/l14d 0 2024-06-15T01:07:26.851 INFO:tasks.workunit.client.0.smithi137.stdout:2/981: truncate d0/dc/d2e/d72/d56/d6e/d104/f11c 509839 0 2024-06-15T01:07:26.855 INFO:tasks.workunit.client.0.smithi137.stdout:6/982: creat d0/d1a/d3e/d57/d32/d12a/f131 x:0 0 0 2024-06-15T01:07:26.856 INFO:tasks.workunit.client.0.smithi137.stdout:2/982: readlink d0/l64 0 2024-06-15T01:07:26.857 INFO:tasks.workunit.client.0.smithi137.stdout:9/830: dwrite df/d12/f3e [0,4194304] 0 2024-06-15T01:07:26.862 INFO:tasks.workunit.client.0.smithi137.stdout:6/983: chown d0/d5/l50 3310915 1 2024-06-15T01:07:26.863 INFO:tasks.workunit.client.0.smithi137.stdout:9/831: chown df/d12/l3b 161082 1 2024-06-15T01:07:26.869 INFO:tasks.workunit.client.0.smithi137.stdout:8/992: dwrite d9/d2a/f6d [0,4194304] 0 2024-06-15T01:07:26.878 INFO:tasks.workunit.client.0.smithi137.stdout:9/832: dwrite df/d12/d29/d79/fc7 [0,4194304] 0 2024-06-15T01:07:26.879 INFO:tasks.workunit.client.0.smithi137.stdout:2/983: dwrite d0/dc/f3d [0,4194304] 0 2024-06-15T01:07:26.880 INFO:tasks.workunit.client.0.smithi137.stdout:6/984: dwrite d0/d1a/d3e/d57/d32/d92/ff2 [0,4194304] 0 2024-06-15T01:07:26.881 INFO:tasks.workunit.client.0.smithi137.stdout:6/985: stat d0/d5/d9/db9/cbe 0 2024-06-15T01:07:26.884 INFO:tasks.workunit.client.0.smithi137.stdout:2/984: stat d0/da/da0/d3a/cdc 0 2024-06-15T01:07:26.886 INFO:tasks.workunit.client.0.smithi137.stdout:2/985: readlink d0/dc/d2e/d72/d56/de9/l124 0 2024-06-15T01:07:26.887 INFO:tasks.workunit.client.0.smithi137.stdout:2/986: readlink d0/de/d2b/lb4 0 2024-06-15T01:07:26.888 INFO:tasks.workunit.client.0.smithi137.stdout:6/986: rmdir d0/d1a/d3e/d57/db6 39 2024-06-15T01:07:26.888 INFO:tasks.workunit.client.0.smithi137.stdout:9/833: fsync df/d12/fb5 0 2024-06-15T01:07:26.892 INFO:tasks.workunit.client.0.smithi137.stdout:2/987: chown d0/de/d2b/d47/c54 1 1 2024-06-15T01:07:26.894 INFO:tasks.workunit.client.0.smithi137.stdout:6/987: mknod d0/d1a/d3e/d57/c132 0 2024-06-15T01:07:26.896 INFO:tasks.workunit.client.0.smithi137.stdout:9/834: fdatasync df/d12/d29/d79/d9a/fe8 0 2024-06-15T01:07:26.897 INFO:tasks.workunit.client.0.smithi137.stdout:9/835: write df/d49/ff5 [137774,66595] 0 2024-06-15T01:07:26.898 INFO:tasks.workunit.client.0.smithi137.stdout:6/988: creat d0/d1a/d3e/d87/de0/f133 x:0 0 0 2024-06-15T01:07:26.899 INFO:tasks.workunit.client.0.smithi137.stdout:6/989: chown d0/d1a/d3e/d57/d1d/da5/fe3 948883 1 2024-06-15T01:07:26.900 INFO:tasks.workunit.client.0.smithi137.stdout:9/836: mknod df/d1a/da7/cfb 0 2024-06-15T01:07:26.902 INFO:tasks.workunit.client.0.smithi137.stdout:6/990: unlink d0/ff3 0 2024-06-15T01:07:26.905 INFO:tasks.workunit.client.0.smithi137.stdout:2/988: dread d0/dc/d22/f39 [0,4194304] 0 2024-06-15T01:07:26.905 INFO:tasks.workunit.client.0.smithi137.stdout:9/837: creat df/d49/ffc x:0 0 0 2024-06-15T01:07:26.908 INFO:tasks.workunit.client.0.smithi137.stdout:9/838: symlink df/d12/d29/d4a/lfd 0 2024-06-15T01:07:26.910 INFO:tasks.workunit.client.0.smithi137.stdout:9/839: write df/d12/d29/d4a/f94 [79641,130142] 0 2024-06-15T01:07:26.911 INFO:tasks.workunit.client.0.smithi137.stdout:9/840: write df/d12/d1b/f74 [533320,59531] 0 2024-06-15T01:07:26.919 INFO:tasks.workunit.client.0.smithi137.stdout:9/841: dwrite df/d12/d15/fa9 [0,4194304] 0 2024-06-15T01:07:26.924 INFO:tasks.workunit.client.0.smithi137.stdout:2/989: truncate d0/da/da0/d3a/dd4/f6a 2554702 0 2024-06-15T01:07:26.926 INFO:tasks.workunit.client.0.smithi137.stdout:9/842: creat df/d2a/d7e/ffe x:0 0 0 2024-06-15T01:07:26.929 INFO:tasks.workunit.client.0.smithi137.stdout:9/843: unlink df/d12/d1b/f35 0 2024-06-15T01:07:26.930 INFO:tasks.workunit.client.0.smithi137.stdout:6/991: getdents d0/d5/d9/d3f/d12b 0 2024-06-15T01:07:26.931 INFO:tasks.workunit.client.0.smithi137.stdout:9/844: chown df/d12/d15/d75/f7c 296 1 2024-06-15T01:07:26.932 INFO:tasks.workunit.client.0.smithi137.stdout:9/845: stat df/f28 0 2024-06-15T01:07:26.936 INFO:tasks.workunit.client.0.smithi137.stdout:6/992: creat d0/d1a/d3e/d87/de0/d109/f134 x:0 0 0 2024-06-15T01:07:26.936 INFO:tasks.workunit.client.0.smithi137.stdout:9/846: creat df/d12/d29/d95/db3/fff x:0 0 0 2024-06-15T01:07:26.941 INFO:tasks.workunit.client.0.smithi137.stdout:2/990: dwrite d0/da/d73/f9a [0,4194304] 0 2024-06-15T01:07:26.943 INFO:tasks.workunit.client.0.smithi137.stdout:6/993: truncate d0/d1a/fc7 532394 0 2024-06-15T01:07:26.947 INFO:tasks.workunit.client.0.smithi137.stdout:6/994: write d0/d1a/f35 [1410961,33003] 0 2024-06-15T01:07:26.948 INFO:tasks.workunit.client.0.smithi137.stdout:9/847: dwrite df/d12/d1b/d31/d42/fd8 [0,4194304] 0 2024-06-15T01:07:26.951 INFO:tasks.workunit.client.0.smithi137.stdout:6/995: truncate d0/d5/f73 2935030 0 2024-06-15T01:07:26.954 INFO:tasks.workunit.client.0.smithi137.stdout:6/996: symlink d0/d1a/d68/ddd/l135 0 2024-06-15T01:07:26.957 INFO:tasks.workunit.client.0.smithi137.stdout:6/997: rename d0/d5/d9/d3f/d12b to d0/d5/d108/d136 0 2024-06-15T01:07:27.414 INFO:tasks.workunit.client.0.smithi137.stdout:7/821: sync 2024-06-15T01:07:27.419 INFO:tasks.workunit.client.0.smithi137.stdout:7/822: link df/d13/d2e/d1a/d6a/dde/l53 df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b/l11c 0 2024-06-15T01:07:27.422 INFO:tasks.workunit.client.0.smithi137.stdout:7/823: truncate df/d13/d2e/d1a/d6a/dde/fe3 561832 0 2024-06-15T01:07:27.424 INFO:tasks.workunit.client.0.smithi137.stdout:7/824: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e/l11d 0 2024-06-15T01:07:27.430 INFO:tasks.workunit.client.0.smithi137.stdout:7/825: dwrite df/d13/d2e/d1a/d6a/fd6 [0,4194304] 0 2024-06-15T01:07:27.431 INFO:tasks.workunit.client.0.smithi137.stdout:7/826: dread - df/d7d/d3b/fb6 zero size 2024-06-15T01:07:27.433 INFO:tasks.workunit.client.0.smithi137.stdout:7/827: write df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fe8 [744843,13878] 0 2024-06-15T01:07:27.441 INFO:tasks.workunit.client.0.smithi137.stdout:7/828: dwrite df/d13/d2e/de1/ffa [0,4194304] 0 2024-06-15T01:07:27.459 INFO:tasks.workunit.client.0.smithi137.stdout:7/829: truncate df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fe8 1841334 0 2024-06-15T01:07:27.460 INFO:tasks.workunit.client.0.smithi137.stdout:7/830: symlink df/d7d/d1c/d1d/d27/l11e 0 2024-06-15T01:07:27.467 INFO:tasks.workunit.client.0.smithi137.stdout:7/831: dwrite df/d7d/d1c/d1d/d79/dbc/ff7 [0,4194304] 0 2024-06-15T01:07:27.474 INFO:tasks.workunit.client.0.smithi137.stdout:7/832: link df/f78 df/d7d/d1c/d1d/d27/d33/d43/de0/f11f 0 2024-06-15T01:07:27.478 INFO:tasks.workunit.client.0.smithi137.stdout:7/833: creat df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b/f120 x:0 0 0 2024-06-15T01:07:27.484 INFO:tasks.workunit.client.0.smithi137.stdout:7/834: rename df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d8e to df/d7d/d121 0 2024-06-15T01:07:27.486 INFO:tasks.workunit.client.0.smithi137.stdout:7/835: unlink df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/c118 0 2024-06-15T01:07:27.487 INFO:tasks.workunit.client.0.smithi137.stdout:7/836: chown df/d13/d2e/d1a/d6a/d90 24 1 2024-06-15T01:07:27.488 INFO:tasks.workunit.client.0.smithi137.stdout:7/837: dread - df/d13/d2e/d1a/d6a/dde/d89/fc9 zero size 2024-06-15T01:07:27.489 INFO:tasks.workunit.client.0.smithi137.stdout:7/838: stat df/d7d/d1c/d1d/d27/d33/d43/d6c/c91 0 2024-06-15T01:07:27.492 INFO:tasks.workunit.client.0.smithi137.stdout:7/839: truncate df/d7d/d1c/d1d/d27/f48 555149 0 2024-06-15T01:07:27.492 INFO:tasks.workunit.client.0.smithi137.stdout:7/840: chown df/d7d 5 1 2024-06-15T01:07:27.493 INFO:tasks.workunit.client.0.smithi137.stdout:7/841: getdents df/d13/d2e/d1a/d6a/dfe 0 2024-06-15T01:07:27.495 INFO:tasks.workunit.client.0.smithi137.stdout:7/842: mkdir df/d7d/d1c/d1d/d27/d33/d43/dc5/d122 0 2024-06-15T01:07:27.497 INFO:tasks.workunit.client.0.smithi137.stdout:7/843: mknod df/d13/d2e/de1/c123 0 2024-06-15T01:07:27.498 INFO:tasks.workunit.client.0.smithi137.stdout:7/844: unlink df/d13/d3c/f92 0 2024-06-15T01:07:27.501 INFO:tasks.workunit.client.0.smithi137.stdout:7/845: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/f124 x:0 0 0 2024-06-15T01:07:27.505 INFO:tasks.workunit.client.0.smithi137.stdout:7/846: rename df/d13/d3c/f6e to df/d7d/d3b/d5b/f125 0 2024-06-15T01:07:27.507 INFO:tasks.workunit.client.0.smithi137.stdout:7/847: mknod df/d13/d2e/d87/c126 0 2024-06-15T01:07:27.514 INFO:tasks.workunit.client.0.smithi137.stdout:7/848: dwrite df/d13/d2e/d1a/d6a/dde/d89/fc8 [0,4194304] 0 2024-06-15T01:07:27.514 INFO:tasks.workunit.client.0.smithi137.stdout:7/849: chown df/d7d/d3b/d5b 8853449 1 2024-06-15T01:07:27.517 INFO:tasks.workunit.client.0.smithi137.stdout:7/850: chown df/ca9 11655 1 2024-06-15T01:07:27.519 INFO:tasks.workunit.client.0.smithi137.stdout:7/851: creat df/d7d/d3b/d5b/da6/f127 x:0 0 0 2024-06-15T01:07:27.522 INFO:tasks.workunit.client.0.smithi137.stdout:7/852: symlink df/d13/d2e/d1a/d6a/dfe/l128 0 2024-06-15T01:07:27.530 INFO:tasks.workunit.client.0.smithi137.stdout:7/853: dread df/d13/d2e/d1a/d6a/dde/d89/f8a [0,4194304] 0 2024-06-15T01:07:27.531 INFO:tasks.workunit.client.0.smithi137.stdout:7/854: stat df/d13/d2e/d1a/d6a/dfe 0 2024-06-15T01:07:27.532 INFO:tasks.workunit.client.0.smithi137.stdout:7/855: read - df/d13/d2e/d1a/fab zero size 2024-06-15T01:07:27.535 INFO:tasks.workunit.client.0.smithi137.stdout:7/856: truncate df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/d10c/f113 2906 0 2024-06-15T01:07:27.536 INFO:tasks.workunit.client.0.smithi137.stdout:7/857: chown df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b/f120 11 1 2024-06-15T01:07:27.537 INFO:tasks.workunit.client.0.smithi137.stdout:7/858: fdatasync df/d7d/d1c/d1d/d27/d33/d43/d6c/f124 0 2024-06-15T01:07:27.544 INFO:tasks.workunit.client.0.smithi137.stdout:7/859: write df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fc1 [5045257,33541] 0 2024-06-15T01:07:27.544 INFO:tasks.workunit.client.0.smithi137.stdout:6/998: sync 2024-06-15T01:07:27.545 INFO:tasks.workunit.client.0.smithi137.stdout:9/848: sync 2024-06-15T01:07:27.545 INFO:tasks.workunit.client.0.smithi137.stdout:8/993: sync 2024-06-15T01:07:27.545 INFO:tasks.workunit.client.0.smithi137.stdout:2/991: sync 2024-06-15T01:07:27.547 INFO:tasks.workunit.client.0.smithi137.stdout:6/999: truncate d0/d1a/d99/ff1 849275 0 2024-06-15T01:07:27.549 INFO:tasks.workunit.client.0.smithi137.stdout:7/860: write df/d7d/d1c/d1d/d79/dbc/df3/fac [314353,64975] 0 2024-06-15T01:07:27.550 INFO:tasks.workunit.client.0.smithi137.stdout:7/861: dread - df/d7d/d3b/d5b/da6/f127 zero size 2024-06-15T01:07:27.551 INFO:tasks.workunit.client.0.smithi137.stdout:8/994: mknod d9/d2a/db9/dc6/d1e/d3f/dab/db7/dbb/c14e 0 2024-06-15T01:07:27.553 INFO:tasks.workunit.client.0.smithi137.stdout:2/992: creat d0/dc/d2e/d72/d56/d6e/f13e x:0 0 0 2024-06-15T01:07:27.553 INFO:tasks.workunit.client.0.smithi137.stdout:9/849: getdents df/d12/d4e/dbc 0 2024-06-15T01:07:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:8/995: chown d9/d2a/db9/dc6/d1e/d67/f6a 1 1 2024-06-15T01:07:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:2/993: mkdir d0/dc/d22/d13f 0 2024-06-15T01:07:27.562 INFO:tasks.workunit.client.0.smithi137.stdout:8/996: dwrite d9/d2a/f69 [0,4194304] 0 2024-06-15T01:07:27.562 INFO:tasks.workunit.client.0.smithi137.stdout:2/994: dread d0/dc/d22/deb/f62 [0,4194304] 0 2024-06-15T01:07:27.563 INFO:tasks.workunit.client.0.smithi137.stdout:9/850: write df/d12/d29/d4a/fba [857782,123906] 0 2024-06-15T01:07:27.565 INFO:tasks.workunit.client.0.smithi137.stdout:8/997: creat d9/d55/f14f x:0 0 0 2024-06-15T01:07:27.567 INFO:tasks.workunit.client.0.smithi137.stdout:2/995: mkdir d0/da/d73/dfe/d140 0 2024-06-15T01:07:27.568 INFO:tasks.workunit.client.0.smithi137.stdout:9/851: chown df/d12/d15/c39 55 1 2024-06-15T01:07:27.570 INFO:tasks.workunit.client.0.smithi137.stdout:2/996: dread - d0/de/d2b/de5/f127 zero size 2024-06-15T01:07:27.570 INFO:tasks.workunit.client.0.smithi137.stdout:7/862: dread df/d7d/d1c/d1d/d79/fd8 [0,4194304] 0 2024-06-15T01:07:27.572 INFO:tasks.workunit.client.0.smithi137.stdout:8/998: rmdir d9/d2a/db9/dc6/d1e/d3f/dab/dd8/d106/ddb 39 2024-06-15T01:07:27.572 INFO:tasks.workunit.client.0.smithi137.stdout:2/997: write d0/da/da0/d3a/dd4/d5c/f11a [411210,70075] 0 2024-06-15T01:07:27.573 INFO:tasks.workunit.client.0.smithi137.stdout:7/863: write df/d7d/d1c/d1d/dc4/f119 [520415,121594] 0 2024-06-15T01:07:27.573 INFO:tasks.workunit.client.0.smithi137.stdout:9/852: symlink df/d12/d29/d4a/l100 0 2024-06-15T01:07:27.575 INFO:tasks.workunit.client.0.smithi137.stdout:9/853: readlink df/d12/d4e/dbc/d3c/lee 0 2024-06-15T01:07:27.576 INFO:tasks.workunit.client.0.smithi137.stdout:7/864: mkdir df/d7d/d3b/d5b/d129 0 2024-06-15T01:07:27.578 INFO:tasks.workunit.client.0.smithi137.stdout:8/999: creat d9/d2a/db9/dc6/d1e/d67/f150 x:0 0 0 2024-06-15T01:07:27.581 INFO:tasks.workunit.client.0.smithi137.stdout:7/865: mknod df/d13/d2e/c12a 0 2024-06-15T01:07:27.581 INFO:tasks.workunit.client.0.smithi137.stdout:9/854: dread df/d12/d15/f73 [0,4194304] 0 2024-06-15T01:07:27.582 INFO:tasks.workunit.client.0.smithi137.stdout:2/998: write d0/dc/dff/f110 [119489,98429] 0 2024-06-15T01:07:27.584 INFO:tasks.workunit.client.0.smithi137.stdout:9/855: mkdir df/d12/d29/d66/d101 0 2024-06-15T01:07:27.586 INFO:tasks.workunit.client.0.smithi137.stdout:2/999: dread d0/dc/d22/deb/def/f10a [0,4194304] 0 2024-06-15T01:07:27.587 INFO:tasks.workunit.client.0.smithi137.stdout:7/866: getdents df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b 0 2024-06-15T01:07:27.589 INFO:tasks.workunit.client.0.smithi137.stdout:7/867: fsync df/d13/d2e/d1a/d6a/dde/d89/fc8 0 2024-06-15T01:07:27.589 INFO:tasks.workunit.client.0.smithi137.stdout:9/856: creat df/d12/d29/d66/d101/f102 x:0 0 0 2024-06-15T01:07:27.591 INFO:tasks.workunit.client.0.smithi137.stdout:9/857: creat df/d49/d72/f103 x:0 0 0 2024-06-15T01:07:27.592 INFO:tasks.workunit.client.0.smithi137.stdout:9/858: dread - df/d12/d29/d66/d101/f102 zero size 2024-06-15T01:07:27.592 INFO:tasks.workunit.client.0.smithi137.stdout:7/868: rmdir df/d7d/d1c/d1d/d79/dbc 39 2024-06-15T01:07:27.593 INFO:tasks.workunit.client.0.smithi137.stdout:9/859: dread - df/d12/d29/d95/db3/fff zero size 2024-06-15T01:07:27.594 INFO:tasks.workunit.client.0.smithi137.stdout:7/869: truncate df/d7d/d1c/f1f 3478290 0 2024-06-15T01:07:27.595 INFO:tasks.workunit.client.0.smithi137.stdout:9/860: read df/d12/d1b/d31/d42/f56 [2987444,88981] 0 2024-06-15T01:07:27.596 INFO:tasks.workunit.client.0.smithi137.stdout:7/870: dread - df/d7d/d3b/d5b/da6/f127 zero size 2024-06-15T01:07:27.596 INFO:tasks.workunit.client.0.smithi137.stdout:9/861: chown df/d13/dcb/dd4 5531609 1 2024-06-15T01:07:27.598 INFO:tasks.workunit.client.0.smithi137.stdout:7/871: dread - df/d7d/d3b/d5b/da6/fbb zero size 2024-06-15T01:07:27.599 INFO:tasks.workunit.client.0.smithi137.stdout:9/862: fdatasync df/d12/d15/f26 0 2024-06-15T01:07:27.603 INFO:tasks.workunit.client.0.smithi137.stdout:9/863: symlink df/d12/d4e/l104 0 2024-06-15T01:07:27.606 INFO:tasks.workunit.client.0.smithi137.stdout:7/872: rmdir df/d7d/d3b/d5b/d129 0 2024-06-15T01:07:27.612 INFO:tasks.workunit.client.0.smithi137.stdout:9/864: dwrite df/d49/d8d/fe9 [0,4194304] 0 2024-06-15T01:07:27.614 INFO:tasks.workunit.client.0.smithi137.stdout:7/873: write df/d13/d2e/d1a/d6a/f9a [309524,122042] 0 2024-06-15T01:07:27.619 INFO:tasks.workunit.client.0.smithi137.stdout:9/865: link df/d12/d29/d66/f86 df/d12/d15/d75/f105 0 2024-06-15T01:07:27.622 INFO:tasks.workunit.client.0.smithi137.stdout:9/866: symlink df/d12/d29/d4a/l106 0 2024-06-15T01:07:27.622 INFO:tasks.workunit.client.0.smithi137.stdout:9/867: readlink df/le1 0 2024-06-15T01:07:27.624 INFO:tasks.workunit.client.0.smithi137.stdout:9/868: fdatasync df/d12/d29/d4a/f70 0 2024-06-15T01:07:27.633 INFO:tasks.workunit.client.0.smithi137.stdout:9/869: dread f2 [0,4194304] 0 2024-06-15T01:07:27.635 INFO:tasks.workunit.client.0.smithi137.stdout:9/870: symlink df/d1a/da7/l107 0 2024-06-15T01:07:27.637 INFO:tasks.workunit.client.0.smithi137.stdout:7/874: dread df/d7d/d3b/f5f [0,4194304] 0 2024-06-15T01:07:27.637 INFO:tasks.workunit.client.0.smithi137.stdout:9/871: mkdir df/d2a/d108 0 2024-06-15T01:07:27.640 INFO:tasks.workunit.client.0.smithi137.stdout:7/875: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/f12b x:0 0 0 2024-06-15T01:07:27.642 INFO:tasks.workunit.client.0.smithi137.stdout:7/876: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/f12c x:0 0 0 2024-06-15T01:07:27.643 INFO:tasks.workunit.client.0.smithi137.stdout:7/877: readlink df/d7d/d3b/dd1/lfd 0 2024-06-15T01:07:27.644 INFO:tasks.workunit.client.0.smithi137.stdout:7/878: chown df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb 600769683 1 2024-06-15T01:07:27.644 INFO:tasks.workunit.client.0.smithi137.stdout:9/872: truncate df/d1a/f87 22463 0 2024-06-15T01:07:27.645 INFO:tasks.workunit.client.0.smithi137.stdout:9/873: dread - df/d12/d4e/dc6/fcc zero size 2024-06-15T01:07:27.648 INFO:tasks.workunit.client.0.smithi137.stdout:9/874: rename df/d49/d4b/d9b/fd7 to df/d1a/f109 0 2024-06-15T01:07:27.649 INFO:tasks.workunit.client.0.smithi137.stdout:7/879: truncate df/d13/d2e/d1a/d6a/dde/d89/fc7 904000 0 2024-06-15T01:07:27.649 INFO:tasks.workunit.client.0.smithi137.stdout:7/880: chown df/d13/l44 829 1 2024-06-15T01:07:27.650 INFO:tasks.workunit.client.0.smithi137.stdout:9/875: mknod df/d12/d29/d4a/c10a 0 2024-06-15T01:07:27.657 INFO:tasks.workunit.client.0.smithi137.stdout:7/881: dwrite df/d7d/d1c/d1d/d79/fcb [0,4194304] 0 2024-06-15T01:07:27.660 INFO:tasks.workunit.client.0.smithi137.stdout:7/882: mkdir df/d13/d2e/d1a/d6a/dde/d12d 0 2024-06-15T01:07:27.669 INFO:tasks.workunit.client.0.smithi137.stdout:7/883: truncate df/d7d/d1c/f9c 248011 0 2024-06-15T01:07:27.670 INFO:tasks.workunit.client.0.smithi137.stdout:9/876: dread df/d12/d29/d4a/f7f [0,4194304] 0 2024-06-15T01:07:27.672 INFO:tasks.workunit.client.0.smithi137.stdout:7/884: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/d114/c12e 0 2024-06-15T01:07:27.676 INFO:tasks.workunit.client.0.smithi137.stdout:7/885: rename df/d13/d2e/d1a/d6a/fd6 to df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/f12f 0 2024-06-15T01:07:27.689 INFO:tasks.workunit.client.0.smithi137.stdout:9/877: dread df/d12/d15/d75/f7c [0,4194304] 0 2024-06-15T01:07:27.693 INFO:tasks.workunit.client.0.smithi137.stdout:9/878: dread df/d12/d29/d4a/f7f [0,4194304] 0 2024-06-15T01:07:27.695 INFO:tasks.workunit.client.0.smithi137.stdout:9/879: fsync df/d12/fc4 0 2024-06-15T01:07:27.705 INFO:tasks.workunit.client.0.smithi137.stdout:9/880: dwrite df/f6b [0,4194304] 0 2024-06-15T01:07:27.707 INFO:tasks.workunit.client.0.smithi137.stdout:9/881: mknod df/d2a/d7e/dbe/c10b 0 2024-06-15T01:07:27.709 INFO:tasks.workunit.client.0.smithi137.stdout:9/882: symlink df/d12/d29/d66/l10c 0 2024-06-15T01:07:27.711 INFO:tasks.workunit.client.0.smithi137.stdout:9/883: unlink df/d12/d1b/d31/d42/fd8 0 2024-06-15T01:07:27.714 INFO:tasks.workunit.client.0.smithi137.stdout:9/884: rename df/d12/d15/c39 to df/d12/d4e/dc6/deb/c10d 0 2024-06-15T01:07:27.716 INFO:tasks.workunit.client.0.smithi137.stdout:9/885: fsync df/d12/d15/f5f 0 2024-06-15T01:07:27.718 INFO:tasks.workunit.client.0.smithi137.stdout:9/886: readlink df/d12/d29/d79/lf0 0 2024-06-15T01:07:27.721 INFO:tasks.workunit.client.0.smithi137.stdout:9/887: dread f5 [0,4194304] 0 2024-06-15T01:07:27.722 INFO:tasks.workunit.client.0.smithi137.stdout:9/888: fdatasync df/d49/ff5 0 2024-06-15T01:07:27.724 INFO:tasks.workunit.client.0.smithi137.stdout:9/889: rmdir df/d49/d72 39 2024-06-15T01:07:27.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:27 smithi137 ceph-mon[29354]: pgmap v29: 65 pgs: 65 active+clean; 3.0 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 82 MiB/s rd, 175 MiB/s wr, 516 op/s 2024-06-15T01:07:27.727 INFO:tasks.workunit.client.0.smithi137.stdout:9/890: mkdir df/d12/d4e/dbc/d3c/d51/d10e 0 2024-06-15T01:07:27.727 INFO:tasks.workunit.client.0.smithi137.stdout:9/891: fsync df/d12/d15/f52 0 2024-06-15T01:07:27.729 INFO:tasks.workunit.client.0.smithi137.stdout:9/892: chown df/c10 1077120 1 2024-06-15T01:07:27.729 INFO:tasks.workunit.client.0.smithi137.stdout:9/893: chown df/d12/d29/fa3 620 1 2024-06-15T01:07:27.735 INFO:tasks.workunit.client.0.smithi137.stdout:9/894: dwrite df/d12/d1b/d31/d42/f56 [0,4194304] 0 2024-06-15T01:07:27.740 INFO:tasks.workunit.client.0.smithi137.stdout:9/895: rename df/d12/d1b/d31 to df/d1a/d10f 0 2024-06-15T01:07:27.743 INFO:tasks.workunit.client.0.smithi137.stdout:9/896: write df/d12/d29/fe0 [42316,32858] 0 2024-06-15T01:07:27.746 INFO:tasks.workunit.client.0.smithi137.stdout:9/897: dread f5 [0,4194304] 0 2024-06-15T01:07:27.750 INFO:tasks.workunit.client.0.smithi137.stdout:9/898: rename df/d12/d15/f52 to df/d12/d15/d75/f110 0 2024-06-15T01:07:27.753 INFO:tasks.workunit.client.0.smithi137.stdout:9/899: symlink df/d2a/d7e/l111 0 2024-06-15T01:07:27.764 INFO:tasks.workunit.client.0.smithi137.stdout:9/900: dread df/d12/f17 [0,4194304] 0 2024-06-15T01:07:27.779 INFO:tasks.workunit.client.0.smithi137.stdout:9/901: dread df/d12/d29/d4a/f6c [4194304,4194304] 0 2024-06-15T01:07:27.780 INFO:tasks.workunit.client.0.smithi137.stdout:9/902: mknod df/d2a/d7e/dbe/c112 0 2024-06-15T01:07:27.782 INFO:tasks.workunit.client.0.smithi137.stdout:9/903: symlink df/d12/d4e/dc6/l113 0 2024-06-15T01:07:27.783 INFO:tasks.workunit.client.0.smithi137.stdout:9/904: stat df/d1a/d10f/d42/f46 0 2024-06-15T01:07:27.783 INFO:tasks.workunit.client.0.smithi137.stdout:9/905: readlink df/la4 0 2024-06-15T01:07:27.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:27 smithi142 ceph-mon[37379]: pgmap v29: 65 pgs: 65 active+clean; 3.0 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 82 MiB/s rd, 175 MiB/s wr, 516 op/s 2024-06-15T01:07:28.166 INFO:tasks.workunit.client.0.smithi137.stdout:9/906: sync 2024-06-15T01:07:28.167 INFO:tasks.workunit.client.0.smithi137.stdout:7/886: sync 2024-06-15T01:07:28.201 INFO:tasks.workunit.client.0.smithi137.stdout:7/887: write df/fb4 [3422162,54537] 0 2024-06-15T01:07:28.208 INFO:tasks.workunit.client.0.smithi137.stdout:7/888: truncate df/d7d/d1c/f9c 16751 0 2024-06-15T01:07:28.214 INFO:tasks.workunit.client.0.smithi137.stdout:7/889: unlink df/d7d/c6f 0 2024-06-15T01:07:28.215 INFO:tasks.workunit.client.0.smithi137.stdout:7/890: readlink df/d13/d2e/d1a/lc2 0 2024-06-15T01:07:28.261 INFO:tasks.workunit.client.0.smithi137.stdout:7/891: creat df/d7d/d3b/f130 x:0 0 0 2024-06-15T01:07:28.265 INFO:tasks.workunit.client.0.smithi137.stdout:7/892: creat df/d7d/d1c/d1d/d79/dbc/f131 x:0 0 0 2024-06-15T01:07:28.268 INFO:tasks.workunit.client.0.smithi137.stdout:7/893: creat df/d13/df5/f132 x:0 0 0 2024-06-15T01:07:28.272 INFO:tasks.workunit.client.0.smithi137.stdout:7/894: rename df/d13/l44 to df/d7d/d3b/d5b/da6/l133 0 2024-06-15T01:07:28.284 INFO:tasks.workunit.client.0.smithi137.stdout:9/907: truncate df/fe4 947163 0 2024-06-15T01:07:28.292 INFO:tasks.workunit.client.0.smithi137.stdout:9/908: dwrite df/d12/d15/f58 [4194304,4194304] 0 2024-06-15T01:07:28.295 INFO:tasks.workunit.client.0.smithi137.stdout:9/909: truncate df/d12/d29/f61 4505454 0 2024-06-15T01:07:28.301 INFO:tasks.workunit.client.0.smithi137.stdout:9/910: getdents df/d13 0 2024-06-15T01:07:28.440 INFO:tasks.workunit.client.0.smithi137.stdout:7/895: write df/f14 [855089,10333] 0 2024-06-15T01:07:28.445 INFO:tasks.workunit.client.0.smithi137.stdout:7/896: rename df/d13/d3c/l41 to df/d7d/d1c/d1d/d27/d33/d43/dc5/d122/l134 0 2024-06-15T01:07:28.448 INFO:tasks.workunit.client.0.smithi137.stdout:7/897: unlink df/d13/d3c/f5e 0 2024-06-15T01:07:28.451 INFO:tasks.workunit.client.0.smithi137.stdout:7/898: creat df/d13/d2e/d1a/f135 x:0 0 0 2024-06-15T01:07:28.454 INFO:tasks.workunit.client.0.smithi137.stdout:7/899: creat df/d7d/d1c/d1d/d27/d33/d43/dc5/f136 x:0 0 0 2024-06-15T01:07:28.456 INFO:tasks.workunit.client.0.smithi137.stdout:7/900: rmdir df/d7d/d1c 39 2024-06-15T01:07:28.457 INFO:tasks.workunit.client.0.smithi137.stdout:7/901: creat df/d7d/d3b/d5b/da6/f137 x:0 0 0 2024-06-15T01:07:28.461 INFO:tasks.workunit.client.0.smithi137.stdout:7/902: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/d114/c138 0 2024-06-15T01:07:28.461 INFO:tasks.workunit.client.0.smithi137.stdout:7/903: readlink df/d7d/d121/lbd 0 2024-06-15T01:07:28.463 INFO:tasks.workunit.client.0.smithi137.stdout:7/904: fdatasync df/d7d/d3b/fb6 0 2024-06-15T01:07:28.469 INFO:tasks.workunit.client.0.smithi137.stdout:7/905: dread df/f32 [0,4194304] 0 2024-06-15T01:07:28.471 INFO:tasks.workunit.client.0.smithi137.stdout:7/906: creat df/d13/d2e/d87/f139 x:0 0 0 2024-06-15T01:07:28.473 INFO:tasks.workunit.client.0.smithi137.stdout:7/907: mknod df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b/c13a 0 2024-06-15T01:07:28.479 INFO:tasks.workunit.client.0.smithi137.stdout:7/908: dwrite df/d13/d2e/d62/fee [0,4194304] 0 2024-06-15T01:07:28.481 INFO:tasks.workunit.client.0.smithi137.stdout:7/909: readlink df/d7d/d3b/dd1/lfd 0 2024-06-15T01:07:28.483 INFO:tasks.workunit.client.0.smithi137.stdout:7/910: creat df/d13/d2e/d1a/de4/f13b x:0 0 0 2024-06-15T01:07:28.486 INFO:tasks.workunit.client.0.smithi137.stdout:7/911: link df/d13/d2e/d1a/d6a/f9a df/d13/d2e/d87/f13c 0 2024-06-15T01:07:28.488 INFO:tasks.workunit.client.0.smithi137.stdout:7/912: write df/d7d/d3b/f5f [5195303,26952] 0 2024-06-15T01:07:28.493 INFO:tasks.workunit.client.0.smithi137.stdout:7/913: getdents df/d13/d3c 0 2024-06-15T01:07:28.495 INFO:tasks.workunit.client.0.smithi137.stdout:7/914: rename df/d7d/d3b/d5b/cf1 to df/d7d/d1c/d1d/d27/c13d 0 2024-06-15T01:07:28.500 INFO:tasks.workunit.client.0.smithi137.stdout:7/915: write df/d13/d2e/d1a/f26 [117514,42496] 0 2024-06-15T01:07:28.504 INFO:tasks.workunit.client.0.smithi137.stdout:7/916: rmdir df/d13/d2e/d1a/d6a/dde/d12d 0 2024-06-15T01:07:28.505 INFO:tasks.workunit.client.0.smithi137.stdout:7/917: write df/d13/d2e/f51 [1302358,7946] 0 2024-06-15T01:07:28.514 INFO:tasks.workunit.client.0.smithi137.stdout:7/918: dwrite df/d7d/d1c/d1d/d79/dbc/df3/f60 [4194304,4194304] 0 2024-06-15T01:07:28.517 INFO:tasks.workunit.client.0.smithi137.stdout:7/919: creat df/d7d/d1c/d1d/d27/d97/f13e x:0 0 0 2024-06-15T01:07:28.522 INFO:tasks.workunit.client.0.smithi137.stdout:7/920: rename df/d7d/d1c/d1d/d27/d97/c9f to df/d7d/d3b/c13f 0 2024-06-15T01:07:28.525 INFO:tasks.workunit.client.0.smithi137.stdout:7/921: read df/d13/d2e/d1a/d6a/dde/d89/fc7 [593539,76382] 0 2024-06-15T01:07:28.781 INFO:tasks.workunit.client.0.smithi137.stdout:9/911: sync 2024-06-15T01:07:28.781 INFO:tasks.workunit.client.0.smithi137.stdout:9/912: stat df/d13/cd9 0 2024-06-15T01:07:28.783 INFO:tasks.workunit.client.0.smithi137.stdout:9/913: creat df/d13/dcb/f114 x:0 0 0 2024-06-15T01:07:28.784 INFO:tasks.workunit.client.0.smithi137.stdout:9/914: fdatasync f2 0 2024-06-15T01:07:28.785 INFO:tasks.workunit.client.0.smithi137.stdout:9/915: stat df/d12/d4e/dbc/d3c/d51/ff9 0 2024-06-15T01:07:28.790 INFO:tasks.workunit.client.0.smithi137.stdout:9/916: dwrite df/d13/f71 [0,4194304] 0 2024-06-15T01:07:28.799 INFO:tasks.workunit.client.0.smithi137.stdout:9/917: dwrite df/d12/d4e/dbc/f92 [0,4194304] 0 2024-06-15T01:07:28.805 INFO:tasks.workunit.client.0.smithi137.stdout:9/918: dread df/d12/f3e [0,4194304] 0 2024-06-15T01:07:28.813 INFO:tasks.workunit.client.0.smithi137.stdout:9/919: dwrite df/d12/d29/d79/fc7 [4194304,4194304] 0 2024-06-15T01:07:28.818 INFO:tasks.workunit.client.0.smithi137.stdout:9/920: link df/d1a/da7/l107 df/d49/d72/l115 0 2024-06-15T01:07:28.823 INFO:tasks.workunit.client.0.smithi137.stdout:9/921: dread df/d12/d29/d4a/f7f [0,4194304] 0 2024-06-15T01:07:28.825 INFO:tasks.workunit.client.0.smithi137.stdout:9/922: unlink df/d2a/c3a 0 2024-06-15T01:07:28.826 INFO:tasks.workunit.client.0.smithi137.stdout:9/923: chown df/c97 3342 1 2024-06-15T01:07:28.830 INFO:tasks.workunit.client.0.smithi137.stdout:9/924: rename df/d12/d15/c37 to df/d12/d4e/dbc/d3c/d51/d10e/c116 0 2024-06-15T01:07:28.837 INFO:tasks.workunit.client.0.smithi137.stdout:9/925: write df/d12/d1b/f2f [4111694,11169] 0 2024-06-15T01:07:28.840 INFO:tasks.workunit.client.0.smithi137.stdout:7/922: sync 2024-06-15T01:07:28.840 INFO:tasks.workunit.client.0.smithi137.stdout:9/926: rename df/d1a/d10f to df/d12/d15/d117 0 2024-06-15T01:07:28.841 INFO:tasks.workunit.client.0.smithi137.stdout:9/927: chown df/d2a/d7e/ffe 1 1 2024-06-15T01:07:28.842 INFO:tasks.workunit.client.0.smithi137.stdout:7/923: fsync df/d13/d2e/d1a/d6a/dde/d89/f8a 0 2024-06-15T01:07:28.843 INFO:tasks.workunit.client.0.smithi137.stdout:9/928: chown df/d12/d4e/dbc/d3c/l6e 2416 1 2024-06-15T01:07:28.845 INFO:tasks.workunit.client.0.smithi137.stdout:7/924: creat df/d13/d2e/d1a/d6a/dfe/f140 x:0 0 0 2024-06-15T01:07:28.846 INFO:tasks.workunit.client.0.smithi137.stdout:7/925: chown df/d13/d2e/d1a/d6a/d90/ddf 203685652 1 2024-06-15T01:07:28.849 INFO:tasks.workunit.client.0.smithi137.stdout:7/926: mkdir df/d7d/d1c/d1d/d27/d33/d43/d6c/d141 0 2024-06-15T01:07:28.850 INFO:tasks.workunit.client.0.smithi137.stdout:7/927: fdatasync df/d13/d2e/d1a/dc0/fc3 0 2024-06-15T01:07:28.851 INFO:tasks.workunit.client.0.smithi137.stdout:9/929: truncate df/d12/f3e 613325 0 2024-06-15T01:07:28.854 INFO:tasks.workunit.client.0.smithi137.stdout:9/930: truncate df/d2a/f34 46421 0 2024-06-15T01:07:28.859 INFO:tasks.workunit.client.0.smithi137.stdout:7/928: dwrite df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/d10c/f113 [0,4194304] 0 2024-06-15T01:07:28.862 INFO:tasks.workunit.client.0.smithi137.stdout:7/929: fdatasync df/d7d/d3b/d5b/da6/fea 0 2024-06-15T01:07:28.863 INFO:tasks.workunit.client.0.smithi137.stdout:7/930: write df/d7d/d3b/f5f [248568,114133] 0 2024-06-15T01:07:28.874 INFO:tasks.workunit.client.0.smithi137.stdout:9/931: truncate df/d12/f3e 345540 0 2024-06-15T01:07:28.875 INFO:tasks.workunit.client.0.smithi137.stdout:9/932: dread - df/d12/d15/d75/ff2 zero size 2024-06-15T01:07:28.876 INFO:tasks.workunit.client.0.smithi137.stdout:9/933: mknod df/d13/c118 0 2024-06-15T01:07:28.883 INFO:tasks.workunit.client.0.smithi137.stdout:9/934: truncate df/d12/d4e/dbc/f92 770092 0 2024-06-15T01:07:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:9/935: write df/d12/d29/d66/fa2 [455720,59918] 0 2024-06-15T01:07:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:9/936: fdatasync df/d12/d15/f58 0 2024-06-15T01:07:28.892 INFO:tasks.workunit.client.0.smithi137.stdout:9/937: write df/d12/d15/d117/d42/f46 [2303339,62542] 0 2024-06-15T01:07:28.898 INFO:tasks.workunit.client.0.smithi137.stdout:9/938: fsync df/d12/d15/d75/f8a 0 2024-06-15T01:07:28.900 INFO:tasks.workunit.client.0.smithi137.stdout:9/939: creat df/d12/d4e/dbc/d3c/d51/f119 x:0 0 0 2024-06-15T01:07:28.902 INFO:tasks.workunit.client.0.smithi137.stdout:9/940: rmdir df/d12/d29/d79/d9a 39 2024-06-15T01:07:28.905 INFO:tasks.workunit.client.0.smithi137.stdout:9/941: symlink df/d12/d4e/dbc/l11a 0 2024-06-15T01:07:28.907 INFO:tasks.workunit.client.0.smithi137.stdout:9/942: rename df/d12/d29/d66/l10c to df/d13/l11b 0 2024-06-15T01:07:28.909 INFO:tasks.workunit.client.0.smithi137.stdout:9/943: creat df/d1a/f11c x:0 0 0 2024-06-15T01:07:28.910 INFO:tasks.workunit.client.0.smithi137.stdout:9/944: chown df/d12/d29/d66/l68 19952 1 2024-06-15T01:07:28.910 INFO:tasks.workunit.client.0.smithi137.stdout:9/945: chown df/d12/d1b 281096 1 2024-06-15T01:07:28.912 INFO:tasks.workunit.client.0.smithi137.stdout:9/946: mknod df/d1a/c11d 0 2024-06-15T01:07:28.913 INFO:tasks.workunit.client.0.smithi137.stdout:9/947: creat df/d12/d29/d66/f11e x:0 0 0 2024-06-15T01:07:28.915 INFO:tasks.workunit.client.0.smithi137.stdout:9/948: mknod df/d49/d4b/d9b/daf/c11f 0 2024-06-15T01:07:28.915 INFO:tasks.workunit.client.0.smithi137.stdout:9/949: chown df/d12/d29/d95/db3/fff 641303899 1 2024-06-15T01:07:28.916 INFO:tasks.workunit.client.0.smithi137.stdout:9/950: chown df/d12/d29/d95/db3 0 1 2024-06-15T01:07:28.918 INFO:tasks.workunit.client.0.smithi137.stdout:9/951: creat df/d49/f120 x:0 0 0 2024-06-15T01:07:28.925 INFO:tasks.workunit.client.0.smithi137.stdout:9/952: write df/d12/f1f [1522794,22294] 0 2024-06-15T01:07:28.926 INFO:tasks.workunit.client.0.smithi137.stdout:9/953: stat df/d12/d4e/la6 0 2024-06-15T01:07:28.929 INFO:tasks.workunit.client.0.smithi137.stdout:9/954: creat df/d12/d29/d79/f121 x:0 0 0 2024-06-15T01:07:28.930 INFO:tasks.workunit.client.0.smithi137.stdout:9/955: truncate df/d12/d15/f73 1087697 0 2024-06-15T01:07:28.931 INFO:tasks.workunit.client.0.smithi137.stdout:9/956: write df/d12/d15/d117/d42/f46 [1634006,87026] 0 2024-06-15T01:07:28.935 INFO:tasks.workunit.client.0.smithi137.stdout:9/957: creat df/d13/d6f/f122 x:0 0 0 2024-06-15T01:07:28.938 INFO:tasks.workunit.client.0.smithi137.stdout:9/958: creat df/d12/f123 x:0 0 0 2024-06-15T01:07:28.939 INFO:tasks.workunit.client.0.smithi137.stdout:9/959: mkdir df/d2a/d124 0 2024-06-15T01:07:28.945 INFO:tasks.workunit.client.0.smithi137.stdout:9/960: dwrite df/d1a/f11c [0,4194304] 0 2024-06-15T01:07:28.948 INFO:tasks.workunit.client.0.smithi137.stdout:9/961: mkdir df/d12/d4e/dc6/deb/d125 0 2024-06-15T01:07:28.951 INFO:tasks.workunit.client.0.smithi137.stdout:9/962: rename df/d1a/da7/cfb to df/d12/d29/d66/d101/c126 0 2024-06-15T01:07:28.953 INFO:tasks.workunit.client.0.smithi137.stdout:9/963: rmdir df/d12/d1b/df3 0 2024-06-15T01:07:28.955 INFO:tasks.workunit.client.0.smithi137.stdout:9/964: rmdir df/d12/d4e/dbc/d3c 39 2024-06-15T01:07:28.957 INFO:tasks.workunit.client.0.smithi137.stdout:9/965: fdatasync df/d13/d6f/f7b 0 2024-06-15T01:07:28.958 INFO:tasks.workunit.client.0.smithi137.stdout:9/966: dread - df/d12/d29/d66/d101/f102 zero size 2024-06-15T01:07:28.958 INFO:tasks.workunit.client.0.smithi137.stdout:9/967: chown df/c10 872208 1 2024-06-15T01:07:28.962 INFO:tasks.workunit.client.0.smithi137.stdout:9/968: link df/f28 df/d12/d29/d95/db3/f127 0 2024-06-15T01:07:28.962 INFO:tasks.workunit.client.0.smithi137.stdout:9/969: readlink df/d1a/l9f 0 2024-06-15T01:07:28.964 INFO:tasks.workunit.client.0.smithi137.stdout:9/970: dread - df/d13/dcb/f114 zero size 2024-06-15T01:07:28.969 INFO:tasks.workunit.client.0.smithi137.stdout:9/971: rename df/d49/d8d to df/d12/d15/d128 0 2024-06-15T01:07:28.970 INFO:tasks.workunit.client.0.smithi137.stdout:9/972: write df/d13/d6f/f122 [627201,112589] 0 2024-06-15T01:07:28.971 INFO:tasks.workunit.client.0.smithi137.stdout:9/973: chown df/d12/d29/d4a/f6c 73 1 2024-06-15T01:07:28.973 INFO:tasks.workunit.client.0.smithi137.stdout:9/974: fdatasync df/d12/d29/d4a/f5d 0 2024-06-15T01:07:28.974 INFO:tasks.workunit.client.0.smithi137.stdout:9/975: chown df/d12/d29/d66/dce/ce3 3628597 1 2024-06-15T01:07:28.977 INFO:tasks.workunit.client.0.smithi137.stdout:9/976: rename df/d49/ff5 to df/d49/d4b/d9b/f129 0 2024-06-15T01:07:28.978 INFO:tasks.workunit.client.0.smithi137.stdout:9/977: dread - df/d13/dcb/dd4/fe5 zero size 2024-06-15T01:07:28.982 INFO:tasks.workunit.client.0.smithi137.stdout:9/978: write df/d49/d4b/f57 [4286642,69092] 0 2024-06-15T01:07:28.985 INFO:tasks.workunit.client.0.smithi137.stdout:9/979: write df/d2a/fb4 [968232,42138] 0 2024-06-15T01:07:28.987 INFO:tasks.workunit.client.0.smithi137.stdout:9/980: truncate df/d12/d29/d79/fc5 4356455 0 2024-06-15T01:07:28.989 INFO:tasks.workunit.client.0.smithi137.stdout:9/981: mkdir df/d49/d12a 0 2024-06-15T01:07:29.001 INFO:tasks.workunit.client.0.smithi137.stdout:9/982: write df/f28 [4789126,34844] 0 2024-06-15T01:07:29.005 INFO:tasks.workunit.client.0.smithi137.stdout:9/983: write df/d12/d15/f63 [250594,123461] 0 2024-06-15T01:07:29.006 INFO:tasks.workunit.client.0.smithi137.stdout:9/984: chown df/le1 3184472 1 2024-06-15T01:07:29.008 INFO:tasks.workunit.client.0.smithi137.stdout:9/985: truncate df/d12/d4e/f5c 4420099 0 2024-06-15T01:07:29.010 INFO:tasks.workunit.client.0.smithi137.stdout:9/986: mkdir df/d12/d15/d128/d12b 0 2024-06-15T01:07:29.012 INFO:tasks.workunit.client.0.smithi137.stdout:9/987: write df/d12/d29/d79/d9a/fe8 [5100995,88840] 0 2024-06-15T01:07:29.015 INFO:tasks.workunit.client.0.smithi137.stdout:9/988: creat df/d13/dcb/dd0/f12c x:0 0 0 2024-06-15T01:07:29.017 INFO:tasks.workunit.client.0.smithi137.stdout:9/989: mkdir df/d49/d4b/d9b/daf/d12d 0 2024-06-15T01:07:29.019 INFO:tasks.workunit.client.0.smithi137.stdout:9/990: symlink df/d12/d15/d128/l12e 0 2024-06-15T01:07:29.033 INFO:tasks.workunit.client.0.smithi137.stdout:9/991: dread df/d49/d4b/f57 [0,4194304] 0 2024-06-15T01:07:29.036 INFO:tasks.workunit.client.0.smithi137.stdout:9/992: unlink df/d49/d4b/d9b/daf/c11f 0 2024-06-15T01:07:29.037 INFO:tasks.workunit.client.0.smithi137.stdout:9/993: fdatasync df/d12/fc4 0 2024-06-15T01:07:29.042 INFO:tasks.workunit.client.0.smithi137.stdout:9/994: dread df/d12/d29/d4a/fba [0,4194304] 0 2024-06-15T01:07:29.044 INFO:tasks.workunit.client.0.smithi137.stdout:9/995: symlink df/d13/d6f/l12f 0 2024-06-15T01:07:29.053 INFO:tasks.workunit.client.0.smithi137.stdout:9/996: dwrite df/d49/d72/fc2 [0,4194304] 0 2024-06-15T01:07:29.059 INFO:tasks.workunit.client.0.smithi137.stdout:9/997: rmdir df/d49/d4b/d9b/daf/d12d 0 2024-06-15T01:07:29.066 INFO:tasks.workunit.client.0.smithi137.stdout:9/998: dwrite df/d12/d29/d4a/f7f [0,4194304] 0 2024-06-15T01:07:29.067 INFO:tasks.workunit.client.0.smithi137.stdout:9/999: read df/d12/fc4 [703336,12792] 0 2024-06-15T01:07:29.264 INFO:tasks.workunit.client.0.smithi137.stdout:7/931: sync 2024-06-15T01:07:29.266 INFO:tasks.workunit.client.0.smithi137.stdout:7/932: read - df/d7d/d1c/d1d/d27/d33/fe5 zero size 2024-06-15T01:07:29.267 INFO:tasks.workunit.client.0.smithi137.stdout:7/933: chown df/d7d/d3b/d5b/fb9 1828832646 1 2024-06-15T01:07:29.275 INFO:tasks.workunit.client.0.smithi137.stdout:7/934: dwrite df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb [0,4194304] 0 2024-06-15T01:07:29.284 INFO:tasks.workunit.client.0.smithi137.stdout:7/935: dwrite df/d7d/d1c/d1d/d27/d33/d43/fff [0,4194304] 0 2024-06-15T01:07:29.302 INFO:tasks.workunit.client.0.smithi137.stdout:7/936: dwrite df/d13/f20 [0,4194304] 0 2024-06-15T01:07:29.304 INFO:tasks.workunit.client.0.smithi137.stdout:7/937: rmdir df/d7d/d1c/d1d/d27/d33 39 2024-06-15T01:07:29.312 INFO:tasks.workunit.client.0.smithi137.stdout:7/938: rename df/d7d/d1c/d1d/d27/d33/d43/dc5/d10b/l11c to df/d13/d2e/d87/l142 0 2024-06-15T01:07:29.315 INFO:tasks.workunit.client.0.smithi137.stdout:7/939: creat df/d13/d2e/f143 x:0 0 0 2024-06-15T01:07:29.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.320+0000 7fafac7d4700 1 -- 172.21.15.137:0/413850834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 msgr2=0x7fafa4103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.320+0000 7fafac7d4700 1 --2- 172.21.15.137:0/413850834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4103a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009870 tx=0x7fafa0009b80 comp rx=0 tx=0).stop 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 -- 172.21.15.137:0/413850834 shutdown_connections 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 --2- 172.21.15.137:0/413850834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 0x7fafa41063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 --2- 172.21.15.137:0/413850834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4103a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 -- 172.21.15.137:0/413850834 >> 172.21.15.137:0/413850834 conn(0x7fafa40fb030 msgr2=0x7fafa40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 -- 172.21.15.137:0/413850834 shutdown_connections 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.321+0000 7fafac7d4700 1 -- 172.21.15.137:0/413850834 wait complete. 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 Processor -- start 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 -- start start 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 0x7fafa40ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa4199e70 con 0x7fafa4103fd0 2024-06-15T01:07:29.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafac7d4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa4199fb0 con 0x7fafa41016a0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafaa570700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafaa570700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:44540/0 (socket says 172.21.15.137:44540) 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafaa570700 1 -- 172.21.15.137:0/3389877531 learned_addr learned my addr 172.21.15.137:0/3389877531 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.322+0000 7fafa9d6f700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 0x7fafa40ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafaa570700 1 -- 172.21.15.137:0/3389877531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 msgr2=0x7fafa40ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafaa570700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 0x7fafa40ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafaa570700 1 -- 172.21.15.137:0/3389877531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0009520 con 0x7fafa41016a0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafaa570700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009230 tx=0x7fafa000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa001c070 con 0x7fafa41016a0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa40ffb90 con 0x7fafa41016a0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fafa000fd10 con 0x7fafa41016a0 2024-06-15T01:07:29.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa0022e40 con 0x7fafa41016a0 2024-06-15T01:07:29.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.323+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa4100050 con 0x7fafa41016a0 2024-06-15T01:07:29.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.324+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa41903e0 con 0x7fafa41016a0 2024-06-15T01:07:29.322 INFO:tasks.workunit.client.0.smithi137.stdout:7/940: link df/d7d/d1c/d1d/d27/d33/d43/d6c/l4d df/d13/d2e/d87/l144 0 2024-06-15T01:07:29.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.325+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7fafa0020070 con 0x7fafa41016a0 2024-06-15T01:07:29.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.326+0000 7faf9b7fe700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 0x7faf9006e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.326+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fafa008c2f0 con 0x7fafa41016a0 2024-06-15T01:07:29.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.326+0000 7fafa9d6f700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 0x7faf9006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.327+0000 7fafa9d6f700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 0x7faf9006e030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf94005fd0 tx=0x7faf94009310 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:29.324 INFO:tasks.workunit.client.0.smithi137.stdout:7/941: symlink df/d7d/d1c/d1d/d27/d33/d43/d6c/l145 0 2024-06-15T01:07:29.326 INFO:tasks.workunit.client.0.smithi137.stdout:7/942: unlink df/d7d/d1c/d1d/d27/d33/d43/d6c/l4d 0 2024-06-15T01:07:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.331+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fafa005ada0 con 0x7fafa41016a0 2024-06-15T01:07:29.330 INFO:tasks.workunit.client.0.smithi137.stdout:7/943: getdents df/d7d/d121/de6 0 2024-06-15T01:07:29.332 INFO:tasks.workunit.client.0.smithi137.stdout:7/944: creat df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/f146 x:0 0 0 2024-06-15T01:07:29.333 INFO:tasks.workunit.client.0.smithi137.stdout:7/945: write df/d7d/d1c/d1d/d79/dbc/df3/fac [201993,96628] 0 2024-06-15T01:07:29.336 INFO:tasks.workunit.client.0.smithi137.stdout:7/946: mknod df/d7d/d1c/d1d/d27/d33/d43/d6c/d114/c147 0 2024-06-15T01:07:29.355 INFO:tasks.workunit.client.0.smithi137.stdout:7/947: dwrite df/d13/d2e/d1a/d6a/dde/d89/fc9 [0,4194304] 0 2024-06-15T01:07:29.360 INFO:tasks.workunit.client.0.smithi137.stdout:7/948: rename df/d7d/d3b/c13f to df/d7d/d1c/d1d/d27/d97/c148 0 2024-06-15T01:07:29.361 INFO:tasks.workunit.client.0.smithi137.stdout:7/949: write df/d7d/d3b/d5b/da6/f127 [79529,81935] 0 2024-06-15T01:07:29.363 INFO:tasks.workunit.client.0.smithi137.stdout:7/950: readlink df/d7d/d3b/d5b/da6/l133 0 2024-06-15T01:07:29.366 INFO:tasks.workunit.client.0.smithi137.stdout:7/951: write df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/db3/fdb [688380,54818] 0 2024-06-15T01:07:29.369 INFO:tasks.workunit.client.0.smithi137.stdout:7/952: creat df/f149 x:0 0 0 2024-06-15T01:07:29.386 INFO:tasks.workunit.client.0.smithi137.stdout:7/953: dread df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d7a/fc1 [0,4194304] 0 2024-06-15T01:07:29.389 INFO:tasks.workunit.client.0.smithi137.stdout:7/954: creat df/d7d/d1c/d1d/d27/d33/d43/de0/f14a x:0 0 0 2024-06-15T01:07:29.391 INFO:tasks.workunit.client.0.smithi137.stdout:7/955: dread df/d13/d2e/d1a/d6a/dde/d89/fc7 [0,4194304] 0 2024-06-15T01:07:29.394 INFO:tasks.workunit.client.0.smithi137.stdout:7/956: fsync df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/f12f 0 2024-06-15T01:07:29.395 INFO:tasks.workunit.client.0.smithi137.stdout:7/957: chown df/d13/d2e/d1a/d6a/d90/dcf 148242849 1 2024-06-15T01:07:29.405 INFO:tasks.workunit.client.0.smithi137.stdout:7/958: dwrite df/d7d/d1c/f1f [0,4194304] 0 2024-06-15T01:07:29.411 INFO:tasks.workunit.client.0.smithi137.stdout:7/959: link df/d7d/d1c/d1d/d79/cb2 df/d13/d2e/d1a/d6a/c14b 0 2024-06-15T01:07:29.414 INFO:tasks.workunit.client.0.smithi137.stdout:7/960: creat df/d7d/d1c/d1d/d27/d33/d43/de0/f14c x:0 0 0 2024-06-15T01:07:29.417 INFO:tasks.workunit.client.0.smithi137.stdout:7/961: getdents df/d7d/de9 0 2024-06-15T01:07:29.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.616+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafa402cef0 con 0x7faf9006bb70 2024-06-15T01:07:29.612 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:29 smithi137 ceph-mon[29354]: pgmap v30: 65 pgs: 65 active+clean; 3.0 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 35 MiB/s rd, 80 MiB/s wr, 284 op/s 2024-06-15T01:07:29.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.617+0000 7faf9b7fe700 1 -- 172.21.15.137:0/3389877531 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fafa402cef0 con 0x7faf9006bb70 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 msgr2=0x7faf9006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 0x7faf9006e030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf94005fd0 tx=0x7faf94009310 comp rx=0 tx=0).stop 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 msgr2=0x7fafa4101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009230 tx=0x7fafa000f460 comp rx=0 tx=0).stop 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 shutdown_connections 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7faf9006bb70 0x7faf9006e030 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa41016a0 0x7fafa4101000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 --2- 172.21.15.137:0/3389877531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa4103fd0 0x7fafa40ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.619+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 >> 172.21.15.137:0/3389877531 conn(0x7fafa40fb030 msgr2=0x7fafa40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.620+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 shutdown_connections 2024-06-15T01:07:29.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.620+0000 7fafac7d4700 1 -- 172.21.15.137:0/3389877531 wait complete. 2024-06-15T01:07:29.618 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-15T01:07:29.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-06-15T01:07:29.626 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:07:29.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 -- 172.21.15.137:0/2415739108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 msgr2=0x7f80a81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 --2- 172.21.15.137:0/2415739108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81008b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f809c00aa20 tx=0x7f809c00ad30 comp rx=0 tx=0).stop 2024-06-15T01:07:29.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 -- 172.21.15.137:0/2415739108 shutdown_connections 2024-06-15T01:07:29.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 --2- 172.21.15.137:0/2415739108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81008b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 --2- 172.21.15.137:0/2415739108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 0x7f80a80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 -- 172.21.15.137:0/2415739108 >> 172.21.15.137:0/2415739108 conn(0x7f80a80fa7f0 msgr2=0x7f80a80fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 -- 172.21.15.137:0/2415739108 shutdown_connections 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 -- 172.21.15.137:0/2415739108 wait complete. 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.727+0000 7f80adceb700 1 Processor -- start 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80adceb700 1 -- start start 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80adceb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 0x7f80a8196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80adceb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80adceb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8196df0 con 0x7f80a8100450 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80adceb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8196f30 con 0x7f80a80ff250 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45000/0 (socket says 172.21.15.137:45000) 2024-06-15T01:07:29.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 -- 172.21.15.137:0/3662323643 learned_addr learned my addr 172.21.15.137:0/3662323643 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a77fe700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 0x7f80a8196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 -- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 msgr2=0x7f80a8196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 0x7f80a8196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 -- 172.21.15.137:0/3662323643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c00a700 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.728+0000 7f80a6ffd700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f809c000c00 tx=0x7f809c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.729+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c004b70 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.729+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f809c004cd0 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.729+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a819b980 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.729+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809c016680 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.729+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a819bef0 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.730+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a81073e0 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.730+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f809c00e450 con 0x7f80a8100450 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.731+0000 7f80a4ff9700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 0x7f809006e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:29.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.731+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f809c012070 con 0x7f80a8100450 2024-06-15T01:07:29.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.731+0000 7f80a77fe700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 0x7f809006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:29.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.731+0000 7f80a77fe700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 0x7f809006e030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80a81002b0 tx=0x7f809800b410 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:29.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:29.735+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f809c0c0650 con 0x7f80a8100450 2024-06-15T01:07:29.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:29 smithi142 ceph-mon[37379]: pgmap v30: 65 pgs: 65 active+clean; 3.0 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 35 MiB/s rd, 80 MiB/s wr, 284 op/s 2024-06-15T01:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.030+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a8104da0 con 0x7f809006bb70 2024-06-15T01:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.031+0000 7f80a4ff9700 1 -- 172.21.15.137:0/3662323643 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f80a8104da0 con 0x7f809006bb70 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.033+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 msgr2=0x7f809006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.033+0000 7f80adceb700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 0x7f809006e030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80a81002b0 tx=0x7f809800b410 comp rx=0 tx=0).stop 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.033+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 msgr2=0x7f80a81967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.033+0000 7f80adceb700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f809c000c00 tx=0x7f809c00cd20 comp rx=0 tx=0).stop 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 shutdown_connections 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f809006bb70 0x7f809006e030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a80ff250 0x7f80a8196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 --2- 172.21.15.137:0/3662323643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80a8100450 0x7f80a81967d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 >> 172.21.15.137:0/3662323643 conn(0x7f80a80fa7f0 msgr2=0x7f80a8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 shutdown_connections 2024-06-15T01:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.034+0000 7f80adceb700 1 -- 172.21.15.137:0/3662323643 wait complete. 2024-06-15T01:07:30.048 INFO:tasks.workunit.client.0.smithi137.stdout:7/962: sync 2024-06-15T01:07:30.051 INFO:tasks.workunit.client.0.smithi137.stdout:7/963: rmdir df/d13/d2e/d1a/d6a/d90/ddf/d110 0 2024-06-15T01:07:30.055 INFO:tasks.workunit.client.0.smithi137.stdout:7/964: rename df/d7d/d1c/ceb to df/d13/d2e/d1a/c14d 0 2024-06-15T01:07:30.060 INFO:tasks.workunit.client.0.smithi137.stdout:7/965: rename df/d13/d2e/d1a/lc2 to df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/d82/l14e 0 2024-06-15T01:07:30.087 INFO:tasks.workunit.client.0.smithi137.stdout:7/966: write df/d7d/d1c/d1d/d27/d33/d43/ff0 [838277,107615] 0 2024-06-15T01:07:30.088 INFO:tasks.workunit.client.0.smithi137.stdout:7/967: chown df/d13/d2e/d1a/de4/f13b 108031 1 2024-06-15T01:07:30.098 INFO:tasks.workunit.client.0.smithi137.stdout:7/968: dwrite df/d13/d2e/d1a/f68 [0,4194304] 0 2024-06-15T01:07:30.099 INFO:tasks.workunit.client.0.smithi137.stdout:7/969: fsync df/d7d/d1c/d1d/d79/dbc/f131 0 2024-06-15T01:07:30.101 INFO:tasks.workunit.client.0.smithi137.stdout:7/970: write df/d7d/d1c/d1d/dc4/f119 [551221,112408] 0 2024-06-15T01:07:30.105 INFO:tasks.workunit.client.0.smithi137.stdout:7/971: fsync df/d7d/d3b/d5b/f125 0 2024-06-15T01:07:30.107 INFO:tasks.workunit.client.0.smithi137.stdout:7/972: chown df/d13/d2e/d1a/d6a/d90/ddf 348 1 2024-06-15T01:07:30.111 INFO:tasks.workunit.client.0.smithi137.stdout:7/973: getdents df/d7d/d1c/d1d/d27 0 2024-06-15T01:07:30.115 INFO:tasks.workunit.client.0.smithi137.stdout:7/974: getdents df/d13/d2e/d1a/de4 0 2024-06-15T01:07:30.117 INFO:tasks.workunit.client.0.smithi137.stdout:7/975: write df/d13/d2e/d1a/d6a/dde/d89/fc9 [3774209,18778] 0 2024-06-15T01:07:30.129 INFO:tasks.workunit.client.0.smithi137.stdout:7/976: dwrite df/d7d/d1c/f9c [0,4194304] 0 2024-06-15T01:07:30.145 INFO:tasks.workunit.client.0.smithi137.stdout:7/977: truncate df/d7d/d1c/d1d/dc4/f119 201886 0 2024-06-15T01:07:30.148 INFO:tasks.workunit.client.0.smithi137.stdout:7/978: mkdir df/d13/d2e/d1a/d6a/d90/ddf/d14f 0 2024-06-15T01:07:30.188 INFO:tasks.workunit.client.0.smithi137.stdout:7/979: dread df/d7d/d3b/f93 [0,4194304] 0 2024-06-15T01:07:30.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 -- 172.21.15.137:0/3733542726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa8101da0 msgr2=0x7f4aa8104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 --2- 172.21.15.137:0/3733542726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa8101da0 0x7f4aa8104230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009910 tx=0x7f4a98009c20 comp rx=0 tx=0).stop 2024-06-15T01:07:30.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 -- 172.21.15.137:0/3733542726 shutdown_connections 2024-06-15T01:07:30.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 --2- 172.21.15.137:0/3733542726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa8101da0 0x7f4aa8104230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 --2- 172.21.15.137:0/3733542726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa80ff430 0x7f4aa8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 -- 172.21.15.137:0/3733542726 >> 172.21.15.137:0/3733542726 conn(0x7f4aa80fb030 msgr2=0x7f4aa80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:30.194 INFO:tasks.workunit.client.0.smithi137.stdout:7/980: creat df/d13/d2e/d1a/d6a/d90/f150 x:0 0 0 2024-06-15T01:07:30.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 -- 172.21.15.137:0/3733542726 shutdown_connections 2024-06-15T01:07:30.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.197+0000 7f4aae6ee700 1 -- 172.21.15.137:0/3733542726 wait complete. 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.198+0000 7f4aae6ee700 1 Processor -- start 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.198+0000 7f4aae6ee700 1 -- start start 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aae6ee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aae6ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa8101da0 0x7f4aa819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aae6ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa819d3a0 con 0x7f4aa8101da0 2024-06-15T01:07:30.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aae6ee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa819d4e0 con 0x7f4aa80ff430 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aa7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aa7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:44580/0 (socket says 172.21.15.137:44580) 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4aa7fff700 1 -- 172.21.15.137:0/2354703183 learned_addr learned my addr 172.21.15.137:0/2354703183 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.199+0000 7f4a9f7fe700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa8101da0 0x7f4aa819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.200+0000 7f4aa7fff700 1 -- 172.21.15.137:0/2354703183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa8101da0 msgr2=0x7f4aa819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.200+0000 7f4aa7fff700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa8101da0 0x7f4aa819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.200+0000 7f4aa7fff700 1 -- 172.21.15.137:0/2354703183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980095f0 con 0x7f4aa80ff430 2024-06-15T01:07:30.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.200+0000 7f4aa7fff700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000d900 tx=0x7f4a9000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:30.197 INFO:tasks.workunit.client.0.smithi137.stdout:7/981: mkdir df/d7d/d1c/d1d/d27/d151 0 2024-06-15T01:07:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.201+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a900041f0 con 0x7f4aa80ff430 2024-06-15T01:07:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.201+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4a90004cc0 con 0x7f4aa80ff430 2024-06-15T01:07:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.201+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a900056d0 con 0x7f4aa80ff430 2024-06-15T01:07:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.201+0000 7f4aae6ee700 1 -- 172.21.15.137:0/2354703183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa81a1f90 con 0x7f4aa80ff430 2024-06-15T01:07:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.201+0000 7f4aae6ee700 1 -- 172.21.15.137:0/2354703183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa81a2400 con 0x7f4aa80ff430 2024-06-15T01:07:30.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.203+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa810fbc0 con 0x7f4aa80ff430 2024-06-15T01:07:30.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.204+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f4a90004750 con 0x7f4aa80ff430 2024-06-15T01:07:30.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.204+0000 7f4aa5ffb700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 0x7f4a9406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.204+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4a9008aef0 con 0x7f4aa80ff430 2024-06-15T01:07:30.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.205+0000 7f4a9f7fe700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 0x7f4a9406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.207+0000 7f4a9f7fe700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 0x7f4a9406e180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b5c0 tx=0x7f4a98005fb0 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:30.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.211+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f4a900599b0 con 0x7f4aa80ff430 2024-06-15T01:07:30.208 INFO:tasks.workunit.client.0.smithi137.stdout:7/982: write df/d7d/d1c/d1d/d79/fd8 [254465,3250] 0 2024-06-15T01:07:30.218 INFO:tasks.workunit.client.0.smithi137.stdout:7/983: rename df/d7d/d1c/d1d/d27/d33/d43/de0/f14a to df/d7d/d121/f152 0 2024-06-15T01:07:30.219 INFO:tasks.workunit.client.0.smithi137.stdout:7/984: write df/d13/d2e/d1a/d6a/dde/d89/fc8 [4869327,69261] 0 2024-06-15T01:07:30.228 INFO:tasks.workunit.client.0.smithi137.stdout:7/985: write df/d7d/d1c/d1d/d27/d33/d43/d6c/fda [966764,122070] 0 2024-06-15T01:07:30.230 INFO:tasks.workunit.client.0.smithi137.stdout:7/986: write df/d7d/d1c/d1d/d79/dbc/ff7 [2062708,79266] 0 2024-06-15T01:07:30.233 INFO:tasks.workunit.client.0.smithi137.stdout:7/987: truncate df/d7d/d1c/d1d/d27/d33/d43/de0/f14c 469799 0 2024-06-15T01:07:30.238 INFO:tasks.workunit.client.0.smithi137.stdout:7/988: dread df/d7d/d1c/d1d/d79/ffc [0,4194304] 0 2024-06-15T01:07:30.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.456+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4aa810fd50 con 0x7f4a9406bcc0 2024-06-15T01:07:30.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.466+0000 7f4aa5ffb700 1 -- 172.21.15.137:0/2354703183 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4aa810fd50 con 0x7f4a9406bcc0 2024-06-15T01:07:30.462 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:07:30.462 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (7m) 46s ago 8m 25.5M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:07:30.462 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (8m) 46s ago 8m 7746k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (7m) 46s ago 7m 7386k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (8m) 46s ago 8m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (7m) 46s ago 7m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (6m) 46s ago 8m 86.7M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (3m) 46s ago 3m 103M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (3m) 46s ago 3m 107M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (3m) 46s ago 3m 12.9M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (2m) 46s ago 2m 15.9M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:9283,8765,8443 running (10m) 46s ago 10m 111M - 18.2.1 5be31c24972a 61486e562eb9 2024-06-15T01:07:30.463 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (66s) 46s ago 7m 462M - 18.2.2-1443-g6f4b0956 e7282a3bffba df5c0f028b61 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (10m) 46s ago 10m 49.7M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (7m) 46s ago 7m 40.4M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (8m) 46s ago 8m 16.4M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 46s ago 7m 16.4M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (6m) 46s ago 6m 111M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (6m) 46s ago 6m 121M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (5m) 46s ago 5m 102M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (5m) 46s ago 5m 124M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (4m) 46s ago 4m 116M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (4m) 46s ago 4m 104M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:07:30.464 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (6m) 46s ago 8m 58.5M - 2.43.0 a07b618ecd1d 904ef44cd0a5 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 msgr2=0x7f4a9406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 0x7f4a9406e180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800b5c0 tx=0x7f4a98005fb0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 msgr2=0x7f4aa819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000d900 tx=0x7f4a9000dcc0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 shutdown_connections 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f4a9406bcc0 0x7f4a9406e180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4aa80ff430 0x7f4aa819c840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 --2- 172.21.15.137:0/2354703183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa8101da0 0x7f4aa819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 >> 172.21.15.137:0/2354703183 conn(0x7f4aa80fb030 msgr2=0x7f4aa80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:30.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 shutdown_connections 2024-06-15T01:07:30.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.468+0000 7f4a9effd700 1 -- 172.21.15.137:0/2354703183 wait complete. 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 -- 172.21.15.137:0/3676638931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6440ffab0 msgr2=0x7fa6440ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/3676638931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6440ffab0 0x7fa6440ffed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa634009910 tx=0x7fa634009c20 comp rx=0 tx=0).stop 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 -- 172.21.15.137:0/3676638931 shutdown_connections 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/3676638931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa644100cb0 0x7fa644101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/3676638931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6440ffab0 0x7fa6440ffed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 -- 172.21.15.137:0/3676638931 >> 172.21.15.137:0/3676638931 conn(0x7fa6440fb050 msgr2=0x7fa6440fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 -- 172.21.15.137:0/3676638931 shutdown_connections 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.591+0000 7fa64a8f2700 1 -- 172.21.15.137:0/3676638931 wait complete. 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 Processor -- start 2024-06-15T01:07:30.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 -- start start 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644196e00 con 0x7fa644100cb0 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa64a8f2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644196f40 con 0x7fa6440ffab0 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa643fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45030/0 (socket says 172.21.15.137:45030) 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa643fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:44590/0 (socket says 172.21.15.137:44590) 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.592+0000 7fa6437fe700 1 -- 172.21.15.137:0/1688440919 learned_addr learned my addr 172.21.15.137:0/1688440919 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa643fff700 1 -- 172.21.15.137:0/1688440919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 msgr2=0x7fa6441967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa643fff700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa643fff700 1 -- 172.21.15.137:0/1688440919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340095f0 con 0x7fa6440ffab0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa6437fe700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa643fff700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa63400bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63401d070 con 0x7fa6440ffab0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64419b990 con 0x7fa6440ffab0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa634005710 con 0x7fa6440ffab0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64419bea0 con 0x7fa6440ffab0 2024-06-15T01:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.593+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa634003980 con 0x7fa6440ffab0 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.594+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6441084d0 con 0x7fa6440ffab0 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.595+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7fa634003ae0 con 0x7fa6440ffab0 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.595+0000 7fa6417fa700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 0x7fa62c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.595+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa63408c1d0 con 0x7fa6440ffab0 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.595+0000 7fa6437fe700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 0x7fa62c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.596+0000 7fa6437fe700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 0x7fa62c06e030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa638011550 tx=0x7fa638009260 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:30.600+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fa63405aba0 con 0x7fa6440ffab0 2024-06-15T01:07:30.695 INFO:tasks.workunit.client.0.smithi137.stdout:7/989: sync 2024-06-15T01:07:30.700 INFO:tasks.workunit.client.0.smithi137.stdout:7/990: link df/d13/d2e/d1a/d6a/d90/ddf/fed df/d7d/d1c/d1d/d27/d33/d43/d6c/d4a/f153 0 2024-06-15T01:07:30.704 INFO:tasks.workunit.client.0.smithi137.stdout:7/991: dwrite df/d13/d2e/d1a/d6a/dde/d89/fc9 [0,4194304] 0 2024-06-15T01:07:30.707 INFO:tasks.workunit.client.0.smithi137.stdout:7/992: dread df/d7d/d1c/d1d/d27/d33/d43/de0/f14c [0,4194304] 0 2024-06-15T01:07:30.710 INFO:tasks.workunit.client.0.smithi137.stdout:7/993: creat df/d7d/d1c/d1d/d27/d151/f154 x:0 0 0 2024-06-15T01:07:30.713 INFO:tasks.workunit.client.0.smithi137.stdout:7/994: fsync df/d13/d2e/f4f 0 2024-06-15T01:07:30.721 INFO:tasks.workunit.client.0.smithi137.stdout:7/995: dwrite df/d7d/d3b/d5b/da6/f137 [0,4194304] 0 2024-06-15T01:07:30.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:30 smithi137 ceph-mon[29354]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:30.732 INFO:tasks.workunit.client.0.smithi137.stdout:7/996: truncate df/d7d/d1c/d1d/d27/d97/fb1 897324 0 2024-06-15T01:07:30.745 INFO:tasks.workunit.client.0.smithi137.stdout:7/997: dwrite df/d13/d2e/f86 [0,4194304] 0 2024-06-15T01:07:30.746 INFO:tasks.workunit.client.0.smithi137.stdout:7/998: fsync df/d13/d2e/d62/fee 0 2024-06-15T01:07:30.753 INFO:tasks.workunit.client.0.smithi137.stdout:7/999: getdents df/d13/d2e/d87 0 2024-06-15T01:07:30.758 INFO:tasks.workunit.client.0.smithi137.stderr:+ rm -rf -- ./tmp.5akVWFJBP9 2024-06-15T01:07:30.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:30 smithi142 ceph-mon[37379]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.038+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa64419c5a0 con 0x7fa6440ffab0 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.039+0000 7fa6417fa700 1 -- 172.21.15.137:0/1688440919 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7fa63405a730 con 0x7fa6440ffab0 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-15T01:07:31.035 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 1 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 1 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:07:31.036 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.042+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 msgr2=0x7fa62c06e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.042+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 0x7fa62c06e030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa638011550 tx=0x7fa638009260 comp rx=0 tx=0).stop 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.042+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 msgr2=0x7fa6441962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.042+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa63400bde0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 shutdown_connections 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7fa62c06bb70 0x7fa62c06e030 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6440ffab0 0x7fa6441962a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 --2- 172.21.15.137:0/1688440919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa644100cb0 0x7fa6441967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 >> 172.21.15.137:0/1688440919 conn(0x7fa6440fb050 msgr2=0x7fa644068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 shutdown_connections 2024-06-15T01:07:31.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.043+0000 7fa64a8f2700 1 -- 172.21.15.137:0/1688440919 wait complete. 2024-06-15T01:07:31.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.210+0000 7f43d6dbe700 1 -- 172.21.15.137:0/512154492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d010a370 msgr2=0x7f43d010c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.210+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/512154492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d010a370 0x7f43d010c7e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009910 tx=0x7f43c4009c20 comp rx=0 tx=0).stop 2024-06-15T01:07:31.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.211+0000 7f43d6dbe700 1 -- 172.21.15.137:0/512154492 shutdown_connections 2024-06-15T01:07:31.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.211+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/512154492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d010a370 0x7f43d010c7e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.211+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/512154492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d0072b80 0x7f43d0109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.211+0000 7f43d6dbe700 1 -- 172.21.15.137:0/512154492 >> 172.21.15.137:0/512154492 conn(0x7f43d006d9f0 msgr2=0x7f43d006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:31.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.212+0000 7f43d6dbe700 1 -- 172.21.15.137:0/512154492 shutdown_connections 2024-06-15T01:07:31.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.213+0000 7f43d6dbe700 1 -- 172.21.15.137:0/512154492 wait complete. 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.213+0000 7f43d6dbe700 1 Processor -- start 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.213+0000 7f43d6dbe700 1 -- start start 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43d6dbe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d0072b80 0x7f43d01a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43d6dbe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43d6dbe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01a5ce0 con 0x7f43d0072b80 2024-06-15T01:07:31.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43d6dbe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01a5e20 con 0x7f43d010a370 2024-06-15T01:07:31.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43cffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:31.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43cffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:44614/0 (socket says 172.21.15.137:44614) 2024-06-15T01:07:31.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.214+0000 7f43cffff700 1 -- 172.21.15.137:0/3022930355 learned_addr learned my addr 172.21.15.137:0/3022930355 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.215+0000 7f43cffff700 1 -- 172.21.15.137:0/3022930355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d0072b80 msgr2=0x7f43d01a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.215+0000 7f43cffff700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d0072b80 0x7f43d01a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.215+0000 7f43cffff700 1 -- 172.21.15.137:0/3022930355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40095f0 con 0x7f43d010a370 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.215+0000 7f43cffff700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43c400bd70 tx=0x7f43c400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.216+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43c401d070 con 0x7f43d010a370 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.216+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f43c400f460 con 0x7f43d010a370 2024-06-15T01:07:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.216+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43c4021640 con 0x7f43d010a370 2024-06-15T01:07:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.216+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d01aa870 con 0x7f43d010a370 2024-06-15T01:07:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.217+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d01aade0 con 0x7f43d010a370 2024-06-15T01:07:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.220+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 89991+0+0 (secure 0 0 0) 0x7f43c400fa40 con 0x7f43d010a370 2024-06-15T01:07:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.219+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d019f250 con 0x7f43d010a370 2024-06-15T01:07:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.221+0000 7f43cdffb700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 0x7f43c006e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:07:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.221+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f43c408c260 con 0x7f43d010a370 2024-06-15T01:07:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.221+0000 7f43d4b5a700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 0x7f43c006e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:07:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.222+0000 7f43d4b5a700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 0x7f43c006e180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0097b0 tx=0x7f43bc006d20 comp rx=0 tx=0).ready entity=mgr.24355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:07:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.228+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f43c405abf0 con 0x7f43d010a370 2024-06-15T01:07:31.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.570+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 --> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43d00611d0 con 0x7f43c006bcc0 2024-06-15T01:07:31.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.571+0000 7f43cdffb700 1 -- 172.21.15.137:0/3022930355 <== mgr.24355 v2:172.21.15.142:6828/2339029036 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f43d00611d0 con 0x7f43c006bcc0 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [], 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "1/2 daemons upgraded", 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "", 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:07:31.570 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:07:31.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 msgr2=0x7f43c006e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 0x7f43c006e180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0097b0 tx=0x7f43bc006d20 comp rx=0 tx=0).stop 2024-06-15T01:07:31.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 msgr2=0x7f43d01a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:07:31.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43c400bd70 tx=0x7f43c400be50 comp rx=0 tx=0).stop 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 shutdown_connections 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:6828/2339029036,v1:172.21.15.142:6829/2339029036] conn(0x7f43c006bcc0 0x7f43c006e180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d0072b80 0x7f43d01a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 --2- 172.21.15.137:0/3022930355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d010a370 0x7f43d01a56c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 >> 172.21.15.137:0/3022930355 conn(0x7f43d006d9f0 msgr2=0x7f43d006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 shutdown_connections 2024-06-15T01:07:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:07:31.574+0000 7f43d6dbe700 1 -- 172.21.15.137:0/3022930355 wait complete. 2024-06-15T01:07:31.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:31 smithi137 ceph-mon[29354]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:31.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:31 smithi137 ceph-mon[29354]: pgmap v31: 65 pgs: 65 active+clean; 2.6 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 38 MiB/s rd, 87 MiB/s wr, 316 op/s 2024-06-15T01:07:31.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:31 smithi137 ceph-mon[29354]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:31.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:31 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1688440919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:07:31.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:31 smithi142 ceph-mon[37379]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:31.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:31 smithi142 ceph-mon[37379]: pgmap v31: 65 pgs: 65 active+clean; 2.6 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 38 MiB/s rd, 87 MiB/s wr, 316 op/s 2024-06-15T01:07:31.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:31 smithi142 ceph-mon[37379]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:31.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:31 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1688440919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:07:33.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:33.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:33.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:33.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:07:33.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:07:33.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:32 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:33.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:07:33.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:33.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:33.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:07:33.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:07:33.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:32 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: Upgrade: Need to upgrade myself (mgr.smithi142.dllryb) 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: Upgrade: Need to upgrade myself (mgr.smithi142.dllryb) 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 41 MiB/s rd, 99 MiB/s wr, 383 op/s 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:07:34.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:33 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: Upgrade: Need to upgrade myself (mgr.smithi142.dllryb) 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: Upgrade: Need to upgrade myself (mgr.smithi142.dllryb) 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 41 MiB/s rd, 99 MiB/s wr, 383 op/s 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi137.nbzbyc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:07:34.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:33 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:35.200 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:34 smithi137 ceph-mon[29354]: Upgrade: Updating mgr.smithi137.nbzbyc 2024-06-15T01:07:35.200 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:34 smithi137 ceph-mon[29354]: Deploying daemon mgr.smithi137.nbzbyc on smithi137 2024-06-15T01:07:35.200 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:34 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:35.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:34 smithi142 ceph-mon[37379]: Upgrade: Updating mgr.smithi137.nbzbyc 2024-06-15T01:07:35.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:34 smithi142 ceph-mon[37379]: Deploying daemon mgr.smithi137.nbzbyc on smithi137 2024-06-15T01:07:35.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:34 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:36.210 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:35 smithi137 ceph-mon[29354]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 23 MiB/s rd, 53 MiB/s wr, 233 op/s 2024-06-15T01:07:36.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:35 smithi142 ceph-mon[37379]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 23 MiB/s rd, 53 MiB/s wr, 233 op/s 2024-06-15T01:07:37.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:37 smithi137 ceph-mon[29354]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 23 MiB/s rd, 55 MiB/s wr, 343 op/s 2024-06-15T01:07:37.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:37 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:37.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:37 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:37.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:37 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:38.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:37 smithi142 ceph-mon[37379]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 23 MiB/s rd, 55 MiB/s wr, 343 op/s 2024-06-15T01:07:38.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:37 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:38.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:37 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:38.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:37 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:40.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:39 smithi137 ceph-mon[29354]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 207 op/s 2024-06-15T01:07:40.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:39 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:07:40.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:39 smithi142 ceph-mon[37379]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 207 op/s 2024-06-15T01:07:40.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:39 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:07:42.104 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:41 smithi137 ceph-mon[29354]: pgmap v36: 65 pgs: 65 active+clean; 1.5 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 299 op/s 2024-06-15T01:07:42.280 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:41 smithi142 ceph-mon[37379]: pgmap v36: 65 pgs: 65 active+clean; 1.5 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 6.6 MiB/s rd, 21 MiB/s wr, 299 op/s 2024-06-15T01:07:44.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:43 smithi137 ceph-mon[29354]: pgmap v37: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 3.3 MiB/s rd, 14 MiB/s wr, 304 op/s 2024-06-15T01:07:44.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:43 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:43 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:43 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:43 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:43 smithi142 ceph-mon[37379]: pgmap v37: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 3.3 MiB/s rd, 14 MiB/s wr, 304 op/s 2024-06-15T01:07:44.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:43 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:43 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:43 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:44.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:43 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: Standby manager daemon smithi137.nbzbyc restarted 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: Standby manager daemon smithi137.nbzbyc started 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/crt"}]: dispatch 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/key"}]: dispatch 2024-06-15T01:07:45.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:44 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: Standby manager daemon smithi137.nbzbyc restarted 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: Standby manager daemon smithi137.nbzbyc started 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/crt"}]: dispatch 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi137.nbzbyc/key"}]: dispatch 2024-06-15T01:07:45.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:44 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:07:46.190 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-15T01:07:46.191 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-15T01:07:46.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:45 smithi137 ceph-mon[29354]: pgmap v38: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 26 KiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-06-15T01:07:46.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:45 smithi137 ceph-mon[29354]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:07:46.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:45 smithi137 ceph-mon[29354]: mgrmap e25: smithi142.dllryb(active, since 70s), standbys: smithi137.nbzbyc 2024-06-15T01:07:46.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:45 smithi142 ceph-mon[37379]: pgmap v38: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 26 KiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-06-15T01:07:46.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:45 smithi142 ceph-mon[37379]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:07:46.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:45 smithi142 ceph-mon[37379]: mgrmap e25: smithi142.dllryb(active, since 70s), standbys: smithi137.nbzbyc 2024-06-15T01:07:48.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: pgmap v39: 65 pgs: 65 active+clean; 784 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 41 KiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-06-15T01:07:48.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd='[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]': finished 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: mgrmap e26: smithi137.nbzbyc(active, starting, since 0.00744244s) 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:07:48.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T01:07:48.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T01:07:48.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T01:07:48.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T01:07:48.320 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:47 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:48.419 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: pgmap v39: 65 pgs: 65 active+clean; 784 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail; 41 KiB/s rd, 3.0 MiB/s wr, 357 op/s 2024-06-15T01:07:48.419 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.419 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.419 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 172.21.15.142:0/3009706893' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24355 ' entity='mgr.smithi142.dllryb' cmd='[{"prefix": "mgr fail", "who": "smithi142.dllryb"}]': finished 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: mgrmap e26: smithi137.nbzbyc(active, starting, since 0.00744244s) 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:07:48.420 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi137.nbzbyc", "id": "smithi137.nbzbyc"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: Manager daemon smithi137.nbzbyc is now available 2024-06-15T01:07:48.421 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:47 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:07:49.102 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:49.103 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:07:49.103 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T01:07:49.103 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:48 smithi142 ceph-mon[37379]: mgrmap e27: smithi137.nbzbyc(active, since 1.00962s) 2024-06-15T01:07:49.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:07:49.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/mirror_snapshot_schedule"}]: dispatch 2024-06-15T01:07:49.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi137.nbzbyc/trash_purge_schedule"}]: dispatch 2024-06-15T01:07:49.333 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:48 smithi137 ceph-mon[29354]: mgrmap e27: smithi137.nbzbyc(active, since 1.00962s) 2024-06-15T01:07:50.247 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:49 smithi137 ceph-mon[29354]: pgmap v3: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:50.248 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:49 smithi137 ceph-mon[29354]: mgrmap e28: smithi137.nbzbyc(active, since 2s) 2024-06-15T01:07:50.268 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:49 smithi142 ceph-mon[37379]: pgmap v3: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:50.268 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:49 smithi142 ceph-mon[37379]: mgrmap e28: smithi137.nbzbyc(active, since 2s) 2024-06-15T01:07:51.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:50 smithi137 ceph-mon[29354]: [15/Jun/2024:01:07:50] ENGINE Bus STARTING 2024-06-15T01:07:51.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:50 smithi137 ceph-mon[29354]: [15/Jun/2024:01:07:50] ENGINE Serving on http://172.21.15.137:8765 2024-06-15T01:07:51.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:50 smithi142 ceph-mon[37379]: [15/Jun/2024:01:07:50] ENGINE Bus STARTING 2024-06-15T01:07:51.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:50 smithi142 ceph-mon[37379]: [15/Jun/2024:01:07:50] ENGINE Serving on http://172.21.15.137:8765 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: [15/Jun/2024:01:07:50] ENGINE Serving on https://172.21.15.137:7150 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: [15/Jun/2024:01:07:50] ENGINE Bus STARTED 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: [15/Jun/2024:01:07:50] ENGINE Error in HTTPServer.serve 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Traceback (most recent call last): 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._run(expiration_interval) 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-15T01:07:52.235 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: s = self.context.wrap_socket( 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: return self.sslsocket_class._create( 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self.do_handshake() 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: self._sslobj.do_handshake() 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:52.236 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:52.237 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:51 smithi137 ceph-mon[29354]: mgrmap e29: smithi137.nbzbyc(active, since 4s) 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: [15/Jun/2024:01:07:50] ENGINE Serving on https://172.21.15.137:7150 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: [15/Jun/2024:01:07:50] ENGINE Bus STARTED 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: [15/Jun/2024:01:07:50] ENGINE Error in HTTPServer.serve 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Traceback (most recent call last): 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._run(expiration_interval) 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-15T01:07:52.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: s = self.context.wrap_socket( 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: return self.sslsocket_class._create( 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self.do_handshake() 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: self._sslobj.do_handshake() 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:52.318 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:52.319 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:51 smithi142 ceph-mon[37379]: mgrmap e29: smithi137.nbzbyc(active, since 4s) 2024-06-15T01:07:53.462 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:53 smithi142 ceph-mon[37379]: pgmap v5: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:53.463 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:53 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:53.463 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:53 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:53.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:53 smithi137 ceph-mon[29354]: pgmap v5: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:53.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:53 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:53.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:53 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:07:54.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:07:54.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:07:54.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:07:54.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:07:54.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:07:54.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:54 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:07:54.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:07:54.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:07:54.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:07:54.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:07:54.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:07:54.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:54 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/2358430581' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:07:55.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:55 smithi137 ceph-mon[29354]: pgmap v6: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:55.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:55 smithi137 ceph-mon[29354]: mgrmap e30: smithi137.nbzbyc(active, since 6s), standbys: smithi142.dllryb 2024-06-15T01:07:55.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:55 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:55.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:55 smithi142 ceph-mon[37379]: pgmap v6: 65 pgs: 65 active+clean; 343 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:07:55.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:55 smithi142 ceph-mon[37379]: mgrmap e30: smithi137.nbzbyc(active, since 6s), standbys: smithi142.dllryb 2024-06-15T01:07:55.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:55 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr metadata", "who": "smithi142.dllryb", "id": "smithi142.dllryb"}]: dispatch 2024-06-15T01:07:57.465 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:57 smithi142 ceph-mon[37379]: pgmap v7: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-06-15T01:07:57.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:57 smithi137 ceph-mon[29354]: pgmap v7: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-06-15T01:07:58.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:07:58 smithi137 ceph-mon[29354]: pgmap v8: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-06-15T01:07:58.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:07:58 smithi142 ceph-mon[37379]: pgmap v8: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-06-15T01:08:00.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:00 smithi137 ceph-mon[29354]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 1.6 MiB/s wr, 428 op/s 2024-06-15T01:08:01.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:00 smithi142 ceph-mon[37379]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 1.6 MiB/s wr, 428 op/s 2024-06-15T01:08:01.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 -- 172.21.15.137:0/1907070586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 msgr2=0x7f9554100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:01.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 --2- 172.21.15.137:0/1907070586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554100190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9548009910 tx=0x7f9548009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:01.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 -- 172.21.15.137:0/1907070586 shutdown_connections 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 --2- 172.21.15.137:0/1907070586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554100190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 --2- 172.21.15.137:0/1907070586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f9554101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 -- 172.21.15.137:0/1907070586 >> 172.21.15.137:0/1907070586 conn(0x7f95540fa7c0 msgr2=0x7f95540fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 -- 172.21.15.137:0/1907070586 shutdown_connections 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.730+0000 7f955a8e0700 1 -- 172.21.15.137:0/1907070586 wait complete. 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 Processor -- start 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 -- start start 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f95541956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554195cd0 con 0x7f95540ffd10 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.731+0000 7f955a8e0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554195e10 con 0x7f9554101140 2024-06-15T01:08:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f9553fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f9553fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54146/0 (socket says 172.21.15.137:54146) 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f9553fff700 1 -- 172.21.15.137:0/2157633336 learned_addr learned my addr 172.21.15.137:0/2157633336 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95537fe700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f95541956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95537fe700 1 -- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 msgr2=0x7f9554195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95537fe700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95537fe700 1 -- 172.21.15.137:0/2157633336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480095f0 con 0x7f9554101140 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f9553fff700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95537fe700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f95541956b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9548000c00 tx=0x7f954800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f954801d070 con 0x7f9554101140 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955419a860 con 0x7f9554101140 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9548005710 con 0x7f9554101140 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9548003980 con 0x7f9554101140 2024-06-15T01:08:01.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.732+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955419ad70 con 0x7f9554101140 2024-06-15T01:08:01.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.734+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95541085b0 con 0x7f9554101140 2024-06-15T01:08:01.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.734+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f9548004ac0 con 0x7f9554101140 2024-06-15T01:08:01.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.734+0000 7f95517fa700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 0x7f953c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:01.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.735+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f954808b6f0 con 0x7f9554101140 2024-06-15T01:08:01.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.735+0000 7f9553fff700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 0x7f953c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:01.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.735+0000 7f9553fff700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 0x7f953c06d580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9544011550 tx=0x7f9544009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:01.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:01.740+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f954805a0f0 con 0x7f9554101140 2024-06-15T01:08:02.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.066+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9554105670 con 0x7f953c06b0c0 2024-06-15T01:08:02.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.067+0000 7f95517fa700 1 -- 172.21.15.137:0/2157633336 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f9554105670 con 0x7f953c06b0c0 2024-06-15T01:08:02.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 msgr2=0x7f953c06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 0x7f953c06d580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9544011550 tx=0x7f9544009260 comp rx=0 tx=0).stop 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 msgr2=0x7f95541956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f95541956b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9548000c00 tx=0x7f954800bde0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 shutdown_connections 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f953c06b0c0 0x7f953c06d580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95540ffd10 0x7f9554195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 --2- 172.21.15.137:0/2157633336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9554101140 0x7f95541956b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 >> 172.21.15.137:0/2157633336 conn(0x7f95540fa7c0 msgr2=0x7f9554103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 shutdown_connections 2024-06-15T01:08:02.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.071+0000 7f955a8e0700 1 -- 172.21.15.137:0/2157633336 wait complete. 2024-06-15T01:08:02.078 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:08:02.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 -- 172.21.15.137:0/1262190774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c0fee80 msgr2=0x7f2c2c0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 --2- 172.21.15.137:0/1262190774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c0ff2a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009910 tx=0x7f2c1c009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 -- 172.21.15.137:0/1262190774 shutdown_connections 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 --2- 172.21.15.137:0/1262190774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c101e60 0x7f2c2c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 --2- 172.21.15.137:0/1262190774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c0ff2a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.203+0000 7f2c32687700 1 -- 172.21.15.137:0/1262190774 >> 172.21.15.137:0/1262190774 conn(0x7f2c2c0fa7c0 msgr2=0x7f2c2c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.204+0000 7f2c32687700 1 -- 172.21.15.137:0/1262190774 shutdown_connections 2024-06-15T01:08:02.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.204+0000 7f2c32687700 1 -- 172.21.15.137:0/1262190774 wait complete. 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.204+0000 7f2c32687700 1 Processor -- start 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.204+0000 7f2c32687700 1 -- start start 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c32687700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c32687700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c101e60 0x7f2c2c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c32687700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c196e30 con 0x7f2c2c101e60 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c32687700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c196f70 con 0x7f2c2c0fee80 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:46984/0 (socket says 172.21.15.137:46984) 2024-06-15T01:08:02.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 -- 172.21.15.137:0/366427529 learned_addr learned my addr 172.21.15.137:0/366427529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2b7fe700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c101e60 0x7f2c2c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 -- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c101e60 msgr2=0x7f2c2c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c101e60 0x7f2c2c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.205+0000 7f2c2bfff700 1 -- 172.21.15.137:0/366427529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0095f0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c2bfff700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000c00 tx=0x7f2c1c00bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1c01d070 con 0x7f2c2c0fee80 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c19b9c0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2c1c0059e0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1c003980 con 0x7f2c2c0fee80 2024-06-15T01:08:02.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.206+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c19bed0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.207+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c1095e0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.208+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f2c1c0041f0 con 0x7f2c2c0fee80 2024-06-15T01:08:02.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.208+0000 7f2c297fa700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 0x7f2c1406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.208+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2c1c08bb10 con 0x7f2c2c0fee80 2024-06-15T01:08:02.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.208+0000 7f2c2b7fe700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 0x7f2c1406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:02.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.209+0000 7f2c2b7fe700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 0x7f2c1406d580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009c60 tx=0x7f2c20009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:02.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.213+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f2c1c05a310 con 0x7f2c2c0fee80 2024-06-15T01:08:02.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.483+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c2c1067b0 con 0x7f2c1406b0c0 2024-06-15T01:08:02.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.484+0000 7f2c297fa700 1 -- 172.21.15.137:0/366427529 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f2c2c1067b0 con 0x7f2c1406b0c0 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 msgr2=0x7f2c1406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 0x7f2c1406d580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009c60 tx=0x7f2c20009260 comp rx=0 tx=0).stop 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 msgr2=0x7f2c2c1962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000c00 tx=0x7f2c1c00bde0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 shutdown_connections 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c1406b0c0 0x7f2c1406d580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0fee80 0x7f2c2c1962d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 --2- 172.21.15.137:0/366427529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c2c101e60 0x7f2c2c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 >> 172.21.15.137:0/366427529 conn(0x7f2c2c0fa7c0 msgr2=0x7f2c2c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 shutdown_connections 2024-06-15T01:08:02.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.486+0000 7f2c32687700 1 -- 172.21.15.137:0/366427529 wait complete. 2024-06-15T01:08:02.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 -- 172.21.15.137:0/262918865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c100bb0 msgr2=0x7ffb7c100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 --2- 172.21.15.137:0/262918865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c100fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b0a0 tx=0x7ffb7400b3b0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 -- 172.21.15.137:0/262918865 shutdown_connections 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 --2- 172.21.15.137:0/262918865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c101db0 0x7ffb7c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 --2- 172.21.15.137:0/262918865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c100fd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.632+0000 7ffb80afe700 1 -- 172.21.15.137:0/262918865 >> 172.21.15.137:0/262918865 conn(0x7ffb7c0fc110 msgr2=0x7ffb7c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.633+0000 7ffb80afe700 1 -- 172.21.15.137:0/262918865 shutdown_connections 2024-06-15T01:08:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.633+0000 7ffb80afe700 1 -- 172.21.15.137:0/262918865 wait complete. 2024-06-15T01:08:02.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 Processor -- start 2024-06-15T01:08:02.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 -- start start 2024-06-15T01:08:02.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c101db0 0x7ffb7c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0736c0 con 0x7ffb7c100bb0 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb80afe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c073800 con 0x7ffb7c101db0 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.634+0000 7ffb7a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.635+0000 7ffb7a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54174/0 (socket says 172.21.15.137:54174) 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.635+0000 7ffb7a59c700 1 -- 172.21.15.137:0/880984112 learned_addr learned my addr 172.21.15.137:0/880984112 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.635+0000 7ffb7a59c700 1 -- 172.21.15.137:0/880984112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c101db0 msgr2=0x7ffb7c073180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.635+0000 7ffb7a59c700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c101db0 0x7ffb7c073180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.635+0000 7ffb7a59c700 1 -- 172.21.15.137:0/880984112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009520 con 0x7ffb7c100bb0 2024-06-15T01:08:02.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.636+0000 7ffb7a59c700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b070 tx=0x7ffb740092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:02.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.637+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb7400e040 con 0x7ffb7c100bb0 2024-06-15T01:08:02.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.637+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffb7401d900 con 0x7ffb7c100bb0 2024-06-15T01:08:02.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.637+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb74009d00 con 0x7ffb7c100bb0 2024-06-15T01:08:02.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.637+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb740045b0 con 0x7ffb7c100bb0 2024-06-15T01:08:02.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.637+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c073d20 con 0x7ffb7c100bb0 2024-06-15T01:08:02.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.639+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c1095d0 con 0x7ffb7c100bb0 2024-06-15T01:08:02.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.640+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7ffb74019070 con 0x7ffb7c100bb0 2024-06-15T01:08:02.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.640+0000 7ffb6b7fe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 0x7ffb6406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:02.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.641+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffb7408c2b0 con 0x7ffb7c100bb0 2024-06-15T01:08:02.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.641+0000 7ffb79d9b700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 0x7ffb6406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:02.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.642+0000 7ffb79d9b700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 0x7ffb6406d600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009ca0 tx=0x7ffb6c009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:02.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.648+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7ffb7405ac30 con 0x7ffb7c100bb0 2024-06-15T01:08:02.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:02 smithi142 ceph-mon[37379]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 392 op/s 2024-06-15T01:08:02.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:02 smithi142 ceph-mon[37379]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.943+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb7c19e790 con 0x7ffb6406b140 2024-06-15T01:08:02.940 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:02 smithi137 ceph-mon[29354]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 392 op/s 2024-06-15T01:08:02.940 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:02 smithi137 ceph-mon[29354]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.953+0000 7ffb6b7fe700 1 -- 172.21.15.137:0/880984112 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ffb7c19e790 con 0x7ffb6406b140 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (7m) 19s ago 9m 25.5M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (9m) 19s ago 9m 7826k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (8m) 11s ago 8m 7486k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (9m) 19s ago 9m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (8m) 11s ago 8m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:08:02.949 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (7m) 19s ago 8m 86.7M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (3m) 19s ago 3m 145M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (3m) 19s ago 3m 157M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (3m) 11s ago 3m 13.7M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (3m) 11s ago 3m 16.6M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (26s) 19s ago 10m 44.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (99s) 11s ago 8m 96.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba df5c0f028b61 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (10m) 19s ago 10m 50.0M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (8m) 11s ago 8m 44.1M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (9m) 19s ago 9m 16.3M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 11s ago 8m 16.4M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (7m) 19s ago 7m 191M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (6m) 19s ago 6m 217M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (6m) 19s ago 6m 166M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:08:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (5m) 11s ago 5m 232M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (5m) 11s ago 5m 214M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (5m) 11s ago 5m 207M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (31s) 19s ago 8m 36.7M - 2.43.0 a07b618ecd1d 59690de6ef6a 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.955+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 msgr2=0x7ffb6406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.955+0000 7ffb80afe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 0x7ffb6406d600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009ca0 tx=0x7ffb6c009260 comp rx=0 tx=0).stop 2024-06-15T01:08:02.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.955+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 msgr2=0x7ffb7c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.955+0000 7ffb80afe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b070 tx=0x7ffb740092e0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 shutdown_connections 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb6406b140 0x7ffb6406d600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c100bb0 0x7ffb7c074b30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 --2- 172.21.15.137:0/880984112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb7c101db0 0x7ffb7c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 >> 172.21.15.137:0/880984112 conn(0x7ffb7c0fc110 msgr2=0x7ffb7c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 shutdown_connections 2024-06-15T01:08:02.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:02.956+0000 7ffb80afe700 1 -- 172.21.15.137:0/880984112 wait complete. 2024-06-15T01:08:03.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.084+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2461750705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b40ffab0 msgr2=0x7f17b40ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.084+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2461750705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b40ffab0 0x7f17b40ffed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009910 tx=0x7f17b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:03.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.084+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2461750705 shutdown_connections 2024-06-15T01:08:03.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.084+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2461750705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b4100cb0 0x7f17b4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.084+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2461750705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b40ffab0 0x7f17b40ffed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.085+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2461750705 >> 172.21.15.137:0/2461750705 conn(0x7f17b40fb030 msgr2=0x7f17b40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:03.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.085+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2461750705 shutdown_connections 2024-06-15T01:08:03.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.085+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2461750705 wait complete. 2024-06-15T01:08:03.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.086+0000 7f17bb4fd700 1 Processor -- start 2024-06-15T01:08:03.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.086+0000 7f17bb4fd700 1 -- start start 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.086+0000 7f17bb4fd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b40ffab0 0x7f17b4195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17bb4fd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17bb4fd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b4195cd0 con 0x7f17b4100cb0 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17bb4fd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b4195e10 con 0x7f17b40ffab0 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b9299700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b40ffab0 0x7f17b4195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54192/0 (socket says 172.21.15.137:54192) 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 -- 172.21.15.137:0/2896008897 learned_addr learned my addr 172.21.15.137:0/2896008897 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 -- 172.21.15.137:0/2896008897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b40ffab0 msgr2=0x7f17b4195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b40ffab0 0x7f17b4195170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 -- 172.21.15.137:0/2896008897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00095f0 con 0x7f17b4100cb0 2024-06-15T01:08:03.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.087+0000 7f17b8a98700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eab0 tx=0x7f17a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.088+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a4013070 con 0x7f17b4100cb0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.088+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b4068470 con 0x7f17b4100cb0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.088+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b4068a40 con 0x7f17b4100cb0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.088+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f17a400bbd0 con 0x7f17b4100cb0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.088+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a4021610 con 0x7f17b4100cb0 2024-06-15T01:08:03.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.089+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1798005320 con 0x7f17b4100cb0 2024-06-15T01:08:03.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.090+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f17a400bd40 con 0x7f17b4100cb0 2024-06-15T01:08:03.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.090+0000 7f17aa7fc700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 0x7f17a006d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.090+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f17a408bcb0 con 0x7f17b4100cb0 2024-06-15T01:08:03.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.091+0000 7f17b9299700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 0x7f17a006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.091+0000 7f17b9299700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 0x7f17a006d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b600 tx=0x7f17b0005fb0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:03.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.099+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f17a405a5c0 con 0x7f17b4100cb0 2024-06-15T01:08:03.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.594+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1798005cc0 con 0x7f17b4100cb0 2024-06-15T01:08:03.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.596+0000 7f17aa7fc700 1 -- 172.21.15.137:0/2896008897 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f17a405a150 con 0x7f17b4100cb0 2024-06-15T01:08:03.591 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:08:03.591 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:08:03.591 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:08:03.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:08:03.592 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:08:03.593 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:08:03.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.599+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 msgr2=0x7f17a006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.599+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 0x7f17a006d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b600 tx=0x7f17b0005fb0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.600+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 msgr2=0x7f17b41956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.600+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17a400eab0 tx=0x7f17a400ee70 comp rx=0 tx=0).stop 2024-06-15T01:08:03.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 shutdown_connections 2024-06-15T01:08:03.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17a006b0c0 0x7f17a006d580 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17b40ffab0 0x7f17b4195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 --2- 172.21.15.137:0/2896008897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17b4100cb0 0x7f17b41956b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 >> 172.21.15.137:0/2896008897 conn(0x7f17b40fb030 msgr2=0x7f17b4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:03.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 shutdown_connections 2024-06-15T01:08:03.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.601+0000 7f17bb4fd700 1 -- 172.21.15.137:0/2896008897 wait complete. 2024-06-15T01:08:03.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.760+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/4277410937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac101e60 msgr2=0x7fe1ac1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.760+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/4277410937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1022c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009910 tx=0x7fe1a0009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:03.757 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:03 smithi137 ceph-mon[29354]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:03.757 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:03 smithi137 ceph-mon[29354]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:03.757 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:03 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:03.757 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:03 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2896008897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/4277410937 shutdown_connections 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/4277410937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1022c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/4277410937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/4277410937 >> 172.21.15.137:0/4277410937 conn(0x7fe1ac0fa7c0 msgr2=0x7fe1ac0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/4277410937 shutdown_connections 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.761+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/4277410937 wait complete. 2024-06-15T01:08:03.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 Processor -- start 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 -- start start 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac196e10 con 0x7fe1ac0fee80 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1b0cc3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac196f50 con 0x7fe1ac101e60 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1aa59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1aa59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54222/0 (socket says 172.21.15.137:54222) 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1aa59c700 1 -- 172.21.15.137:0/1886272630 learned_addr learned my addr 172.21.15.137:0/1886272630 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.762+0000 7fe1a9d9b700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1a9d9b700 1 -- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 msgr2=0x7fe1ac1962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1a9d9b700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac1962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1a9d9b700 1 -- 172.21.15.137:0/1886272630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00095f0 con 0x7fe1ac101e60 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1a9d9b700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1967f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009910 tx=0x7fe1a000bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a001d070 con 0x7fe1ac101e60 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe1a00056f0 con 0x7fe1ac101e60 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac19b9a0 con 0x7fe1ac101e60 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a00039d0 con 0x7fe1ac101e60 2024-06-15T01:08:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.763+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac19beb0 con 0x7fe1ac101e60 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.764+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac109650 con 0x7fe1ac101e60 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.765+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7fe1a00041f0 con 0x7fe1ac101e60 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.765+0000 7fe19b7fe700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 0x7fe19406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.766+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe1a008ba10 con 0x7fe1ac101e60 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.766+0000 7fe1aa59c700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 0x7fe19406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.766+0000 7fe1aa59c700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 0x7fe19406d580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005950 tx=0x7fe19c009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:03.770+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fe1a005a250 con 0x7fe1ac101e60 2024-06-15T01:08:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.043+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1ac1067b0 con 0x7fe19406b0c0 2024-06-15T01:08:04.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.044+0000 7fe19b7fe700 1 -- 172.21.15.137:0/1886272630 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fe1ac1067b0 con 0x7fe19406b0c0 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "2/2 daemons upgraded", 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "", 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:08:04.040 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 msgr2=0x7fe19406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 0x7fe19406d580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005950 tx=0x7fe19c009260 comp rx=0 tx=0).stop 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 msgr2=0x7fe1ac1967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1967f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009910 tx=0x7fe1a000bde0 comp rx=0 tx=0).stop 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 shutdown_connections 2024-06-15T01:08:04.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe19406b0c0 0x7fe19406d580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:04.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac0fee80 0x7fe1ac1962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:04.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 --2- 172.21.15.137:0/1886272630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ac101e60 0x7fe1ac1967f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:04.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 >> 172.21.15.137:0/1886272630 conn(0x7fe1ac0fa7c0 msgr2=0x7fe1ac105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:04.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 shutdown_connections 2024-06-15T01:08:04.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:04.046+0000 7fe1b0cc3700 1 -- 172.21.15.137:0/1886272630 wait complete. 2024-06-15T01:08:04.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:03 smithi142 ceph-mon[37379]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:04.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:03 smithi142 ceph-mon[37379]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:04.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:03 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:04.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:03 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2896008897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:04.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:04 smithi137 ceph-mon[29354]: pgmap v11: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 392 op/s 2024-06-15T01:08:04.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:04 smithi137 ceph-mon[29354]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:05.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:04 smithi142 ceph-mon[37379]: pgmap v11: 65 pgs: 65 active+clean; 287 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 392 op/s 2024-06-15T01:08:05.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:04 smithi142 ceph-mon[37379]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:06.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:06 smithi137 ceph-mon[29354]: pgmap v12: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s rd, 2.2 MiB/s wr, 607 op/s 2024-06-15T01:08:07.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:06 smithi142 ceph-mon[37379]: pgmap v12: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s rd, 2.2 MiB/s wr, 607 op/s 2024-06-15T01:08:08.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:08 smithi137 ceph-mon[29354]: pgmap v13: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.4 MiB/s wr, 404 op/s 2024-06-15T01:08:08.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:08 smithi142 ceph-mon[37379]: pgmap v13: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.4 MiB/s wr, 404 op/s 2024-06-15T01:08:10.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:10 smithi137 ceph-mon[29354]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 531 op/s 2024-06-15T01:08:11.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:10 smithi142 ceph-mon[37379]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 531 op/s 2024-06-15T01:08:12.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:12 smithi137 ceph-mon[29354]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-15T01:08:13.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:12 smithi142 ceph-mon[37379]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-15T01:08:14.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:14 smithi137 ceph-mon[29354]: pgmap v16: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-15T01:08:15.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:14 smithi142 ceph-mon[37379]: pgmap v16: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-15T01:08:16.418 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.418 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.418 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.418 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:16.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:17.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:17 smithi137 ceph-mon[29354]: pgmap v17: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 534 op/s 2024-06-15T01:08:17.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:17 smithi142 ceph-mon[37379]: pgmap v17: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 534 op/s 2024-06-15T01:08:18.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:18 smithi137 ceph-mon[29354]: pgmap v18: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-06-15T01:08:18.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:18 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:18.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:18 smithi142 ceph-mon[37379]: pgmap v18: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-06-15T01:08:18.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:18 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:20.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:20 smithi137 ceph-mon[29354]: pgmap v19: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.0 MiB/s wr, 535 op/s 2024-06-15T01:08:21.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:20 smithi142 ceph-mon[37379]: pgmap v19: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.0 MiB/s wr, 535 op/s 2024-06-15T01:08:22.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:22 smithi137 ceph-mon[29354]: pgmap v20: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 409 op/s 2024-06-15T01:08:23.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:22 smithi142 ceph-mon[37379]: pgmap v20: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 409 op/s 2024-06-15T01:08:24.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:24 smithi137 ceph-mon[29354]: pgmap v21: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 419 op/s 2024-06-15T01:08:25.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:24 smithi142 ceph-mon[37379]: pgmap v21: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 419 op/s 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:08:26.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-06-15T01:08:26.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:08:26.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:08:26.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:26 smithi137 ceph-mon[29354]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:08:27.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:27.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:08:27.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: Updating smithi137:/etc/ceph/ceph.conf 2024-06-15T01:08:27.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: Updating smithi142:/etc/ceph/ceph.conf 2024-06-15T01:08:27.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-06-15T01:08:27.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:08:27.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-15T01:08:27.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:26 smithi142 ceph-mon[37379]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:27.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:08:27.975 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:27 smithi137 ceph-mon[29354]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: Updating smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: Updating smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:28.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:28.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:28.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: Reconfiguring prometheus.smithi137 (dependencies changed)... 2024-06-15T01:08:28.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:27 smithi142 ceph-mon[37379]: Reconfiguring daemon prometheus.smithi137 on smithi137 2024-06-15T01:08:28.937 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:28 smithi137 ceph-mon[29354]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 393 op/s 2024-06-15T01:08:29.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:28 smithi142 ceph-mon[37379]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 393 op/s 2024-06-15T01:08:30.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:30 smithi142 ceph-mon[37379]: pgmap v24: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.9 MiB/s wr, 589 op/s 2024-06-15T01:08:30.863 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-06-15T01:08:30.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-06-15T01:08:30.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:30 smithi137 ceph-mon[29354]: pgmap v24: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.9 MiB/s wr, 589 op/s 2024-06-15T01:08:31.199 DEBUG:teuthology.parallel:result is None 2024-06-15T01:08:32.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:32 smithi137 ceph-mon[29354]: pgmap v25: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 373 op/s 2024-06-15T01:08:33.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:32 smithi142 ceph-mon[37379]: pgmap v25: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 373 op/s 2024-06-15T01:08:33.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:33 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:34.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:33 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 -- 172.21.15.137:0/3798185841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc101e00 msgr2=0x7f92bc102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3798185841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc101e00 0x7f92bc102260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009910 tx=0x7f92ac009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 -- 172.21.15.137:0/3798185841 shutdown_connections 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3798185841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc101e00 0x7f92bc102260 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3798185841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc100c00 0x7f92bc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 -- 172.21.15.137:0/3798185841 >> 172.21.15.137:0/3798185841 conn(0x7f92bc0fc180 msgr2=0x7f92bc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 -- 172.21.15.137:0/3798185841 shutdown_connections 2024-06-15T01:08:34.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.178+0000 7f92c3f57700 1 -- 172.21.15.137:0/3798185841 wait complete. 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 Processor -- start 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 -- start start 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 0x7f92bc071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 0x7f92bc072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc072920 con 0x7f92bc100c00 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.179+0000 7f92c3f57700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc1a4020 con 0x7f92bc101e00 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c1cf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 0x7f92bc071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c1cf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 0x7f92bc071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37062/0 (socket says 172.21.15.137:37062) 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c14f2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 0x7f92bc072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c1cf3700 1 -- 172.21.15.137:0/3937968975 learned_addr learned my addr 172.21.15.137:0/3937968975 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c14f2700 1 -- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 msgr2=0x7f92bc071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c14f2700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 0x7f92bc071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c14f2700 1 -- 172.21.15.137:0/3937968975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0095f0 con 0x7f92bc101e00 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c14f2700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 0x7f92bc072300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009910 tx=0x7f92ac00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92ac01d070 con 0x7f92bc101e00 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f92ac00f460 con 0x7f92bc101e00 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92ac021640 con 0x7f92bc101e00 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.180+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc1a41c0 con 0x7f92bc101e00 2024-06-15T01:08:34.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.181+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc1a46e0 con 0x7f92bc101e00 2024-06-15T01:08:34.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.182+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc109620 con 0x7f92bc101e00 2024-06-15T01:08:34.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.182+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f92ac02b440 con 0x7f92bc101e00 2024-06-15T01:08:34.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.183+0000 7f92b2ffd700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 0x7f92a806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.183+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f92ac08b790 con 0x7f92bc101e00 2024-06-15T01:08:34.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.183+0000 7f92c1cf3700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 0x7f92a806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.183+0000 7f92c1cf3700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 0x7f92a806d580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92b800ac60 tx=0x7f92b800a3b0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:34.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.187+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f92ac059f50 con 0x7f92bc101e00 2024-06-15T01:08:34.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.511+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92bc106750 con 0x7f92a806b0c0 2024-06-15T01:08:34.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.513+0000 7f92b2ffd700 1 -- 172.21.15.137:0/3937968975 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f92bc106750 con 0x7f92a806b0c0 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 msgr2=0x7f92a806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 0x7f92a806d580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92b800ac60 tx=0x7f92b800a3b0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 msgr2=0x7f92bc072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 0x7f92bc072300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009910 tx=0x7f92ac00bd70 comp rx=0 tx=0).stop 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 shutdown_connections 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f92a806b0c0 0x7f92a806d580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92bc100c00 0x7f92bc071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 --2- 172.21.15.137:0/3937968975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92bc101e00 0x7f92bc072300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.515+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 >> 172.21.15.137:0/3937968975 conn(0x7f92bc0fc180 msgr2=0x7f92bc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:34.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.516+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 shutdown_connections 2024-06-15T01:08:34.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.516+0000 7f92c3f57700 1 -- 172.21.15.137:0/3937968975 wait complete. 2024-06-15T01:08:34.522 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:08:34.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 -- 172.21.15.137:0/2668716878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 msgr2=0x7f2064103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 --2- 172.21.15.137:0/2668716878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064103a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2058009910 tx=0x7f2058009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:34.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 -- 172.21.15.137:0/2668716878 shutdown_connections 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 --2- 172.21.15.137:0/2668716878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 0x7f2064106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 --2- 172.21.15.137:0/2668716878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064103a30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 -- 172.21.15.137:0/2668716878 >> 172.21.15.137:0/2668716878 conn(0x7f20640fb030 msgr2=0x7f20640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 -- 172.21.15.137:0/2668716878 shutdown_connections 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.652+0000 7f206afc4700 1 -- 172.21.15.137:0/2668716878 wait complete. 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 Processor -- start 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 -- start start 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 0x7f2064196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064196d80 con 0x7f2064101640 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f206afc4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064196ec0 con 0x7f2064103f70 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f2068d60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f2068d60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37082/0 (socket says 172.21.15.137:37082) 2024-06-15T01:08:34.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f2068d60700 1 -- 172.21.15.137:0/2489526103 learned_addr learned my addr 172.21.15.137:0/2489526103 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:34.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f2063fff700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 0x7f2064196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.653+0000 7f2068d60700 1 -- 172.21.15.137:0/2489526103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 msgr2=0x7f2064196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:34.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2068d60700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 0x7f2064196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:34.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2068d60700 1 -- 172.21.15.137:0/2489526103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580095f0 con 0x7f2064101640 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2068d60700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2058006010 tx=0x7f205800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205801d070 con 0x7f2064101640 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206419b910 con 0x7f2064101640 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f205800f460 con 0x7f2064101640 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2058021640 con 0x7f2064101640 2024-06-15T01:08:34.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.654+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206419be80 con 0x7f2064101640 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.655+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20641903e0 con 0x7f2064101640 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.656+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f205800f9b0 con 0x7f2064101640 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.656+0000 7f2061ffb700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 0x7f205406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.657+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f205808b690 con 0x7f2064101640 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.657+0000 7f2063fff700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 0x7f205406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:34.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.657+0000 7f2063fff700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 0x7f205406d580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2050005950 tx=0x7f205000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:34.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:34.665+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f2058059ef0 con 0x7f2064101640 2024-06-15T01:08:34.942 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:34 smithi137 ceph-mon[29354]: pgmap v26: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 373 op/s 2024-06-15T01:08:35.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.028+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20640611d0 con 0x7f205406b0c0 2024-06-15T01:08:35.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.029+0000 7f2061ffb700 1 -- 172.21.15.137:0/2489526103 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f20640611d0 con 0x7f205406b0c0 2024-06-15T01:08:35.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 msgr2=0x7f205406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 0x7f205406d580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2050005950 tx=0x7f205000b410 comp rx=0 tx=0).stop 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 msgr2=0x7f2064196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2058006010 tx=0x7f205800bd70 comp rx=0 tx=0).stop 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 shutdown_connections 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f205406b0c0 0x7f205406d580 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2064101640 0x7f2064196220 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 --2- 172.21.15.137:0/2489526103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064103f70 0x7f2064196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 >> 172.21.15.137:0/2489526103 conn(0x7f20640fb030 msgr2=0x7f20640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:35.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 shutdown_connections 2024-06-15T01:08:35.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.032+0000 7f206afc4700 1 -- 172.21.15.137:0/2489526103 wait complete. 2024-06-15T01:08:35.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:34 smithi142 ceph-mon[37379]: pgmap v26: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 373 op/s 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/347247989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d180ffab0 msgr2=0x7f2d180ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/347247989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d180ffab0 0x7f2d180ffed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009910 tx=0x7f2d08009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/347247989 shutdown_connections 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/347247989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d18100cb0 0x7f2d18101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/347247989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d180ffab0 0x7f2d180ffed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/347247989 >> 172.21.15.137:0/347247989 conn(0x7f2d180fb030 msgr2=0x7f2d180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/347247989 shutdown_connections 2024-06-15T01:08:35.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.174+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/347247989 wait complete. 2024-06-15T01:08:35.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.175+0000 7f2d1e4c2700 1 Processor -- start 2024-06-15T01:08:35.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.175+0000 7f2d1e4c2700 1 -- start start 2024-06-15T01:08:35.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d1e4c2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d180ffab0 0x7f2d18198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d1e4c2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d1e4c2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18198fe0 con 0x7f2d180ffab0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d1e4c2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18199120 con 0x7f2d18100cb0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:50264/0 (socket says 172.21.15.137:50264) 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 -- 172.21.15.137:0/115683089 learned_addr learned my addr 172.21.15.137:0/115683089 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d17fff700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d180ffab0 0x7f2d18198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 -- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d180ffab0 msgr2=0x7f2d18198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d180ffab0 0x7f2d18198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 -- 172.21.15.137:0/115683089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080095f0 con 0x7f2d18100cb0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.176+0000 7f2d177fe700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e920 tx=0x7f2d0c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.177+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0c009790 con 0x7f2d18100cb0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.177+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1819dbd0 con 0x7f2d18100cb0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.177+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2d0c00cba0 con 0x7f2d18100cb0 2024-06-15T01:08:35.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.177+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0c018650 con 0x7f2d18100cb0 2024-06-15T01:08:35.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.177+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1819e0e0 con 0x7f2d18100cb0 2024-06-15T01:08:35.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.178+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d181073e0 con 0x7f2d18100cb0 2024-06-15T01:08:35.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.179+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f2d0c010430 con 0x7f2d18100cb0 2024-06-15T01:08:35.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.179+0000 7f2d157fa700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 0x7f2d0006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.180+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2d0c014070 con 0x7f2d18100cb0 2024-06-15T01:08:35.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.180+0000 7f2d17fff700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 0x7f2d0006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.180+0000 7f2d17fff700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 0x7f2d0006d570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000c00 tx=0x7f2d08009f70 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:35.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.184+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f2d0c052650 con 0x7f2d18100cb0 2024-06-15T01:08:35.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.477+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d18105170 con 0x7f2d0006b0b0 2024-06-15T01:08:35.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.491+0000 7f2d157fa700 1 -- 172.21.15.137:0/115683089 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2d18105170 con 0x7f2d0006b0b0 2024-06-15T01:08:35.485 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:08:35.485 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (8m) 20s ago 9m 25.5M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:08:35.485 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (10m) 20s ago 10m 7838k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (8m) 43s ago 8m 7486k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (9m) 20s ago 9m 6656k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (8m) 43s ago 8m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (8m) 20s ago 9m 86.7M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (4m) 20s ago 4m 146M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (4m) 20s ago 4m 161M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (4m) 43s ago 4m 13.7M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (4m) 43s ago 4m 16.6M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (58s) 20s ago 11m 460M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (2m) 43s ago 8m 96.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba df5c0f028b61 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (11m) 20s ago 11m 53.5M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (8m) 43s ago 8m 44.1M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:08:35.486 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (9m) 20s ago 9m 16.3M - 1.5.0 0da6a335fe13 143619d9bda2 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 43s ago 8m 16.4M - 1.5.0 0da6a335fe13 502297b4f172 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (7m) 20s ago 7m 193M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (7m) 20s ago 7m 217M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (6m) 20s ago 6m 169M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (6m) 43s ago 6m 232M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (6m) 43s ago 6m 214M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (5m) 43s ago 5m 207M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:08:35.487 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (64s) 20s ago 9m 44.8M - 2.43.0 a07b618ecd1d 59690de6ef6a 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 msgr2=0x7f2d0006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 0x7f2d0006d570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000c00 tx=0x7f2d08009f70 comp rx=0 tx=0).stop 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 msgr2=0x7f2d181989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e920 tx=0x7f2d0c00ece0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 shutdown_connections 2024-06-15T01:08:35.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d0006b0b0 0x7f2d0006d570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d180ffab0 0x7f2d18198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 --2- 172.21.15.137:0/115683089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d18100cb0 0x7f2d181989c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 >> 172.21.15.137:0/115683089 conn(0x7f2d180fb030 msgr2=0x7f2d180fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:35.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 shutdown_connections 2024-06-15T01:08:35.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.494+0000 7f2d1e4c2700 1 -- 172.21.15.137:0/115683089 wait complete. 2024-06-15T01:08:35.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 -- 172.21.15.137:0/1583808671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 msgr2=0x7f97c00ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 --2- 172.21.15.137:0/1583808671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c00ff080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009910 tx=0x7f97b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:35.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 -- 172.21.15.137:0/1583808671 shutdown_connections 2024-06-15T01:08:35.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 --2- 172.21.15.137:0/1583808671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c00ff080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 --2- 172.21.15.137:0/1583808671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c01025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 -- 172.21.15.137:0/1583808671 >> 172.21.15.137:0/1583808671 conn(0x7f97c00fa7c0 msgr2=0x7f97c00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:35.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 -- 172.21.15.137:0/1583808671 shutdown_connections 2024-06-15T01:08:35.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.633+0000 7f97c861a700 1 -- 172.21.15.137:0/1583808671 wait complete. 2024-06-15T01:08:35.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 Processor -- start 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 -- start start 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c00ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0100430 con 0x7f97c00fec20 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.634+0000 7f97c861a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0100570 con 0x7f97c01021a0 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c63b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c63b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37102/0 (socket says 172.21.15.137:37102) 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c5bb5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c00ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c63b6700 1 -- 172.21.15.137:0/1363720266 learned_addr learned my addr 172.21.15.137:0/1363720266 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c5bb5700 1 -- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 msgr2=0x7f97c01018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c5bb5700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:35.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c5bb5700 1 -- 172.21.15.137:0/1363720266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00095f0 con 0x7f97c01021a0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c63b6700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c5bb5700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c00ffef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009910 tx=0x7f97b000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b001d070 con 0x7f97c01021a0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f97b00048e0 con 0x7f97c01021a0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.635+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c01007f0 con 0x7f97c01021a0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.636+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b0003af0 con 0x7f97c01021a0 2024-06-15T01:08:35.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.636+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c0100c80 con 0x7f97c01021a0 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.637+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c01095f0 con 0x7f97c01021a0 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.637+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f97b00041f0 con 0x7f97c01021a0 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.638+0000 7f97b77fe700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 0x7f97ac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.638+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f97b008bed0 con 0x7f97c01021a0 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.638+0000 7f97c63b6700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 0x7f97ac06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:35.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.638+0000 7f97c63b6700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 0x7f97ac06d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005950 tx=0x7f97bc009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:35.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:35.643+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f97b005a850 con 0x7f97c01021a0 2024-06-15T01:08:35.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:35 smithi137 ceph-mon[29354]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:35.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:35 smithi137 ceph-mon[29354]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:36.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:35 smithi142 ceph-mon[37379]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:36.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:35 smithi142 ceph-mon[37379]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:36.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.079+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f97c019e510 con 0x7f97c01021a0 2024-06-15T01:08:36.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.079+0000 7f97b77fe700 1 -- 172.21.15.137:0/1363720266 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f97b0026070 con 0x7f97c01021a0 2024-06-15T01:08:36.073 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:08:36.073 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:08:36.073 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-15T01:08:36.074 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:08:36.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:08:36.075 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:08:36.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 msgr2=0x7f97ac06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 0x7f97ac06d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005950 tx=0x7f97bc009260 comp rx=0 tx=0).stop 2024-06-15T01:08:36.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 msgr2=0x7f97c00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c00ffef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009910 tx=0x7f97b000bd80 comp rx=0 tx=0).stop 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 shutdown_connections 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f97ac06b100 0x7f97ac06d5c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c00fec20 0x7f97c01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 --2- 172.21.15.137:0/1363720266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c01021a0 0x7f97c00ffef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 >> 172.21.15.137:0/1363720266 conn(0x7f97c00fa7c0 msgr2=0x7f97c00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 shutdown_connections 2024-06-15T01:08:36.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.082+0000 7f97c861a700 1 -- 172.21.15.137:0/1363720266 wait complete. 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 -- 172.21.15.137:0/2566425479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650103f70 msgr2=0x7f1650106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 --2- 172.21.15.137:0/2566425479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650103f70 0x7f1650106360 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f163c009910 tx=0x7f163c009c20 comp rx=0 tx=0).stop 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 -- 172.21.15.137:0/2566425479 shutdown_connections 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 --2- 172.21.15.137:0/2566425479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650103f70 0x7f1650106360 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 --2- 172.21.15.137:0/2566425479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650101640 0x7f1650103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 -- 172.21.15.137:0/2566425479 >> 172.21.15.137:0/2566425479 conn(0x7f16500fb030 msgr2=0x7f16500fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:36.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 -- 172.21.15.137:0/2566425479 shutdown_connections 2024-06-15T01:08:36.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 -- 172.21.15.137:0/2566425479 wait complete. 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.212+0000 7f1656c89700 1 Processor -- start 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1656c89700 1 -- start start 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1656c89700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1656c89700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650103f70 0x7f16500ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1656c89700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16500ffb30 con 0x7f1650101640 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1656c89700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16500ffc70 con 0x7f1650103f70 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:36.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57756/0 (socket says 172.21.15.137:57756) 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 -- 172.21.15.137:0/940870221 learned_addr learned my addr 172.21.15.137:0/940870221 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f164ffff700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650103f70 0x7f16500ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 -- 172.21.15.137:0/940870221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650103f70 msgr2=0x7f16500ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650103f70 0x7f16500ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 -- 172.21.15.137:0/940870221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0095f0 con 0x7f1650101640 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.213+0000 7f1654a25700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f164400d8d0 tx=0x7f164400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.214+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16440098a0 con 0x7f1650101640 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.214+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1644009e90 con 0x7f1650101640 2024-06-15T01:08:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.214+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16500fff50 con 0x7f1650101640 2024-06-15T01:08:36.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.214+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165019e2b0 con 0x7f1650101640 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.214+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164400f5f0 con 0x7f1650101640 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.215+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 30) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f1644009a00 con 0x7f1650101640 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.216+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16501903e0 con 0x7f1650101640 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.216+0000 7f164dffb700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 0x7f164006d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.216+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f164408a270 con 0x7f1650101640 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.216+0000 7f164ffff700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 0x7f164006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:08:36.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.217+0000 7f164ffff700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 0x7f164006d580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c00b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:08:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.223+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f1644058b50 con 0x7f1650101640 2024-06-15T01:08:36.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.492+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16500611d0 con 0x7f164006b0c0 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.493+0000 7f164dffb700 1 -- 172.21.15.137:0/940870221 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f16500611d0 con 0x7f164006b0c0 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:08:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:08:36.488 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "2/2 daemons upgraded", 2024-06-15T01:08:36.488 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mgr daemons", 2024-06-15T01:08:36.488 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:08:36.488 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 msgr2=0x7f164006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 0x7f164006d580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c00b540 comp rx=0 tx=0).stop 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 msgr2=0x7f1650100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f164400d8d0 tx=0x7f164400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 shutdown_connections 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f164006b0c0 0x7f164006d580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1650101640 0x7f1650100fa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 --2- 172.21.15.137:0/940870221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1650103f70 0x7f16500ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:08:36.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 >> 172.21.15.137:0/940870221 conn(0x7f16500fb030 msgr2=0x7f16500fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:08:36.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 shutdown_connections 2024-06-15T01:08:36.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:08:36.496+0000 7f1656c89700 1 -- 172.21.15.137:0/940870221 wait complete. 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: pgmap v27: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 545 op/s 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:36 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1363720266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: pgmap v27: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 545 op/s 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:36 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1363720266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:38.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:37 smithi137 ceph-mon[29354]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:38.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:37 smithi142 ceph-mon[37379]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:08:39.022 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: pgmap v28: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-06-15T01:08:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: Upgrade: Updating mgr.smithi142.dllryb 2024-06-15T01:08:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:08:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:08:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:39.023 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:38 smithi142 ceph-mon[37379]: Deploying daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:08:39.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: pgmap v28: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-06-15T01:08:39.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: Upgrade: Updating mgr.smithi142.dllryb 2024-06-15T01:08:39.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.dllryb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-15T01:08:39.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-15T01:08:39.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:39.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:38 smithi137 ceph-mon[29354]: Deploying daemon mgr.smithi142.dllryb on smithi142 2024-06-15T01:08:40.355 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-06-15T01:08:40.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-15T01:08:40.698 DEBUG:teuthology.parallel:result is None 2024-06-15T01:08:40.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-15T01:08:40.756 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-15T01:08:40.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-06-15T01:08:40.804 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-06-15T01:08:40.804 DEBUG:teuthology.parallel:result is None 2024-06-15T01:08:40.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:40 smithi137 ceph-mon[29354]: pgmap v29: 65 pgs: 65 active+clean; 268 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 504 op/s 2024-06-15T01:08:41.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:40 smithi142 ceph-mon[37379]: pgmap v29: 65 pgs: 65 active+clean; 268 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 504 op/s 2024-06-15T01:08:43.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: pgmap v30: 65 pgs: 65 active+clean; 268 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1016 KiB/s wr, 308 op/s 2024-06-15T01:08:43.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:43.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:43 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: pgmap v30: 65 pgs: 65 active+clean; 268 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1016 KiB/s wr, 308 op/s 2024-06-15T01:08:43.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:43.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:43.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:43 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:45.274 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:45 smithi142 ceph-mon[37379]: pgmap v31: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1023 KiB/s wr, 328 op/s 2024-06-15T01:08:45.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:45 smithi137 ceph-mon[29354]: pgmap v31: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1023 KiB/s wr, 328 op/s 2024-06-15T01:08:46.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:46 smithi142 ceph-mon[37379]: pgmap v32: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.2 MiB/s wr, 399 op/s 2024-06-15T01:08:46.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.793 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.793 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.793 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:46 smithi137 ceph-mon[29354]: pgmap v32: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.2 MiB/s wr, 399 op/s 2024-06-15T01:08:46.793 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:46.793 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb restarted 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:08:47.696 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:47 smithi142 ceph-mon[37379]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:08:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb restarted 2024-06-15T01:08:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: Standby manager daemon smithi142.dllryb started 2024-06-15T01:08:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/crt"}]: dispatch 2024-06-15T01:08:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-15T01:08:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.dllryb/key"}]: dispatch 2024-06-15T01:08:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:47 smithi137 ceph-mon[29354]: from='mgr.? 172.21.15.142:0/4185484445' entity='mgr.smithi142.dllryb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: mgrmap e31: smithi137.nbzbyc(active, since 59s), standbys: smithi142.dllryb 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: pgmap v33: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 625 KiB/s wr, 216 op/s 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:48.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: Failed to load ceph-mgr modules: k8sevents 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: mgrmap e31: smithi137.nbzbyc(active, since 59s), standbys: smithi142.dllryb 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: pgmap v33: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 625 KiB/s wr, 216 op/s 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:48.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi137.nbzbyc"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi137.nbzbyc"}]': finished 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.dllryb"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.dllryb"}]': finished 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.226 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:08:50.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi137.nbzbyc"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi137.nbzbyc"}]': finished 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.dllryb"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.dllryb"}]': finished 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.567 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:08:50.568 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all mgr 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all rgw 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all rbd-mirror 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all cephfs-mirror 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all iscsi 2024-06-15T01:08:51.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all nfs 2024-06-15T01:08:51.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all nvmeof 2024-06-15T01:08:51.475 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:51 smithi137 ceph-mon[29354]: pgmap v34: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 626 KiB/s wr, 217 op/s 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all mgr 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all rgw 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all rbd-mirror 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all cephfs-mirror 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all iscsi 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all nfs 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all nvmeof 2024-06-15T01:08:51.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:51 smithi142 ceph-mon[37379]: pgmap v34: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 626 KiB/s wr, 217 op/s 2024-06-15T01:08:52.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:52 smithi137 ceph-mon[29354]: Upgrade: Updating node-exporter.smithi137 (1/2) 2024-06-15T01:08:52.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:52 smithi137 ceph-mon[29354]: Deploying daemon node-exporter.smithi137 on smithi137 2024-06-15T01:08:52.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:52 smithi142 ceph-mon[37379]: Upgrade: Updating node-exporter.smithi137 (1/2) 2024-06-15T01:08:52.566 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:52 smithi142 ceph-mon[37379]: Deploying daemon node-exporter.smithi137 on smithi137 2024-06-15T01:08:53.299 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:53 smithi142 ceph-mon[37379]: pgmap v35: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 214 KiB/s wr, 91 op/s 2024-06-15T01:08:53.300 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:53 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:53.300 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:53 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:53.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:53 smithi137 ceph-mon[29354]: pgmap v35: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 214 KiB/s wr, 91 op/s 2024-06-15T01:08:53.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:53 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:53.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:53 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:55.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:55 smithi142 ceph-mon[37379]: pgmap v36: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 214 KiB/s wr, 91 op/s 2024-06-15T01:08:55.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:55 smithi142 ceph-mon[37379]: Upgrade: Updating node-exporter.smithi142 (2/2) 2024-06-15T01:08:55.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:55 smithi142 ceph-mon[37379]: Deploying daemon node-exporter.smithi142 on smithi142 2024-06-15T01:08:55.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:55 smithi137 ceph-mon[29354]: pgmap v36: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 214 KiB/s wr, 91 op/s 2024-06-15T01:08:55.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:55 smithi137 ceph-mon[29354]: Upgrade: Updating node-exporter.smithi142 (2/2) 2024-06-15T01:08:55.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:55 smithi137 ceph-mon[29354]: Deploying daemon node-exporter.smithi142 on smithi142 2024-06-15T01:08:57.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:57 smithi142 ceph-mon[37379]: pgmap v37: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 207 KiB/s wr, 72 op/s 2024-06-15T01:08:57.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:57 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:57.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:57 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:57.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:57 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:57.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:57 smithi137 ceph-mon[29354]: pgmap v37: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 207 KiB/s wr, 72 op/s 2024-06-15T01:08:57.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:57 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:57.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:57 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:08:57.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:57 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:08:58.796 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:08:58 smithi137 ceph-mon[29354]: pgmap v38: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 511 B/s wr, 0 op/s 2024-06-15T01:08:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:08:58 smithi142 ceph-mon[37379]: pgmap v38: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 511 B/s wr, 0 op/s 2024-06-15T01:09:00.638 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.638 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.638 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:00 smithi142 ceph-mon[37379]: pgmap v39: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 511 B/s wr, 0 op/s 2024-06-15T01:09:00.638 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.638 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.706 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:00 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.706 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:00 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.706 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:00 smithi137 ceph-mon[29354]: pgmap v39: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 511 B/s wr, 0 op/s 2024-06-15T01:09:00.707 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:00 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:00.707 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:00 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:02 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:02 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:02 smithi137 ceph-mon[29354]: pgmap v40: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:09:02.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:02 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.474 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:02 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:02 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:02 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:02 smithi142 ceph-mon[37379]: pgmap v40: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:09:02.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:02 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:02.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:02 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:04.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:03 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:04.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:03 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:04.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:03 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:04.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:03 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:04.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:03 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:04.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:03 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:05.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:04 smithi137 ceph-mon[29354]: pgmap v41: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:09:05.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:04 smithi142 ceph-mon[37379]: pgmap v41: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:06.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:06 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:06 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 -- 172.21.15.137:0/2989616052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8100bb0 msgr2=0x7fecb8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 --2- 172.21.15.137:0/2989616052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8100bb0 0x7fecb8100fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feca8009910 tx=0x7feca8009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 -- 172.21.15.137:0/2989616052 shutdown_connections 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 --2- 172.21.15.137:0/2989616052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101db0 0x7fecb8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 --2- 172.21.15.137:0/2989616052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8100bb0 0x7fecb8100fd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 -- 172.21.15.137:0/2989616052 >> 172.21.15.137:0/2989616052 conn(0x7fecb80fc110 msgr2=0x7fecb80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 -- 172.21.15.137:0/2989616052 shutdown_connections 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.603+0000 7fecbe76f700 1 -- 172.21.15.137:0/2989616052 wait complete. 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 Processor -- start 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 -- start start 2024-06-15T01:09:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8100bb0 0x7fecb81963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb8196f10 con 0x7fecb8101db0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.604+0000 7fecbe76f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb8197050 con 0x7fecb8100bb0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49918/0 (socket says 172.21.15.137:49918) 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 -- 172.21.15.137:0/356612504 learned_addr learned my addr 172.21.15.137:0/356612504 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb7fff700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8100bb0 0x7fecb81963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 -- 172.21.15.137:0/356612504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8100bb0 msgr2=0x7fecb81963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8100bb0 0x7fecb81963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 -- 172.21.15.137:0/356612504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80095f0 con 0x7fecb8101db0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb77fe700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b4b0 tx=0x7fecac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecac011800 con 0x7fecb8101db0 2024-06-15T01:09:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb819bb00 con 0x7fecb8101db0 2024-06-15T01:09:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fecac011df0 con 0x7fecb8101db0 2024-06-15T01:09:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecac010e40 con 0x7fecb8101db0 2024-06-15T01:09:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.605+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb819c0d0 con 0x7fecb8101db0 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.606+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb8109650 con 0x7fecb8101db0 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.607+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 31) v1 ==== 89566+0+0 (secure 0 0 0) 0x7fecac010380 con 0x7fecb8101db0 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.607+0000 7fecb57fa700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 0x7feca006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.607+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fecac089f70 con 0x7fecb8101db0 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.607+0000 7fecb7fff700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 0x7feca006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.608+0000 7fecb7fff700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 0x7feca006d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feca8006010 tx=0x7feca800b580 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:06.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.612+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fecac0bd650 con 0x7fecb8101db0 2024-06-15T01:09:06.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.842+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecb8106700 con 0x7feca006b050 2024-06-15T01:09:06.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.843+0000 7fecb57fa700 1 -- 172.21.15.137:0/356612504 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fecb8106700 con 0x7feca006b050 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 msgr2=0x7feca006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 0x7feca006d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feca8006010 tx=0x7feca800b580 comp rx=0 tx=0).stop 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 msgr2=0x7fecb81968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b4b0 tx=0x7fecac00b870 comp rx=0 tx=0).stop 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 shutdown_connections 2024-06-15T01:09:06.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feca006b050 0x7feca006d510 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8100bb0 0x7fecb81963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 --2- 172.21.15.137:0/356612504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecb8101db0 0x7fecb81968f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 >> 172.21.15.137:0/356612504 conn(0x7fecb80fc110 msgr2=0x7fecb8104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:06.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 shutdown_connections 2024-06-15T01:09:06.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.845+0000 7fecbe76f700 1 -- 172.21.15.137:0/356612504 wait complete. 2024-06-15T01:09:06.850 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 -- 172.21.15.137:0/2033893481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c8100cb0 msgr2=0x7f64c8101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/2033893481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c8100cb0 0x7f64c8101110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009910 tx=0x7f64b8009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 -- 172.21.15.137:0/2033893481 shutdown_connections 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/2033893481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c8100cb0 0x7f64c8101110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/2033893481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c80ffab0 0x7f64c80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 -- 172.21.15.137:0/2033893481 >> 172.21.15.137:0/2033893481 conn(0x7f64c80fb030 msgr2=0x7f64c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:06.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 -- 172.21.15.137:0/2033893481 shutdown_connections 2024-06-15T01:09:06.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.958+0000 7f64cf2f6700 1 -- 172.21.15.137:0/2033893481 wait complete. 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 Processor -- start 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 -- start start 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c8100cb0 0x7f64c8196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8196db0 con 0x7f64c80ffab0 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cf2f6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8196ef0 con 0x7f64c8100cb0 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cd092700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cd092700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49928/0 (socket says 172.21.15.137:49928) 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cd092700 1 -- 172.21.15.137:0/1475971655 learned_addr learned my addr 172.21.15.137:0/1475971655 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cc891700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c8100cb0 0x7f64c8196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cd092700 1 -- 172.21.15.137:0/1475971655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c8100cb0 msgr2=0x7f64c8196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.959+0000 7f64cd092700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c8100cb0 0x7f64c8196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64cd092700 1 -- 172.21.15.137:0/1475971655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80095f0 con 0x7f64c80ffab0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64cd092700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f64c400e8f0 tx=0x7f64c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c4009790 con 0x7f64c80ffab0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f64c40044b0 con 0x7f64c80ffab0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c4003d30 con 0x7f64c80ffab0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c819b9a0 con 0x7f64c80ffab0 2024-06-15T01:09:06.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.960+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c819bf70 con 0x7f64c80ffab0 2024-06-15T01:09:06.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.961+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c81084d0 con 0x7f64c80ffab0 2024-06-15T01:09:06.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.961+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 31) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f64c4003fe0 con 0x7f64c80ffab0 2024-06-15T01:09:06.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.962+0000 7f64be7fc700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 0x7f64b406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:06.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.962+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f64c4014070 con 0x7f64c80ffab0 2024-06-15T01:09:06.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.962+0000 7f64cc891700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 0x7f64b406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:06.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.962+0000 7f64cc891700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 0x7f64b406d580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005cd0 tx=0x7f64b8005c40 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:06.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:06.966+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f64c4058ff0 con 0x7f64c80ffab0 2024-06-15T01:09:07.166 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:07 smithi137 ceph-mon[29354]: pgmap v42: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:07.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.183+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64c8068ae0 con 0x7f64b406b0c0 2024-06-15T01:09:07.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.184+0000 7f64be7fc700 1 -- 172.21.15.137:0/1475971655 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f64c8068ae0 con 0x7f64b406b0c0 2024-06-15T01:09:07.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 msgr2=0x7f64b406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 0x7f64b406d580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005cd0 tx=0x7f64b8005c40 comp rx=0 tx=0).stop 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 msgr2=0x7f64c8196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f64c400e8f0 tx=0x7f64c400ecb0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 shutdown_connections 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f64b406b0c0 0x7f64b406d580 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64c80ffab0 0x7f64c8196250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 --2- 172.21.15.137:0/1475971655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c8100cb0 0x7f64c8196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 >> 172.21.15.137:0/1475971655 conn(0x7f64c80fb030 msgr2=0x7f64c80683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 shutdown_connections 2024-06-15T01:09:07.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.186+0000 7f64cf2f6700 1 -- 172.21.15.137:0/1475971655 wait complete. 2024-06-15T01:09:07.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 -- 172.21.15.137:0/2606930978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c0ff290 msgr2=0x7f6f5c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 --2- 172.21.15.137:0/2606930978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c0ff6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009910 tx=0x7f6f58009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 -- 172.21.15.137:0/2606930978 shutdown_connections 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 --2- 172.21.15.137:0/2606930978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c100490 0x7f6f5c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 --2- 172.21.15.137:0/2606930978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c0ff6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 -- 172.21.15.137:0/2606930978 >> 172.21.15.137:0/2606930978 conn(0x7f6f5c0fa810 msgr2=0x7f6f5c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 -- 172.21.15.137:0/2606930978 shutdown_connections 2024-06-15T01:09:07.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.291+0000 7f6f641f5700 1 -- 172.21.15.137:0/2606930978 wait complete. 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 Processor -- start 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 -- start start 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c19b680 con 0x7f6f5c100490 2024-06-15T01:09:07.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f641f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c19b7c0 con 0x7f6f5c0ff290 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f61790700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f61790700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49948/0 (socket says 172.21.15.137:49948) 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f61790700 1 -- 172.21.15.137:0/1749175206 learned_addr learned my addr 172.21.15.137:0/1749175206 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.292+0000 7f6f61f91700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f61790700 1 -- 172.21.15.137:0/1749175206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c0ff290 msgr2=0x7f6f5c109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f61790700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c109170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f61790700 1 -- 172.21.15.137:0/1749175206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580095f0 con 0x7f6f5c100490 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f61790700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c8a0 tx=0x7f6f4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4c0078e0 con 0x7f6f5c100490 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c19baa0 con 0x7f6f5c100490 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6f4c00f450 con 0x7f6f5c100490 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4c00e5c0 con 0x7f6f5c100490 2024-06-15T01:09:07.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.293+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c19c070 con 0x7f6f5c100490 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.294+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c107cb0 con 0x7f6f5c100490 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.294+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 31) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f6f4c00f5c0 con 0x7f6f5c100490 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.295+0000 7f6f52ffd700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 0x7f6f4806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.295+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6f4c089f80 con 0x7f6f5c100490 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.295+0000 7f6f61f91700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 0x7f6f4806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.295+0000 7f6f61f91700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 0x7f6f4806d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f58006010 tx=0x7f6f58005dc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:07.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.299+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f6f4c058860 con 0x7f6f5c100490 2024-06-15T01:09:07.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:07 smithi142 ceph-mon[37379]: pgmap v42: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:07.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.511+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f5c1046d0 con 0x7f6f4806b0b0 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.521+0000 7f6f52ffd700 1 -- 172.21.15.137:0/1749175206 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6f5c1046d0 con 0x7f6f4806b0b0 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (8m) 6s ago 10m 25.5M - 0.25.0 c8568f914cd2 4751365554ed 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (10m) 6s ago 10m 7931k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (9m) 8s ago 9m 7574k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (10m) 6s ago 10m 6660k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (9m) 8s ago 9m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (8m) 6s ago 9m 85.5M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (4m) 6s ago 4m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:09:07.516 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (4m) 6s ago 4m 131M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (4m) 8s ago 4m 14.2M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (4m) 8s ago 4m 17.1M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (90s) 6s ago 11m 481M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (25s) 8s ago 9m 436M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (11m) 6s ago 11m 55.1M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (9m) 8s ago 9m 44.7M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (15s) 6s ago 10m 7302k - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (11s) 8s ago 9m 4038k - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (8m) 6s ago 8m 221M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (7m) 6s ago 7m 243M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (7m) 6s ago 7m 194M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (7m) 8s ago 7m 264M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:09:07.517 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (6m) 8s ago 6m 239M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:09:07.518 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (6m) 8s ago 6m 220M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:09:07.518 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (32s) 6s ago 9m 50.2M - 2.43.0 a07b618ecd1d f5b8c62f3847 2024-06-15T01:09:07.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 msgr2=0x7f6f4806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 0x7f6f4806d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f58006010 tx=0x7f6f58005dc0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 msgr2=0x7f6f5c19b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c8a0 tx=0x7f6f4c00cbb0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 shutdown_connections 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f4806b0b0 0x7f6f4806d570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c0ff290 0x7f6f5c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 --2- 172.21.15.137:0/1749175206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f5c100490 0x7f6f5c19b0b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 >> 172.21.15.137:0/1749175206 conn(0x7f6f5c0fa810 msgr2=0x7f6f5c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 shutdown_connections 2024-06-15T01:09:07.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.523+0000 7f6f641f5700 1 -- 172.21.15.137:0/1749175206 wait complete. 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 -- 172.21.15.137:0/3953498545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30100c00 msgr2=0x7f0e30101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 --2- 172.21.15.137:0/3953498545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30100c00 0x7f0e30101020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009910 tx=0x7f0e20009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 -- 172.21.15.137:0/3953498545 shutdown_connections 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 --2- 172.21.15.137:0/3953498545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30101e00 0x7f0e30102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 --2- 172.21.15.137:0/3953498545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30100c00 0x7f0e30101020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 -- 172.21.15.137:0/3953498545 >> 172.21.15.137:0/3953498545 conn(0x7f0e300fc180 msgr2=0x7f0e300fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 -- 172.21.15.137:0/3953498545 shutdown_connections 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 -- 172.21.15.137:0/3953498545 wait complete. 2024-06-15T01:09:07.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.627+0000 7f0e35578700 1 Processor -- start 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e35578700 1 -- start start 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e35578700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30100c00 0x7f0e30074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e35578700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e35578700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300736c0 con 0x7f0e30101e00 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e35578700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30073800 con 0x7f0e30100c00 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49960/0 (socket says 172.21.15.137:49960) 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 -- 172.21.15.137:0/3967427718 learned_addr learned my addr 172.21.15.137:0/3967427718 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2effd700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30100c00 0x7f0e30074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 -- 172.21.15.137:0/3967427718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30100c00 msgr2=0x7f0e30074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30100c00 0x7f0e30074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 -- 172.21.15.137:0/3967427718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200095f0 con 0x7f0e30101e00 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e2e7fc700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400d8d0 tx=0x7f0e2400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.628+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e240098a0 con 0x7f0e30101e00 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.629+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e30073ae0 con 0x7f0e30101e00 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.629+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0e24009e90 con 0x7f0e30101e00 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.629+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2400f5f0 con 0x7f0e30101e00 2024-06-15T01:09:07.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.629+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3019e300 con 0x7f0e30101e00 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.630+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e30109620 con 0x7f0e30101e00 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.630+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 31) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f0e24010460 con 0x7f0e30101e00 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.631+0000 7f0e17fff700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 0x7f0e1806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.631+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0e2408a330 con 0x7f0e30101e00 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.631+0000 7f0e2effd700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 0x7f0e1806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.631+0000 7f0e2effd700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 0x7f0e1806d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2000b560 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:07.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.635+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f0e24058b90 con 0x7f0e30101e00 2024-06-15T01:09:07.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.914+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0e301097e0 con 0x7f0e30101e00 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.914+0000 7f0e17fff700 1 -- 172.21.15.137:0/3967427718 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f0e24010710 con 0x7f0e30101e00 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:09:07.911 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:09:07.912 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 msgr2=0x7f0e1806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 0x7f0e1806d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2000b560 comp rx=0 tx=0).stop 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 msgr2=0x7f0e30073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400d8d0 tx=0x7f0e2400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 shutdown_connections 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0e1806b0b0 0x7f0e1806d570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e30100c00 0x7f0e30074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 --2- 172.21.15.137:0/3967427718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e30101e00 0x7f0e30073180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 >> 172.21.15.137:0/3967427718 conn(0x7f0e300fc180 msgr2=0x7f0e30105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:07.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 shutdown_connections 2024-06-15T01:09:07.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:07.916+0000 7f0e35578700 1 -- 172.21.15.137:0/3967427718 wait complete. 2024-06-15T01:09:08.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 -- 172.21.15.137:0/3916164424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994100be0 msgr2=0x7f6994101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:08.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 --2- 172.21.15.137:0/3916164424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994100be0 0x7f6994101000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6984009910 tx=0x7f6984009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:08.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 -- 172.21.15.137:0/3916164424 shutdown_connections 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 --2- 172.21.15.137:0/3916164424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994101de0 0x7f6994102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 --2- 172.21.15.137:0/3916164424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994100be0 0x7f6994101000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 -- 172.21.15.137:0/3916164424 >> 172.21.15.137:0/3916164424 conn(0x7f69940fc140 msgr2=0x7f69940fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 -- 172.21.15.137:0/3916164424 shutdown_connections 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.024+0000 7f699a730700 1 -- 172.21.15.137:0/3916164424 wait complete. 2024-06-15T01:09:08.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 Processor -- start 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 -- start start 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 0x7f69941962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994196e50 con 0x7f6994101de0 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f699a730700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994196f90 con 0x7f6994100be0 2024-06-15T01:09:08.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f69937fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f69937fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49980/0 (socket says 172.21.15.137:49980) 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f69937fe700 1 -- 172.21.15.137:0/115079728 learned_addr learned my addr 172.21.15.137:0/115079728 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.025+0000 7f6993fff700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 0x7f69941962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69937fe700 1 -- 172.21.15.137:0/115079728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 msgr2=0x7f69941962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69937fe700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 0x7f69941962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69937fe700 1 -- 172.21.15.137:0/115079728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840095f0 con 0x7f6994101de0 2024-06-15T01:09:08.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f6993fff700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 0x7f69941962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:09:08.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69937fe700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f698800d900 tx=0x7f698800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:08.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6988004a00 con 0x7f6994101de0 2024-06-15T01:09:08.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6988005020 con 0x7f6994101de0 2024-06-15T01:09:08.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699419ba40 con 0x7f6994101de0 2024-06-15T01:09:08.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6988009c00 con 0x7f6994101de0 2024-06-15T01:09:08.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.026+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699419c010 con 0x7f6994101de0 2024-06-15T01:09:08.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.027+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6994109620 con 0x7f6994101de0 2024-06-15T01:09:08.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.027+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 31) v1 ==== 89566+0+0 (secure 0 0 0) 0x7f6988005190 con 0x7f6994101de0 2024-06-15T01:09:08.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.028+0000 7f69917fa700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 0x7f697c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:08.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.028+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f698808a620 con 0x7f6994101de0 2024-06-15T01:09:08.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.028+0000 7f6993fff700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 0x7f697c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:08.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.028+0000 7f6993fff700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 0x7f697c06d580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6984006010 tx=0x7f6984005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:08.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.032+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f6988058e80 con 0x7f6994101de0 2024-06-15T01:09:08.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:08 smithi137 ceph-mon[29354]: Upgrade: Updating prometheus.smithi137 2024-06-15T01:09:08.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:08 smithi137 ceph-mon[29354]: Deploying daemon prometheus.smithi137 on smithi137 2024-06-15T01:09:08.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:08 smithi137 ceph-mon[29354]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:08.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:08 smithi137 ceph-mon[29354]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:08.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:08 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3967427718' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:08.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.244+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6994106730 con 0x7f697c06b0c0 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.245+0000 7f69917fa700 1 -- 172.21.15.137:0/115079728 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f6994106730 con 0x7f697c06b0c0 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:09:08.240 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "2/2 daemons upgraded", 2024-06-15T01:09:08.241 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading prometheus daemons", 2024-06-15T01:09:08.241 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:09:08.241 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:09:08.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.247+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 msgr2=0x7f697c06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:08.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.247+0000 7f699a730700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 0x7f697c06d580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6984006010 tx=0x7f6984005a90 comp rx=0 tx=0).stop 2024-06-15T01:09:08.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.247+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 msgr2=0x7f6994196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:08.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.247+0000 7f699a730700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f698800d900 tx=0x7f698800dcc0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 shutdown_connections 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f697c06b0c0 0x7f697c06d580 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6994100be0 0x7f69941962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 --2- 172.21.15.137:0/115079728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6994101de0 0x7f6994196830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 >> 172.21.15.137:0/115079728 conn(0x7f69940fc140 msgr2=0x7f6994105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 shutdown_connections 2024-06-15T01:09:08.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:08.248+0000 7f699a730700 1 -- 172.21.15.137:0/115079728 wait complete. 2024-06-15T01:09:08.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:08 smithi142 ceph-mon[37379]: Upgrade: Updating prometheus.smithi137 2024-06-15T01:09:08.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:08 smithi142 ceph-mon[37379]: Deploying daemon prometheus.smithi137 on smithi137 2024-06-15T01:09:08.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:08 smithi142 ceph-mon[37379]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:08.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:08 smithi142 ceph-mon[37379]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:08.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:08 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3967427718' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:09.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:09 smithi137 ceph-mon[29354]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:09.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:09 smithi137 ceph-mon[29354]: pgmap v43: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:09.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:09 smithi142 ceph-mon[37379]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:09.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:09 smithi142 ceph-mon[37379]: pgmap v43: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:10.306 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:10 smithi137 ceph-mon[29354]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:10.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:10 smithi142 ceph-mon[37379]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:11 smithi137 ceph-mon[29354]: pgmap v44: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:11.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:11 smithi142 ceph-mon[37379]: pgmap v44: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:13.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:12 smithi137 ceph-mon[29354]: pgmap v45: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:13.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:12 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:13.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:12 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:13.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:12 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:13.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:12 smithi142 ceph-mon[37379]: pgmap v45: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:13.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:12 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:13.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:12 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:13.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:12 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:15.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:14 smithi137 ceph-mon[29354]: pgmap v46: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:15.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:14 smithi142 ceph-mon[37379]: pgmap v46: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:17.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:16 smithi137 ceph-mon[29354]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:17.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:17.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:16 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:17.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:16 smithi142 ceph-mon[37379]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:17.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:17.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:16 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:18.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:18 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:18.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:18 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:18.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:18 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:18.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:18 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:18.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:18 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:18.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:18 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:19.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:19 smithi137 ceph-mon[29354]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:19.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:19 smithi142 ceph-mon[37379]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:20.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: mgrmap e32: smithi137.nbzbyc(active, since 91s), standbys: smithi142.dllryb 2024-06-15T01:09:20.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: pgmap v49: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:20.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:20 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: mgrmap e32: smithi137.nbzbyc(active, since 91s), standbys: smithi142.dllryb 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: pgmap v49: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:20.817 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:20 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:21.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:21 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:09:21.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:21 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-15T01:09:22.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:22 smithi137 ceph-mon[29354]: pgmap v50: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:22.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:22 smithi137 ceph-mon[29354]: Upgrade: Updating alertmanager.smithi137 2024-06-15T01:09:22.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:22 smithi137 ceph-mon[29354]: Deploying daemon alertmanager.smithi137 on smithi137 2024-06-15T01:09:22.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:22 smithi142 ceph-mon[37379]: pgmap v50: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:22.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:22 smithi142 ceph-mon[37379]: Upgrade: Updating alertmanager.smithi137 2024-06-15T01:09:22.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:22 smithi142 ceph-mon[37379]: Deploying daemon alertmanager.smithi137 on smithi137 2024-06-15T01:09:24.866 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:24 smithi137 ceph-mon[29354]: pgmap v51: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:25.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:24 smithi142 ceph-mon[37379]: pgmap v51: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:26.893 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:26 smithi137 ceph-mon[29354]: pgmap v52: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:27.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:26 smithi142 ceph-mon[37379]: pgmap v52: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:28.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:28 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:28.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:28 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:28.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:28 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:28.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:28 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:28.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:28 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:28.341 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:28 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:29.195 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:29 smithi137 ceph-mon[29354]: pgmap v53: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:29.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:29 smithi142 ceph-mon[37379]: pgmap v53: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:31.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:31 smithi142 ceph-mon[37379]: pgmap v54: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:31.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:31 smithi137 ceph-mon[29354]: pgmap v54: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-06-15T01:09:32.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:32 smithi137 ceph-mon[29354]: pgmap v55: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:32.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:32 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:32.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:32 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:32.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:32 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:32.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:32 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:33.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:32 smithi142 ceph-mon[37379]: pgmap v55: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:33.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:32 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:33.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:32 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:33.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:32 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:33.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:32 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:33.770 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:33 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:34.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:33 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:34.974 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:34 smithi137 ceph-mon[29354]: pgmap v56: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:35.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:34 smithi142 ceph-mon[37379]: pgmap v56: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:37.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: pgmap v57: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:37.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:37.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:36 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: pgmap v57: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:37.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.317 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:36 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:37.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:37 smithi137 ceph-mon[29354]: Upgrade: Updating grafana.smithi137 2024-06-15T01:09:37.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:37 smithi137 ceph-mon[29354]: Deploying daemon grafana.smithi137 on smithi137 2024-06-15T01:09:38.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:37 smithi142 ceph-mon[37379]: Upgrade: Updating grafana.smithi137 2024-06-15T01:09:38.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:37 smithi142 ceph-mon[37379]: Deploying daemon grafana.smithi137 on smithi137 2024-06-15T01:09:38.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 -- 172.21.15.137:0/1659687067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 msgr2=0x7f13740ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1659687067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13740ffed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1364009910 tx=0x7f1364009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 -- 172.21.15.137:0/1659687067 shutdown_connections 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1659687067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f1374101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1659687067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13740ffed0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 -- 172.21.15.137:0/1659687067 >> 172.21.15.137:0/1659687067 conn(0x7f13740fb050 msgr2=0x7f13740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 -- 172.21.15.137:0/1659687067 shutdown_connections 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.357+0000 7f1379dd7700 1 -- 172.21.15.137:0/1659687067 wait complete. 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 Processor -- start 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 -- start start 2024-06-15T01:09:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f13741968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374196f10 con 0x7f13740ffab0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f1379dd7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374197050 con 0x7f1374100cb0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36442/0 (socket says 172.21.15.137:36442) 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 -- 172.21.15.137:0/1533791769 learned_addr learned my addr 172.21.15.137:0/1533791769 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f136bfff700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f13741968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 -- 172.21.15.137:0/1533791769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 msgr2=0x7f13741968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f13741968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f13737fe700 1 -- 172.21.15.137:0/1533791769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13640095f0 con 0x7f13740ffab0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.358+0000 7f136bfff700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f13741968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f13737fe700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f136400bda0 tx=0x7f136400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136401d070 con 0x7f13740ffab0 2024-06-15T01:09:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1364022470 con 0x7f13740ffab0 2024-06-15T01:09:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136400f670 con 0x7f13740ffab0 2024-06-15T01:09:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137419baa0 con 0x7f13740ffab0 2024-06-15T01:09:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.359+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137419c010 con 0x7f13740ffab0 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.360+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374108550 con 0x7f13740ffab0 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.360+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f1364022a50 con 0x7f13740ffab0 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.361+0000 7f13717fa700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 0x7f136006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.361+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f136408b850 con 0x7f13740ffab0 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.361+0000 7f136bfff700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 0x7f136006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.361+0000 7f136bfff700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 0x7f136006d440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f135c007900 tx=0x7f135c008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:38.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.365+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f136405a4a0 con 0x7f13740ffab0 2024-06-15T01:09:38.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.582+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13740693b0 con 0x7f136006af80 2024-06-15T01:09:38.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.583+0000 7f13717fa700 1 -- 172.21.15.137:0/1533791769 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f13740693b0 con 0x7f136006af80 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 msgr2=0x7f136006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 0x7f136006d440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f135c007900 tx=0x7f135c008040 comp rx=0 tx=0).stop 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 msgr2=0x7f13741963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f136400bda0 tx=0x7f136400be80 comp rx=0 tx=0).stop 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 shutdown_connections 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136006af80 0x7f136006d440 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffab0 0x7f13741963b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 --2- 172.21.15.137:0/1533791769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374100cb0 0x7f13741968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 >> 172.21.15.137:0/1533791769 conn(0x7f13740fb050 msgr2=0x7f1374068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:38.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 shutdown_connections 2024-06-15T01:09:38.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.585+0000 7f1379dd7700 1 -- 172.21.15.137:0/1533791769 wait complete. 2024-06-15T01:09:38.588 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 -- 172.21.15.137:0/3530428536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d801016e0 msgr2=0x7f3d80103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 --2- 172.21.15.137:0/3530428536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d801016e0 0x7f3d80103ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009910 tx=0x7f3d70009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 -- 172.21.15.137:0/3530428536 shutdown_connections 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 --2- 172.21.15.137:0/3530428536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d80104010 0x7f3d80106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 --2- 172.21.15.137:0/3530428536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d801016e0 0x7f3d80103ad0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 -- 172.21.15.137:0/3530428536 >> 172.21.15.137:0/3530428536 conn(0x7f3d800fb050 msgr2=0x7f3d800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 -- 172.21.15.137:0/3530428536 shutdown_connections 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.693+0000 7f3d86488700 1 -- 172.21.15.137:0/3530428536 wait complete. 2024-06-15T01:09:38.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 Processor -- start 2024-06-15T01:09:38.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 -- start start 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d801016e0 0x7f3d80196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80196dc0 con 0x7f3d80104010 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d86488700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80196f00 con 0x7f3d801016e0 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d7f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d7f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36468/0 (socket says 172.21.15.137:36468) 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.694+0000 7f3d7f7fe700 1 -- 172.21.15.137:0/352194833 learned_addr learned my addr 172.21.15.137:0/352194833 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7ffff700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d801016e0 0x7f3d80196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7f7fe700 1 -- 172.21.15.137:0/352194833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d801016e0 msgr2=0x7f3d80196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7f7fe700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d801016e0 0x7f3d80196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7f7fe700 1 -- 172.21.15.137:0/352194833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d700095f0 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7f7fe700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c8f0 tx=0x7f3d7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d74007a70 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8019b9b0 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3d74004cc0 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d74005580 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.695+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8019bf80 con 0x7f3d80104010 2024-06-15T01:09:38.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.696+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d80190410 con 0x7f3d80104010 2024-06-15T01:09:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.697+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f3d74004750 con 0x7f3d80104010 2024-06-15T01:09:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.697+0000 7f3d7d7fa700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 0x7f3d6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.697+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3d74013070 con 0x7f3d80104010 2024-06-15T01:09:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.697+0000 7f3d7ffff700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 0x7f3d6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.697+0000 7f3d7ffff700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 0x7f3d6806d440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d70006010 tx=0x7f3d700058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:38.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.701+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f3d74058c00 con 0x7f3d80104010 2024-06-15T01:09:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.917+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d800611d0 con 0x7f3d6806af80 2024-06-15T01:09:38.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.918+0000 7f3d7d7fa700 1 -- 172.21.15.137:0/352194833 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f3d800611d0 con 0x7f3d6806af80 2024-06-15T01:09:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.920+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 msgr2=0x7f3d6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.920+0000 7f3d86488700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 0x7f3d6806d440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d70006010 tx=0x7f3d700058e0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.920+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 msgr2=0x7f3d801967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.920+0000 7f3d86488700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c8f0 tx=0x7f3d7400ccb0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 shutdown_connections 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d6806af80 0x7f3d6806d440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d801016e0 0x7f3d80196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 --2- 172.21.15.137:0/352194833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d80104010 0x7f3d801967a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 >> 172.21.15.137:0/352194833 conn(0x7f3d800fb050 msgr2=0x7f3d800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 shutdown_connections 2024-06-15T01:09:38.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:38.921+0000 7f3d86488700 1 -- 172.21.15.137:0/352194833 wait complete. 2024-06-15T01:09:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.024+0000 7f6086e1e700 1 -- 172.21.15.137:0/2858179221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080103f70 msgr2=0x7f6080106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.024+0000 7f6086e1e700 1 --2- 172.21.15.137:0/2858179221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080103f70 0x7f6080106360 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f606c009910 tx=0x7f606c009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- 172.21.15.137:0/2858179221 shutdown_connections 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 --2- 172.21.15.137:0/2858179221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080103f70 0x7f6080106360 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 --2- 172.21.15.137:0/2858179221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080101640 0x7f6080103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- 172.21.15.137:0/2858179221 >> 172.21.15.137:0/2858179221 conn(0x7f60800fb030 msgr2=0x7f60800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- 172.21.15.137:0/2858179221 shutdown_connections 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- 172.21.15.137:0/2858179221 wait complete. 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 Processor -- start 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- start start 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080103f70 0x7f60800ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800ffb30 con 0x7f6080101640 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.025+0000 7f6086e1e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800ffc70 con 0x7f6080103f70 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36482/0 (socket says 172.21.15.137:36482) 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 -- 172.21.15.137:0/1149980514 learned_addr learned my addr 172.21.15.137:0/1149980514 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f607ffff700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080103f70 0x7f60800ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 -- 172.21.15.137:0/1149980514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080103f70 msgr2=0x7f60800ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080103f70 0x7f60800ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 -- 172.21.15.137:0/1149980514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0095f0 con 0x7f6080101640 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6084bba700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f607400d8d0 tx=0x7f607400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60740098a0 con 0x7f6080101640 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6074009e90 con 0x7f6080101640 2024-06-15T01:09:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60800fff50 con 0x7f6080101640 2024-06-15T01:09:39.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607400f5f0 con 0x7f6080101640 2024-06-15T01:09:39.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.026+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608019e2b0 con 0x7f6080101640 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.027+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60801903e0 con 0x7f6080101640 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.028+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f6074010460 con 0x7f6080101640 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.028+0000 7f607dffb700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 0x7f607006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.028+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f607408a010 con 0x7f6080101640 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.028+0000 7f607ffff700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 0x7f607006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.029+0000 7f607ffff700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 0x7f607006d440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f606c006010 tx=0x7f606c00b560 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.033+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f6074058b10 con 0x7f6080101640 2024-06-15T01:09:39.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:38 smithi137 ceph-mon[29354]: pgmap v58: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:39.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.235+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60800611d0 con 0x7f607006af80 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.246+0000 7f607dffb700 1 -- 172.21.15.137:0/1149980514 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f60800611d0 con 0x7f607006af80 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (12s) 7s ago 10m 15.0M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (11m) 7s ago 11m 7972k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (9m) 39s ago 9m 7574k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (11m) 7s ago 11m 6660k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (9m) 39s ago 9m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (9m) 7s ago 10m 87.6M - 9.4.7 954c08fa6188 b729e5c1698f 2024-06-15T01:09:39.240 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (5m) 7s ago 5m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (5m) 7s ago 5m 131M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (5m) 39s ago 5m 14.2M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (5m) 39s ago 5m 17.1M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (2m) 7s ago 12m 484M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (57s) 39s ago 9m 436M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (12m) 7s ago 12m 56.5M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (9m) 39s ago 9m 44.7M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (46s) 7s ago 10m 8108k - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (42s) 39s ago 9m 4038k - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (8m) 7s ago 8m 222M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:09:39.241 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (8m) 7s ago 8m 243M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:09:39.242 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (7m) 7s ago 7m 195M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:09:39.242 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (7m) 39s ago 7m 264M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:09:39.242 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (7m) 39s ago 7m 239M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:09:39.242 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (6m) 39s ago 6m 220M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:09:39.242 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (27s) 7s ago 10m 53.2M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 msgr2=0x7f607006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 0x7f607006d440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f606c006010 tx=0x7f606c00b560 comp rx=0 tx=0).stop 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 msgr2=0x7f6080100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f607400d8d0 tx=0x7f607400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 shutdown_connections 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f607006af80 0x7f607006d440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6080101640 0x7f6080100fa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 --2- 172.21.15.137:0/1149980514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6080103f70 0x7f60800ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 >> 172.21.15.137:0/1149980514 conn(0x7f60800fb030 msgr2=0x7f60800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 shutdown_connections 2024-06-15T01:09:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.248+0000 7f6086e1e700 1 -- 172.21.15.137:0/1149980514 wait complete. 2024-06-15T01:09:39.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:38 smithi142 ceph-mon[37379]: pgmap v58: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:39.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2462237667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 msgr2=0x7f249c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2462237667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c100e50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2498009910 tx=0x7f2498009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2462237667 shutdown_connections 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2462237667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2462237667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c100e50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2462237667 >> 172.21.15.137:0/2462237667 conn(0x7f249c0fbfd0 msgr2=0x7f249c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2462237667 shutdown_connections 2024-06-15T01:09:39.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.353+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2462237667 wait complete. 2024-06-15T01:09:39.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 Processor -- start 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 -- start start 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c196c20 con 0x7f249c100a30 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a2f9e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c196d60 con 0x7f249c101c30 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36494/0 (socket says 172.21.15.137:36494) 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 -- 172.21.15.137:0/2444327544 learned_addr learned my addr 172.21.15.137:0/2444327544 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a179b700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 -- 172.21.15.137:0/2444327544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 msgr2=0x7f249c196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 -- 172.21.15.137:0/2444327544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980095f0 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a179b700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.354+0000 7f24a1f9c700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f249800bd70 tx=0x7f249800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.355+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f249801d070 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.355+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c19b7b0 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.355+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f249800f460 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.355+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2498021640 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.355+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c19bd20 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.356+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c109450 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.356+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f249800fa20 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.356+0000 7f2492ffd700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 0x7f248806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.357+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f249808b5f0 con 0x7f249c100a30 2024-06-15T01:09:39.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.357+0000 7f24a179b700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 0x7f248806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.357+0000 7f24a179b700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 0x7f248806d440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f248c005fd0 tx=0x7f248c005dc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.361+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f249805a070 con 0x7f249c100a30 2024-06-15T01:09:39.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.638+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f249c19c3c0 con 0x7f249c100a30 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.639+0000 7f2492ffd700 1 -- 172.21.15.137:0/2444327544 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f2498059c00 con 0x7f249c100a30 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:39.633 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:09:39.634 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 msgr2=0x7f248806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 0x7f248806d440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f248c005fd0 tx=0x7f248c005dc0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 msgr2=0x7f249c1960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f249800bd70 tx=0x7f249800be50 comp rx=0 tx=0).stop 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 shutdown_connections 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f248806af80 0x7f248806d440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f249c100a30 0x7f249c1960c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 --2- 172.21.15.137:0/2444327544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f249c101c30 0x7f249c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 >> 172.21.15.137:0/2444327544 conn(0x7f249c0fbfd0 msgr2=0x7f249c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 shutdown_connections 2024-06-15T01:09:39.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.641+0000 7f24a2f9e700 1 -- 172.21.15.137:0/2444327544 wait complete. 2024-06-15T01:09:39.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 -- 172.21.15.137:0/2303539932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c100cb0 msgr2=0x7f9b7c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2303539932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c101110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009910 tx=0x7f9b6c009c20 comp rx=0 tx=0).stop 2024-06-15T01:09:39.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 -- 172.21.15.137:0/2303539932 shutdown_connections 2024-06-15T01:09:39.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2303539932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c101110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2303539932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 -- 172.21.15.137:0/2303539932 >> 172.21.15.137:0/2303539932 conn(0x7f9b7c0fb030 msgr2=0x7f9b7c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 -- 172.21.15.137:0/2303539932 shutdown_connections 2024-06-15T01:09:39.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.748+0000 7f9b847f4700 1 -- 172.21.15.137:0/2303539932 wait complete. 2024-06-15T01:09:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 Processor -- start 2024-06-15T01:09:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 -- start start 2024-06-15T01:09:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c19b580 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b847f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c19b6c0 con 0x7f9b7c100cb0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b82590700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b82590700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36502/0 (socket says 172.21.15.137:36502) 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b82590700 1 -- 172.21.15.137:0/2092407143 learned_addr learned my addr 172.21.15.137:0/2092407143 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.749+0000 7f9b81d8f700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b82590700 1 -- 172.21.15.137:0/2092407143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c100cb0 msgr2=0x7f9b7c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b82590700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b82590700 1 -- 172.21.15.137:0/2092407143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0095f0 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b82590700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c8f0 tx=0x7f9b7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78007a70 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b78007bd0 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c19b9a0 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78018630 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.750+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c19bf70 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.751+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c1084d0 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.751+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f9b7800f450 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.752+0000 7f9b737fe700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 0x7f9b6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.752+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9b7808a180 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.752+0000 7f9b81d8f700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 0x7f9b6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:09:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.753+0000 7f9b81d8f700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 0x7f9b6806d440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0694f0 tx=0x7f9b6c00b580 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:09:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.756+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f9b78058c00 con 0x7f9b7c0ffab0 2024-06-15T01:09:39.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:39 smithi137 ceph-mon[29354]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:39.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:39 smithi137 ceph-mon[29354]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:39.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:39 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2444327544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:39.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.984+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b7c105600 con 0x7f9b6806af80 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.986+0000 7f9b737fe700 1 -- 172.21.15.137:0/2092407143 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f9b7c105600 con 0x7f9b6806af80 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "2/2 daemons upgraded", 2024-06-15T01:09:39.980 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading grafana daemons", 2024-06-15T01:09:39.981 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:09:39.981 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 msgr2=0x7f9b6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 0x7f9b6806d440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0694f0 tx=0x7f9b6c00b580 comp rx=0 tx=0).stop 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 msgr2=0x7f9b7c068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c8f0 tx=0x7f9b7800ccb0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 shutdown_connections 2024-06-15T01:09:39.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b6806af80 0x7f9b6806d440 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b7c0ffab0 0x7f9b7c068f70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 --2- 172.21.15.137:0/2092407143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:09:39.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 >> 172.21.15.137:0/2092407143 conn(0x7f9b7c0fb030 msgr2=0x7f9b7c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:09:39.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 shutdown_connections 2024-06-15T01:09:39.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:09:39.988+0000 7f9b847f4700 1 -- 172.21.15.137:0/2092407143 wait complete. 2024-06-15T01:09:40.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:39 smithi142 ceph-mon[37379]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:40.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:39 smithi142 ceph-mon[37379]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:40.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:39 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2444327544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:41.101 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:40 smithi137 ceph-mon[29354]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:41.102 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:40 smithi137 ceph-mon[29354]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:41.102 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:40 smithi137 ceph-mon[29354]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:41.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:40 smithi142 ceph-mon[37379]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:41.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:40 smithi142 ceph-mon[37379]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:41.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:40 smithi142 ceph-mon[37379]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:42 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:43.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:42 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:45.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:44 smithi137 ceph-mon[29354]: pgmap v61: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:45.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:44 smithi142 ceph-mon[37379]: pgmap v61: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:47.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:46 smithi142 ceph-mon[37379]: pgmap v62: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:47.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:47.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:46 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:47.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:46 smithi137 ceph-mon[29354]: pgmap v62: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:47.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:47.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:46 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:48.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:48.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:48.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:48 smithi137 ceph-mon[29354]: pgmap v63: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:48.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:48 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:48.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:48.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:48.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:48 smithi142 ceph-mon[37379]: pgmap v63: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:48.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: pgmap v64: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:50.973 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:50 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: pgmap v64: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:51.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:50 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.977 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: Upgrade: Finalizing container_image settings 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-15T01:09:51.978 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-15T01:09:51.979 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-06-15T01:09:51.980 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: Upgrade: Complete! 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:51.981 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:51 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:52.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.066 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: Upgrade: Finalizing container_image settings 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-15T01:09:52.067 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: Upgrade: Complete! 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:09:52.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:51 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:53.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:53 smithi137 ceph-mon[29354]: pgmap v65: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:53.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:53 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:53.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:53 smithi142 ceph-mon[37379]: pgmap v65: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:53.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:53 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:09:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:55 smithi142 ceph-mon[37379]: pgmap v66: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:55.473 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:55 smithi137 ceph-mon[29354]: pgmap v66: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:57.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:57 smithi142 ceph-mon[37379]: pgmap v67: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:57.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:57 smithi137 ceph-mon[29354]: pgmap v67: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:09:58 smithi142 ceph-mon[37379]: pgmap v68: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:09:58.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:09:58 smithi137 ceph-mon[29354]: pgmap v68: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:01.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:00 smithi142 ceph-mon[37379]: pgmap v69: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:01.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:00 smithi142 ceph-mon[37379]: overall HEALTH_OK 2024-06-15T01:10:01.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:00 smithi137 ceph-mon[29354]: pgmap v69: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:01.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:00 smithi137 ceph-mon[29354]: overall HEALTH_OK 2024-06-15T01:10:02.984 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:02 smithi137 ceph-mon[29354]: pgmap v70: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:03.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:02 smithi142 ceph-mon[37379]: pgmap v70: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:03.770 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:03 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:04.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:03 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:05.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:04 smithi142 ceph-mon[37379]: pgmap v71: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:04 smithi137 ceph-mon[29354]: pgmap v71: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:07.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:06 smithi142 ceph-mon[37379]: pgmap v72: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:07.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:06 smithi137 ceph-mon[29354]: pgmap v72: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:08.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:08 smithi142 ceph-mon[37379]: pgmap v73: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:08.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:08 smithi137 ceph-mon[29354]: pgmap v73: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 -- 172.21.15.137:0/3492578617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 msgr2=0x7f5070100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 --2- 172.21.15.137:0/3492578617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070100fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5058009910 tx=0x7f5058009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 -- 172.21.15.137:0/3492578617 shutdown_connections 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 --2- 172.21.15.137:0/3492578617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 0x7f5070102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 --2- 172.21.15.137:0/3492578617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070100fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 -- 172.21.15.137:0/3492578617 >> 172.21.15.137:0/3492578617 conn(0x7f50700fc140 msgr2=0x7f50700fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 -- 172.21.15.137:0/3492578617 shutdown_connections 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 -- 172.21.15.137:0/3492578617 wait complete. 2024-06-15T01:10:10.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.110+0000 7f5075256700 1 Processor -- start 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f5075256700 1 -- start start 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f5075256700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f5075256700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 0x7f5070073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f5075256700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700736c0 con 0x7f5070100bc0 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f5075256700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070073800 con 0x7f5070101dc0 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34878/0 (socket says 172.21.15.137:34878) 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 -- 172.21.15.137:0/1933876242 learned_addr learned my addr 172.21.15.137:0/1933876242 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506e59c700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 0x7f5070073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 -- 172.21.15.137:0/1933876242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 msgr2=0x7f5070073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 0x7f5070073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.111+0000 7f506ed9d700 1 -- 172.21.15.137:0/1933876242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580095f0 con 0x7f5070100bc0 2024-06-15T01:10:10.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f506ed9d700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f505800bda0 tx=0x7f505800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:10.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505801d070 con 0x7f5070100bc0 2024-06-15T01:10:10.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5058005710 con 0x7f5070100bc0 2024-06-15T01:10:10.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5070073a80 con 0x7f5070100bc0 2024-06-15T01:10:10.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5058017400 con 0x7f5070100bc0 2024-06-15T01:10:10.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.112+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070073f70 con 0x7f5070100bc0 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.113+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50701095e0 con 0x7f5070100bc0 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.113+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f505800f460 con 0x7f5070100bc0 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.114+0000 7f5067fff700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 0x7f505c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.114+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f505808b650 con 0x7f5070100bc0 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.114+0000 7f506e59c700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 0x7f505c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:10.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.114+0000 7f506e59c700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 0x7f505c06d440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5060005950 tx=0x7f506000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:10.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.118+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f505805a0d0 con 0x7f5070100bc0 2024-06-15T01:10:10.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.330+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5070106710 con 0x7f505c06af80 2024-06-15T01:10:10.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.331+0000 7f5067fff700 1 -- 172.21.15.137:0/1933876242 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5070106710 con 0x7f505c06af80 2024-06-15T01:10:10.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.333+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 msgr2=0x7f505c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:10.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.333+0000 7f5075256700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 0x7f505c06d440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5060005950 tx=0x7f506000b410 comp rx=0 tx=0).stop 2024-06-15T01:10:10.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.333+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 msgr2=0x7f5070074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.333+0000 7f5075256700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f505800bda0 tx=0x7f505800bdd0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 shutdown_connections 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f505c06af80 0x7f505c06d440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5070100bc0 0x7f5070074b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 --2- 172.21.15.137:0/1933876242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5070101dc0 0x7f5070073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 >> 172.21.15.137:0/1933876242 conn(0x7f50700fc140 msgr2=0x7f5070104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 shutdown_connections 2024-06-15T01:10:10.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:10.334+0000 7f5075256700 1 -- 172.21.15.137:0/1933876242 wait complete. 2024-06-15T01:10:10.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:10 smithi137 ceph-mon[29354]: pgmap v74: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:11.039 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-06-15T01:10:11.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:10 smithi142 ceph-mon[37379]: pgmap v74: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:11.287 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:12.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:11 smithi142 ceph-mon[37379]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:12.195 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:11 smithi137 ceph-mon[29354]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:12.852 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:12 smithi137 ceph-mon[29354]: pgmap v75: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:13.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:12 smithi142 ceph-mon[37379]: pgmap v75: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 -- 172.21.15.137:0/1967769729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 msgr2=0x7f1f98105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/1967769729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f98105e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009910 tx=0x7f1f88009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 -- 172.21.15.137:0/1967769729 shutdown_connections 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/1967769729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 0x7f1f980ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/1967769729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f98105e20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 -- 172.21.15.137:0/1967769729 >> 172.21.15.137:0/1967769729 conn(0x7f1f980fb530 msgr2=0x7f1f980fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 -- 172.21.15.137:0/1967769729 shutdown_connections 2024-06-15T01:10:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.077+0000 7f1f9e15c700 1 -- 172.21.15.137:0/1967769729 wait complete. 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.078+0000 7f1f9e15c700 1 Processor -- start 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.078+0000 7f1f9e15c700 1 -- start start 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 0x7f1f9819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819d1c0 con 0x7f1f98105a40 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98196670 con 0x7f1f980ffa40 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34886/0 (socket says 172.21.15.137:34886) 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 -- 172.21.15.137:0/3811825732 learned_addr learned my addr 172.21.15.137:0/3811825732 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f977fe700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 0x7f1f9819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 -- 172.21.15.137:0/3811825732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 msgr2=0x7f1f9819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 0x7f1f9819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 -- 172.21.15.137:0/3811825732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880095f0 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f96ffd700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b4b0 tx=0x7f1f8c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8c011800 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f98196950 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f8c011e00 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8c010e60 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.079+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98196ea0 con 0x7f1f98105a40 2024-06-15T01:10:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.080+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9804ea90 con 0x7f1f98105a40 2024-06-15T01:10:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.081+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f1f8c011960 con 0x7f1f98105a40 2024-06-15T01:10:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.081+0000 7f1f94ff9700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 0x7f1f8006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.081+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1f8c089d00 con 0x7f1f98105a40 2024-06-15T01:10:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.082+0000 7f1f977fe700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 0x7f1f8006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.082+0000 7f1f977fe700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 0x7f1f8006d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006010 tx=0x7f1f8801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.086+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f1f8c08d030 con 0x7f1f98105a40 2024-06-15T01:10:13.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.359+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f98197b80 con 0x7f1f98105a40 2024-06-15T01:10:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.360+0000 7f1f94ff9700 1 -- 172.21.15.137:0/3811825732 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f1f8c058780 con 0x7f1f98105a40 2024-06-15T01:10:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 msgr2=0x7f1f8006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 0x7f1f8006d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006010 tx=0x7f1f8801a040 comp rx=0 tx=0).stop 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 msgr2=0x7f1f9819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b4b0 tx=0x7f1f8c00b7c0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 shutdown_connections 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f8006af80 0x7f1f8006d440 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f980ffa40 0x7f1f9819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 --2- 172.21.15.137:0/3811825732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f98105a40 0x7f1f9819cb30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 >> 172.21.15.137:0/3811825732 conn(0x7f1f980fb530 msgr2=0x7f1f98103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 shutdown_connections 2024-06-15T01:10:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:13.362+0000 7f1f9e15c700 1 -- 172.21.15.137:0/3811825732 wait complete. 2024-06-15T01:10:13.367 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:10:14.038 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:13 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/3811825732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:14.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-06-15T01:10:14.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:13 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/3811825732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:14.274 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:14.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:14 smithi137 ceph-mon[29354]: pgmap v76: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:15.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:14 smithi142 ceph-mon[37379]: pgmap v76: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 -- 172.21.15.137:0/2460133364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610101d10 msgr2=0x7fa6101020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 --2- 172.21.15.137:0/2460133364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610101d10 0x7fa6101020f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa604009910 tx=0x7fa604009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 -- 172.21.15.137:0/2460133364 shutdown_connections 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 --2- 172.21.15.137:0/2460133364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610102630 0x7fa61010ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 --2- 172.21.15.137:0/2460133364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610101d10 0x7fa6101020f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 -- 172.21.15.137:0/2460133364 >> 172.21.15.137:0/2460133364 conn(0x7fa6100fb540 msgr2=0x7fa6100fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 -- 172.21.15.137:0/2460133364 shutdown_connections 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.957+0000 7fa616d4a700 1 -- 172.21.15.137:0/2460133364 wait complete. 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 Processor -- start 2024-06-15T01:10:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 -- start start 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 0x7fa6100ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101003f0 con 0x7fa610102630 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa616d4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610100530 con 0x7fa610101d10 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34916/0 (socket says 172.21.15.137:34916) 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 -- 172.21.15.137:0/1319283023 learned_addr learned my addr 172.21.15.137:0/1319283023 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa614ae6700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 0x7fa6100ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 -- 172.21.15.137:0/1319283023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 msgr2=0x7fa6100ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 0x7fa6100ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa60ffff700 1 -- 172.21.15.137:0/1319283023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040095f0 con 0x7fa610102630 2024-06-15T01:10:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.958+0000 7fa614ae6700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 0x7fa6100ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa60ffff700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00b3d0 tx=0x7fa5fc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5fc011800 con 0x7fa610102630 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6101a2990 con 0x7fa610102630 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5fc011e00 con 0x7fa610102630 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5fc010e30 con 0x7fa610102630 2024-06-15T01:10:15.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.959+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6101a2da0 con 0x7fa610102630 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.960+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610108220 con 0x7fa610102630 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.960+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7fa5fc011960 con 0x7fa610102630 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.961+0000 7fa60dffb700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 0x7fa60006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.961+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa5fc089cb0 con 0x7fa610102630 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.961+0000 7fa614ae6700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 0x7fa60006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.961+0000 7fa614ae6700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 0x7fa60006d440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b5c0 tx=0x7fa60401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:15.965+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fa5fc054ca0 con 0x7fa610102630 2024-06-15T01:10:16.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.237+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa61004ea90 con 0x7fa610102630 2024-06-15T01:10:16.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.237+0000 7fa60dffb700 1 -- 172.21.15.137:0/1319283023 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fa5fc0582c0 con 0x7fa610102630 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 msgr2=0x7fa60006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 0x7fa60006d440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b5c0 tx=0x7fa60401a040 comp rx=0 tx=0).stop 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 msgr2=0x7fa6100ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00b3d0 tx=0x7fa5fc00b790 comp rx=0 tx=0).stop 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 shutdown_connections 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa60006af80 0x7fa60006d440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa610101d10 0x7fa6100ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 --2- 172.21.15.137:0/1319283023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa610102630 0x7fa6100ffeb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 >> 172.21.15.137:0/1319283023 conn(0x7fa6100fb540 msgr2=0x7fa610105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 shutdown_connections 2024-06-15T01:10:16.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:16.239+0000 7fa616d4a700 1 -- 172.21.15.137:0/1319283023 wait complete. 2024-06-15T01:10:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)" 2024-06-15T01:10:16.878 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:16 smithi137 ceph-mon[29354]: pgmap v77: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:16.878 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:16 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1319283023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:16.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-06-15T01:10:17.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:16 smithi142 ceph-mon[37379]: pgmap v77: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:17.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:16 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1319283023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:18.557 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:18 smithi137 ceph-mon[29354]: pgmap v78: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:18.557 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:18 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:18.746 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:18 smithi142 ceph-mon[37379]: pgmap v78: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:18.746 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:18 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:18.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 -- 172.21.15.137:0/225990723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660100dd0 msgr2=0x7f46601011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:18.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 --2- 172.21.15.137:0/225990723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660100dd0 0x7f46601011b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4650009910 tx=0x7f4650009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 -- 172.21.15.137:0/225990723 shutdown_connections 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 --2- 172.21.15.137:0/225990723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660101780 0x7f46601057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 --2- 172.21.15.137:0/225990723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660100dd0 0x7f46601011b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 -- 172.21.15.137:0/225990723 >> 172.21.15.137:0/225990723 conn(0x7f46600fc640 msgr2=0x7f46600fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 -- 172.21.15.137:0/225990723 shutdown_connections 2024-06-15T01:10:18.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.789+0000 7f4665cb5700 1 -- 172.21.15.137:0/225990723 wait complete. 2024-06-15T01:10:18.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 Processor -- start 2024-06-15T01:10:18.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 -- start start 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 0x7f466019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466019d0c0 con 0x7f4660101780 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4665cb5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660196570 con 0x7f4660100dd0 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4657fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4657fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51628/0 (socket says 172.21.15.137:51628) 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f4657fff700 1 -- 172.21.15.137:0/665501288 learned_addr learned my addr 172.21.15.137:0/665501288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.790+0000 7f465f7fe700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 0x7f466019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4657fff700 1 -- 172.21.15.137:0/665501288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 msgr2=0x7f466019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4657fff700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 0x7f466019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4657fff700 1 -- 172.21.15.137:0/665501288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500095f0 con 0x7f4660101780 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f465f7fe700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 0x7f466019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4657fff700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f464800d900 tx=0x7f464800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4648009900 con 0x7f4660101780 2024-06-15T01:10:18.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4648010460 con 0x7f4660101780 2024-06-15T01:10:18.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4660196850 con 0x7f4660101780 2024-06-15T01:10:18.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464800f5f0 con 0x7f4660101780 2024-06-15T01:10:18.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.791+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4660196da0 con 0x7f4660101780 2024-06-15T01:10:18.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.792+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660109170 con 0x7f4660101780 2024-06-15T01:10:18.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.792+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f46480105d0 con 0x7f4660101780 2024-06-15T01:10:18.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.793+0000 7f465d7fa700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 0x7f464c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:18.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.793+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4648089f40 con 0x7f4660101780 2024-06-15T01:10:18.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.793+0000 7f465f7fe700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 0x7f464c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:18.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.793+0000 7f465f7fe700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 0x7f464c06d440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f465000b5c0 tx=0x7f465001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:18.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:18.797+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f46480bd650 con 0x7f4660101780 2024-06-15T01:10:19.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.069+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4660197010 con 0x7f4660101780 2024-06-15T01:10:19.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.070+0000 7f465d7fa700 1 -- 172.21.15.137:0/665501288 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f46480202c0 con 0x7f4660101780 2024-06-15T01:10:19.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 msgr2=0x7f464c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:19.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 0x7f464c06d440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f465000b5c0 tx=0x7f465001a040 comp rx=0 tx=0).stop 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 msgr2=0x7f466019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f464800d900 tx=0x7f464800dcc0 comp rx=0 tx=0).stop 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 shutdown_connections 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464c06af80 0x7f464c06d440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4660100dd0 0x7f466019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 --2- 172.21.15.137:0/665501288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4660101780 0x7f466019ca30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 >> 172.21.15.137:0/665501288 conn(0x7f46600fc640 msgr2=0x7f46600fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 shutdown_connections 2024-06-15T01:10:19.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:19.072+0000 7f4665cb5700 1 -- 172.21.15.137:0/665501288 wait complete. 2024-06-15T01:10:19.076 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:10:19.693 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:19 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/665501288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:19.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-06-15T01:10:19.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:19 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/665501288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:19.924 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:20.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:20 smithi137 ceph-mon[29354]: pgmap v79: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:20.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:20 smithi142 ceph-mon[37379]: pgmap v79: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:21.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 -- 172.21.15.137:0/3021625949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 msgr2=0x7f43c00ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:21.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 --2- 172.21.15.137:0/3021625949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c00ffeb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009910 tx=0x7f43b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:21.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 -- 172.21.15.137:0/3021625949 shutdown_connections 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 --2- 172.21.15.137:0/3021625949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c00ffeb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 --2- 172.21.15.137:0/3021625949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 -- 172.21.15.137:0/3021625949 >> 172.21.15.137:0/3021625949 conn(0x7f43c00fb540 msgr2=0x7f43c00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 -- 172.21.15.137:0/3021625949 shutdown_connections 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 -- 172.21.15.137:0/3021625949 wait complete. 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.619+0000 7f43c81f0700 1 Processor -- start 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 -- start start 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0102080 con 0x7f43c00ffa30 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01021c0 con 0x7f43c0105a30 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51654/0 (socket says 172.21.15.137:51654) 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 -- 172.21.15.137:0/2653876183 learned_addr learned my addr 172.21.15.137:0/2653876183 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c578b700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 -- 172.21.15.137:0/2653876183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 msgr2=0x7f43c0101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 -- 172.21.15.137:0/2653876183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00095f0 con 0x7f43c00ffa30 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c578b700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:10:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c5f8c700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8d0 tx=0x7f43bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:21.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc009940 con 0x7f43c00ffa30 2024-06-15T01:10:21.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c01a2840 con 0x7f43c00ffa30 2024-06-15T01:10:21.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43bc010460 con 0x7f43c00ffa30 2024-06-15T01:10:21.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.620+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc00f5f0 con 0x7f43c00ffa30 2024-06-15T01:10:21.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.621+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c01a2c50 con 0x7f43c00ffa30 2024-06-15T01:10:21.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.622+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c004ea90 con 0x7f43c00ffa30 2024-06-15T01:10:21.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.622+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f43bc0105d0 con 0x7f43c00ffa30 2024-06-15T01:10:21.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.622+0000 7f43b6ffd700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 0x7f43ac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:21.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.623+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f43bc08a220 con 0x7f43c00ffa30 2024-06-15T01:10:21.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.623+0000 7f43c578b700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 0x7f43ac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:21.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.623+0000 7f43c578b700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 0x7f43ac06d440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b5c0 tx=0x7f43b0005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:21.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.627+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f43bc055210 con 0x7f43c00ffa30 2024-06-15T01:10:21.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:21.836+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}) v1 -- 0x7f43c00fbf50 con 0x7f43ac06af80 2024-06-15T01:10:22.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:22 smithi137 ceph-mon[29354]: pgmap v80: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:22.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:22 smithi137 ceph-mon[29354]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:23.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.032+0000 7f43b6ffd700 1 -- 172.21.15.137:0/2653876183 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5393 (secure 0 0 0) 0x7f43c00fbf50 con 0x7f43ac06af80 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 msgr2=0x7f43ac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 0x7f43ac06d440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b5c0 tx=0x7f43b0005960 comp rx=0 tx=0).stop 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 msgr2=0x7f43c0101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8d0 tx=0x7f43bc00dc90 comp rx=0 tx=0).stop 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 shutdown_connections 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43ac06af80 0x7f43ac06d440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:23.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c00ffa30 0x7f43c0101600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:23.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 --2- 172.21.15.137:0/2653876183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43c0105a30 0x7f43c0101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:23.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.034+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 >> 172.21.15.137:0/2653876183 conn(0x7f43c00fb540 msgr2=0x7f43c0103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:23.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.035+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 shutdown_connections 2024-06-15T01:10:23.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:23.035+0000 7f43c81f0700 1 -- 172.21.15.137:0/2653876183 wait complete. 2024-06-15T01:10:23.039 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:10:23.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:22 smithi142 ceph-mon[37379]: pgmap v80: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:23.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:22 smithi142 ceph-mon[37379]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:23.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch ps' 2024-06-15T01:10:23.863 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:24.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:24 smithi137 ceph-mon[29354]: pgmap v81: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:25.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:24 smithi142 ceph-mon[37379]: pgmap v81: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 -- 172.21.15.137:0/3316974428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1218105a40 msgr2=0x7f1218105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 --2- 172.21.15.137:0/3316974428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1218105a40 0x7f1218105e20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1208009910 tx=0x7f1208009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 -- 172.21.15.137:0/3316974428 shutdown_connections 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 --2- 172.21.15.137:0/3316974428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12180ffa40 0x7f12180ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 --2- 172.21.15.137:0/3316974428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1218105a40 0x7f1218105e20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 -- 172.21.15.137:0/3316974428 >> 172.21.15.137:0/3316974428 conn(0x7f12180fb570 msgr2=0x7f12180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 -- 172.21.15.137:0/3316974428 shutdown_connections 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.417+0000 7f121e38c700 1 -- 172.21.15.137:0/3316974428 wait complete. 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 Processor -- start 2024-06-15T01:10:25.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 -- start start 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1218105a40 0x7f1218195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218069d50 con 0x7f12180ffa40 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f121e38c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218195d60 con 0x7f1218105a40 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f1217fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f1217fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51666/0 (socket says 172.21.15.137:51666) 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.418+0000 7f1217fff700 1 -- 172.21.15.137:0/246552771 learned_addr learned my addr 172.21.15.137:0/246552771 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f12177fe700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1218105a40 0x7f1218195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f1217fff700 1 -- 172.21.15.137:0/246552771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1218105a40 msgr2=0x7f1218195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f1217fff700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1218105a40 0x7f1218195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f1217fff700 1 -- 172.21.15.137:0/246552771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080095f0 con 0x7f12180ffa40 2024-06-15T01:10:25.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f1217fff700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:25.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120801d070 con 0x7f12180ffa40 2024-06-15T01:10:25.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f120800f460 con 0x7f12180ffa40 2024-06-15T01:10:25.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1218195fe0 con 0x7f12180ffa40 2024-06-15T01:10:25.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1208021640 con 0x7f12180ffa40 2024-06-15T01:10:25.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.419+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12181a2b70 con 0x7f12180ffa40 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.420+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1218108040 con 0x7f12180ffa40 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.421+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f120800f9e0 con 0x7f12180ffa40 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.421+0000 7f12157fa700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 0x7f120006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.421+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f120808b370 con 0x7f12180ffa40 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.421+0000 7f12177fe700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 0x7f120006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:25.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.422+0000 7f12177fe700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 0x7f120006d440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12180696f0 tx=0x7f120c00a380 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:25.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.426+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f1208059df0 con 0x7f12180ffa40 2024-06-15T01:10:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.625+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f12180611d0 con 0x7f120006af80 2024-06-15T01:10:25.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.635+0000 7f12157fa700 1 -- 172.21.15.137:0/246552771 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f12180611d0 con 0x7f120006af80 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (59s) 38s ago 11m 15.0M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (11m) 38s ago 11m 7994k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (10m) 86s ago 10m 7574k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (11m) 38s ago 11m 6660k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (10m) 86s ago 10m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:10:25.629 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (44s) 38s ago 11m 41.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (5m) 38s ago 5m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (6m) 38s ago 6m 131M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (6m) 86s ago 5m 14.2M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (5m) 86s ago 5m 17.1M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (2m) 38s ago 13m 485M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (103s) 86s ago 10m 436M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (13m) 38s ago 13m 57.9M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (10m) 86s ago 10m 44.7M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (93s) 38s ago 11m 7864k - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (89s) 86s ago 10m 4038k - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (9m) 38s ago 9m 223M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:10:25.630 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (9m) 38s ago 9m 243M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:10:25.631 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (8m) 38s ago 8m 195M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:10:25.631 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (8m) 86s ago 8m 264M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:10:25.631 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (7m) 86s ago 7m 239M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:10:25.631 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (7m) 86s ago 7m 220M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:10:25.631 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (74s) 38s ago 11m 57.9M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.637+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 msgr2=0x7f120006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.637+0000 7f121e38c700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 0x7f120006d440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12180696f0 tx=0x7f120c00a380 comp rx=0 tx=0).stop 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.637+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 msgr2=0x7f12181952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.637+0000 7f121e38c700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120800bd70 comp rx=0 tx=0).stop 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 shutdown_connections 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f120006af80 0x7f120006d440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12180ffa40 0x7f12181952e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 --2- 172.21.15.137:0/246552771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1218105a40 0x7f1218195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 >> 172.21.15.137:0/246552771 conn(0x7f12180fb570 msgr2=0x7f12180fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 shutdown_connections 2024-06-15T01:10:25.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:25.638+0000 7f121e38c700 1 -- 172.21.15.137:0/246552771 wait complete. 2024-06-15T01:10:26.267 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-15T01:10:26.268 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:10:26.268 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-06-15T01:10:26.475 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:26.932 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:26 smithi137 ceph-mon[29354]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:26.933 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:26 smithi137 ceph-mon[29354]: pgmap v82: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:27.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:26 smithi142 ceph-mon[37379]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:27.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:26 smithi142 ceph-mon[37379]: pgmap v82: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:28.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.201+0000 7f7baac56700 1 -- 172.21.15.137:0/1962858598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 msgr2=0x7f7ba4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:28.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.201+0000 7f7baac56700 1 --2- 172.21.15.137:0/1962858598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba4100050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009910 tx=0x7f7b90009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:28.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- 172.21.15.137:0/1962858598 shutdown_connections 2024-06-15T01:10:28.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 --2- 172.21.15.137:0/1962858598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba4100050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 --2- 172.21.15.137:0/1962858598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 0x7f7ba4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- 172.21.15.137:0/1962858598 >> 172.21.15.137:0/1962858598 conn(0x7f7ba40755a0 msgr2=0x7f7ba40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- 172.21.15.137:0/1962858598 shutdown_connections 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- 172.21.15.137:0/1962858598 wait complete. 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 Processor -- start 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- start start 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 0x7f7ba419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419d0e0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7baac56700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4196590 con 0x7f7ba4105a40 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.202+0000 7f7ba89f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52138/0 (socket says 172.21.15.137:52138) 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 -- 172.21.15.137:0/2214058585 learned_addr learned my addr 172.21.15.137:0/2214058585 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:28.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba3fff700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 0x7f7ba419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 -- 172.21.15.137:0/2214058585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 msgr2=0x7f7ba419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 0x7f7ba419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 -- 172.21.15.137:0/2214058585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900095f0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba89f2700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c8a0 tx=0x7f7b9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9800cea0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba4196870 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b98004830 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b98005500 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.203+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba4196dc0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.204+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba4108fe0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.205+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f7b980049a0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.205+0000 7f7ba1ffb700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 0x7f7b9406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.206+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7b98089ef0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.206+0000 7f7ba3fff700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 0x7f7b9406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:28.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.206+0000 7f7ba3fff700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 0x7f7b9406d440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4104ef0 tx=0x7f7b9001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:28.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.211+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f7b98058970 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.401+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f7ba404ea90 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.406+0000 7f7ba1ffb700 1 -- 172.21.15.137:0/2214058585 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v45)=0 v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7b9801d0c0 con 0x7f7ba40ffbd0 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 msgr2=0x7f7b9406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 0x7f7b9406d440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4104ef0 tx=0x7f7b9001a040 comp rx=0 tx=0).stop 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 msgr2=0x7f7ba419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c8a0 tx=0x7f7b9800cc60 comp rx=0 tx=0).stop 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 shutdown_connections 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b9406af80 0x7f7b9406d440 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba40ffbd0 0x7f7ba419c510 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 --2- 172.21.15.137:0/2214058585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba4105a40 0x7f7ba419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 >> 172.21.15.137:0/2214058585 conn(0x7f7ba40755a0 msgr2=0x7f7ba4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 shutdown_connections 2024-06-15T01:10:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:28.408+0000 7f7baac56700 1 -- 172.21.15.137:0/2214058585 wait complete. 2024-06-15T01:10:28.781 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:28 smithi137 ceph-mon[29354]: pgmap v83: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:28.781 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:28 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/2214058585' entity='client.admin' 2024-06-15T01:10:28.781 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:28 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:28.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:28 smithi142 ceph-mon[37379]: pgmap v83: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:28.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:28 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/2214058585' entity='client.admin' 2024-06-15T01:10:28.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:28 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:29.049 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-15T01:10:29.049 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:10:29.049 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-15T01:10:29.254 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:29.648 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:29 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:29.648 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:29 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:29.648 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:29 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:29.648 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:29 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:29 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:29 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:29 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:29 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:30.704 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:30 smithi137 ceph-mon[29354]: pgmap v84: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:30.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:30 smithi142 ceph-mon[37379]: pgmap v84: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:30.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 -- 172.21.15.137:0/1265634368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0540ff2b0 msgr2=0x7fb0540ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:30.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1265634368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0540ff2b0 0x7fb0540ff710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb04800aa20 tx=0x7fb04800ad30 comp rx=0 tx=0).stop 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 -- 172.21.15.137:0/1265634368 shutdown_connections 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1265634368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0540ff2b0 0x7fb0540ff710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1265634368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0541052c0 0x7fb0541056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 -- 172.21.15.137:0/1265634368 >> 172.21.15.137:0/1265634368 conn(0x7fb054074e00 msgr2=0x7fb054075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 -- 172.21.15.137:0/1265634368 shutdown_connections 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.953+0000 7fb05c22a700 1 -- 172.21.15.137:0/1265634368 wait complete. 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 Processor -- start 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 -- start start 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0540ff2b0 0x7fb054100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541019b0 con 0x7fb0541052c0 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb05c22a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054101af0 con 0x7fb0540ff2b0 2024-06-15T01:10:30.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb0597c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb0597c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52156/0 (socket says 172.21.15.137:52156) 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb0597c5700 1 -- 172.21.15.137:0/1841883911 learned_addr learned my addr 172.21.15.137:0/1841883911 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.954+0000 7fb059fc6700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0540ff2b0 0x7fb054100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb0597c5700 1 -- 172.21.15.137:0/1841883911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0540ff2b0 msgr2=0x7fb054100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb0597c5700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0540ff2b0 0x7fb054100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb0597c5700 1 -- 172.21.15.137:0/1841883911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04800a700 con 0x7fb0541052c0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb0597c5700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb04800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04800ccc0 con 0x7fb0541052c0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb048005070 con 0x7fb0541052c0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb054101d70 con 0x7fb0541052c0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb048016820 con 0x7fb0541052c0 2024-06-15T01:10:30.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.955+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0541a2b60 con 0x7fb0541052c0 2024-06-15T01:10:30.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.956+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0541097d0 con 0x7fb0541052c0 2024-06-15T01:10:30.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.956+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7fb0480051e0 con 0x7fb0541052c0 2024-06-15T01:10:30.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.957+0000 7fb046ffd700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 0x7fb04006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:30.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.957+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb048012070 con 0x7fb0541052c0 2024-06-15T01:10:30.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.957+0000 7fb059fc6700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 0x7fb04006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:30.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.957+0000 7fb059fc6700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 0x7fb04006d440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb050005950 tx=0x7fb05000a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:30.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:30.961+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fb048056f20 con 0x7fb0541052c0 2024-06-15T01:10:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.156+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fb05404ea90 con 0x7fb0541052c0 2024-06-15T01:10:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.156+0000 7fb046ffd700 1 -- 172.21.15.137:0/1841883911 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v45)=0 v45) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb04805a540 con 0x7fb0541052c0 2024-06-15T01:10:31.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 msgr2=0x7fb04006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 0x7fb04006d440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb050005950 tx=0x7fb05000a300 comp rx=0 tx=0).stop 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 msgr2=0x7fb054101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb04800fd90 comp rx=0 tx=0).stop 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 shutdown_connections 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb04006af80 0x7fb04006d440 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0540ff2b0 0x7fb054100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 --2- 172.21.15.137:0/1841883911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0541052c0 0x7fb054101470 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 >> 172.21.15.137:0/1841883911 conn(0x7fb054074e00 msgr2=0x7fb0541037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 shutdown_connections 2024-06-15T01:10:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:31.158+0000 7fb05c22a700 1 -- 172.21.15.137:0/1841883911 wait complete. 2024-06-15T01:10:31.802 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-15T01:10:32.008 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:33.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:32 smithi142 ceph-mon[37379]: pgmap v85: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:33.085 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:32 smithi137 ceph-mon[29354]: pgmap v85: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:33.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 -- 172.21.15.137:0/844468830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 msgr2=0x7f3a341057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:33.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/844468830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a341057c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009910 tx=0x7f3a24009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 -- 172.21.15.137:0/844468830 shutdown_connections 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/844468830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a341057c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/844468830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 0x7f3a341011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 -- 172.21.15.137:0/844468830 >> 172.21.15.137:0/844468830 conn(0x7f3a340fc650 msgr2=0x7f3a340fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 -- 172.21.15.137:0/844468830 shutdown_connections 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.635+0000 7f3a3b40b700 1 -- 172.21.15.137:0/844468830 wait complete. 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 Processor -- start 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 -- start start 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 0x7f3a341966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a341972a0 con 0x7f3a34101770 2024-06-15T01:10:33.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a3b40b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a34199fd0 con 0x7f3a34100dc0 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a389a6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a389a6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52168/0 (socket says 172.21.15.137:52168) 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a391a7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 0x7f3a341966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.636+0000 7f3a389a6700 1 -- 172.21.15.137:0/1808569523 learned_addr learned my addr 172.21.15.137:0/1808569523 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a389a6700 1 -- 172.21.15.137:0/1808569523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 msgr2=0x7f3a341966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a389a6700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 0x7f3a341966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a389a6700 1 -- 172.21.15.137:0/1808569523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240095f0 con 0x7f3a34101770 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a389a6700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a240092e0 tx=0x7f3a2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a2401d070 con 0x7f3a34101770 2024-06-15T01:10:33.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3419a250 con 0x7f3a34101770 2024-06-15T01:10:33.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a24005dc0 con 0x7f3a34101770 2024-06-15T01:10:33.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a2400fea0 con 0x7f3a34101770 2024-06-15T01:10:33.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.637+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3419a740 con 0x7f3a34101770 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.638+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a341091c0 con 0x7f3a34101770 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.639+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f3a24003a60 con 0x7f3a34101770 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.639+0000 7f3a2a7fc700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 0x7f3a2006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.639+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3a2408b310 con 0x7f3a34101770 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.639+0000 7f3a391a7700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 0x7f3a2006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:33.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.639+0000 7f3a391a7700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 0x7f3a2006d440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a30005950 tx=0x7f3a3000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:33.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.644+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f3a24056300 con 0x7f3a34101770 2024-06-15T01:10:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.839+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f3a3404ea90 con 0x7f3a34101770 2024-06-15T01:10:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.840+0000 7f3a2a7fc700 1 -- 172.21.15.137:0/1808569523 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v45)=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3a24059920 con 0x7f3a34101770 2024-06-15T01:10:33.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 msgr2=0x7f3a2006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:33.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 0x7f3a2006d440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a30005950 tx=0x7f3a3000b410 comp rx=0 tx=0).stop 2024-06-15T01:10:33.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 msgr2=0x7f3a34196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a240092e0 tx=0x7f3a2400bd70 comp rx=0 tx=0).stop 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 shutdown_connections 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a2006af80 0x7f3a2006d440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a34100dc0 0x7f3a341966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 --2- 172.21.15.137:0/1808569523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a34101770 0x7f3a34196c10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 >> 172.21.15.137:0/1808569523 conn(0x7f3a340fc650 msgr2=0x7f3a340fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 shutdown_connections 2024-06-15T01:10:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:33.842+0000 7f3a3b40b700 1 -- 172.21.15.137:0/1808569523 wait complete. 2024-06-15T01:10:34.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:33 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:34.174 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:33 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:34.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-15T01:10:34.697 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:35.061 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:34 smithi137 ceph-mon[29354]: pgmap v86: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:35.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:34 smithi142 ceph-mon[37379]: pgmap v86: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 -- 172.21.15.137:0/1273387632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680730b0 msgr2=0x7fe668073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 --2- 172.21.15.137:0/1273387632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680730b0 0x7fe668073490 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe664009910 tx=0x7fe664009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 -- 172.21.15.137:0/1273387632 shutdown_connections 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 --2- 172.21.15.137:0/1273387632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680739d0 0x7fe66810cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 --2- 172.21.15.137:0/1273387632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680730b0 0x7fe668073490 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 -- 172.21.15.137:0/1273387632 >> 172.21.15.137:0/1273387632 conn(0x7fe6680fc650 msgr2=0x7fe6680fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 -- 172.21.15.137:0/1273387632 shutdown_connections 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.316+0000 7fe66f63f700 1 -- 172.21.15.137:0/1273387632 wait complete. 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 Processor -- start 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 -- start start 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680730b0 0x7fe6681989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe668199570 con 0x7fe6680739d0 2024-06-15T01:10:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66f63f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66819d380 con 0x7fe6680730b0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66cbda700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66cbda700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58790/0 (socket says 172.21.15.137:58790) 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66cbda700 1 -- 172.21.15.137:0/335184629 learned_addr learned my addr 172.21.15.137:0/335184629 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.317+0000 7fe66d3db700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680730b0 0x7fe6681989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66cbda700 1 -- 172.21.15.137:0/335184629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680730b0 msgr2=0x7fe6681989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66cbda700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680730b0 0x7fe6681989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66cbda700 1 -- 172.21.15.137:0/335184629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6640095f0 con 0x7fe6680739d0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66cbda700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b4b0 tx=0x7fe65800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe658011800 con 0x7fe6680739d0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66819d660 con 0x7fe6680739d0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe658011e00 con 0x7fe6680739d0 2024-06-15T01:10:36.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe658010e60 con 0x7fe6680739d0 2024-06-15T01:10:36.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.318+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66819dbb0 con 0x7fe6680739d0 2024-06-15T01:10:36.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.319+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66810a5e0 con 0x7fe6680739d0 2024-06-15T01:10:36.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.319+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7fe658011960 con 0x7fe6680739d0 2024-06-15T01:10:36.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.320+0000 7fe65e7fc700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 0x7fe65406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:36.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.320+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe658089d00 con 0x7fe6680739d0 2024-06-15T01:10:36.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.320+0000 7fe66d3db700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 0x7fe65406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:36.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.320+0000 7fe66d3db700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 0x7fe65406d440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe664006010 tx=0x7fe66401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:36.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.324+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fe65808d030 con 0x7fe6680739d0 2024-06-15T01:10:36.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.514+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fe66804ea90 con 0x7fe6680739d0 2024-06-15T01:10:36.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.514+0000 7fe65e7fc700 1 -- 172.21.15.137:0/335184629 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v45)=0 v45) v1 ==== 135+0+0 (secure 0 0 0) 0x7fe658058780 con 0x7fe6680739d0 2024-06-15T01:10:36.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 msgr2=0x7fe65406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:36.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 0x7fe65406d440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe664006010 tx=0x7fe66401a040 comp rx=0 tx=0).stop 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 msgr2=0x7fe668198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b4b0 tx=0x7fe65800b7c0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 shutdown_connections 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe65406af80 0x7fe65406d440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6680730b0 0x7fe6681989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 --2- 172.21.15.137:0/335184629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6680739d0 0x7fe668198ee0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 >> 172.21.15.137:0/335184629 conn(0x7fe6680fc650 msgr2=0x7fe668107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 shutdown_connections 2024-06-15T01:10:36.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:36.516+0000 7fe66f63f700 1 -- 172.21.15.137:0/335184629 wait complete. 2024-06-15T01:10:36.890 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:36 smithi137 ceph-mon[29354]: pgmap v87: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:37.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:36 smithi142 ceph-mon[37379]: pgmap v87: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:37.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-06-15T01:10:37.443 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:38.644 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:38 smithi137 ceph-mon[29354]: pgmap v88: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:38.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:38 smithi142 ceph-mon[37379]: pgmap v88: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.988+0000 7fc770edc700 1 -- 172.21.15.137:0/3191519765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 msgr2=0x7fc76c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.988+0000 7fc770edc700 1 --2- 172.21.15.137:0/3191519765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c1011b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009910 tx=0x7fc75c009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 -- 172.21.15.137:0/3191519765 shutdown_connections 2024-06-15T01:10:38.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 --2- 172.21.15.137:0/3191519765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 0x7fc76c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:38.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 --2- 172.21.15.137:0/3191519765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c1011b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:38.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 -- 172.21.15.137:0/3191519765 >> 172.21.15.137:0/3191519765 conn(0x7fc76c0fc640 msgr2=0x7fc76c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:38.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 -- 172.21.15.137:0/3191519765 shutdown_connections 2024-06-15T01:10:38.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.989+0000 7fc770edc700 1 -- 172.21.15.137:0/3191519765 wait complete. 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 Processor -- start 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 -- start start 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 0x7fc76c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c073b10 con 0x7fc76c100dd0 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc770edc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c073c50 con 0x7fc76c101780 2024-06-15T01:10:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58800/0 (socket says 172.21.15.137:58800) 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 -- 172.21.15.137:0/1322881834 learned_addr learned my addr 172.21.15.137:0/1322881834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc769d9b700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 0x7fc76c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 -- 172.21.15.137:0/1322881834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 msgr2=0x7fc76c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 0x7fc76c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 -- 172.21.15.137:0/1322881834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0095f0 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.990+0000 7fc76a59c700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006010 tx=0x7fc75c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.991+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc75c01d070 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.991+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c073ed0 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.991+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc75c005710 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.991+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc75c00fe20 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.991+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c1a2c20 con 0x7fc76c100dd0 2024-06-15T01:10:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.992+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c109170 con 0x7fc76c100dd0 2024-06-15T01:10:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.992+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7fc75c003a60 con 0x7fc76c100dd0 2024-06-15T01:10:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.993+0000 7fc75b7fe700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 0x7fc75406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.993+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc75c08b350 con 0x7fc76c100dd0 2024-06-15T01:10:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.993+0000 7fc769d9b700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 0x7fc75406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.993+0000 7fc769d9b700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 0x7fc75406d440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc76c074880 tx=0x7fc76000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:38.997+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fc75c056460 con 0x7fc76c100dd0 2024-06-15T01:10:39.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.204+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}) v1 -- 0x7fc76c02cd80 con 0x7fc75406af80 2024-06-15T01:10:39.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.209+0000 7fc75b7fe700 1 -- 172.21.15.137:0/1322881834 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fc76c02cd80 con 0x7fc75406af80 2024-06-15T01:10:39.202 INFO:teuthology.orchestra.run.smithi137.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 msgr2=0x7fc75406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 0x7fc75406d440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc76c074880 tx=0x7fc76000b410 comp rx=0 tx=0).stop 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 msgr2=0x7fc76c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006010 tx=0x7fc75c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 shutdown_connections 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc75406af80 0x7fc75406d440 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc76c100dd0 0x7fc76c073090 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 --2- 172.21.15.137:0/1322881834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc76c101780 0x7fc76c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.211+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 >> 172.21.15.137:0/1322881834 conn(0x7fc76c0fc640 msgr2=0x7fc76c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.212+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 shutdown_connections 2024-06-15T01:10:39.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:39.212+0000 7fc770edc700 1 -- 172.21.15.137:0/1322881834 wait complete. 2024-06-15T01:10:39.866 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-15T01:10:39.867 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:10:39.867 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-06-15T01:10:40.374 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:40.374 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:10:40.374 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:40.374 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:40.375 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:40.375 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:40.375 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:40.375 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:40 smithi137 ceph-mon[29354]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:40.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:40.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:10:40.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:40.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:40.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:40 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:41.115 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:10:41.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:10:41.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: pgmap v89: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:41.353 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:41.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:41.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.354 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:41 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: pgmap v89: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:41.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:41.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:41 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-15T01:10:42.451 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:42 smithi137 ceph-mon[29354]: Upgrade: Target is version 18.2.2-1443-g6f4b0956 (reef) 2024-06-15T01:10:42.451 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:42 smithi137 ceph-mon[29354]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0'] 2024-06-15T01:10:42.452 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:42 smithi137 ceph-mon[29354]: Upgrade: Setting container_image for all mgr 2024-06-15T01:10:42.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:42 smithi142 ceph-mon[37379]: Upgrade: Target is version 18.2.2-1443-g6f4b0956 (reef) 2024-06-15T01:10:42.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:42 smithi142 ceph-mon[37379]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0'] 2024-06-15T01:10:42.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:42 smithi142 ceph-mon[37379]: Upgrade: Setting container_image for all mgr 2024-06-15T01:10:43.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:43 smithi137 ceph-mon[29354]: pgmap v90: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:43.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:43 smithi142 ceph-mon[37379]: pgmap v90: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 -- 172.21.15.137:0/881829838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 msgr2=0x7f0118105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 --2- 172.21.15.137:0/881829838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118105e20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0114009910 tx=0x7f0114009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 -- 172.21.15.137:0/881829838 shutdown_connections 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 --2- 172.21.15.137:0/881829838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 0x7f01180ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 --2- 172.21.15.137:0/881829838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118105e20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 -- 172.21.15.137:0/881829838 >> 172.21.15.137:0/881829838 conn(0x7f01180fb550 msgr2=0x7f01180fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 -- 172.21.15.137:0/881829838 shutdown_connections 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.756+0000 7f011f6dd700 1 -- 172.21.15.137:0/881829838 wait complete. 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 Processor -- start 2024-06-15T01:10:43.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 -- start start 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 0x7f0118195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01181961b0 con 0x7f0118105a40 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011f6dd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118199f40 con 0x7f01180ffa40 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011cc78700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011cc78700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58826/0 (socket says 172.21.15.137:58826) 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011cc78700 1 -- 172.21.15.137:0/1855373450 learned_addr learned my addr 172.21.15.137:0/1855373450 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011d479700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 0x7f0118195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.757+0000 7f011cc78700 1 -- 172.21.15.137:0/1855373450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 msgr2=0x7f0118195590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f011cc78700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 0x7f0118195590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f011cc78700 1 -- 172.21.15.137:0/1855373450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140095f0 con 0x7f0118105a40 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f011cc78700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f010800c8a0 tx=0x7f010800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010800fa50 con 0x7f0118105a40 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f010800fbb0 con 0x7f0118105a40 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011819a1c0 con 0x7f0118105a40 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0108007610 con 0x7f0118105a40 2024-06-15T01:10:43.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.758+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011819a710 con 0x7f0118105a40 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.759+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011804ea90 con 0x7f0118105a40 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.760+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f01080179e0 con 0x7f0118105a40 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.760+0000 7f010e7fc700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 0x7f010406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.760+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f010808bc20 con 0x7f0118105a40 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.760+0000 7f011d479700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 0x7f010406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:43.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.761+0000 7f011d479700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 0x7f010406d440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0114005f50 tx=0x7f011401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:43.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.765+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f010805a7d0 con 0x7f0118105a40 2024-06-15T01:10:43.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.984+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f011819ad30 con 0x7f010406af80 2024-06-15T01:10:43.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.985+0000 7f010e7fc700 1 -- 172.21.15.137:0/1855373450 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f011819ad30 con 0x7f010406af80 2024-06-15T01:10:43.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 msgr2=0x7f010406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:43.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 0x7f010406d440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0114005f50 tx=0x7f011401a040 comp rx=0 tx=0).stop 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 msgr2=0x7f0118195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f010800c8a0 tx=0x7f010800cbb0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 shutdown_connections 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f010406af80 0x7f010406d440 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01180ffa40 0x7f0118195590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 --2- 172.21.15.137:0/1855373450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0118105a40 0x7f0118195ad0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.987+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 >> 172.21.15.137:0/1855373450 conn(0x7f01180fb550 msgr2=0x7f01180fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.988+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 shutdown_connections 2024-06-15T01:10:43.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:43.988+0000 7f011f6dd700 1 -- 172.21.15.137:0/1855373450 wait complete. 2024-06-15T01:10:43.992 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 -- 172.21.15.137:0/3894458817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca40105a40 msgr2=0x7fca40105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 --2- 172.21.15.137:0/3894458817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca40105a40 0x7fca40105e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca34009910 tx=0x7fca34009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 -- 172.21.15.137:0/3894458817 shutdown_connections 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 --2- 172.21.15.137:0/3894458817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca400ffbd0 0x7fca40100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 --2- 172.21.15.137:0/3894458817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca40105a40 0x7fca40105e20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 -- 172.21.15.137:0/3894458817 >> 172.21.15.137:0/3894458817 conn(0x7fca400755a0 msgr2=0x7fca400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 -- 172.21.15.137:0/3894458817 shutdown_connections 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.097+0000 7fca46be1700 1 -- 172.21.15.137:0/3894458817 wait complete. 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 Processor -- start 2024-06-15T01:10:44.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 -- start start 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40105a40 0x7fca40197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40198100 con 0x7fca400ffbd0 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca46be1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40198270 con 0x7fca40105a40 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca4497d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca4497d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58840/0 (socket says 172.21.15.137:58840) 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca4497d700 1 -- 172.21.15.137:0/4139826591 learned_addr learned my addr 172.21.15.137:0/4139826591 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.098+0000 7fca3ffff700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40105a40 0x7fca40197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca4497d700 1 -- 172.21.15.137:0/4139826591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40105a40 msgr2=0x7fca40197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca4497d700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40105a40 0x7fca40197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca4497d700 1 -- 172.21.15.137:0/4139826591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340095f0 con 0x7fca400ffbd0 2024-06-15T01:10:44.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca4497d700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca34006010 tx=0x7fca3400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3401d070 con 0x7fca400ffbd0 2024-06-15T01:10:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca3400f460 con 0x7fca400ffbd0 2024-06-15T01:10:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca34021620 con 0x7fca400ffbd0 2024-06-15T01:10:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca401a0610 con 0x7fca400ffbd0 2024-06-15T01:10:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.099+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca401a0a80 con 0x7fca400ffbd0 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.100+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca40108040 con 0x7fca400ffbd0 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.101+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7fca3400f5d0 con 0x7fca400ffbd0 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.101+0000 7fca3dffb700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 0x7fca3006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.101+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fca3408b390 con 0x7fca400ffbd0 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.101+0000 7fca3ffff700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 0x7fca3006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.102+0000 7fca3ffff700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 0x7fca3006d440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca40068ce0 tx=0x7fca2c009190 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.106+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7fca34056380 con 0x7fca400ffbd0 2024-06-15T01:10:44.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: Upgrade: Updating mon.smithi137 2024-06-15T01:10:44.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:44.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:10:44.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:10:44.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:44.267 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: Deploying daemon mon.smithi137 on smithi137 2024-06-15T01:10:44.267 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: pgmap v91: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:44.267 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:44 smithi137 ceph-mon[29354]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:44.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.329+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca400611d0 con 0x7fca3006af80 2024-06-15T01:10:44.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.331+0000 7fca3dffb700 1 -- 172.21.15.137:0/4139826591 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fca400611d0 con 0x7fca3006af80 2024-06-15T01:10:44.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 msgr2=0x7fca3006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 0x7fca3006d440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca40068ce0 tx=0x7fca2c009190 comp rx=0 tx=0).stop 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 msgr2=0x7fca401975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca34006010 tx=0x7fca3400bd70 comp rx=0 tx=0).stop 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 shutdown_connections 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fca3006af80 0x7fca3006d440 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca400ffbd0 0x7fca401975a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 --2- 172.21.15.137:0/4139826591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40105a40 0x7fca40197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 >> 172.21.15.137:0/4139826591 conn(0x7fca400755a0 msgr2=0x7fca400fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 shutdown_connections 2024-06-15T01:10:44.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.333+0000 7fca46be1700 1 -- 172.21.15.137:0/4139826591 wait complete. 2024-06-15T01:10:44.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 -- 172.21.15.137:0/1220097548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1014105b50 msgr2=0x7f1014105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 --2- 172.21.15.137:0/1220097548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1014105b50 0x7f1014105f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1004009910 tx=0x7f1004009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 -- 172.21.15.137:0/1220097548 shutdown_connections 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 --2- 172.21.15.137:0/1220097548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140fef70 0x7f10140ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 --2- 172.21.15.137:0/1220097548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1014105b50 0x7f1014105f30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 -- 172.21.15.137:0/1220097548 >> 172.21.15.137:0/1220097548 conn(0x7f1014075e30 msgr2=0x7f10140fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 -- 172.21.15.137:0/1220097548 shutdown_connections 2024-06-15T01:10:44.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.443+0000 7f101b52c700 1 -- 172.21.15.137:0/1220097548 wait complete. 2024-06-15T01:10:44.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 Processor -- start 2024-06-15T01:10:44.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 -- start start 2024-06-15T01:10:44.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1014105b50 0x7f101419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419d0e0 con 0x7f10140fef70 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f101b52c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1014196590 con 0x7f1014105b50 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f10192c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f10192c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58850/0 (socket says 172.21.15.137:58850) 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.444+0000 7f10192c8700 1 -- 172.21.15.137:0/2909194763 learned_addr learned my addr 172.21.15.137:0/2909194763 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f1018ac7700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1014105b50 0x7f101419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f10192c8700 1 -- 172.21.15.137:0/2909194763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1014105b50 msgr2=0x7f101419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f10192c8700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1014105b50 0x7f101419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f10192c8700 1 -- 172.21.15.137:0/2909194763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040095f0 con 0x7f10140fef70 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f10192c8700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f100400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100401d070 con 0x7f10140fef70 2024-06-15T01:10:44.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1014196810 con 0x7f10140fef70 2024-06-15T01:10:44.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f100400f460 con 0x7f10140fef70 2024-06-15T01:10:44.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1004021620 con 0x7f10140fef70 2024-06-15T01:10:44.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.445+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014196d00 con 0x7f10140fef70 2024-06-15T01:10:44.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.446+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014109120 con 0x7f10140fef70 2024-06-15T01:10:44.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.447+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f100400f5d0 con 0x7f10140fef70 2024-06-15T01:10:44.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.447+0000 7f100a7fc700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 0x7f100006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.447+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f100408b380 con 0x7f10140fef70 2024-06-15T01:10:44.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.447+0000 7f1018ac7700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 0x7f100006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.448+0000 7f1018ac7700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 0x7f100006d440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1014103710 tx=0x7f1010009190 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.452+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f1004059f20 con 0x7f10140fef70 2024-06-15T01:10:44.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: Upgrade: Updating mon.smithi137 2024-06-15T01:10:44.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:44.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:10:44.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:10:44.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:44.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: Deploying daemon mon.smithi137 on smithi137 2024-06-15T01:10:44.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: pgmap v91: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:44.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:44 smithi142 ceph-mon[37379]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:44.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.668+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1014197030 con 0x7f100006af80 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.679+0000 7f100a7fc700 1 -- 172.21.15.137:0/2909194763 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1014197030 con 0x7f100006af80 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (78s) 57s ago 11m 15.0M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (12m) 57s ago 12m 7994k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (11m) 105s ago 11m 7574k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (12m) 57s ago 12m 6660k - 18.2.1 5be31c24972a 589dd2d3c3ae 2024-06-15T01:10:44.672 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (10m) 105s ago 10m 6660k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (63s) 57s ago 11m 41.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (6m) 57s ago 6m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (6m) 57s ago 6m 131M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (6m) 105s ago 6m 14.2M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (6m) 105s ago 6m 17.1M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (3m) 57s ago 13m 485M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (2m) 105s ago 10m 436M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (13m) 57s ago 13m 57.9M 2048M 18.2.1 5be31c24972a 2462064093be 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (10m) 105s ago 10m 44.7M 2048M 18.2.1 5be31c24972a 211281b609be 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (112s) 57s ago 12m 7864k - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (108s) 105s ago 10m 4038k - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (10m) 57s ago 10m 223M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (9m) 57s ago 9m 243M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:10:44.673 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (9m) 57s ago 9m 195M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:10:44.674 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (8m) 105s ago 8m 264M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:10:44.674 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (8m) 105s ago 8m 239M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:10:44.674 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (7m) 105s ago 7m 220M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:10:44.674 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (93s) 57s ago 11m 57.9M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:10:44.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 msgr2=0x7f100006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 0x7f100006d440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1014103710 tx=0x7f1010009190 comp rx=0 tx=0).stop 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 msgr2=0x7f101419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f100400bd70 comp rx=0 tx=0).stop 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 shutdown_connections 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f100006af80 0x7f100006d440 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10140fef70 0x7f101419c510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 --2- 172.21.15.137:0/2909194763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1014105b50 0x7f101419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 >> 172.21.15.137:0/2909194763 conn(0x7f1014075e30 msgr2=0x7f1014104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:44.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 shutdown_connections 2024-06-15T01:10:44.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.681+0000 7f101b52c700 1 -- 172.21.15.137:0/2909194763 wait complete. 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 -- 172.21.15.137:0/3186731700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 msgr2=0x7f4d14100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 --2- 172.21.15.137:0/3186731700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d14100050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 -- 172.21.15.137:0/3186731700 shutdown_connections 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 --2- 172.21.15.137:0/3186731700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d14100050 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 --2- 172.21.15.137:0/3186731700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 0x7f4d14105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 -- 172.21.15.137:0/3186731700 >> 172.21.15.137:0/3186731700 conn(0x7f4d140755a0 msgr2=0x7f4d140759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 -- 172.21.15.137:0/3186731700 shutdown_connections 2024-06-15T01:10:44.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.787+0000 7f4d1a483700 1 -- 172.21.15.137:0/3186731700 wait complete. 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.788+0000 7f4d1a483700 1 Processor -- start 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.788+0000 7f4d1a483700 1 -- start start 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.788+0000 7f4d1a483700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.788+0000 7f4d1a483700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 0x7f4d1419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d1a483700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1419d0d0 con 0x7f4d140ffbd0 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d1a483700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14196580 con 0x7f4d14105a40 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58856/0 (socket says 172.21.15.137:58856) 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 -- 172.21.15.137:0/1695607511 learned_addr learned my addr 172.21.15.137:0/1695607511 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d137fe700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 0x7f4d1419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 -- 172.21.15.137:0/1695607511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 msgr2=0x7f4d1419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 0x7f4d1419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:44.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 -- 172.21.15.137:0/1695607511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d080095f0 con 0x7f4d140ffbd0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d13fff700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b4b0 tx=0x7f4d0400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d04011800 con 0x7f4d140ffbd0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d14196860 con 0x7f4d140ffbd0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d04011e00 con 0x7f4d140ffbd0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d04010e60 con 0x7f4d140ffbd0 2024-06-15T01:10:44.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.789+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d14196db0 con 0x7f4d140ffbd0 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.790+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d14109070 con 0x7f4d140ffbd0 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.791+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f4d04011960 con 0x7f4d140ffbd0 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.792+0000 7f4d117fa700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.792+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4d04089c70 con 0x7f4d140ffbd0 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.792+0000 7f4d137fe700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:44.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.792+0000 7f4d137fe700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d14104e40 tx=0x7f4d0801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:44.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:44.796+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f4d04058770 con 0x7f4d140ffbd0 2024-06-15T01:10:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.082+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d1404ea90 con 0x7f4d140ffbd0 2024-06-15T01:10:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.082+0000 7f4d117fa700 1 -- 172.21.15.137:0/1695607511 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f4d04058300 con 0x7f4d140ffbd0 2024-06-15T01:10:45.075 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:10:45.076 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-06-15T01:10:45.077 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:10:45.077 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:10:45.077 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 msgr2=0x7f4cfc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d14104e40 tx=0x7f4d0801a040 comp rx=0 tx=0).stop 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 msgr2=0x7f4d1419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b4b0 tx=0x7f4d0400b7c0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 shutdown_connections 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ffbd0 0x7f4d1419c500 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 --2- 172.21.15.137:0/1695607511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d14105a40 0x7f4d1419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 >> 172.21.15.137:0/1695607511 conn(0x7f4d140755a0 msgr2=0x7f4d14103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 shutdown_connections 2024-06-15T01:10:45.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.085+0000 7f4d1a483700 1 -- 172.21.15.137:0/1695607511 wait complete. 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 -- 172.21.15.137:0/3900708834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 msgr2=0x7f1204068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 --2- 172.21.15.137:0/3900708834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204068a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009910 tx=0x7f11f4009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 -- 172.21.15.137:0/3900708834 shutdown_connections 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 --2- 172.21.15.137:0/3900708834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 0x7f1204105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 --2- 172.21.15.137:0/3900708834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204068a50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 -- 172.21.15.137:0/3900708834 >> 172.21.15.137:0/3900708834 conn(0x7f12040755a0 msgr2=0x7f12040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 -- 172.21.15.137:0/3900708834 shutdown_connections 2024-06-15T01:10:45.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.200+0000 7f1209169700 1 -- 172.21.15.137:0/3900708834 wait complete. 2024-06-15T01:10:45.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 Processor -- start 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 -- start start 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 0x7f1204196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204197290 con 0x7f1204068670 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.201+0000 7f1209169700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120419b020 con 0x7f1204069020 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58870/0 (socket says 172.21.15.137:58870) 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 -- 172.21.15.137:0/4112748975 learned_addr learned my addr 172.21.15.137:0/4112748975 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f120259c700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 0x7f1204196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 -- 172.21.15.137:0/4112748975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 msgr2=0x7f1204196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 0x7f1204196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 -- 172.21.15.137:0/4112748975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40095f0 con 0x7f1204068670 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1202d9d700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f11f4000c00 tx=0x7f11f400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f401d070 con 0x7f1204068670 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11f4005dc0 con 0x7f1204068670 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f4003940 con 0x7f1204068670 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.202+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120419b2a0 con 0x7f1204068670 2024-06-15T01:10:45.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.203+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120419b730 con 0x7f1204068670 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.204+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1204109130 con 0x7f1204068670 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.204+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f11f4004ae0 con 0x7f1204068670 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.204+0000 7f11ebfff700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 0x7f11ec06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.205+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f11f408b6c0 con 0x7f1204068670 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.205+0000 7f120259c700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 0x7f11ec06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.205+0000 7f120259c700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 0x7f11ec06d440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1204197c90 tx=0x7f11f8009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.209+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f11f40566b0 con 0x7f1204068670 2024-06-15T01:10:45.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.465+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f120404ea90 con 0x7f1204068670 2024-06-15T01:10:45.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.466+0000 7f11ebfff700 1 -- 172.21.15.137:0/4112748975 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f11f4059cd0 con 0x7f1204068670 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:10:45.460 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:10:45.461 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:45.462 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 msgr2=0x7f11ec06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 0x7f11ec06d440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1204197c90 tx=0x7f11f8009260 comp rx=0 tx=0).stop 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 msgr2=0x7f1204196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f11f4000c00 tx=0x7f11f400bde0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 shutdown_connections 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f11ec06af80 0x7f11ec06d440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1204068670 0x7f1204196670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 --2- 172.21.15.137:0/4112748975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1204069020 0x7f1204196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 >> 172.21.15.137:0/4112748975 conn(0x7f12040755a0 msgr2=0x7f12040fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 shutdown_connections 2024-06-15T01:10:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.469+0000 7f1209169700 1 -- 172.21.15.137:0/4112748975 wait complete. 2024-06-15T01:10:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:10:45.534 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 ceph-mon[29354]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:45.534 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 ceph-mon[29354]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:45.534 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 ceph-mon[29354]: from='client.? 172.21.15.137:0/1695607511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:45.534 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 systemd[1]: Stopping Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:10:45.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:45 smithi142 ceph-mon[37379]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:45.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:45 smithi142 ceph-mon[37379]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:45.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:45 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1695607511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:45.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 -- 172.21.15.137:0/3006416089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 msgr2=0x7f28a0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/3006416089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a0100050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2894009910 tx=0x7f2894009c20 comp rx=0 tx=0).stop 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 -- 172.21.15.137:0/3006416089 shutdown_connections 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/3006416089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a0100050 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/3006416089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 0x7f28a0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 -- 172.21.15.137:0/3006416089 >> 172.21.15.137:0/3006416089 conn(0x7f28a00755a0 msgr2=0x7f28a00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 -- 172.21.15.137:0/3006416089 shutdown_connections 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.580+0000 7f28a4fc0700 1 -- 172.21.15.137:0/3006416089 wait complete. 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 Processor -- start 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 -- start start 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 0x7f28a0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0198100 con 0x7f28a0105a40 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f28a4fc0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0198240 con 0x7f28a00ffbd0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f289dd9b700 1 -- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 msgr2=0x7f28a0197b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:3300/0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f289dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 0x7f28a0197b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f289e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f289e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:40042/0 (socket says 172.21.15.137:40042) 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.581+0000 7f289e59c700 1 -- 172.21.15.137:0/2297734589 learned_addr learned my addr 172.21.15.137:0/2297734589 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f289e59c700 1 -- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 msgr2=0x7f28a0197b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f289e59c700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 0x7f28a0197b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f289e59c700 1 -- 172.21.15.137:0/2297734589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940095f0 con 0x7f28a00ffbd0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f289e59c700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f289000b700 tx=0x7f289000bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2890011800 con 0x7f28a00ffbd0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2890011e00 con 0x7f28a00ffbd0 2024-06-15T01:10:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a01a0660 con 0x7f28a00ffbd0 2024-06-15T01:10:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f289000f2e0 con 0x7f28a00ffbd0 2024-06-15T01:10:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.582+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a01a0ad0 con 0x7f28a00ffbd0 2024-06-15T01:10:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.583+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a0102ce0 con 0x7f28a00ffbd0 2024-06-15T01:10:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.584+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f2890010380 con 0x7f28a00ffbd0 2024-06-15T01:10:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.584+0000 7f288f7fe700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 0x7f288806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.584+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f289008a150 con 0x7f28a00ffbd0 2024-06-15T01:10:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.584+0000 7f289dd9b700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 0x7f288806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.585+0000 7f289dd9b700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 0x7f288806d3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f289400b540 tx=0x7f2894009fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.589+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f2890055120 con 0x7f28a00ffbd0 2024-06-15T01:10:45.798 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[29350]: 2024-06-15T01:10:45.540+0000 7f0e49c35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi137 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:10:45.798 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[29350]: 2024-06-15T01:10:45.540+0000 7f0e49c35700 -1 mon.smithi137@0(leader) e2 *** Got Signal Terminated *** 2024-06-15T01:10:45.798 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:45 smithi137 podman[112078]: 2024-06-15 01:10:45.693674609 +0000 UTC m=+0.282557740 container died 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-15T01:10:45.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.804+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28a002cdf0 con 0x7f288806aee0 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.806+0000 7f288f7fe700 1 -- 172.21.15.137:0/2297734589 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f28a002cdf0 con 0x7f288806aee0 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "2/23 daemons upgraded", 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mon daemons", 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:10:45.799 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 msgr2=0x7f288806d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 0x7f288806d3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f289400b540 tx=0x7f2894009fd0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 msgr2=0x7f28a01975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f289000b700 tx=0x7f289000bac0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 shutdown_connections 2024-06-15T01:10:45.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f288806aee0 0x7f288806d3a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28a00ffbd0 0x7f28a01975f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 --2- 172.21.15.137:0/2297734589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28a0105a40 0x7f28a0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 >> 172.21.15.137:0/2297734589 conn(0x7f28a00755a0 msgr2=0x7f28a00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 shutdown_connections 2024-06-15T01:10:45.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.808+0000 7f28a4fc0700 1 -- 172.21.15.137:0/2297734589 wait complete. 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 -- 172.21.15.137:0/20024944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654101c80 msgr2=0x7f4654102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 --2- 172.21.15.137:0/20024944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654101c80 0x7f4654102060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f465000b0a0 tx=0x7f465000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 -- 172.21.15.137:0/20024944 shutdown_connections 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 --2- 172.21.15.137:0/20024944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654102630 0x7f465410ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 --2- 172.21.15.137:0/20024944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654101c80 0x7f4654102060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 -- 172.21.15.137:0/20024944 >> 172.21.15.137:0/20024944 conn(0x7f46540fb540 msgr2=0x7f46540fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 -- 172.21.15.137:0/20024944 shutdown_connections 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.913+0000 7f465c741700 1 -- 172.21.15.137:0/20024944 wait complete. 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 Processor -- start 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 -- start start 2024-06-15T01:10:45.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 0x7f46540ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46541003f0 con 0x7f4654101c80 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465c741700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4654100530 con 0x7f4654102630 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465a4dd700 1 -- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 msgr2=0x7f46540ff970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:3300/0 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f465a4dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 0x7f46540ff970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f4659cdc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f4659cdc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.137:40060/0 (socket says 172.21.15.137:40060) 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.914+0000 7f4659cdc700 1 -- 172.21.15.137:0/1364665889 learned_addr learned my addr 172.21.15.137:0/1364665889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f4659cdc700 1 -- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 msgr2=0x7f46540ff970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f4659cdc700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 0x7f46540ff970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f4659cdc700 1 -- 172.21.15.137:0/1364665889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650009d00 con 0x7f4654102630 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f4659cdc700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f464400b4d0 tx=0x7f464400b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4644011800 con 0x7f4654102630 2024-06-15T01:10:45.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46541a28c0 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4644011e00 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464400f2e0 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.915+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46541a2cd0 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.916+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4654108220 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.917+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 89464+0+0 (secure 0 0 0) 0x7f4644010380 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.917+0000 7f464b7fe700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 0x7f464006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.917+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4644089d60 con 0x7f4654102630 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.917+0000 7f465a4dd700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 0x7f464006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:10:45.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.918+0000 7f465a4dd700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 0x7f464006d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4650015040 tx=0x7f4650000f40 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:10:45.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:45.922+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181823 (secure 0 0 0) 0x7f4644054cf0 con 0x7f4654102630 2024-06-15T01:10:46.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.211+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f465404ea90 con 0x7f4654102630 2024-06-15T01:10:46.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.212+0000 7f464b7fe700 1 -- 172.21.15.137:0/1364665889 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4644058310 con 0x7f4654102630 2024-06-15T01:10:46.205 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_OK 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 msgr2=0x7f464006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 0x7f464006d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4650015040 tx=0x7f4650000f40 comp rx=0 tx=0).stop 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 msgr2=0x7f46540ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f464400b4d0 tx=0x7f464400b890 comp rx=0 tx=0).stop 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 shutdown_connections 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f464006af30 0x7f464006d3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4654101c80 0x7f46540ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:46.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 --2- 172.21.15.137:0/1364665889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4654102630 0x7f46540ffeb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:10:46.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 >> 172.21.15.137:0/1364665889 conn(0x7f46540fb540 msgr2=0x7f4654105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:10:46.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 shutdown_connections 2024-06-15T01:10:46.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:10:46.215+0000 7f465c741700 1 -- 172.21.15.137:0/1364665889 wait complete. 2024-06-15T01:10:46.435 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 podman[112078]: 2024-06-15 01:10:46.157842078 +0000 UTC m=+0.746725207 container cleanup 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, GIT_CLEAN=True, ceph=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-15T01:10:46.435 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 bash[112078]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137 2024-06-15T01:10:46.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 podman[112114]: 2024-06-15 01:10:46.443076056 +0000 UTC m=+0.749684913 container remove 2462064093be6e53f1d743c92902517729b460dfaf15b10d6c5a7cd23deca5e7 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD) 2024-06-15T01:10:46.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service: Deactivated successfully. 2024-06-15T01:10:46.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 systemd[1]: Stopped Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:10:46.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service: Consumed 13.832s CPU time. 2024-06-15T01:10:47.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:46 smithi137 systemd[1]: Starting Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:10:47.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 podman[112276]: 2024-06-15 01:10:46.931053225 +0000 UTC m=+0.018052253 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:10:47.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 podman[112276]: 2024-06-15 01:10:47.035822165 +0000 UTC m=+0.122821217 container create d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-15T01:10:47.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 podman[112276]: 2024-06-15 01:10:47.313306243 +0000 UTC m=+0.400305254 container init d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 podman[112276]: 2024-06-15 01:10:47.316597184 +0000 UTC m=+0.403596194 container start d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: set uid:gid to 167:167 (ceph:ceph) 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable), process ceph-mon, pid 2 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: pidfile_write: ignore empty --pid-file 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: load: jerasure load: lrc 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: RocksDB version: 7.9.2 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Git sha 0 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Compile date 2024-06-08 16:28:17 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: DB SUMMARY 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: DB Session ID: M4KGVXLUGHL09GKF54LH 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: CURRENT file: CURRENT 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: IDENTITY file: IDENTITY 2024-06-15T01:10:47.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi137/store.db dir, Total Num: 1, files: 000032.sst 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi137/store.db: 000030.log size: 3599098 ; 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.error_if_exists: 0 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.create_if_missing: 0 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.paranoid_checks: 1 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.env: 0x55f5fa0bdba0 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.fs: PosixFileSystem 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.info_log: 0x55f5fbea0dc0 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_file_opening_threads: 16 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.statistics: (nil) 2024-06-15T01:10:47.724 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.use_fsync: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_log_file_size: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.keep_log_file_num: 1000 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.recycle_log_file_num: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_fallocate: 1 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_mmap_reads: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_mmap_writes: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.use_direct_reads: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.create_missing_column_families: 0 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.db_log_dir: 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.wal_dir: 2024-06-15T01:10:47.725 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.advise_random_on_open: 1 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.db_write_buffer_size: 0 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.write_buffer_manager: 0x55f5fbe72140 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.rate_limiter: (nil) 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-15T01:10:47.726 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.wal_recovery_mode: 2 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enable_thread_tracking: 0 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enable_pipelined_write: 0 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.unordered_write: 0 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-15T01:10:47.727 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.row_cache: None 2024-06-15T01:10:47.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.wal_filter: None 2024-06-15T01:10:47.728 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_ingest_behind: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.two_write_queues: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.manual_wal_flush: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.wal_compression: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.atomic_flush: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.log_readahead_size: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.best_efforts_recovery: 0 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-15T01:10:47.729 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.allow_data_in_errors: 0 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.db_host_id: __hostname__ 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_background_jobs: 2 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_background_compactions: -1 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_subcompactions: 1 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_total_wal_size: 0 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-15T01:10:47.730 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_open_files: -1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bytes_per_sync: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_readahead_size: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_background_flushes: -1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Compression algorithms supported: 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kZSTD supported: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kXpressCompression supported: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kBZip2Compression supported: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kLZ4Compression supported: 1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kZlibCompression supported: 1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kLZ4HCCompression supported: 1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: kSnappyCompression supported: 1 2024-06-15T01:10:47.731 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi137/store.db/MANIFEST-000015 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.merge_operator: 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_filter: None 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_filter_factory: None 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.sst_partitioner_factory: None 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f5fbea0a60) 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-15T01:10:47.732 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_type: 0 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: data_block_index_type: 0 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_shortening: 1 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: checksum: 4 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: no_block_cache: 0 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache: 0x55f5fbe97350 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_options: 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: capacity : 536870912 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: num_shard_bits : 4 2024-06-15T01:10:47.733 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: strict_capacity_limit : 0 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_cache_compressed: (nil) 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: persistent_cache: (nil) 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_size: 4096 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_size_deviation: 10 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_restart_interval: 16 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: index_block_restart_interval: 1 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: metadata_block_size: 4096 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: partition_filters: 0 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: use_delta_encoding: 1 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: filter_policy: bloomfilter 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: whole_key_filtering: 1 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: verify_compression: 0 2024-06-15T01:10:47.734 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: format_version: 5 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: enable_index_compression: 1 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: block_align: 0 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: max_auto_readahead_size: 262144 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: prepopulate_block_cache: 0 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: initial_auto_readahead_size: 8192 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.write_buffer_size: 33554432 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_write_buffer_number: 2 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression: NoCompression 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression: Disabled 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.prefix_extractor: nullptr 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-15T01:10:47.735 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.num_levels: 7 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-15T01:10:47.736 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.level: 32767 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.strategy: 0 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.enabled: false 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.target_file_size_base: 67108864 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-15T01:10:47.737 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-15T01:10:47.738 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.arena_block_size: 1048576 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.disable_auto_compactions: 0 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-15T01:10:47.739 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.table_properties_collectors: 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.inplace_update_support: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.bloom_locality: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.max_successive_merges: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.paranoid_file_checks: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.force_consistency_checks: 1 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.report_bg_io_stats: 0 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.ttl: 2592000 2024-06-15T01:10:47.740 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enable_blob_files: false 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.min_blob_size: 0 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_file_size: 268435456 2024-06-15T01:10:47.741 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.blob_file_starting_level: 0 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi137/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12107, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 420d8cc3-0575-4f23-8f0f-236050d5091b 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413847342909, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-06-15T01:10:47.742 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413847353059, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3092308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12108, "largest_seqno": 15034, "table_properties": {"data_size": 3081187, "index_size": 6708, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3397, "raw_key_size": 29662, "raw_average_key_size": 22, "raw_value_size": 3055221, "raw_average_value_size": 2280, "num_data_blocks": 305, "num_entries": 1340, "num_filter_entries": 1340, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718413847, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "420d8cc3-0575-4f23-8f0f-236050d5091b", "db_session_id": "M4KGVXLUGHL09GKF54LH", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413847353199, "job": 1, "event": "recovery_finished"} 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi137/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f5fbec0e00 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: DB pointer 0x55f5fbfb4000 2024-06-15T01:10:47.743 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** DB Stats ** 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** Compaction Stats [default] ** 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T01:10:47.744 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: L0 1/0 2.95 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 651.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: L6 1/0 5.71 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Sum 2/0 8.66 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 651.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 651.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** Compaction Stats [default] ** 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 651.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T01:10:47.745 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Cumulative compaction: 0.00 GB write, 204.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Interval compaction: 0.00 GB write, 204.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Block cache BinnedLRUCache@0x55f5fbe97350#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: starting mon.smithi137 rank 0 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi137 fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:10:47.746 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???) e2 preinit fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).mds e21 new map 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).mds e21 print_map 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: e21 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: legacy client fscid: 1 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Filesystem 'cephfs' (1) 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: fs_name cephfs 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: epoch 21 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: tableserver 0 2024-06-15T01:10:47.747 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: root 0 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: session_timeout 60 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: session_autoclose 300 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: max_file_size 1099511627776 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: max_xattr_size 65536 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: required_client_features {} 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: last_failure 0 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: last_failure_osd_epoch 47 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: max_mds 2 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: in 0,1 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: up {0=14480,1=14496} 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: failed 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: damaged 2024-06-15T01:10:47.748 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: stopped 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: data_pools [3] 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: metadata_pool 2 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: inline_data disabled 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: balancer 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: bal_rank_mask -1 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: standby_count_wanted 1 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: [mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: [mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: Standby daemons: 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: [mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout: [mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:10:47.749 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 ceph-mon[112295]: mon.smithi137@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 bash[112276]: d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 2024-06-15T01:10:47.750 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:47 smithi137 systemd[1]: Started Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: pgmap v92: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1364665889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:10:49.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: pgmap v93: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: mon.smithi137 calling monitor election 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: mgrmap e32: smithi137.nbzbyc(active, since 3m), standbys: smithi142.dllryb 2024-06-15T01:10:49.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: overall HEALTH_OK 2024-06-15T01:10:49.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='' 2024-06-15T01:10:49.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:49.225 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:49.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:10:49.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:10:49.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: pgmap v92: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='client.? 172.21.15.137:0/1364665889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: pgmap v93: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: mon.smithi137 calling monitor election 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:10:49.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: mgrmap e32: smithi137.nbzbyc(active, since 3m), standbys: smithi142.dllryb 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: overall HEALTH_OK 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='' 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:48 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:51.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:50 smithi137 ceph-mon[112295]: pgmap v94: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:51.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:50 smithi142 ceph-mon[37379]: pgmap v94: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:53.095 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:52 smithi137 ceph-mon[112295]: pgmap v95: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:53.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:52 smithi142 ceph-mon[37379]: pgmap v95: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:54.847 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:54 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:54.847 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:54 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:54.847 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:54 smithi137 ceph-mon[112295]: pgmap v96: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:54.848 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:54 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:54.848 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:54 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:55.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:54 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:55.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:54 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:55.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:54 smithi142 ceph-mon[37379]: pgmap v96: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:55.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:54 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:55.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:54 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:57.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:56 smithi142 ceph-mon[37379]: pgmap v97: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:57.067 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:56 smithi137 ceph-mon[112295]: pgmap v97: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: Detected new or changed devices on smithi137 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:58.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:58.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-15T01:10:58.723 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:10:58 smithi137 ceph-mon[112295]: pgmap v98: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:10:58.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: Detected new or changed devices on smithi137 2024-06-15T01:10:58.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-06-15T01:10:58.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:10:58 smithi142 ceph-mon[37379]: pgmap v98: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:00.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: Upgrade: Updating mon.smithi142 2024-06-15T01:11:00.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:00.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:11:00.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:11:00.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:00.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-mon[37379]: Deploying daemon mon.smithi142 on smithi142 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: Upgrade: Updating mon.smithi142 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: from='mgr.24425 ' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:00.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:00 smithi137 ceph-mon[112295]: Deploying daemon mon.smithi142 on smithi142 2024-06-15T01:11:00.753 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 systemd[1]: Stopping Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:11:01.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142[37367]: 2024-06-15T01:11:00.760+0000 7f7aa39b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi142 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:11:01.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142[37367]: 2024-06-15T01:11:00.760+0000 7f7aa39b5700 -1 mon.smithi142@1(peon) e2 *** Got Signal Terminated *** 2024-06-15T01:11:01.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:00 smithi142 podman[81972]: 2024-06-15 01:11:00.893592098 +0000 UTC m=+0.234947755 container died 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-06-15T01:11:01.553 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 podman[81972]: 2024-06-15 01:11:01.251826948 +0000 UTC m=+0.593182603 container cleanup 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-06-15T01:11:01.553 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 bash[81972]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142 2024-06-15T01:11:01.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 podman[81985]: 2024-06-15 01:11:01.56132066 +0000 UTC m=+0.668581764 container remove 211281b609be26cd986636ddb62fc7503c54484894508d530bdbb1b30752b934 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-06-15T01:11:01.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service: Deactivated successfully. 2024-06-15T01:11:01.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 systemd[1]: Stopped Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:11:01.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service: Consumed 7.390s CPU time. 2024-06-15T01:11:02.235 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:01 smithi142 systemd[1]: Starting Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:11:02.235 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 podman[82116]: 2024-06-15 01:11:02.035032646 +0000 UTC m=+0.017741748 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:11:02.235 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 podman[82116]: 2024-06-15 01:11:02.145554687 +0000 UTC m=+0.128263784 container create abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3) 2024-06-15T01:11:02.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 podman[82116]: 2024-06-15 01:11:02.356966424 +0000 UTC m=+0.339675522 container init abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:11:02.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 podman[82116]: 2024-06-15 01:11:02.360214391 +0000 UTC m=+0.342923498 container start abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True) 2024-06-15T01:11:02.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: set uid:gid to 167:167 (ceph:ceph) 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable), process ceph-mon, pid 2 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: pidfile_write: ignore empty --pid-file 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: load: jerasure load: lrc 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: RocksDB version: 7.9.2 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Git sha 0 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Compile date 2024-06-08 16:28:17 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: DB SUMMARY 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: DB Session ID: 0XP34RW2HQUSSYVDKX6E 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: CURRENT file: CURRENT 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: IDENTITY file: IDENTITY 2024-06-15T01:11:02.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi142/store.db dir, Total Num: 1, files: 000027.sst 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi142/store.db: 000025.log size: 5955535 ; 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.error_if_exists: 0 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.create_if_missing: 0 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.paranoid_checks: 1 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.env: 0x55851893dba0 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.fs: PosixFileSystem 2024-06-15T01:11:02.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.info_log: 0x55851a1bcdc0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_file_opening_threads: 16 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.statistics: (nil) 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.use_fsync: 0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_log_file_size: 0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.keep_log_file_num: 1000 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.recycle_log_file_num: 0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_fallocate: 1 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_mmap_reads: 0 2024-06-15T01:11:02.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_mmap_writes: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.use_direct_reads: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.create_missing_column_families: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.db_log_dir: 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.wal_dir: 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.advise_random_on_open: 1 2024-06-15T01:11:02.403 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.db_write_buffer_size: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.write_buffer_manager: 0x55851a18e140 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.rate_limiter: (nil) 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.wal_recovery_mode: 2 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enable_thread_tracking: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enable_pipelined_write: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.unordered_write: 0 2024-06-15T01:11:02.404 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-15T01:11:02.405 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-15T01:11:02.405 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-15T01:11:02.405 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-15T01:11:02.405 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.row_cache: None 2024-06-15T01:11:02.405 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.wal_filter: None 2024-06-15T01:11:02.406 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-15T01:11:02.406 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_ingest_behind: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.two_write_queues: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.manual_wal_flush: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.wal_compression: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.atomic_flush: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.log_readahead_size: 0 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-15T01:11:02.407 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.best_efforts_recovery: 0 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.allow_data_in_errors: 0 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.db_host_id: __hostname__ 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_background_jobs: 2 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_background_compactions: -1 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_subcompactions: 1 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_total_wal_size: 0 2024-06-15T01:11:02.408 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_open_files: -1 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bytes_per_sync: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_readahead_size: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_background_flushes: -1 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Compression algorithms supported: 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kZSTD supported: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kXpressCompression supported: 0 2024-06-15T01:11:02.409 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kBZip2Compression supported: 0 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kLZ4Compression supported: 1 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kZlibCompression supported: 1 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kLZ4HCCompression supported: 1 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: kSnappyCompression supported: 1 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000010 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.merge_operator: 2024-06-15T01:11:02.410 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_filter: None 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_filter_factory: None 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.sst_partitioner_factory: None 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55851a1bca60) 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_type: 0 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_index_type: 0 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_shortening: 1 2024-06-15T01:11:02.411 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: checksum: 4 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: no_block_cache: 0 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache: 0x55851a1b3350 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_options: 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: capacity : 536870912 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_shard_bits : 4 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: strict_capacity_limit : 0 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_compressed: (nil) 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: persistent_cache: (nil) 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size: 4096 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size_deviation: 10 2024-06-15T01:11:02.412 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_restart_interval: 16 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_block_restart_interval: 1 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_block_size: 4096 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: partition_filters: 0 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: use_delta_encoding: 1 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: filter_policy: bloomfilter 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: whole_key_filtering: 1 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: verify_compression: 0 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: format_version: 5 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_index_compression: 1 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_align: 0 2024-06-15T01:11:02.413 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_auto_readahead_size: 262144 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: prepopulate_block_cache: 0 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: initial_auto_readahead_size: 8192 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.write_buffer_size: 33554432 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_write_buffer_number: 2 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression: NoCompression 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression: Disabled 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.prefix_extractor: nullptr 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.num_levels: 7 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-15T01:11:02.414 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-15T01:11:02.415 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.level: 32767 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.strategy: 0 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.enabled: false 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.target_file_size_base: 67108864 2024-06-15T01:11:02.416 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-15T01:11:02.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.arena_block_size: 1048576 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.disable_auto_compactions: 0 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-15T01:11:02.418 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.table_properties_collectors: 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.inplace_update_support: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.bloom_locality: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.max_successive_merges: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-15T01:11:02.419 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.paranoid_file_checks: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.force_consistency_checks: 1 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.report_bg_io_stats: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.ttl: 2592000 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enable_blob_files: false 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.min_blob_size: 0 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_file_size: 268435456 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-15T01:11:02.420 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.blob_file_starting_level: 0 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12125, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eda2365a-5a11-455e-8015-8e14ca1e9b1f 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413862387156, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413862402799, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3396533, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12130, "largest_seqno": 15616, "table_properties": {"data_size": 3384448, "index_size": 7416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3653, "raw_key_size": 31914, "raw_average_key_size": 22, "raw_value_size": 3356411, "raw_average_value_size": 2335, "num_data_blocks": 337, "num_entries": 1437, "num_filter_entries": 1437, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718413862, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eda2365a-5a11-455e-8015-8e14ca1e9b1f", "db_session_id": "0XP34RW2HQUSSYVDKX6E", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718413862402958, "job": 1, "event": "recovery_finished"} 2024-06-15T01:11:02.421 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi142/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55851a1dce00 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: DB pointer 0x55851a2d0000 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** DB Stats ** 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T01:11:02.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L0 1/0 3.24 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L6 1/0 5.71 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Sum 2/0 8.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:11:02.423 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 631.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-15T01:11:02.424 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative compaction: 0.00 GB write, 173.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval compaction: 0.00 GB write, 173.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache BinnedLRUCache@0x55851a1b3350#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: starting mon.smithi142 rank 1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi142 fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:11:02.425 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???) e2 preinit fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).mds e21 new map 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).mds e21 print_map 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: e21 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: legacy client fscid: 1 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Filesystem 'cephfs' (1) 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: fs_name cephfs 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: epoch 21 2024-06-15T01:11:02.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: tableserver 0 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: root 0 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: session_timeout 60 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: session_autoclose 300 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_file_size 1099511627776 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_xattr_size 65536 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: required_client_features {} 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: last_failure 0 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: last_failure_osd_epoch 47 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_mds 2 2024-06-15T01:11:02.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: in 0,1 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: up {0=14480,1=14496} 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: failed 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: damaged 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: stopped 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_pools [3] 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_pool 2 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: inline_data disabled 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: balancer 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: bal_rank_mask -1 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: standby_count_wanted 1 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:02.428 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Standby daemons: 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-06-15T01:11:02.429 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 ceph-mon[82136]: mon.smithi142@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-06-15T01:11:02.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 bash[82116]: abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 2024-06-15T01:11:02.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:02 smithi142 systemd[1]: Started Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:11:03.801 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:03 smithi142 ceph-mon[82136]: pgmap v99: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:03.801 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:11:03.801 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:11:04.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:03 smithi137 ceph-mon[112295]: pgmap v99: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:04.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi137"}]: dispatch 2024-06-15T01:11:04.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-06-15T01:11:05.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: pgmap v100: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:05.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: pgmap v101: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: mon.smithi142 calling monitor election 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: mon.smithi137 calling monitor election 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: mgrmap e32: smithi137.nbzbyc(active, since 3m), standbys: smithi142.dllryb 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: overall HEALTH_OK 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:05.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: pgmap v100: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: pgmap v101: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: mon.smithi142 calling monitor election 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: mon.smithi137 calling monitor election 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: mon.smithi137 is new leader, mons smithi137,smithi142 in quorum (ranks 0,1) 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: monmap e2: 2 mons at {smithi137=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: osdmap e49: 6 total, 6 up, 6 in 2024-06-15T01:11:05.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: mgrmap e32: smithi137.nbzbyc(active, since 3m), standbys: smithi142.dllryb 2024-06-15T01:11:05.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: overall HEALTH_OK 2024-06-15T01:11:05.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:05.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:05.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:07.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:06 smithi142 ceph-mon[82136]: pgmap v102: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:07.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:06 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:07.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:06 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:07.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:06 smithi137 ceph-mon[112295]: pgmap v102: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:07.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:06 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:07.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:06 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:08.294 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:07 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:08.294 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:07 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:08.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:07 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:08.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:07 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:09.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:08 smithi142 ceph-mon[82136]: pgmap v103: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:09.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:08 smithi137 ceph-mon[112295]: pgmap v103: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:11.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: pgmap v104: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:11.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: Detected new or changed devices on smithi142 2024-06-15T01:11:11.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.223 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi137"}]: dispatch 2024-06-15T01:11:11.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi137"}]': finished 2024-06-15T01:11:11.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]: dispatch 2024-06-15T01:11:11.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]': finished 2024-06-15T01:11:11.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: pgmap v104: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:11.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: Detected new or changed devices on smithi142 2024-06-15T01:11:11.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi137"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi137"}]': finished 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]: dispatch 2024-06-15T01:11:11.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]': finished 2024-06-15T01:11:12.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:12 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all mon 2024-06-15T01:11:12.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:12.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:11:12.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:12.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:12 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all mon 2024-06-15T01:11:12.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:12.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi137", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:11:12.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:13.310 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:13 smithi137 ceph-mon[112295]: Upgrade: Updating crash.smithi137 (1/2) 2024-06-15T01:11:13.311 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:13 smithi137 ceph-mon[112295]: Deploying daemon crash.smithi137 on smithi137 2024-06-15T01:11:13.311 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:13 smithi137 ceph-mon[112295]: pgmap v105: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:13.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:13 smithi142 ceph-mon[82136]: Upgrade: Updating crash.smithi137 (1/2) 2024-06-15T01:11:13.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:13 smithi142 ceph-mon[82136]: Deploying daemon crash.smithi137 on smithi137 2024-06-15T01:11:13.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:13 smithi142 ceph-mon[82136]: pgmap v105: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:15.438 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:15 smithi142 ceph-mon[82136]: pgmap v106: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:15.438 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:15.438 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:15.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:15 smithi137 ceph-mon[112295]: pgmap v106: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:15.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:15.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:16.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 -- 172.21.15.137:0/3878426901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8100da0 msgr2=0x7f48f8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 --2- 172.21.15.137:0/3878426901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8100da0 0x7f48f8101180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009910 tx=0x7f48e8009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:16.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 -- 172.21.15.137:0/3878426901 shutdown_connections 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 --2- 172.21.15.137:0/3878426901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8101750 0x7f48f81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 --2- 172.21.15.137:0/3878426901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8100da0 0x7f48f8101180 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 -- 172.21.15.137:0/3878426901 >> 172.21.15.137:0/3878426901 conn(0x7f48f80fc630 msgr2=0x7f48f80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 -- 172.21.15.137:0/3878426901 shutdown_connections 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.328+0000 7f48fcded700 1 -- 172.21.15.137:0/3878426901 wait complete. 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 Processor -- start 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 -- start start 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8100da0 0x7f48f8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8073b10 con 0x7f48f8101750 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48fcded700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8073c50 con 0x7f48f8100da0 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48f5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48f5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37670/0 (socket says 172.21.15.137:37670) 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48f5d9b700 1 -- 172.21.15.137:0/3911512767 learned_addr learned my addr 172.21.15.137:0/3911512767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.329+0000 7f48f659c700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8100da0 0x7f48f8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48f5d9b700 1 -- 172.21.15.137:0/3911512767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8100da0 msgr2=0x7f48f8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48f5d9b700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8100da0 0x7f48f8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48f5d9b700 1 -- 172.21.15.137:0/3911512767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e80095f0 con 0x7f48f8101750 2024-06-15T01:11:16.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48f5d9b700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d900 tx=0x7f48ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:16.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ec009900 con 0x7f48f8101750 2024-06-15T01:11:16.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48ec010460 con 0x7f48f8101750 2024-06-15T01:11:16.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ec00f5f0 con 0x7f48f8101750 2024-06-15T01:11:16.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f81a2860 con 0x7f48f8101750 2024-06-15T01:11:16.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.330+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f81a2c10 con 0x7f48f8101750 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.331+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8109140 con 0x7f48f8101750 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.332+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f48ec0105d0 con 0x7f48f8101750 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.332+0000 7f48e77fe700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 0x7f48e006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.332+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f48ec089cb0 con 0x7f48f8101750 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.332+0000 7f48f659c700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 0x7f48e006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.333+0000 7f48f659c700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 0x7f48e006d440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48e800b5c0 tx=0x7f48e801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:16.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.336+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f48ec054ca0 con 0x7f48f8101750 2024-06-15T01:11:16.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:16 smithi137 ceph-mon[112295]: from='client.34100 -' entity='client.crash.smithi137' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:16.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:16 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:16.360 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:16 smithi137 ceph-mon[112295]: pgmap v107: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:16.446 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:16 smithi142 ceph-mon[82136]: from='client.34100 -' entity='client.crash.smithi137' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:16.447 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:16 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:16.447 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:16 smithi142 ceph-mon[82136]: pgmap v107: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:16.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.575+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48f802ce30 con 0x7f48e006af80 2024-06-15T01:11:16.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.576+0000 7f48e77fe700 1 -- 172.21.15.137:0/3911512767 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f48f802ce30 con 0x7f48e006af80 2024-06-15T01:11:16.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.578+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 msgr2=0x7f48e006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.578+0000 7f48fcded700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 0x7f48e006d440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48e800b5c0 tx=0x7f48e801a040 comp rx=0 tx=0).stop 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 msgr2=0x7f48f80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d900 tx=0x7f48ec00dc10 comp rx=0 tx=0).stop 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 shutdown_connections 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48e006af80 0x7f48e006d440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f8100da0 0x7f48f8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 --2- 172.21.15.137:0/3911512767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48f8101750 0x7f48f80735d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 >> 172.21.15.137:0/3911512767 conn(0x7f48f80fc630 msgr2=0x7f48f80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 shutdown_connections 2024-06-15T01:11:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.579+0000 7f48fcded700 1 -- 172.21.15.137:0/3911512767 wait complete. 2024-06-15T01:11:16.582 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:11:16.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 -- 172.21.15.137:0/221508995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 msgr2=0x7f22c0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/221508995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0101180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009910 tx=0x7f22b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 -- 172.21.15.137:0/221508995 shutdown_connections 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/221508995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/221508995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0101180 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 -- 172.21.15.137:0/221508995 >> 172.21.15.137:0/221508995 conn(0x7f22c00fc630 msgr2=0x7f22c00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.686+0000 7f22c5e5d700 1 -- 172.21.15.137:0/221508995 shutdown_connections 2024-06-15T01:11:16.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.687+0000 7f22c5e5d700 1 -- 172.21.15.137:0/221508995 wait complete. 2024-06-15T01:11:16.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.687+0000 7f22c5e5d700 1 Processor -- start 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.687+0000 7f22c5e5d700 1 -- start start 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22c5e5d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22c5e5d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22c5e5d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c0073b10 con 0x7f22c0100da0 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22c5e5d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c0073c50 con 0x7f22c0101750 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22bf7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22bf7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37696/0 (socket says 172.21.15.137:37696) 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22bf7fe700 1 -- 172.21.15.137:0/2200875173 learned_addr learned my addr 172.21.15.137:0/2200875173 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22beffd700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22beffd700 1 -- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 msgr2=0x7f22c0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22beffd700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22beffd700 1 -- 172.21.15.137:0/2200875173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00095f0 con 0x7f22c0101750 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.688+0000 7f22beffd700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c00735d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22b400e940 tx=0x7f22b400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:16.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.689+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b400c990 con 0x7f22c0101750 2024-06-15T01:11:16.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.689+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22b4010430 con 0x7f22c0101750 2024-06-15T01:11:16.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.689+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c01a2860 con 0x7f22c0101750 2024-06-15T01:11:16.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.689+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b4018650 con 0x7f22c0101750 2024-06-15T01:11:16.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.689+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c01a2c70 con 0x7f22c0101750 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.690+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c0109140 con 0x7f22c0101750 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.690+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f22b40187b0 con 0x7f22c0101750 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.691+0000 7f22bcff9700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 0x7f22a806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.691+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f22b4014070 con 0x7f22c0101750 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.691+0000 7f22bf7fe700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 0x7f22a806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:16.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.691+0000 7f22bf7fe700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 0x7f22a806d440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009fd0 tx=0x7f22b0005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:16.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.695+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f22b4052900 con 0x7f22c0101750 2024-06-15T01:11:16.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.929+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22c0073de0 con 0x7f22a806af80 2024-06-15T01:11:16.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.931+0000 7f22bcff9700 1 -- 172.21.15.137:0/2200875173 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f22c0073de0 con 0x7f22a806af80 2024-06-15T01:11:16.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 msgr2=0x7f22a806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 0x7f22a806d440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009fd0 tx=0x7f22b0005a90 comp rx=0 tx=0).stop 2024-06-15T01:11:16.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 msgr2=0x7f22c00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:16.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c00735d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22b400e940 tx=0x7f22b400ec50 comp rx=0 tx=0).stop 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 shutdown_connections 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22a806af80 0x7f22a806d440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22c0100da0 0x7f22c0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 --2- 172.21.15.137:0/2200875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22c0101750 0x7f22c00735d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 >> 172.21.15.137:0/2200875173 conn(0x7f22c00fc630 msgr2=0x7f22c00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.933+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 shutdown_connections 2024-06-15T01:11:16.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:16.934+0000 7f22c5e5d700 1 -- 172.21.15.137:0/2200875173 wait complete. 2024-06-15T01:11:17.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- 172.21.15.137:0/529197312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 msgr2=0x7f2a04104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 --2- 172.21.15.137:0/529197312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04104720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009910 tx=0x7f29f4009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:17.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- 172.21.15.137:0/529197312 shutdown_connections 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 --2- 172.21.15.137:0/529197312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04104720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 --2- 172.21.15.137:0/529197312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 0x7f2a04100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- 172.21.15.137:0/529197312 >> 172.21.15.137:0/529197312 conn(0x7f2a04074e00 msgr2=0x7f2a04075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- 172.21.15.137:0/529197312 shutdown_connections 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- 172.21.15.137:0/529197312 wait complete. 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 Processor -- start 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.040+0000 7f2a0c721700 1 -- start start 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0c721700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 0x7f2a04196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0c721700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0c721700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04197270 con 0x7f2a04100fd0 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0c721700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0419b000 con 0x7f2a04100620 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37722/0 (socket says 172.21.15.137:37722) 2024-06-15T01:11:17.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0a4bd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 0x7f2a04196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 -- 172.21.15.137:0/3469552616 learned_addr learned my addr 172.21.15.137:0/3469552616 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 -- 172.21.15.137:0/3469552616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 msgr2=0x7f2a04196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 0x7f2a04196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 -- 172.21.15.137:0/3469552616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40095f0 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a09cbc700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29f401d070 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0419b280 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29f400bed0 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.041+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29f400fd90 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.042+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0419b710 con 0x7f2a04100fd0 2024-06-15T01:11:17.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.042+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04109010 con 0x7f2a04100fd0 2024-06-15T01:11:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.043+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f29f4021410 con 0x7f2a04100fd0 2024-06-15T01:11:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.044+0000 7f29fb7fe700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 0x7f29f006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.044+0000 7f2a0a4bd700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 0x7f29f006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.044+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f29f408b370 con 0x7f2a04100fd0 2024-06-15T01:11:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.044+0000 7f2a0a4bd700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 0x7f29f006d440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009c60 tx=0x7f2a00009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.048+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f29f4056360 con 0x7f2a04100fd0 2024-06-15T01:11:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.267+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a0402cdb0 con 0x7f29f006af80 2024-06-15T01:11:17.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.277+0000 7f29fb7fe700 1 -- 172.21.15.137:0/3469552616 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2a0402cdb0 con 0x7f29f006af80 2024-06-15T01:11:17.269 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:11:17.269 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (110s) 23s ago 12m 15.1M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (12m) 23s ago 12m 8078k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (11m) 10s ago 11m 7742k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 starting - - - - 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (11m) 10s ago 11m 6665k - 18.2.1 5be31c24972a d0d5d520430f 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (95s) 23s ago 12m 80.7M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (6m) 23s ago 6m 116M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (6m) 23s ago 6m 131M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (6m) 10s ago 6m 15.3M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (6m) 10s ago 6m 18.6M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (3m) 23s ago 13m 488M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (2m) 10s ago 11m 438M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (30s) 23s ago 14m 23.1M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (15s) 10s ago 11m 22.1M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:11:17.270 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (2m) 23s ago 12m 15.4M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (2m) 10s ago 11m 17.2M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (10m) 23s ago 10m 223M 1216M 18.2.1 5be31c24972a b029f42bcc3f 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (10m) 23s ago 10m 244M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (9m) 23s ago 9m 197M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (9m) 10s ago 9m 267M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (8m) 10s ago 8m 241M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (8m) 10s ago 8m 223M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:11:17.271 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (2m) 23s ago 11m 63.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 msgr2=0x7f29f006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 0x7f29f006d440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009c60 tx=0x7f2a00009260 comp rx=0 tx=0).stop 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 msgr2=0x7f2a04196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f400ba70 comp rx=0 tx=0).stop 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 shutdown_connections 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f29f006af80 0x7f29f006d440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a04100620 0x7f2a04196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 --2- 172.21.15.137:0/3469552616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a04100fd0 0x7f2a04196b90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 >> 172.21.15.137:0/3469552616 conn(0x7f2a04074e00 msgr2=0x7f2a04103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 shutdown_connections 2024-06-15T01:11:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.279+0000 7f2a0c721700 1 -- 172.21.15.137:0/3469552616 wait complete. 2024-06-15T01:11:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH) 2024-06-15T01:11:17.346 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: Upgrade: Updating crash.smithi142 (2/2) 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: Deploying daemon crash.smithi142 on smithi142 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:17.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:17 smithi137 ceph-mon[112295]: from='client.44107 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- 172.21.15.137:0/693155570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 msgr2=0x7f0f680ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 --2- 172.21.15.137:0/693155570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f680ffcf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009910 tx=0x7f0f64009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- 172.21.15.137:0/693155570 shutdown_connections 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 --2- 172.21.15.137:0/693155570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 --2- 172.21.15.137:0/693155570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f680ffcf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- 172.21.15.137:0/693155570 >> 172.21.15.137:0/693155570 conn(0x7f0f680fb550 msgr2=0x7f0f680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- 172.21.15.137:0/693155570 shutdown_connections 2024-06-15T01:11:17.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- 172.21.15.137:0/693155570 wait complete. 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 Processor -- start 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.386+0000 7f0f7058e700 1 -- start start 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6819f1d0 con 0x7f0f680ff910 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f68198680 con 0x7f0f68100230 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37740/0 (socket says 172.21.15.137:37740) 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 -- 172.21.15.137:0/2625636777 learned_addr learned my addr 172.21.15.137:0/2625636777 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6db29700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 -- 172.21.15.137:0/2625636777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 msgr2=0x7f0f6819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 -- 172.21.15.137:0/2625636777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f640095f0 con 0x7f0f680ff910 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6db29700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f6e32a700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0f64006010 tx=0x7f0f6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6401d070 con 0x7f0f680ff910 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0f6400f460 con 0x7f0f680ff910 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f64021640 con 0x7f0f680ff910 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f68198900 con 0x7f0f680ff910 2024-06-15T01:11:17.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.387+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f68198df0 con 0x7f0f680ff910 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.388+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6810a6f0 con 0x7f0f680ff910 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.389+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0f64003a60 con 0x7f0f680ff910 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.389+0000 7f0f5f7fe700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 0x7f0f5406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.389+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0f6408b420 con 0x7f0f680ff910 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.389+0000 7f0f6db29700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 0x7f0f5406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.390+0000 7f0f6db29700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 0x7f0f5406d440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f680fbc00 tx=0x7f0f58005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.394+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f64059f20 con 0x7f0f680ff910 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH) 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: Upgrade: Updating crash.smithi142 (2/2) 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: Deploying daemon crash.smithi142 on smithi142 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:17.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:17 smithi142 ceph-mon[82136]: from='client.44107 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:17.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.665+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f6804ea90 con 0x7f0f680ff910 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.665+0000 7f0f5f7fe700 1 -- 172.21.15.137:0/2625636777 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7f0f64059ab0 con 0x7f0f680ff910 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:17.658 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:11:17.659 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:11:17.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 msgr2=0x7f0f5406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 0x7f0f5406d440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f680fbc00 tx=0x7f0f58005f90 comp rx=0 tx=0).stop 2024-06-15T01:11:17.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 msgr2=0x7f0f6819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0f64006010 tx=0x7f0f6400be20 comp rx=0 tx=0).stop 2024-06-15T01:11:17.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 shutdown_connections 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f5406af80 0x7f0f5406d440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f680ff910 0x7f0f6819e600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 --2- 172.21.15.137:0/2625636777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68100230 0x7f0f6819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.667+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 >> 172.21.15.137:0/2625636777 conn(0x7f0f680fb550 msgr2=0x7f0f680fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.668+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 shutdown_connections 2024-06-15T01:11:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.668+0000 7f0f7058e700 1 -- 172.21.15.137:0/2625636777 wait complete. 2024-06-15T01:11:17.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 -- 172.21.15.137:0/3390456927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb8105a40 msgr2=0x7f0bb8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3390456927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb8105a40 0x7f0bb8105e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009910 tx=0x7f0ba8009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:17.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 -- 172.21.15.137:0/3390456927 shutdown_connections 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3390456927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb80ffa40 0x7f0bb80ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3390456927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb8105a40 0x7f0bb8105e20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 -- 172.21.15.137:0/3390456927 >> 172.21.15.137:0/3390456927 conn(0x7f0bb80fb530 msgr2=0x7f0bb80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 -- 172.21.15.137:0/3390456927 shutdown_connections 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.771+0000 7f0bbe744700 1 -- 172.21.15.137:0/3390456927 wait complete. 2024-06-15T01:11:17.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 Processor -- start 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 -- start start 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8105a40 0x7f0bb8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8197290 con 0x7f0bb80ffa40 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bbe744700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb819b020 con 0x7f0bb8105a40 2024-06-15T01:11:17.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bb7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bb77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8105a40 0x7f0bb8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bb7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37766/0 (socket says 172.21.15.137:37766) 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.772+0000 7f0bb7fff700 1 -- 172.21.15.137:0/3536805470 learned_addr learned my addr 172.21.15.137:0/3536805470 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb7fff700 1 -- 172.21.15.137:0/3536805470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8105a40 msgr2=0x7f0bb8196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb7fff700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8105a40 0x7f0bb8196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb7fff700 1 -- 172.21.15.137:0/3536805470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80095f0 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb7fff700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8006010 tx=0x7f0ba800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba801d070 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb819b2a0 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ba8005710 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba800fe20 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.773+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb819b790 con 0x7f0bb80ffa40 2024-06-15T01:11:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.774+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb8109050 con 0x7f0bb80ffa40 2024-06-15T01:11:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.775+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0ba8003a60 con 0x7f0bb80ffa40 2024-06-15T01:11:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.775+0000 7f0bb57fa700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 0x7f0ba006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.775+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0ba808b380 con 0x7f0bb80ffa40 2024-06-15T01:11:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.775+0000 7f0bb77fe700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 0x7f0ba006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.775+0000 7f0bb77fe700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 0x7f0ba006d440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8197c90 tx=0x7f0bac00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:17.779+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ba8056370 con 0x7f0bb80ffa40 2024-06-15T01:11:18.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.015+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0bb802cd30 con 0x7f0bb80ffa40 2024-06-15T01:11:18.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.016+0000 7f0bb57fa700 1 -- 172.21.15.137:0/3536805470 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f0ba8059990 con 0x7f0bb80ffa40 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:11:18.009 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:11:18.010 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:18.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:18.012 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:11:18.012 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:18.012 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:18.012 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 msgr2=0x7f0ba006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 0x7f0ba006d440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8197c90 tx=0x7f0bac00b410 comp rx=0 tx=0).stop 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 msgr2=0x7f0bb8196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8006010 tx=0x7f0ba800bd70 comp rx=0 tx=0).stop 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 shutdown_connections 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ba006af80 0x7f0ba006d440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb80ffa40 0x7f0bb8196670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 --2- 172.21.15.137:0/3536805470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bb8105a40 0x7f0bb8196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 >> 172.21.15.137:0/3536805470 conn(0x7f0bb80fb530 msgr2=0x7f0bb8069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 shutdown_connections 2024-06-15T01:11:18.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.018+0000 7f0bbe744700 1 -- 172.21.15.137:0/3536805470 wait complete. 2024-06-15T01:11:18.014 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 -- 172.21.15.137:0/2429329745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9150105a50 msgr2=0x7f9150105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 --2- 172.21.15.137:0/2429329745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9150105a50 0x7f9150105e30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9140009910 tx=0x7f9140009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 -- 172.21.15.137:0/2429329745 shutdown_connections 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 --2- 172.21.15.137:0/2429329745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91500ffa50 0x7f91500ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 --2- 172.21.15.137:0/2429329745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9150105a50 0x7f9150105e30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.123+0000 7f915535c700 1 -- 172.21.15.137:0/2429329745 >> 172.21.15.137:0/2429329745 conn(0x7f91500fb540 msgr2=0x7f91500fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 -- 172.21.15.137:0/2429329745 shutdown_connections 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 -- 172.21.15.137:0/2429329745 wait complete. 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 Processor -- start 2024-06-15T01:11:18.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 -- start start 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150105a50 0x7f9150196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150197290 con 0x7f91500ffa50 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.124+0000 7f915535c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915019b020 con 0x7f9150105a50 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37776/0 (socket says 172.21.15.137:37776) 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 -- 172.21.15.137:0/2909030909 learned_addr learned my addr 172.21.15.137:0/2909030909 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914e7fc700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150105a50 0x7f9150196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 -- 172.21.15.137:0/2909030909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150105a50 msgr2=0x7f9150196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150105a50 0x7f9150196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 -- 172.21.15.137:0/2909030909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400095f0 con 0x7f91500ffa50 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f914effd700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91400092e0 tx=0x7f914000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:18.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914001d070 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9140005e00 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915019b2a0 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914000fe90 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.125+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915019b790 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.126+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9150109060 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.127+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f914000f6d0 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.127+0000 7f9147fff700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 0x7f913c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.127+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f914008b460 con 0x7f91500ffa50 2024-06-15T01:11:18.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.127+0000 7f914e7fc700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 0x7f913c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.128+0000 7f914e7fc700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 0x7f913c06d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9150197c90 tx=0x7f913800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:18.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.131+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9140056450 con 0x7f91500ffa50 2024-06-15T01:11:18.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.340+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f915002ccc0 con 0x7f913c06af80 2024-06-15T01:11:18.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.342+0000 7f9147fff700 1 -- 172.21.15.137:0/2909030909 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f915002ccc0 con 0x7f913c06af80 2024-06-15T01:11:18.334 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:11:18.334 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "4/23 daemons upgraded", 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading crash daemons", 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:11:18.335 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:11:18.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 msgr2=0x7f913c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 0x7f913c06d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9150197c90 tx=0x7f913800b410 comp rx=0 tx=0).stop 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 msgr2=0x7f9150196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91400092e0 tx=0x7f914000bd70 comp rx=0 tx=0).stop 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 shutdown_connections 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f913c06af80 0x7f913c06d440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91500ffa50 0x7f9150196670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 --2- 172.21.15.137:0/2909030909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9150105a50 0x7f9150196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 >> 172.21.15.137:0/2909030909 conn(0x7f91500fb540 msgr2=0x7f9150069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 shutdown_connections 2024-06-15T01:11:18.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.344+0000 7f915535c700 1 -- 172.21.15.137:0/2909030909 wait complete. 2024-06-15T01:11:18.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 -- 172.21.15.137:0/4055402665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab41026d0 msgr2=0x7ffab410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/4055402665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab41026d0 0x7ffab410abc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009910 tx=0x7ffaa0009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 -- 172.21.15.137:0/4055402665 shutdown_connections 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/4055402665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab41026d0 0x7ffab410abc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/4055402665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab4101db0 0x7ffab4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 -- 172.21.15.137:0/4055402665 >> 172.21.15.137:0/4055402665 conn(0x7ffab40fb540 msgr2=0x7ffab40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 -- 172.21.15.137:0/4055402665 shutdown_connections 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.448+0000 7ffabaf4f700 1 -- 172.21.15.137:0/4055402665 wait complete. 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 Processor -- start 2024-06-15T01:11:18.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 -- start start 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab41026d0 0x7ffab40fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4100490 con 0x7ffab4101db0 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffabaf4f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab41005d0 con 0x7ffab41026d0 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffab8ceb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffab8ceb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37798/0 (socket says 172.21.15.137:37798) 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffab8ceb700 1 -- 172.21.15.137:0/466895994 learned_addr learned my addr 172.21.15.137:0/466895994 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.449+0000 7ffab3fff700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab41026d0 0x7ffab40fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab8ceb700 1 -- 172.21.15.137:0/466895994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab41026d0 msgr2=0x7ffab40fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab8ceb700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab41026d0 0x7ffab40fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab8ceb700 1 -- 172.21.15.137:0/466895994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00095f0 con 0x7ffab4101db0 2024-06-15T01:11:18.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab8ceb700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800b4b0 tx=0x7ffaa800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa8011800 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffaa8011e00 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa8010e60 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab41a2990 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.450+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab41a2d40 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.451+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab4108250 con 0x7ffab4101db0 2024-06-15T01:11:18.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.451+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ffaa8011960 con 0x7ffab4101db0 2024-06-15T01:11:18.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.452+0000 7ffab1ffb700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 0x7ffaa406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:18.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.452+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffaa8089c50 con 0x7ffab4101db0 2024-06-15T01:11:18.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.452+0000 7ffab3fff700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 0x7ffaa406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:18.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.452+0000 7ffab3fff700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 0x7ffaa406d440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0006010 tx=0x7ffaa001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:18.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.456+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffaa8054cc0 con 0x7ffab4101db0 2024-06-15T01:11:18.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: pgmap v108: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:18.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: from='client.34112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:18.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2625636777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:18.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:18.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:18.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:18 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3536805470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:11:18.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.730+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffab4107c30 con 0x7ffab4101db0 2024-06-15T01:11:18.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.730+0000 7ffab1ffb700 1 -- 172.21.15.137:0/466895994 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+228 (secure 0 0 0) 0x7ffaa8015100 con 0x7ffab4101db0 2024-06-15T01:11:18.723 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 1 mgr modules have recently crashed 2024-06-15T01:11:18.723 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 1 mgr modules have recently crashed 2024-06-15T01:11:18.723 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:11:18.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 msgr2=0x7ffaa406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 0x7ffaa406d440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0006010 tx=0x7ffaa001a040 comp rx=0 tx=0).stop 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 msgr2=0x7ffab40ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800b4b0 tx=0x7ffaa800b7c0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 shutdown_connections 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa406af80 0x7ffaa406d440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffab4101db0 0x7ffab40ffa10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 --2- 172.21.15.137:0/466895994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab41026d0 0x7ffab40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 >> 172.21.15.137:0/466895994 conn(0x7ffab40fb540 msgr2=0x7ffab4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 shutdown_connections 2024-06-15T01:11:18.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:18.733+0000 7ffabaf4f700 1 -- 172.21.15.137:0/466895994 wait complete. 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: pgmap v108: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: from='client.34112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2625636777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:18.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:18 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3536805470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:11:19.736 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:19 smithi142 ceph-mon[82136]: from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:19.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:19 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/466895994' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:11:19.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:19 smithi137 ceph-mon[112295]: from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:19.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:19 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/466895994' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: pgmap v109: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='client.44127 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: pgmap v110: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:21.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: pgmap v109: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:21.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='client.44127 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: pgmap v110: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:21.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:22.189 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:22 smithi137 ceph-mon[112295]: from='client.44131 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:22.189 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:22 smithi137 ceph-mon[112295]: pgmap v111: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:22.189 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:22 smithi137 ceph-mon[112295]: from='client.44135 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:22.189 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:22 smithi137 ceph-mon[112295]: pgmap v112: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:22.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:22 smithi142 ceph-mon[82136]: from='client.44131 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:22.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:22 smithi142 ceph-mon[82136]: pgmap v111: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:22.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:22 smithi142 ceph-mon[82136]: from='client.44135 -' entity='client.crash.smithi142' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:22.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:22 smithi142 ceph-mon[82136]: pgmap v112: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:23.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:23 smithi142 ceph-mon[82136]: Health check update: 4 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH) 2024-06-15T01:11:23.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:23.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:23.446 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:23 smithi137 ceph-mon[112295]: Health check update: 4 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH) 2024-06-15T01:11:23.446 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:23.447 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:24.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:24 smithi137 ceph-mon[112295]: pgmap v113: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:24.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:24 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:24.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:24 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:24.493 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:24 smithi142 ceph-mon[82136]: pgmap v113: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:24.493 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:24 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:24.494 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:24 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.869 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:25 smithi137 ceph-mon[112295]: pgmap v114: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:25.869 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.869 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.869 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.869 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.979 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:25 smithi142 ceph-mon[82136]: pgmap v114: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:25.979 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.980 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.980 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:25.980 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:27.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:27 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:27.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:27 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:27.722 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:27 smithi137 ceph-mon[112295]: pgmap v115: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:27.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:27 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:27.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:27 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:27.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:27 smithi142 ceph-mon[82136]: pgmap v115: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: pgmap v116: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all crash 2024-06-15T01:11:29.600 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi137"}]: dispatch 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi137"}]': finished 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]: dispatch 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]': finished 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-15T01:11:29.601 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:29 smithi137 ceph-mon[112295]: Upgrade: osd.0 is safe to restart 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: pgmap v116: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all crash 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi137"}]: dispatch 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi137"}]': finished 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]: dispatch 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]': finished 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-06-15T01:11:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:29 smithi142 ceph-mon[82136]: Upgrade: osd.0 is safe to restart 2024-06-15T01:11:31.149 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:30 smithi137 ceph-mon[112295]: Upgrade: Updating osd.0 2024-06-15T01:11:31.150 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:30 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:31.150 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:30 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-15T01:11:31.150 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:30 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:31.150 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:30 smithi137 ceph-mon[112295]: Deploying daemon osd.0 on smithi137 2024-06-15T01:11:31.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:30 smithi142 ceph-mon[82136]: Upgrade: Updating osd.0 2024-06-15T01:11:31.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:30 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:31.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:30 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-15T01:11:31.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:30 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:31.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:30 smithi142 ceph-mon[82136]: Deploying daemon osd.0 on smithi137 2024-06-15T01:11:32.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:31 smithi137 ceph-mon[112295]: pgmap v117: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:32.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:31 smithi142 ceph-mon[82136]: pgmap v117: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:32.972 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:32 smithi137 systemd[1]: Stopping Ceph osd.0 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:11:33.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:11:33.093+0000 7fd2445e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:11:33.472 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:11:33.093+0000 7fd2445e7700 -1 osd.0 49 *** Got signal Terminated *** 2024-06-15T01:11:33.472 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[48155]: 2024-06-15T01:11:33.093+0000 7fd2445e7700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:11:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-mon[112295]: pgmap v118: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:33 smithi137 ceph-mon[112295]: osd.0 marked itself down and dead 2024-06-15T01:11:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:33 smithi142 ceph-mon[82136]: pgmap v118: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:34.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:33 smithi142 ceph-mon[82136]: osd.0 marked itself down and dead 2024-06-15T01:11:34.570 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:34 smithi137 podman[120874]: 2024-06-15 01:11:34.291283621 +0000 UTC m=+1.318879684 container died b029f42bcc3fffdee1b3cee0d77f9eae04991234cea5ffeb45b577cce2816b92 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1) 2024-06-15T01:11:34.918 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:34 smithi137 podman[120874]: 2024-06-15 01:11:34.661071182 +0000 UTC m=+1.688667245 container cleanup b029f42bcc3fffdee1b3cee0d77f9eae04991234cea5ffeb45b577cce2816b92 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-15T01:11:34.918 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:34 smithi137 bash[120874]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0 2024-06-15T01:11:35.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:34 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:11:35.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:34 smithi137 ceph-mon[112295]: osdmap e50: 6 total, 5 up, 6 in 2024-06-15T01:11:35.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:34 smithi137 ceph-mon[112295]: osdmap e51: 6 total, 5 up, 6 in 2024-06-15T01:11:35.222 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:34 smithi137 podman[120890]: 2024-06-15 01:11:34.928931434 +0000 UTC m=+0.637723447 container remove b029f42bcc3fffdee1b3cee0d77f9eae04991234cea5ffeb45b577cce2816b92 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1) 2024-06-15T01:11:35.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:34 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:11:35.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:34 smithi142 ceph-mon[82136]: osdmap e50: 6 total, 5 up, 6 in 2024-06-15T01:11:35.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:34 smithi142 ceph-mon[82136]: osdmap e51: 6 total, 5 up, 6 in 2024-06-15T01:11:35.558 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.174837696 +0000 UTC m=+0.017332064 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:11:35.558 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.305640839 +0000 UTC m=+0.148135238 container create 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux ) 2024-06-15T01:11:35.558 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.566684317 +0000 UTC m=+0.409178683 container init 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-15T01:11:35.815 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.570172392 +0000 UTC m=+0.412666750 container start 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, CEPH_POINT_RELEASE=) 2024-06-15T01:11:35.815 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.658062356 +0000 UTC m=+0.500556729 container attach 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-15T01:11:36.161 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:35 smithi137 ceph-mon[112295]: pgmap v120: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:36.161 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:35 smithi137 podman[120979]: 2024-06-15 01:11:35.824049773 +0000 UTC m=+0.666544133 container died 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-15T01:11:36.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:35 smithi142 ceph-mon[82136]: pgmap v120: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:36.427 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 podman[121001]: 2024-06-15 01:11:36.41162647 +0000 UTC m=+0.581210881 container remove 48563182f5f1d170c716e8ad92c00fc75d93c055a563610ba07e43fe7b3a95aa (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:11:36.721 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.0.service: Deactivated successfully. 2024-06-15T01:11:36.721 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 systemd[1]: Stopped Ceph osd.0 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:11:36.721 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.0.service: Consumed 1min 1.742s CPU time. 2024-06-15T01:11:36.721 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 systemd[1]: Starting Ceph osd.0 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:11:37.140 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 podman[121102]: 2024-06-15 01:11:36.768760804 +0000 UTC m=+0.017473837 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:11:37.140 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:36 smithi137 podman[121102]: 2024-06-15 01:11:36.905045965 +0000 UTC m=+0.153758983 container create 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-15T01:11:37.140 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 podman[121102]: 2024-06-15 01:11:37.149037498 +0000 UTC m=+0.397750534 container init 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-06-15T01:11:37.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 podman[121102]: 2024-06-15 01:11:37.152505285 +0000 UTC m=+0.401218312 container start 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-15T01:11:37.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 podman[121102]: 2024-06-15 01:11:37.232281508 +0000 UTC m=+0.480994535 container attach 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, ceph=True) 2024-06-15T01:11:37.900 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-15T01:11:37.900 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-15T01:11:37.900 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-15T01:11:37.900 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate[121117]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 bash[121102]: --> ceph-volume raw activate successful for osd ID: 0 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 conmon[121117]: conmon 20e79762aa2339ba41ca : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f.scope/container/memory.events 2024-06-15T01:11:37.901 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:37 smithi137 podman[121102]: 2024-06-15 01:11:37.569786766 +0000 UTC m=+0.818499790 container died 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-15T01:11:38.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:37 smithi137 ceph-mon[112295]: pgmap v122: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:37 smithi142 ceph-mon[82136]: pgmap v122: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:38.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 podman[121253]: 2024-06-15 01:11:38.167119787 +0000 UTC m=+0.587956055 container remove 20e79762aa2339ba41ca588f1fc2f172c11e9d7e76a68bb519487634bc82cc8f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-15T01:11:38.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 podman[121314]: 2024-06-15 01:11:38.282845611 +0000 UTC m=+0.017321186 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:11:38.472 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 podman[121314]: 2024-06-15 01:11:38.40421584 +0000 UTC m=+0.138703177 container create e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:11:38.971 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 podman[121314]: 2024-06-15 01:11:38.665413559 +0000 UTC m=+0.399889142 container init e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:11:38.972 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 podman[121314]: 2024-06-15 01:11:38.668882915 +0000 UTC m=+0.403358471 container start e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-15T01:11:38.972 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 ceph-osd[121334]: -- 172.21.15.137:0/3047054176 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d8decf51e0 con 0x55d8dfa76000 2024-06-15T01:11:38.972 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 bash[121314]: e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb 2024-06-15T01:11:38.972 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:38 smithi137 systemd[1]: Started Ceph osd.0 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:11:39.460 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:39 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:11:39.212+0000 7f0bb3147640 -1 Falling back to public interface 2024-06-15T01:11:40.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:39 smithi142 ceph-mon[82136]: pgmap v123: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:40.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:40.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:40.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:40.070 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:39 smithi137 ceph-mon[112295]: pgmap v123: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:40.071 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:40.071 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:40.071 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:41.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:40 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 64/327 objects degraded (19.572%), 19 pgs degraded (PG_DEGRADED) 2024-06-15T01:11:41.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:40 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 64/327 objects degraded (19.572%), 19 pgs degraded (PG_DEGRADED) 2024-06-15T01:11:42.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:41 smithi137 ceph-mon[112295]: pgmap v124: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:42.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:41 smithi142 ceph-mon[82136]: pgmap v124: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:43.768 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:43 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:11:43.665+0000 7f0bb3147640 -1 osd.0 0 read_superblock omap replica is missing. 2024-06-15T01:11:44.060 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:43 smithi137 ceph-mon[112295]: pgmap v125: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:44.061 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:43 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:44.061 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:43 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:44.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:43 smithi142 ceph-mon[82136]: pgmap v125: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:44.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:43 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:44.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:43 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:44.471 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:11:44.068+0000 7f0bb3147640 -1 osd.0 49 log_to_monitors true 2024-06-15T01:11:45.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:44 smithi142 ceph-mon[82136]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-15T01:11:45.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:45.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:45.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:44 smithi137 ceph-mon[112295]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-15T01:11:45.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:45.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:45.796 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:11:45.692+0000 7f0ba6ed8640 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:11:46.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-mon[112295]: pgmap v126: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:46.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-mon[112295]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-15T01:11:46.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-mon[112295]: osdmap e52: 6 total, 5 up, 6 in 2024-06-15T01:11:46.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-mon[112295]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:11:46.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:45 smithi137 ceph-mon[112295]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' 2024-06-15T01:11:46.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:45 smithi142 ceph-mon[82136]: pgmap v126: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:46.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:45 smithi142 ceph-mon[82136]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-15T01:11:46.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:45 smithi142 ceph-mon[82136]: osdmap e52: 6 total, 5 up, 6 in 2024-06-15T01:11:46.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:45 smithi142 ceph-mon[82136]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:11:46.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:45 smithi142 ceph-mon[82136]: from='osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906]' entity='osd.0' 2024-06-15T01:11:47.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:46 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:11:47.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:46 smithi137 ceph-mon[112295]: osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906] boot 2024-06-15T01:11:47.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:46 smithi137 ceph-mon[112295]: osdmap e53: 6 total, 6 up, 6 in 2024-06-15T01:11:47.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:46 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:11:47.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:46 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:11:47.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:46 smithi142 ceph-mon[82136]: osd.0 [v2:172.21.15.137:6802/4060871906,v1:172.21.15.137:6803/4060871906] boot 2024-06-15T01:11:47.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:46 smithi142 ceph-mon[82136]: osdmap e53: 6 total, 6 up, 6 in 2024-06-15T01:11:47.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:46 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: pgmap v128: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:11:48.222 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:47 smithi137 ceph-mon[112295]: osdmap e54: 6 total, 6 up, 6 in 2024-06-15T01:11:48.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: pgmap v128: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:48.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:11:48.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:47 smithi142 ceph-mon[82136]: osdmap e54: 6 total, 6 up, 6 in 2024-06-15T01:11:48.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 -- 172.21.15.137:0/483081860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4101770 msgr2=0x7f2db41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/483081860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4101770 0x7f2db41057c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009910 tx=0x7f2da8009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 -- 172.21.15.137:0/483081860 shutdown_connections 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/483081860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4101770 0x7f2db41057c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/483081860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4100dc0 0x7f2db41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 -- 172.21.15.137:0/483081860 >> 172.21.15.137:0/483081860 conn(0x7f2db40fc650 msgr2=0x7f2db40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 -- 172.21.15.137:0/483081860 shutdown_connections 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 -- 172.21.15.137:0/483081860 wait complete. 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.846+0000 7f2db9ad9700 1 Processor -- start 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db9ad9700 1 -- start start 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db9ad9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db9ad9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 0x7f2db4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db9ad9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41972a0 con 0x7f2db4100dc0 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db9ad9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4199fd0 con 0x7f2db4101770 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54442/0 (socket says 172.21.15.137:54442) 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 -- 172.21.15.137:0/2063013010 learned_addr learned my addr 172.21.15.137:0/2063013010 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db2ffd700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 0x7f2db4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 -- 172.21.15.137:0/2063013010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 msgr2=0x7f2db4196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 0x7f2db4196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 -- 172.21.15.137:0/2063013010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80095f0 con 0x7f2db4100dc0 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db2ffd700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 0x7f2db4196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.847+0000 7f2db37fe700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d8d0 tx=0x7f2da400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.848+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2da4009940 con 0x7f2db4100dc0 2024-06-15T01:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.848+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2da4010460 con 0x7f2db4100dc0 2024-06-15T01:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.848+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db419a2b0 con 0x7f2db4100dc0 2024-06-15T01:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.848+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2da400f5f0 con 0x7f2db4100dc0 2024-06-15T01:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.848+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db419a7d0 con 0x7f2db4100dc0 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.849+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db41091c0 con 0x7f2db4100dc0 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.849+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2da40105d0 con 0x7f2db4100dc0 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.850+0000 7f2db0ff9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 0x7f2d9c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.850+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2da408a220 con 0x7f2db4100dc0 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.850+0000 7f2db2ffd700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 0x7f2d9c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.850+0000 7f2db2ffd700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 0x7f2d9c06d440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2db4197df0 tx=0x7f2da801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:48.854+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2da4055210 con 0x7f2db4100dc0 2024-06-15T01:11:49.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.075+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2db419a9f0 con 0x7f2d9c06af80 2024-06-15T01:11:49.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.076+0000 7f2db0ff9700 1 -- 172.21.15.137:0/2063013010 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2db419a9f0 con 0x7f2d9c06af80 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.078+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 msgr2=0x7f2d9c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 0x7f2d9c06d440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2db4197df0 tx=0x7f2da801a040 comp rx=0 tx=0).stop 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 msgr2=0x7f2db41966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d8d0 tx=0x7f2da400dc90 comp rx=0 tx=0).stop 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 shutdown_connections 2024-06-15T01:11:49.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d9c06af80 0x7f2d9c06d440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2db4100dc0 0x7f2db41966d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 --2- 172.21.15.137:0/2063013010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2db4101770 0x7f2db4196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 >> 172.21.15.137:0/2063013010 conn(0x7f2db40fc650 msgr2=0x7f2db40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 shutdown_connections 2024-06-15T01:11:49.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.079+0000 7f2db9ad9700 1 -- 172.21.15.137:0/2063013010 wait complete. 2024-06-15T01:11:49.082 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:11:49.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 -- 172.21.15.137:0/1315068803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884101790 msgr2=0x7fb8841057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1315068803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884101790 0x7fb8841057e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009910 tx=0x7fb86c009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:49.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 -- 172.21.15.137:0/1315068803 shutdown_connections 2024-06-15T01:11:49.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1315068803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884101790 0x7fb8841057e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1315068803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884100de0 0x7fb8841011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 -- 172.21.15.137:0/1315068803 >> 172.21.15.137:0/1315068803 conn(0x7fb8840fc670 msgr2=0x7fb8840fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.178 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:49 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:11:49.178 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:49 smithi137 ceph-mon[112295]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-15T01:11:49.178 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:49.178 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:49.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 -- 172.21.15.137:0/1315068803 shutdown_connections 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.185+0000 7fb8888a9700 1 -- 172.21.15.137:0/1315068803 wait complete. 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 Processor -- start 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 -- start start 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884101790 0x7fb884196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8841972a0 con 0x7fb884100de0 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb8888a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884199fd0 con 0x7fb884101790 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb88259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb88259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54464/0 (socket says 172.21.15.137:54464) 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb88259c700 1 -- 172.21.15.137:0/1667698201 learned_addr learned my addr 172.21.15.137:0/1667698201 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.186+0000 7fb879d9b700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884101790 0x7fb884196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb88259c700 1 -- 172.21.15.137:0/1667698201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884101790 msgr2=0x7fb884196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb88259c700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884101790 0x7fb884196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb88259c700 1 -- 172.21.15.137:0/1667698201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0095f0 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb88259c700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb87400d8d0 tx=0x7fb87400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb874009940 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb874010460 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88419a2b0 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87400f5f0 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.187+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88419a7d0 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.188+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8841091e0 con 0x7fb884100de0 2024-06-15T01:11:49.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.188+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb874009aa0 con 0x7fb884100de0 2024-06-15T01:11:49.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.189+0000 7fb87bfff700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 0x7fb87006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.189+0000 7fb879d9b700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 0x7fb87006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.189+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb87408a100 con 0x7fb884100de0 2024-06-15T01:11:49.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.189+0000 7fb879d9b700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 0x7fb87006d440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb884197df0 tx=0x7fb86c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.193+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb874055170 con 0x7fb884100de0 2024-06-15T01:11:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:49 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:11:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:49 smithi142 ceph-mon[82136]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-06-15T01:11:49.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:49.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:11:49.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.403+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb88419aa40 con 0x7fb87006af80 2024-06-15T01:11:49.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.404+0000 7fb87bfff700 1 -- 172.21.15.137:0/1667698201 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb88419aa40 con 0x7fb87006af80 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 msgr2=0x7fb87006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 0x7fb87006d440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb884197df0 tx=0x7fb86c01a040 comp rx=0 tx=0).stop 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 msgr2=0x7fb8841966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb87400d8d0 tx=0x7fb87400dc90 comp rx=0 tx=0).stop 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 shutdown_connections 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb87006af80 0x7fb87006d440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb884100de0 0x7fb8841966d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 --2- 172.21.15.137:0/1667698201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb884101790 0x7fb884196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 >> 172.21.15.137:0/1667698201 conn(0x7fb8840fc670 msgr2=0x7fb8840fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 shutdown_connections 2024-06-15T01:11:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.406+0000 7fb8888a9700 1 -- 172.21.15.137:0/1667698201 wait complete. 2024-06-15T01:11:49.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 -- 172.21.15.137:0/1650848675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5480ffa40 msgr2=0x7ff5480ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 --2- 172.21.15.137:0/1650848675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5480ffa40 0x7ff5480ffec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff538009910 tx=0x7ff538009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 -- 172.21.15.137:0/1650848675 shutdown_connections 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 --2- 172.21.15.137:0/1650848675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5480ffa40 0x7ff5480ffec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 --2- 172.21.15.137:0/1650848675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff548105a40 0x7ff548105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 -- 172.21.15.137:0/1650848675 >> 172.21.15.137:0/1650848675 conn(0x7ff5480fb570 msgr2=0x7ff5480fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 -- 172.21.15.137:0/1650848675 shutdown_connections 2024-06-15T01:11:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.513+0000 7ff54d525700 1 -- 172.21.15.137:0/1650848675 wait complete. 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 Processor -- start 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 -- start start 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5480ffa40 0x7ff54819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54819f1f0 con 0x7ff548105a40 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff54d525700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5481986a0 con 0x7ff5480ffa40 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff5467fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff5467fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54486/0 (socket says 172.21.15.137:54486) 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff5467fc700 1 -- 172.21.15.137:0/1410811193 learned_addr learned my addr 172.21.15.137:0/1410811193 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.514+0000 7ff546ffd700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5480ffa40 0x7ff54819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff5467fc700 1 -- 172.21.15.137:0/1410811193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5480ffa40 msgr2=0x7ff54819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff5467fc700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5480ffa40 0x7ff54819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff5467fc700 1 -- 172.21.15.137:0/1410811193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380095f0 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff5467fc700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff53800b5c0 tx=0x7ff53800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53801d070 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff53800f460 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff548198920 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff538021640 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.515+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff548198e10 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.516+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54804ea90 con 0x7ff548105a40 2024-06-15T01:11:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.517+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff53800f9b0 con 0x7ff548105a40 2024-06-15T01:11:49.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.517+0000 7ff52ffff700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 0x7ff53006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.517+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff53808b460 con 0x7ff548105a40 2024-06-15T01:11:49.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.517+0000 7ff546ffd700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 0x7ff53006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.517+0000 7ff546ffd700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 0x7ff53006d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005fd0 tx=0x7ff53c00a560 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.521+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff538059ee0 con 0x7ff548105a40 2024-06-15T01:11:49.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.722+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff548199220 con 0x7ff53006af80 2024-06-15T01:11:49.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.731+0000 7ff52ffff700 1 -- 172.21.15.137:0/1410811193 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff548199220 con 0x7ff53006af80 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (2m) 6s ago 12m 17.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (13m) 6s ago 13m 8144k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (12m) 27s ago 12m 7768k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (35s) 6s ago 13m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (30s) 27s ago 12m 37.6M - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (2m) 6s ago 12m 80.9M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (7m) 6s ago 7m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (7m) 6s ago 7m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:11:49.724 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (7m) 27s ago 7m 15.4M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (7m) 27s ago 7m 18.7M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (4m) 6s ago 14m 491M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (3m) 27s ago 11m 438M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (62s) 6s ago 14m 38.4M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (47s) 27s ago 11m 28.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (2m) 6s ago 13m 15.5M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (2m) 27s ago 11m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (11s) 6s ago 11m 11.4M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (10m) 6s ago 10m 245M 1216M 18.2.1 5be31c24972a 35d852fe7c9d 2024-06-15T01:11:49.725 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (10m) 6s ago 10m 196M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:11:49.726 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (9m) 27s ago 9m 267M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:11:49.726 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (9m) 27s ago 9m 241M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:11:49.726 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (8m) 27s ago 8m 223M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:11:49.726 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (2m) 6s ago 12m 73.3M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.733+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 msgr2=0x7ff53006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.733+0000 7ff54d525700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 0x7ff53006d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005fd0 tx=0x7ff53c00a560 comp rx=0 tx=0).stop 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.733+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 msgr2=0x7ff54819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.733+0000 7ff54d525700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff53800b5c0 tx=0x7ff53800be20 comp rx=0 tx=0).stop 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 shutdown_connections 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff53006af80 0x7ff53006d440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5480ffa40 0x7ff54819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 --2- 172.21.15.137:0/1410811193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff548105a40 0x7ff54819eb60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 >> 172.21.15.137:0/1410811193 conn(0x7ff5480fb570 msgr2=0x7ff5480fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 shutdown_connections 2024-06-15T01:11:49.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.734+0000 7ff54d525700 1 -- 172.21.15.137:0/1410811193 wait complete. 2024-06-15T01:11:49.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 -- 172.21.15.137:0/3852830660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c105b10 msgr2=0x7fb06c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 --2- 172.21.15.137:0/3852830660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c105b10 0x7fb06c105ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb060009910 tx=0x7fb060009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:49.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 -- 172.21.15.137:0/3852830660 shutdown_connections 2024-06-15T01:11:49.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 --2- 172.21.15.137:0/3852830660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c100420 0x7fb06c100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 --2- 172.21.15.137:0/3852830660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c105b10 0x7fb06c105ef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 -- 172.21.15.137:0/3852830660 >> 172.21.15.137:0/3852830660 conn(0x7fb06c074bf0 msgr2=0x7fb06c075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 -- 172.21.15.137:0/3852830660 shutdown_connections 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.836+0000 7fb071b60700 1 -- 172.21.15.137:0/3852830660 wait complete. 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 Processor -- start 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 -- start start 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c105b10 0x7fb06c195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c196160 con 0x7fb06c100420 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.837+0000 7fb071b60700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c199ef0 con 0x7fb06c105b10 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54504/0 (socket says 172.21.15.137:54504) 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 -- 172.21.15.137:0/4033237767 learned_addr learned my addr 172.21.15.137:0/4033237767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06affd700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c105b10 0x7fb06c195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 -- 172.21.15.137:0/4033237767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c105b10 msgr2=0x7fb06c195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c105b10 0x7fb06c195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 -- 172.21.15.137:0/4033237767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600095f0 con 0x7fb06c100420 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb06b7fe700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb06000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06001d070 con 0x7fb06c100420 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb06000f460 con 0x7fb06c100420 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb060021620 con 0x7fb06c100420 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c19a090 con 0x7fb06c100420 2024-06-15T01:11:49.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.838+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c19a580 con 0x7fb06c100420 2024-06-15T01:11:49.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.839+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c04ea90 con 0x7fb06c100420 2024-06-15T01:11:49.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.840+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb06000f5d0 con 0x7fb06c100420 2024-06-15T01:11:49.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.840+0000 7fb068ff9700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 0x7fb05406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:49.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.840+0000 7fb06affd700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 0x7fb05406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.840+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb06008b460 con 0x7fb06c100420 2024-06-15T01:11:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.841+0000 7fb06affd700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 0x7fb05406d440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb06c196b60 tx=0x7fb05c00a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:49.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:49.845+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb060056570 con 0x7fb06c100420 2024-06-15T01:11:50.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.115+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb06c19aae0 con 0x7fb06c100420 2024-06-15T01:11:50.108 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:50 smithi137 ceph-mon[112295]: pgmap v131: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:50.108 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:50 smithi137 ceph-mon[112295]: from='client.34146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:50.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.116+0000 7fb068ff9700 1 -- 172.21.15.137:0/4033237767 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fb060059b90 con 0x7fb06c100420 2024-06-15T01:11:50.108 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:11:50.108 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 1 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-06-15T01:11:50.109 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 5 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 msgr2=0x7fb05406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 0x7fb05406d440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb06c196b60 tx=0x7fb05c00a300 comp rx=0 tx=0).stop 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 msgr2=0x7fb06c195540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb06000bd70 comp rx=0 tx=0).stop 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 shutdown_connections 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb05406af80 0x7fb05406d440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb06c100420 0x7fb06c195540 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 --2- 172.21.15.137:0/4033237767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb06c105b10 0x7fb06c195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 >> 172.21.15.137:0/4033237767 conn(0x7fb06c074bf0 msgr2=0x7fb06c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 shutdown_connections 2024-06-15T01:11:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.118+0000 7fb071b60700 1 -- 172.21.15.137:0/4033237767 wait complete. 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.220+0000 7fa384cd7700 1 -- 172.21.15.137:0/1074495257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 msgr2=0x7fa380105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.220+0000 7fa384cd7700 1 --2- 172.21.15.137:0/1074495257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa380105870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa374009910 tx=0x7fa374009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 -- 172.21.15.137:0/1074495257 shutdown_connections 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 --2- 172.21.15.137:0/1074495257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa380105870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 --2- 172.21.15.137:0/1074495257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa3800ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 -- 172.21.15.137:0/1074495257 >> 172.21.15.137:0/1074495257 conn(0x7fa380074df0 msgr2=0x7fa380075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 -- 172.21.15.137:0/1074495257 shutdown_connections 2024-06-15T01:11:50.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 -- 172.21.15.137:0/1074495257 wait complete. 2024-06-15T01:11:50.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 Processor -- start 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.221+0000 7fa384cd7700 1 -- start start 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa38019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019d0c0 con 0x7fa3800ffee0 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380196570 con 0x7fa3800ff530 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54514/0 (socket says 172.21.15.137:54514) 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 -- 172.21.15.137:0/2805616116 learned_addr learned my addr 172.21.15.137:0/2805616116 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37e59c700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa38019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 -- 172.21.15.137:0/2805616116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 msgr2=0x7fa38019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa38019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 -- 172.21.15.137:0/2805616116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740095f0 con 0x7fa3800ffee0 2024-06-15T01:11:50.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37e59c700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa38019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa37dd9b700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa37400bd70 tx=0x7fa37400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa37401d070 con 0x7fa3800ffee0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa37400f460 con 0x7fa3800ffee0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3801967f0 con 0x7fa3800ffee0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa374021640 con 0x7fa3800ffee0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.222+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa380196ce0 con 0x7fa3800ffee0 2024-06-15T01:11:50.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.223+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa380109150 con 0x7fa3800ffee0 2024-06-15T01:11:50.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.224+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa374003a60 con 0x7fa3800ffee0 2024-06-15T01:11:50.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.224+0000 7fa36f7fe700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 0x7fa36806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.224+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa37408b300 con 0x7fa3800ffee0 2024-06-15T01:11:50.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.224+0000 7fa37e59c700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 0x7fa36806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.225+0000 7fa37e59c700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 0x7fa36806d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370005ee0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.229+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa374059e00 con 0x7fa3800ffee0 2024-06-15T01:11:50.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:50 smithi142 ceph-mon[82136]: pgmap v131: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64/327 objects degraded (19.572%) 2024-06-15T01:11:50.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:50 smithi142 ceph-mon[82136]: from='client.34146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:50.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.461+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa380197010 con 0x7fa3800ffee0 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.461+0000 7fa36f7fe700 1 -- 172.21.15.137:0/2805616116 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fa374059990 con 0x7fa3800ffee0 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:11:50.454 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:11:50.455 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:50.456 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:50.457 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:11:50.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:11:50.457 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:50.457 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 msgr2=0x7fa36806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 0x7fa36806d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370005ee0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 msgr2=0x7fa38019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa37400bd70 tx=0x7fa37400be50 comp rx=0 tx=0).stop 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 shutdown_connections 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa36806af80 0x7fa36806d440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.464+0000 7fa384cd7700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800ff530 0x7fa38019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.465+0000 7fa384cd7700 1 --2- 172.21.15.137:0/2805616116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3800ffee0 0x7fa38019ca30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.465+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 >> 172.21.15.137:0/2805616116 conn(0x7fa380074df0 msgr2=0x7fa3801050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.465+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 shutdown_connections 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.465+0000 7fa384cd7700 1 -- 172.21.15.137:0/2805616116 wait complete. 2024-06-15T01:11:50.458 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:11:50.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 -- 172.21.15.137:0/259553758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 msgr2=0x7f07740ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 --2- 172.21.15.137:0/259553758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f07740ffec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0760009910 tx=0x7f0760009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:50.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 -- 172.21.15.137:0/259553758 shutdown_connections 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 --2- 172.21.15.137:0/259553758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f07740ffec0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 --2- 172.21.15.137:0/259553758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 0x7f0774105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.571+0000 7f077afd9700 1 -- 172.21.15.137:0/259553758 >> 172.21.15.137:0/259553758 conn(0x7f07740fb530 msgr2=0x7f07740fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 -- 172.21.15.137:0/259553758 shutdown_connections 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 -- 172.21.15.137:0/259553758 wait complete. 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 Processor -- start 2024-06-15T01:11:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 -- start start 2024-06-15T01:11:50.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 0x7f0774197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774198120 con 0x7f07740ffa40 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.572+0000 7f077afd9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774198290 con 0x7f0774105a40 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54528/0 (socket says 172.21.15.137:54528) 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 -- 172.21.15.137:0/2496439163 learned_addr learned my addr 172.21.15.137:0/2496439163 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0773fff700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 0x7f0774197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 -- 172.21.15.137:0/2496439163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 msgr2=0x7f0774197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 0x7f0774197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 -- 172.21.15.137:0/2496439163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600095f0 con 0x7f07740ffa40 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0778d75700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f076800d8d0 tx=0x7f076800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07680098a0 con 0x7f07740ffa40 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0768010460 con 0x7f07740ffa40 2024-06-15T01:11:50.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07741a0570 con 0x7f07740ffa40 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0768009ea0 con 0x7f07740ffa40 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.573+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07741a0ac0 con 0x7f07740ffa40 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.574+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0774107fd0 con 0x7f07740ffa40 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.575+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f07680105d0 con 0x7f07740ffa40 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.575+0000 7f0771ffb700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 0x7f076406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.575+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f076808a090 con 0x7f07740ffa40 2024-06-15T01:11:50.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.575+0000 7f0773fff700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 0x7f076406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.576+0000 7f0773fff700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 0x7f076406d440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0774069a20 tx=0x7f076001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.579+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0768055080 con 0x7f07740ffa40 2024-06-15T01:11:50.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.791+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07740611d0 con 0x7f076406af80 2024-06-15T01:11:50.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.792+0000 7f0771ffb700 1 -- 172.21.15.137:0/2496439163 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f07740611d0 con 0x7f076406af80 2024-06-15T01:11:50.784 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "7/23 daemons upgraded", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading osd daemons", 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:11:50.785 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 msgr2=0x7f076406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 0x7f076406d440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0774069a20 tx=0x7f076001a040 comp rx=0 tx=0).stop 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 msgr2=0x7f0774197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f076800d8d0 tx=0x7f076800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 shutdown_connections 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f076406af80 0x7f076406d440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07740ffa40 0x7f0774197500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 --2- 172.21.15.137:0/2496439163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774105a40 0x7f0774197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.794+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 >> 172.21.15.137:0/2496439163 conn(0x7f07740fb530 msgr2=0x7f07740fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.795+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 shutdown_connections 2024-06-15T01:11:50.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.795+0000 7f077afd9700 1 -- 172.21.15.137:0/2496439163 wait complete. 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 -- 172.21.15.137:0/3133677225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac105a40 msgr2=0x7f3dac105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 --2- 172.21.15.137:0/3133677225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac105a40 0x7f3dac105e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009910 tx=0x7f3d9c009c20 comp rx=0 tx=0).stop 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 -- 172.21.15.137:0/3133677225 shutdown_connections 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 --2- 172.21.15.137:0/3133677225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 --2- 172.21.15.137:0/3133677225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac105a40 0x7f3dac105e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 -- 172.21.15.137:0/3133677225 >> 172.21.15.137:0/3133677225 conn(0x7f3dac0755a0 msgr2=0x7f3dac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 -- 172.21.15.137:0/3133677225 shutdown_connections 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.899+0000 7f3db23a3700 1 -- 172.21.15.137:0/3133677225 wait complete. 2024-06-15T01:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 Processor -- start 2024-06-15T01:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 -- start start 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac105a40 0x7f3dac101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac102120 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3db23a3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac102290 con 0x7f3dac105a40 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54536/0 (socket says 172.21.15.137:54536) 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 -- 172.21.15.137:0/491647805 learned_addr learned my addr 172.21.15.137:0/491647805 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dab7fe700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac105a40 0x7f3dac101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 -- 172.21.15.137:0/491647805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac105a40 msgr2=0x7f3dac101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac105a40 0x7f3dac101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.900+0000 7f3dabfff700 1 -- 172.21.15.137:0/491647805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0095f0 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3dabfff700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0092e0 tx=0x7f3d9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d9c01d070 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d9c005e00 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac1a2810 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d9c00fe60 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.901+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac1a2bc0 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.902+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac109050 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.902+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3d9c00f6d0 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.903+0000 7f3da97fa700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 0x7f3d9806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:11:50.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.903+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3d9c08b470 con 0x7f3dac0ffbd0 2024-06-15T01:11:50.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.903+0000 7f3dab7fe700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 0x7f3d9806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:11:50.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.903+0000 7f3dab7fe700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 0x7f3d9806d440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005950 tx=0x7f3d940058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:11:50.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:50.907+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3d9c056580 con 0x7f3dac0ffbd0 2024-06-15T01:11:51.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.179+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3dac02cc70 con 0x7f3dac0ffbd0 2024-06-15T01:11:51.171 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:51 smithi137 ceph-mon[112295]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:51.171 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:51 smithi137 ceph-mon[112295]: from='client.34154 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:51.171 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4033237767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:51.172 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2805616116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.179+0000 7f3da97fa700 1 -- 172.21.15.137:0/491647805 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+567 (secure 0 0 0) 0x7f3d9c00f460 con 0x7f3dac0ffbd0 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 4 mgr modules have recently crashed 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:11:51.172 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 msgr2=0x7f3d9806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 0x7f3d9806d440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005950 tx=0x7f3d940058e0 comp rx=0 tx=0).stop 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 msgr2=0x7f3dac101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0092e0 tx=0x7f3d9c00be20 comp rx=0 tx=0).stop 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 shutdown_connections 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9806af80 0x7f3d9806d440 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac0ffbd0 0x7f3dac101610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 --2- 172.21.15.137:0/491647805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac105a40 0x7f3dac101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 >> 172.21.15.137:0/491647805 conn(0x7f3dac0755a0 msgr2=0x7f3dac068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:11:51.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 shutdown_connections 2024-06-15T01:11:51.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:11:51.181+0000 7f3db23a3700 1 -- 172.21.15.137:0/491647805 wait complete. 2024-06-15T01:11:51.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:51 smithi142 ceph-mon[82136]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:51.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:51 smithi142 ceph-mon[82136]: from='client.34154 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:51.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4033237767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:11:51.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2805616116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:11:52.306 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:52 smithi137 ceph-mon[112295]: pgmap v132: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:52.306 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:52 smithi137 ceph-mon[112295]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:52.306 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:52 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 64/327 objects degraded (19.572%), 19 pgs degraded) 2024-06-15T01:11:52.306 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:52 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/491647805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:11:52.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:52 smithi142 ceph-mon[82136]: pgmap v132: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:52.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:52 smithi142 ceph-mon[82136]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:11:52.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:52 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 64/327 objects degraded (19.572%), 19 pgs degraded) 2024-06-15T01:11:52.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:52 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/491647805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:11:54.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:54 smithi137 ceph-mon[112295]: pgmap v133: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:54.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:54 smithi142 ceph-mon[82136]: pgmap v133: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:56.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:56 smithi142 ceph-mon[82136]: pgmap v134: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:56 smithi137 ceph-mon[112295]: pgmap v134: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:58.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:58 smithi142 ceph-mon[82136]: pgmap v135: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:58.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:11:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:11:58.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:58 smithi137 ceph-mon[112295]: pgmap v135: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:11:58.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:11:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:00.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:00 smithi142 ceph-mon[82136]: pgmap v136: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:00.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:00 smithi137 ceph-mon[112295]: pgmap v136: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:02.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:02 smithi137 ceph-mon[112295]: pgmap v137: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:02.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:02 smithi142 ceph-mon[82136]: pgmap v137: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:03.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:12:03.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:03.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:03.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:12:03.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:03.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: Upgrade: osd.1 is safe to restart 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: pgmap v138: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-15T01:12:04.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:04.338 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-06-15T01:12:04.338 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: Upgrade: osd.1 is safe to restart 2024-06-15T01:12:04.338 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: pgmap v138: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:04.338 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:04.338 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-15T01:12:04.339 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:05.164 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:05 smithi137 ceph-mon[112295]: Upgrade: Updating osd.1 2024-06-15T01:12:05.164 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:05 smithi137 ceph-mon[112295]: Deploying daemon osd.1 on smithi137 2024-06-15T01:12:05.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:05 smithi142 ceph-mon[82136]: Upgrade: Updating osd.1 2024-06-15T01:12:05.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:05 smithi142 ceph-mon[82136]: Deploying daemon osd.1 on smithi137 2024-06-15T01:12:06.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:06 smithi137 ceph-mon[112295]: pgmap v139: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:06.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:06 smithi142 ceph-mon[82136]: pgmap v139: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:06.855 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:06 smithi137 systemd[1]: Stopping Ceph osd.1 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:12:07.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:07 smithi137 ceph-mon[112295]: osd.1 marked itself down and dead 2024-06-15T01:12:07.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:06 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[56025]: 2024-06-15T01:12:06.862+0000 7f85c66a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:12:07.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:06 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[56025]: 2024-06-15T01:12:06.862+0000 7f85c66a4700 -1 osd.1 54 *** Got signal Terminated *** 2024-06-15T01:12:07.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:06 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[56025]: 2024-06-15T01:12:06.862+0000 7f85c66a4700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:12:07.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:07 smithi142 ceph-mon[82136]: osd.1 marked itself down and dead 2024-06-15T01:12:07.755 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:07 smithi137 podman[126247]: 2024-06-15 01:12:07.49622823 +0000 UTC m=+0.757872694 container died 35d852fe7c9dccc0baadd24c6f59a6201fe7b0f0cd24b704b0df85ccc844be33 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-15T01:12:08.043 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:07 smithi137 podman[126247]: 2024-06-15 01:12:07.863110589 +0000 UTC m=+1.124755050 container cleanup 35d852fe7c9dccc0baadd24c6f59a6201fe7b0f0cd24b704b0df85ccc844be33 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20240222) 2024-06-15T01:12:08.043 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:07 smithi137 bash[126247]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1 2024-06-15T01:12:08.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:08 smithi142 ceph-mon[82136]: pgmap v140: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:08.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:08 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:12:08.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:08 smithi142 ceph-mon[82136]: osdmap e55: 6 total, 5 up, 6 in 2024-06-15T01:12:08.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:08 smithi137 ceph-mon[112295]: pgmap v140: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:08.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:08 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:12:08.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:08 smithi137 ceph-mon[112295]: osdmap e55: 6 total, 5 up, 6 in 2024-06-15T01:12:08.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:08 smithi137 podman[126261]: 2024-06-15 01:12:08.281453445 +0000 UTC m=+0.786639418 container remove 35d852fe7c9dccc0baadd24c6f59a6201fe7b0f0cd24b704b0df85ccc844be33 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-15T01:12:08.927 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:08 smithi137 podman[126351]: 2024-06-15 01:12:08.539242727 +0000 UTC m=+0.016771257 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:08.927 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:08 smithi137 podman[126351]: 2024-06-15 01:12:08.657279457 +0000 UTC m=+0.134807978 container create c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-15T01:12:08.927 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:08 smithi137 podman[126351]: 2024-06-15 01:12:08.935134537 +0000 UTC m=+0.412663057 container init c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD) 2024-06-15T01:12:09.179 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:09 smithi137 ceph-mon[112295]: osdmap e56: 6 total, 5 up, 6 in 2024-06-15T01:12:09.179 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:08 smithi137 podman[126351]: 2024-06-15 01:12:08.939392264 +0000 UTC m=+0.416920794 container start c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:12:09.179 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 podman[126351]: 2024-06-15 01:12:09.009955229 +0000 UTC m=+0.487483759 container attach c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-15T01:12:09.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:09 smithi142 ceph-mon[82136]: osdmap e56: 6 total, 5 up, 6 in 2024-06-15T01:12:09.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 podman[126351]: 2024-06-15 01:12:09.188226325 +0000 UTC m=+0.665754849 container died c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-15T01:12:09.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 podman[126374]: 2024-06-15 01:12:09.797219838 +0000 UTC m=+0.602495073 container remove c872b636623fb2ace3af129c30651a1e1abf6bb5c09193c27efa2efebea73dbe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=) 2024-06-15T01:12:09.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.1.service: Deactivated successfully. 2024-06-15T01:12:09.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 systemd[1]: Stopped Ceph osd.1 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:12:09.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:09 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.1.service: Consumed 1min 10.919s CPU time. 2024-06-15T01:12:10.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:10 smithi142 ceph-mon[82136]: pgmap v143: 65 pgs: 12 stale+active+clean, 53 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:10.372 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-mon[112295]: pgmap v143: 65 pgs: 12 stale+active+clean, 53 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:10.372 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 systemd[1]: Starting Ceph osd.1 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:12:10.372 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.149140256 +0000 UTC m=+0.017310221 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:10.372 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.273576955 +0000 UTC m=+0.141746894 container create 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-15T01:12:10.721 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.518211221 +0000 UTC m=+0.386381180 container init 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-15T01:12:10.721 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.521882373 +0000 UTC m=+0.390052318 container start 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-15T01:12:10.721 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.627178699 +0000 UTC m=+0.495348645 container attach 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-06-15T01:12:11.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:11 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-15T01:12:11.221 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate[126489]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 bash[126475]: --> ceph-volume raw activate successful for osd ID: 1 2024-06-15T01:12:11.222 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:10 smithi137 podman[126475]: 2024-06-15 01:12:10.941267362 +0000 UTC m=+0.809437303 container died 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:12:11.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:11 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:11.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:11 smithi137 podman[126615]: 2024-06-15 01:12:11.722535609 +0000 UTC m=+0.771779854 container remove 16389e6861b6cf755939f01d6045f6c7a60636409c5fc3784028d18ec2b5e901 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:12:11.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:11 smithi137 podman[126677]: 2024-06-15 01:12:11.842579234 +0000 UTC m=+0.017968362 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:11.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:11 smithi137 podman[126677]: 2024-06-15 01:12:11.965229648 +0000 UTC m=+0.140618818 container create 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-15T01:12:12.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:12 smithi137 ceph-mon[112295]: pgmap v144: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:12.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 podman[126677]: 2024-06-15 01:12:12.251153263 +0000 UTC m=+0.426542387 container init 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2) 2024-06-15T01:12:12.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 podman[126677]: 2024-06-15 01:12:12.254227984 +0000 UTC m=+0.429617115 container start 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-15T01:12:12.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 ceph-osd[126695]: -- 172.21.15.137:0/2242331843 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628432e51e0 con 0x562844064000 2024-06-15T01:12:12.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 bash[126677]: 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e 2024-06-15T01:12:12.471 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 systemd[1]: Started Ceph osd.1 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:12:12.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:12 smithi142 ceph-mon[82136]: pgmap v144: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:12.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:12 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:12:12.808+0000 7f5ee9f36640 -1 Falling back to public interface 2024-06-15T01:12:13.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:13 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:13.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:13 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:13.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:13 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:13.649 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:13 smithi137 ceph-mon[112295]: pgmap v145: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:13.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:13 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:13.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:13 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:13.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:13 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:13.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:13 smithi142 ceph-mon[82136]: pgmap v145: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:15.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:15 smithi137 ceph-mon[112295]: pgmap v146: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:16.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:15 smithi142 ceph-mon[82136]: pgmap v146: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:17.706 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:17 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:12:17.352+0000 7f5ee9f36640 -1 osd.1 0 read_superblock omap replica is missing. 2024-06-15T01:12:17.970 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:17 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:12:17.818+0000 7f5ee9f36640 -1 osd.1 54 log_to_monitors true 2024-06-15T01:12:17.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:17 smithi137 ceph-mon[112295]: pgmap v147: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:17.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:17.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:17 smithi142 ceph-mon[82136]: pgmap v147: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:18.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.971 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:12:18.741+0000 7f5ee1ccf640 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:12:18.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-mon[112295]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-15T01:12:18.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:18.972 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:19.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:18 smithi142 ceph-mon[82136]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-15T01:12:19.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:19.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:19.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:19.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:19.867 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:19 smithi137 ceph-mon[112295]: pgmap v148: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:19.868 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:19 smithi137 ceph-mon[112295]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-15T01:12:19.868 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:19 smithi137 ceph-mon[112295]: osdmap e57: 6 total, 5 up, 6 in 2024-06-15T01:12:19.868 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:19 smithi137 ceph-mon[112295]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:12:20.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:19 smithi142 ceph-mon[82136]: pgmap v148: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:20.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:19 smithi142 ceph-mon[82136]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-15T01:12:20.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:19 smithi142 ceph-mon[82136]: osdmap e57: 6 total, 5 up, 6 in 2024-06-15T01:12:20.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:19 smithi142 ceph-mon[82136]: from='osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:12:21.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:20 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:12:21.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:20 smithi142 ceph-mon[82136]: osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669] boot 2024-06-15T01:12:21.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:20 smithi142 ceph-mon[82136]: osdmap e58: 6 total, 6 up, 6 in 2024-06-15T01:12:21.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:12:21.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:20 smithi142 ceph-mon[82136]: osdmap e59: 6 total, 6 up, 6 in 2024-06-15T01:12:21.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:20 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:12:21.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:20 smithi137 ceph-mon[112295]: osd.1 [v2:172.21.15.137:6810/1628106669,v1:172.21.15.137:6811/1628106669] boot 2024-06-15T01:12:21.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:20 smithi137 ceph-mon[112295]: osdmap e58: 6 total, 6 up, 6 in 2024-06-15T01:12:21.138 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-15T01:12:21.139 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:20 smithi137 ceph-mon[112295]: osdmap e59: 6 total, 6 up, 6 in 2024-06-15T01:12:21.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/4054543804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 msgr2=0x7fa5e0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/4054543804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0068a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009910 tx=0x7fa5dc009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/4054543804 shutdown_connections 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/4054543804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0069020 0x7fa5e0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/4054543804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0068a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/4054543804 >> 172.21.15.137:0/4054543804 conn(0x7fa5e00755a0 msgr2=0x7fa5e00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/4054543804 shutdown_connections 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/4054543804 wait complete. 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 Processor -- start 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.296+0000 7fa5e7e0d700 1 -- start start 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e7e0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e7e0d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0069020 0x7fa5e0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e7e0d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0197290 con 0x7fa5e0068670 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e7e0d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e019afb0 con 0x7fa5e0069020 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36534/0 (socket says 172.21.15.137:36534) 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 -- 172.21.15.137:0/661493235 learned_addr learned my addr 172.21.15.137:0/661493235 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 -- 172.21.15.137:0/661493235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0069020 msgr2=0x7fa5e0196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0069020 0x7fa5e0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 -- 172.21.15.137:0/661493235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0095f0 con 0x7fa5e0068670 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e5ba9700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc006010 tx=0x7fa5dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5dc01d070 con 0x7fa5e0068670 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e019b230 con 0x7fa5e0068670 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.297+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5dc00f460 con 0x7fa5e0068670 2024-06-15T01:12:21.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.298+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5dc021620 con 0x7fa5e0068670 2024-06-15T01:12:21.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.298+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e019b720 con 0x7fa5e0068670 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.299+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0109130 con 0x7fa5e0068670 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.299+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa5dc00fa40 con 0x7fa5e0068670 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.300+0000 7fa5d6ffd700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 0x7fa5cc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.300+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa5dc08b3a0 con 0x7fa5e0068670 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.300+0000 7fa5e53a8700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 0x7fa5cc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.300+0000 7fa5e53a8700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 0x7fa5cc06d440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005fd0 tx=0x7fa5d0005f60 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.308+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa5dc0563b0 con 0x7fa5e0068670 2024-06-15T01:12:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.529+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5e019bac0 con 0x7fa5cc06af80 2024-06-15T01:12:21.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.530+0000 7fa5d6ffd700 1 -- 172.21.15.137:0/661493235 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa5e019bac0 con 0x7fa5cc06af80 2024-06-15T01:12:21.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 msgr2=0x7fa5cc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 0x7fa5cc06d440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005fd0 tx=0x7fa5d0005f60 comp rx=0 tx=0).stop 2024-06-15T01:12:21.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 msgr2=0x7fa5e0196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc006010 tx=0x7fa5dc00be20 comp rx=0 tx=0).stop 2024-06-15T01:12:21.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 shutdown_connections 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa5cc06af80 0x7fa5cc06d440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5e0068670 0x7fa5e0196670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 --2- 172.21.15.137:0/661493235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0069020 0x7fa5e0196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 >> 172.21.15.137:0/661493235 conn(0x7fa5e00755a0 msgr2=0x7fa5e00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 shutdown_connections 2024-06-15T01:12:21.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.532+0000 7fa5e7e0d700 1 -- 172.21.15.137:0/661493235 wait complete. 2024-06-15T01:12:21.535 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:12:21.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.638+0000 7fcfbcd36700 1 -- 172.21.15.137:0/1241003165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 msgr2=0x7fcfb8105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.638+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/1241003165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8105e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009910 tx=0x7fcfa8009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:21.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.638+0000 7fcfbcd36700 1 -- 172.21.15.137:0/1241003165 shutdown_connections 2024-06-15T01:12:21.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.638+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/1241003165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 0x7fcfb80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.638+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/1241003165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8105e30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.639+0000 7fcfbcd36700 1 -- 172.21.15.137:0/1241003165 >> 172.21.15.137:0/1241003165 conn(0x7fcfb80fb540 msgr2=0x7fcfb80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.639+0000 7fcfbcd36700 1 -- 172.21.15.137:0/1241003165 shutdown_connections 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.639+0000 7fcfbcd36700 1 -- 172.21.15.137:0/1241003165 wait complete. 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.639+0000 7fcfbcd36700 1 Processor -- start 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.639+0000 7fcfbcd36700 1 -- start start 2024-06-15T01:12:21.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 0x7fcfb8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb8197290 con 0x7fcfb8105a50 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819b020 con 0x7fcfb80ffa50 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36562/0 (socket says 172.21.15.137:36562) 2024-06-15T01:12:21.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/3343319310 learned_addr learned my addr 172.21.15.137:0/3343319310 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb659c700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 0x7fcfb8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/3343319310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 msgr2=0x7fcfb8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 0x7fcfb8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 -- 172.21.15.137:0/3343319310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80095f0 con 0x7fcfb8105a50 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfb5d9b700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d8d0 tx=0x7fcfac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfac0098a0 con 0x7fcfb8105a50 2024-06-15T01:12:21.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfac010460 con 0x7fcfb8105a50 2024-06-15T01:12:21.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb819b300 con 0x7fcfb8105a50 2024-06-15T01:12:21.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfac009ea0 con 0x7fcfb8105a50 2024-06-15T01:12:21.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.640+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb819b850 con 0x7fcfb8105a50 2024-06-15T01:12:21.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.642+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb8109060 con 0x7fcfb8105a50 2024-06-15T01:12:21.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.642+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcfac00f6c0 con 0x7fcfb8105a50 2024-06-15T01:12:21.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.642+0000 7fcfa77fe700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 0x7fcfa006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.642+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcfac08a0c0 con 0x7fcfb8105a50 2024-06-15T01:12:21.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.642+0000 7fcfb659c700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 0x7fcfa006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.643+0000 7fcfb659c700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 0x7fcfa006d440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006010 tx=0x7fcfa800b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.646+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfac0550b0 con 0x7fcfb8105a50 2024-06-15T01:12:21.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.867+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfb819bae0 con 0x7fcfa006af80 2024-06-15T01:12:21.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.869+0000 7fcfa77fe700 1 -- 172.21.15.137:0/3343319310 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcfb819bae0 con 0x7fcfa006af80 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 msgr2=0x7fcfa006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 0x7fcfa006d440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006010 tx=0x7fcfa800b540 comp rx=0 tx=0).stop 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 msgr2=0x7fcfb8196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d8d0 tx=0x7fcfac00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 shutdown_connections 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfa006af80 0x7fcfa006d440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb80ffa50 0x7fcfb8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 --2- 172.21.15.137:0/3343319310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfb8105a50 0x7fcfb8196bb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 >> 172.21.15.137:0/3343319310 conn(0x7fcfb80fb540 msgr2=0x7fcfb8069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:21.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 shutdown_connections 2024-06-15T01:12:21.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.871+0000 7fcfbcd36700 1 -- 172.21.15.137:0/3343319310 wait complete. 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- 172.21.15.137:0/2299945284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 msgr2=0x7fefb40ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/2299945284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb40ffeb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009910 tx=0x7fefa4009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- 172.21.15.137:0/2299945284 shutdown_connections 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/2299945284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb40ffeb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/2299945284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 0x7fefb4105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- 172.21.15.137:0/2299945284 >> 172.21.15.137:0/2299945284 conn(0x7fefb40fb520 msgr2=0x7fefb40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- 172.21.15.137:0/2299945284 shutdown_connections 2024-06-15T01:12:21.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- 172.21.15.137:0/2299945284 wait complete. 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 Processor -- start 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.976+0000 7fefbbe0e700 1 -- start start 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 0x7fefb41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4198090 con 0x7fefb40ffa30 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb41981d0 con 0x7fefb4105a30 2024-06-15T01:12:21.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36580/0 (socket says 172.21.15.137:36580) 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 -- 172.21.15.137:0/3943975171 learned_addr learned my addr 172.21.15.137:0/3943975171 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb93a9700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 0x7fefb41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 -- 172.21.15.137:0/3943975171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 msgr2=0x7fefb41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 0x7fefb41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 -- 172.21.15.137:0/3943975171 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40095f0 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefb9baa700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d8d0 tx=0x7fefb000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb00098a0 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefb0010460 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb41a04e0 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb0009ea0 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.977+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb41a0a00 con 0x7fefb40ffa30 2024-06-15T01:12:21.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.978+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb4107fc0 con 0x7fefb40ffa30 2024-06-15T01:12:21.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.979+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fefb00105d0 con 0x7fefb40ffa30 2024-06-15T01:12:21.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.979+0000 7fefaaffd700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 0x7fefa006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:21.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.979+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fefb0089fc0 con 0x7fefb40ffa30 2024-06-15T01:12:21.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.979+0000 7fefb93a9700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 0x7fefa006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:21.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.980+0000 7fefb93a9700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 0x7fefa006d440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b5c0 tx=0x7fefa401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:21.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:21.983+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fefb0055030 con 0x7fefb40ffa30 2024-06-15T01:12:22.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.186+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fefb402cd80 con 0x7fefa006af80 2024-06-15T01:12:22.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.195+0000 7fefaaffd700 1 -- 172.21.15.137:0/3943975171 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fefb402cd80 con 0x7fefa006af80 2024-06-15T01:12:22.187 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (2m) 4s ago 13m 28.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (13m) 4s ago 13m 8190k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (12m) 59s ago 12m 7768k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (67s) 4s ago 13m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (62s) 59s ago 12m 37.6M - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (2m) 4s ago 13m 79.5M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (7m) 4s ago 7m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (7m) 4s ago 7m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (7m) 59s ago 7m 15.4M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (7m) 59s ago 7m 18.7M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (4m) 4s ago 15m 492M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (3m) 59s ago 12m 438M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (95s) 4s ago 15m 40.5M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (80s) 59s ago 12m 28.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:12:22.188 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (3m) 4s ago 13m 15.7M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (3m) 59s ago 12m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (43s) 4s ago 11m 92.9M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (10s) 4s ago 11m 11.5M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (10m) 4s ago 10m 196M 1216M 18.2.1 5be31c24972a 80527c938f72 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (10m) 59s ago 10m 267M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (9m) 59s ago 9m 241M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (9m) 59s ago 9m 223M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:12:22.189 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (3m) 4s ago 12m 73.3M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:12:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 msgr2=0x7fefa006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 0x7fefa006d440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b5c0 tx=0x7fefa401a040 comp rx=0 tx=0).stop 2024-06-15T01:12:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 msgr2=0x7fefb4197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d8d0 tx=0x7fefb000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 shutdown_connections 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefa006af80 0x7fefa006d440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefb40ffa30 0x7fefb4197470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 --2- 172.21.15.137:0/3943975171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefb4105a30 0x7fefb41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 >> 172.21.15.137:0/3943975171 conn(0x7fefb40fb520 msgr2=0x7fefb40fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 shutdown_connections 2024-06-15T01:12:22.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.198+0000 7fefbbe0e700 1 -- 172.21.15.137:0/3943975171 wait complete. 2024-06-15T01:12:22.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 -- 172.21.15.137:0/2198653158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744105a40 msgr2=0x7f5744105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 --2- 172.21.15.137:0/2198653158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744105a40 0x7f5744105e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f5734009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:22.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 -- 172.21.15.137:0/2198653158 shutdown_connections 2024-06-15T01:12:22.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 --2- 172.21.15.137:0/2198653158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57440ffbd0 0x7f5744100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 --2- 172.21.15.137:0/2198653158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744105a40 0x7f5744105e20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 -- 172.21.15.137:0/2198653158 >> 172.21.15.137:0/2198653158 conn(0x7f57440755a0 msgr2=0x7f57440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:22.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 -- 172.21.15.137:0/2198653158 shutdown_connections 2024-06-15T01:12:22.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.302+0000 7f5749d06700 1 -- 172.21.15.137:0/2198653158 wait complete. 2024-06-15T01:12:22.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 Processor -- start 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 -- start start 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5744105a40 0x7f5744101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744102120 con 0x7f57440ffbd0 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f5749d06700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744102290 con 0x7f5744105a40 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f57437fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f57437fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36596/0 (socket says 172.21.15.137:36596) 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f57437fe700 1 -- 172.21.15.137:0/3042769364 learned_addr learned my addr 172.21.15.137:0/3042769364 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.303+0000 7f573bfff700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5744105a40 0x7f5744101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57437fe700 1 -- 172.21.15.137:0/3042769364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5744105a40 msgr2=0x7f5744101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57437fe700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5744105a40 0x7f5744101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57437fe700 1 -- 172.21.15.137:0/3042769364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340095f0 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57437fe700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f573400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573401d070 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5734005710 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57441a2810 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573400fe20 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.304+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57441a2bc0 con 0x7f57440ffbd0 2024-06-15T01:12:22.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.305+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744109050 con 0x7f57440ffbd0 2024-06-15T01:12:22.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.305+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5734003a60 con 0x7f57440ffbd0 2024-06-15T01:12:22.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.306+0000 7f57417fa700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 0x7f573006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.306+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f573408b2b0 con 0x7f57440ffbd0 2024-06-15T01:12:22.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.306+0000 7f573bfff700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 0x7f573006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.306+0000 7f573bfff700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 0x7f573006d440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5744102e00 tx=0x7f572c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:22.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.310+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f57340562a0 con 0x7f57440ffbd0 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: pgmap v151: 65 pgs: 2 peering, 16 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:22 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:22.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: pgmap v151: 65 pgs: 2 peering, 16 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:22.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:22.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:22 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:22.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.583+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5744108a30 con 0x7f57440ffbd0 2024-06-15T01:12:22.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.583+0000 7f57417fa700 1 -- 172.21.15.137:0/3042769364 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f57340598c0 con 0x7f57440ffbd0 2024-06-15T01:12:22.575 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:12:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:12:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.585+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 msgr2=0x7f573006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.585+0000 7f5749d06700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 0x7f573006d440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5744102e00 tx=0x7f572c00b410 comp rx=0 tx=0).stop 2024-06-15T01:12:22.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.585+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 msgr2=0x7f5744101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.585+0000 7f5749d06700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f573400bd70 comp rx=0 tx=0).stop 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.585+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 shutdown_connections 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f573006af80 0x7f573006d440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57440ffbd0 0x7f5744101610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 --2- 172.21.15.137:0/3042769364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5744105a40 0x7f5744101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 >> 172.21.15.137:0/3042769364 conn(0x7f57440755a0 msgr2=0x7f5744068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 shutdown_connections 2024-06-15T01:12:22.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.586+0000 7f5749d06700 1 -- 172.21.15.137:0/3042769364 wait complete. 2024-06-15T01:12:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.689+0000 7fdc5490c700 1 -- 172.21.15.137:0/3692983952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc500ffa60 msgr2=0x7fdc500ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.689+0000 7fdc5490c700 1 --2- 172.21.15.137:0/3692983952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc500ffa60 0x7fdc500ffee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009910 tx=0x7fdc44009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 -- 172.21.15.137:0/3692983952 shutdown_connections 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 --2- 172.21.15.137:0/3692983952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc500ffa60 0x7fdc500ffee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 --2- 172.21.15.137:0/3692983952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc50105a60 0x7fdc50105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 -- 172.21.15.137:0/3692983952 >> 172.21.15.137:0/3692983952 conn(0x7fdc500fb570 msgr2=0x7fdc500fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 -- 172.21.15.137:0/3692983952 shutdown_connections 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 -- 172.21.15.137:0/3692983952 wait complete. 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 Processor -- start 2024-06-15T01:12:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.690+0000 7fdc5490c700 1 -- start start 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 0x7fdc50196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50197250 con 0x7fdc50105a60 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5019afe0 con 0x7fdc500ffa60 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36608/0 (socket says 172.21.15.137:36608) 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 -- 172.21.15.137:0/1862307581 learned_addr learned my addr 172.21.15.137:0/1862307581 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4e59c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 0x7fdc50196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 -- 172.21.15.137:0/1862307581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 msgr2=0x7fdc50196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 0x7fdc50196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 -- 172.21.15.137:0/1862307581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440095f0 con 0x7fdc50105a60 2024-06-15T01:12:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4e59c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 0x7fdc50196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc4dd9b700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc440092e0 tx=0x7fdc4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4401d070 con 0x7fdc50105a60 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5019b260 con 0x7fdc50105a60 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc44005e00 con 0x7fdc50105a60 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4400fe60 con 0x7fdc50105a60 2024-06-15T01:12:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.691+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5019b750 con 0x7fdc50105a60 2024-06-15T01:12:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.692+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc50109000 con 0x7fdc50105a60 2024-06-15T01:12:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.693+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fdc4400f6d0 con 0x7fdc50105a60 2024-06-15T01:12:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.693+0000 7fdc3f7fe700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 0x7fdc3806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.693+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdc4408b470 con 0x7fdc50105a60 2024-06-15T01:12:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.693+0000 7fdc4e59c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 0x7fdc3806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.694+0000 7fdc4e59c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 0x7fdc3806d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005950 tx=0x7fdc400058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.698+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdc44056580 con 0x7fdc50105a60 2024-06-15T01:12:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.942+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc5004ea90 con 0x7fdc50105a60 2024-06-15T01:12:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.942+0000 7fdc3f7fe700 1 -- 172.21.15.137:0/1862307581 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fdc4400f460 con 0x7fdc50105a60 2024-06-15T01:12:22.934 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:12:22.934 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:12:22.935 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:12:22.936 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:22.937 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 msgr2=0x7fdc3806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 0x7fdc3806d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005950 tx=0x7fdc400058e0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 msgr2=0x7fdc50196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc440092e0 tx=0x7fdc4400be20 comp rx=0 tx=0).stop 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 shutdown_connections 2024-06-15T01:12:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdc3806af80 0x7fdc3806d440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc500ffa60 0x7fdc50196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 --2- 172.21.15.137:0/1862307581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50105a60 0x7fdc50196b70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.944+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 >> 172.21.15.137:0/1862307581 conn(0x7fdc500fb570 msgr2=0x7fdc50069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.945+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 shutdown_connections 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:22.945+0000 7fdc5490c700 1 -- 172.21.15.137:0/1862307581 wait complete. 2024-06-15T01:12:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 -- 172.21.15.137:0/724730401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 msgr2=0x7f5c14105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/724730401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c14105e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009910 tx=0x7f5c04009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 -- 172.21.15.137:0/724730401 shutdown_connections 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/724730401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c140ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/724730401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c14105e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 -- 172.21.15.137:0/724730401 >> 172.21.15.137:0/724730401 conn(0x7f5c140fb560 msgr2=0x7f5c140fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:23.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 -- 172.21.15.137:0/724730401 shutdown_connections 2024-06-15T01:12:23.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.050+0000 7f5c1bcca700 1 -- 172.21.15.137:0/724730401 wait complete. 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 Processor -- start 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 -- start start 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c1419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1419d010 con 0x7f5c14105a70 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c1bcca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141964d0 con 0x7f5c140ffa70 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36636/0 (socket says 172.21.15.137:36636) 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 -- 172.21.15.137:0/3523870323 learned_addr learned my addr 172.21.15.137:0/3523870323 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19a66700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c1419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 -- 172.21.15.137:0/3523870323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 msgr2=0x7f5c1419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c1419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19265700 1 -- 172.21.15.137:0/3523870323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040095f0 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.051+0000 7f5c19a66700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c1419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c19265700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d8d0 tx=0x7f5c1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c10009940 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c10010460 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c141967b0 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c1000f5f0 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.052+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c14196d00 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.053+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c14109080 con 0x7f5c14105a70 2024-06-15T01:12:23.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.053+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5c100105d0 con 0x7f5c14105a70 2024-06-15T01:12:23.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.054+0000 7f5c0affd700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 0x7f5c0006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.054+0000 7f5c19a66700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 0x7f5c0006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.054+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5c1008a220 con 0x7f5c14105a70 2024-06-15T01:12:23.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.054+0000 7f5c19a66700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 0x7f5c0006d440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b5c0 tx=0x7f5c04005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:23.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.058+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c10058ca0 con 0x7f5c14105a70 2024-06-15T01:12:23.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3042769364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:23.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:23 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1862307581' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:12:23.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.268+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c14196f20 con 0x7f5c0006af80 2024-06-15T01:12:23.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.270+0000 7f5c0affd700 1 -- 172.21.15.137:0/3523870323 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5c14196f20 con 0x7f5c0006af80 2024-06-15T01:12:23.261 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:12:23.262 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:12:23.263 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:12:23.263 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "8/23 daemons upgraded", 2024-06-15T01:12:23.263 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading osd daemons", 2024-06-15T01:12:23.263 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:12:23.263 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 msgr2=0x7f5c0006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 0x7f5c0006d440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b5c0 tx=0x7f5c04005fd0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 msgr2=0x7f5c1419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d8d0 tx=0x7f5c1000dc90 comp rx=0 tx=0).stop 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 shutdown_connections 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0006af80 0x7f5c0006d440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c140ffa70 0x7f5c1419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 --2- 172.21.15.137:0/3523870323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c14105a70 0x7f5c1419c930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 >> 172.21.15.137:0/3523870323 conn(0x7f5c140fb560 msgr2=0x7f5c14068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:23.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 shutdown_connections 2024-06-15T01:12:23.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.272+0000 7f5c1bcca700 1 -- 172.21.15.137:0/3523870323 wait complete. 2024-06-15T01:12:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 -- 172.21.15.137:0/1071828206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ffee0 msgr2=0x7f95d4105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 --2- 172.21.15.137:0/1071828206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ffee0 0x7f95d4105870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009910 tx=0x7f95c4009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 -- 172.21.15.137:0/1071828206 shutdown_connections 2024-06-15T01:12:23.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 --2- 172.21.15.137:0/1071828206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ffee0 0x7f95d4105870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 --2- 172.21.15.137:0/1071828206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ff530 0x7f95d40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 -- 172.21.15.137:0/1071828206 >> 172.21.15.137:0/1071828206 conn(0x7f95d4074df0 msgr2=0x7f95d4075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:23.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 -- 172.21.15.137:0/1071828206 shutdown_connections 2024-06-15T01:12:23.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.376+0000 7f95d891b700 1 -- 172.21.15.137:0/1071828206 wait complete. 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 Processor -- start 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 -- start start 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ffee0 0x7f95d419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d419d030 con 0x7f95d40ff530 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d891b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4196570 con 0x7f95d40ffee0 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36650/0 (socket says 172.21.15.137:36650) 2024-06-15T01:12:23.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d259c700 1 -- 172.21.15.137:0/2907435024 learned_addr learned my addr 172.21.15.137:0/2907435024 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.377+0000 7f95d1d9b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ffee0 0x7f95d419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d259c700 1 -- 172.21.15.137:0/2907435024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ffee0 msgr2=0x7f95d419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d259c700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ffee0 0x7f95d419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d259c700 1 -- 172.21.15.137:0/2907435024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40095f0 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d259c700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c930 tx=0x7f95c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c8007ad0 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95c800ce80 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d4196850 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c8018630 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.378+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d4196da0 con 0x7f95d40ff530 2024-06-15T01:12:23.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.379+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d4109150 con 0x7f95d40ff530 2024-06-15T01:12:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.379+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f95c8007c30 con 0x7f95d40ff530 2024-06-15T01:12:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.380+0000 7f95c37fe700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 0x7f95bc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.380+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f95c808a1c0 con 0x7f95d40ff530 2024-06-15T01:12:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.380+0000 7f95d1d9b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 0x7f95bc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:23.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.380+0000 7f95d1d9b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 0x7f95bc06d440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d40fe120 tx=0x7f95c401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:23.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.384+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f95c8058c40 con 0x7f95d40ff530 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:23.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3042769364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:23.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:23 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1862307581' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.654+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95d4196fc0 con 0x7f95d40ff530 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.655+0000 7f95c37fe700 1 -- 172.21.15.137:0/2907435024 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1627 (secure 0 0 0) 0x7f95c80587d0 con 0x7f95d40ff530 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded; 4 mgr modules have recently crashed 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-06-15T01:12:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:12:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 msgr2=0x7f95bc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 0x7f95bc06d440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d40fe120 tx=0x7f95c401a040 comp rx=0 tx=0).stop 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 msgr2=0x7f95d419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:23.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c930 tx=0x7f95c800cc40 comp rx=0 tx=0).stop 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 shutdown_connections 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f95bc06af80 0x7f95bc06d440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95d40ff530 0x7f95d419c4f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 --2- 172.21.15.137:0/2907435024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d40ffee0 0x7f95d419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.657+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 >> 172.21.15.137:0/2907435024 conn(0x7f95d4074df0 msgr2=0x7f95d41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.658+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 shutdown_connections 2024-06-15T01:12:23.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:23.658+0000 7f95d891b700 1 -- 172.21.15.137:0/2907435024 wait complete. 2024-06-15T01:12:24.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:24 smithi137 ceph-mon[112295]: pgmap v153: 65 pgs: 2 peering, 16 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:24.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:24 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2907435024' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:12:24.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:24 smithi142 ceph-mon[82136]: pgmap v153: 65 pgs: 2 peering, 16 active+undersized, 16 active+undersized+degraded, 31 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/327 objects degraded (15.596%) 2024-06-15T01:12:24.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:24 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2907435024' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:12:25.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:25 smithi137 ceph-mon[112295]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:25.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:25 smithi142 ceph-mon[82136]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:26.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:26 smithi137 ceph-mon[112295]: pgmap v154: 65 pgs: 2 peering, 63 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:26.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:26 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded) 2024-06-15T01:12:26.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:26 smithi142 ceph-mon[82136]: pgmap v154: 65 pgs: 2 peering, 63 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:26.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:26 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/327 objects degraded (15.596%), 16 pgs degraded) 2024-06-15T01:12:28.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:28 smithi137 ceph-mon[112295]: pgmap v155: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:28.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:28 smithi142 ceph-mon[82136]: pgmap v155: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:30.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:30 smithi137 ceph-mon[112295]: pgmap v156: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:30.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:30 smithi142 ceph-mon[82136]: pgmap v156: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:32.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:32 smithi137 ceph-mon[112295]: pgmap v157: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:32.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:32 smithi142 ceph-mon[82136]: pgmap v157: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:34.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:34 smithi137 ceph-mon[112295]: pgmap v158: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:34.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:34 smithi142 ceph-mon[82136]: pgmap v158: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:34.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:36.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:36 smithi142 ceph-mon[82136]: pgmap v159: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:36.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:36 smithi137 ceph-mon[112295]: pgmap v159: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:37.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:37 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:37.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:37 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:38.256 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:38.256 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: Upgrade: osd.2 is safe to restart 2024-06-15T01:12:38.257 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: pgmap v160: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:38.257 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:38.257 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-15T01:12:38.257 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:38 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: Upgrade: osd.2 is safe to restart 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: pgmap v160: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-15T01:12:38.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:38 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:39.278 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:39 smithi137 ceph-mon[112295]: Upgrade: Updating osd.2 2024-06-15T01:12:39.278 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:39 smithi137 ceph-mon[112295]: Deploying daemon osd.2 on smithi137 2024-06-15T01:12:39.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:39 smithi142 ceph-mon[82136]: Upgrade: Updating osd.2 2024-06-15T01:12:39.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:39 smithi142 ceph-mon[82136]: Deploying daemon osd.2 on smithi137 2024-06-15T01:12:40.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:40 smithi142 ceph-mon[82136]: pgmap v161: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:40.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:40 smithi137 ceph-mon[112295]: pgmap v161: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:40.990 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:40 smithi137 systemd[1]: Stopping Ceph osd.2 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:12:41.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:41 smithi142 ceph-mon[82136]: osd.2 marked itself down and dead 2024-06-15T01:12:41.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:41 smithi137 ceph-mon[112295]: osd.2 marked itself down and dead 2024-06-15T01:12:41.471 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:40 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[63329]: 2024-06-15T01:12:40.998+0000 7fe304a1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:12:41.471 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:40 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[63329]: 2024-06-15T01:12:40.998+0000 7fe304a1f700 -1 osd.2 59 *** Got signal Terminated *** 2024-06-15T01:12:41.471 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:40 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[63329]: 2024-06-15T01:12:40.998+0000 7fe304a1f700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:12:41.863 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:41 smithi137 podman[132013]: 2024-06-15 01:12:41.539182657 +0000 UTC m=+0.650302645 container died 80527c938f727a007a6e7c3d4b5b16dd5d673cde72a8d9d7deb347a4be63851c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-15T01:12:42.165 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:42 smithi137 ceph-mon[112295]: pgmap v162: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:42.166 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:42 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:12:42.166 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:42 smithi137 ceph-mon[112295]: osdmap e60: 6 total, 5 up, 6 in 2024-06-15T01:12:42.166 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:41 smithi137 podman[132013]: 2024-06-15 01:12:41.965047797 +0000 UTC m=+1.076167788 container cleanup 80527c938f727a007a6e7c3d4b5b16dd5d673cde72a8d9d7deb347a4be63851c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-15T01:12:42.166 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:41 smithi137 bash[132013]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2 2024-06-15T01:12:42.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:42 smithi142 ceph-mon[82136]: pgmap v162: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:42.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:42 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:12:42.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:42 smithi142 ceph-mon[82136]: osdmap e60: 6 total, 5 up, 6 in 2024-06-15T01:12:42.470 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:42 smithi137 podman[132029]: 2024-06-15 01:12:42.174861786 +0000 UTC m=+0.634071605 container remove 80527c938f727a007a6e7c3d4b5b16dd5d673cde72a8d9d7deb347a4be63851c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1) 2024-06-15T01:12:42.920 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:42 smithi137 podman[132117]: 2024-06-15 01:12:42.489532819 +0000 UTC m=+0.017487862 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:42.920 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:42 smithi137 podman[132117]: 2024-06-15 01:12:42.630297799 +0000 UTC m=+0.158252834 container create d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.35.3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-15T01:12:42.920 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:42 smithi137 podman[132117]: 2024-06-15 01:12:42.928690034 +0000 UTC m=+0.456645075 container init d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-15T01:12:43.188 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:43 smithi137 ceph-mon[112295]: osdmap e61: 6 total, 5 up, 6 in 2024-06-15T01:12:43.188 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:42 smithi137 podman[132117]: 2024-06-15 01:12:42.93229249 +0000 UTC m=+0.460247531 container start d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3) 2024-06-15T01:12:43.188 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 podman[132117]: 2024-06-15 01:12:43.00363272 +0000 UTC m=+0.531587753 container attach d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528) 2024-06-15T01:12:43.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:43 smithi142 ceph-mon[82136]: osdmap e61: 6 total, 5 up, 6 in 2024-06-15T01:12:43.470 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 podman[132117]: 2024-06-15 01:12:43.197842655 +0000 UTC m=+0.725797690 container died d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-15T01:12:43.919 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 podman[132141]: 2024-06-15 01:12:43.707032113 +0000 UTC m=+0.503507564 container remove d013cbec8e9c939899e165ab219967842418065da8d6a7afa5245a5f54a6512b (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-06-15T01:12:43.919 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.2.service: Deactivated successfully. 2024-06-15T01:12:43.919 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 systemd[1]: Stopped Ceph osd.2 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:12:43.919 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.2.service: Consumed 56.886s CPU time. 2024-06-15T01:12:44.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-mon[112295]: pgmap v165: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:44.221 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:43 smithi137 systemd[1]: Starting Ceph osd.2 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:12:44.221 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.047407748 +0000 UTC m=+0.017438448 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:44.221 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.150364338 +0000 UTC m=+0.120395068 container create e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True) 2024-06-15T01:12:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:44 smithi142 ceph-mon[82136]: pgmap v165: 65 pgs: 9 stale+active+clean, 56 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:12:44.720 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.394696759 +0000 UTC m=+0.364727460 container init e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS) 2024-06-15T01:12:44.721 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.398005907 +0000 UTC m=+0.368036603 container start e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True) 2024-06-15T01:12:44.721 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.46961976 +0000 UTC m=+0.439650462 container attach e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3) 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-15T01:12:45.080 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-15T01:12:45.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-15T01:12:45.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate[132252]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-15T01:12:45.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 bash[132238]: --> ceph-volume raw activate successful for osd ID: 2 2024-06-15T01:12:45.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:44 smithi137 podman[132238]: 2024-06-15 01:12:44.814372759 +0000 UTC m=+0.784403448 container died e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-15T01:12:45.456 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:45 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:45.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:45 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:45.721 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 podman[132374]: 2024-06-15 01:12:45.465503971 +0000 UTC m=+0.645578158 container remove e71e9f1373a01d27709ba1ab56eba66c291634055268fbc5f7c03616231eb5d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:12:45.721 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 podman[132433]: 2024-06-15 01:12:45.579760553 +0000 UTC m=+0.017409377 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:12:45.721 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 podman[132433]: 2024-06-15 01:12:45.708306164 +0000 UTC m=+0.145955030 container create 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:12:46.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 podman[132433]: 2024-06-15 01:12:45.944299749 +0000 UTC m=+0.381948580 container init 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=) 2024-06-15T01:12:46.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 podman[132433]: 2024-06-15 01:12:45.947619779 +0000 UTC m=+0.385268593 container start 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3) 2024-06-15T01:12:46.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:45 smithi137 ceph-osd[132451]: -- 172.21.15.137:0/2430390075 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621a0c5f1e0 con 0x5621a19de000 2024-06-15T01:12:46.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:46 smithi137 bash[132433]: 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 2024-06-15T01:12:46.081 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:46 smithi137 systemd[1]: Started Ceph osd.2 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:12:46.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:46 smithi137 ceph-mon[112295]: pgmap v166: 65 pgs: 11 active+undersized, 3 stale+active+clean, 8 active+undersized+degraded, 43 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/327 objects degraded (8.563%) 2024-06-15T01:12:46.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:46 smithi142 ceph-mon[82136]: pgmap v166: 65 pgs: 11 active+undersized, 3 stale+active+clean, 8 active+undersized+degraded, 43 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/327 objects degraded (8.563%) 2024-06-15T01:12:46.796 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:46 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:12:46.494+0000 7f1e03c70640 -1 Falling back to public interface 2024-06-15T01:12:47.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:47.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:47.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:47.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:47.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:47.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:48.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:48 smithi137 ceph-mon[112295]: pgmap v167: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:48.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:48.344 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:48 smithi142 ceph-mon[82136]: pgmap v167: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:48.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:48.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:12:50.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:50 smithi137 ceph-mon[112295]: pgmap v168: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:50.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:50 smithi142 ceph-mon[82136]: pgmap v168: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:51.220 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:50 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:12:50.912+0000 7f1e03c70640 -1 osd.2 0 read_superblock omap replica is missing. 2024-06-15T01:12:51.720 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:51 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:12:51.352+0000 7f1e03c70640 -1 osd.2 59 log_to_monitors true 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: pgmap v169: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.305 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: pgmap v169: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-15T01:12:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:53.312 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-mon[112295]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-15T01:12:53.312 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-mon[112295]: osdmap e62: 6 total, 5 up, 6 in 2024-06-15T01:12:53.313 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-mon[112295]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:12:53.313 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 40/327 objects degraded (12.232%), 12 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:53.313 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-mon[112295]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' 2024-06-15T01:12:53.313 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:12:53 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:12:53.054+0000 7f1df7a01640 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:12:53.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:53 smithi142 ceph-mon[82136]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-15T01:12:53.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:53 smithi142 ceph-mon[82136]: osdmap e62: 6 total, 5 up, 6 in 2024-06-15T01:12:53.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:53 smithi142 ceph-mon[82136]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-06-15T01:12:53.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:53 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 40/327 objects degraded (12.232%), 12 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:53.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:53 smithi142 ceph-mon[82136]: from='osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480]' entity='osd.2' 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- 172.21.15.137:0/1682415552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0100dc0 msgr2=0x7f13b01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1682415552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0100dc0 0x7f13b01011a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009910 tx=0x7f13a4009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- 172.21.15.137:0/1682415552 shutdown_connections 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1682415552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0101770 0x7f13b01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1682415552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0100dc0 0x7f13b01011a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- 172.21.15.137:0/1682415552 >> 172.21.15.137:0/1682415552 conn(0x7f13b00fc650 msgr2=0x7f13b00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:53.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- 172.21.15.137:0/1682415552 shutdown_connections 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- 172.21.15.137:0/1682415552 wait complete. 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 Processor -- start 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.768+0000 7f13b70d5700 1 -- start start 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b70d5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0100dc0 0x7f13b0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b70d5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b70d5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0197290 con 0x7f13b0101770 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b70d5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b019b020 con 0x7f13b0100dc0 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47428/0 (socket says 172.21.15.137:47428) 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 -- 172.21.15.137:0/1505950281 learned_addr learned my addr 172.21.15.137:0/1505950281 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:53.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b4e71700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0100dc0 0x7f13b0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 -- 172.21.15.137:0/1505950281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0100dc0 msgr2=0x7f13b0196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0100dc0 0x7f13b0196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 -- 172.21.15.137:0/1505950281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40095f0 con 0x7f13b0101770 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13affff700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c8f0 tx=0x7f139c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f139c007a50 con 0x7f13b0101770 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f139c007bb0 con 0x7f13b0101770 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f139c018630 con 0x7f13b0101770 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.769+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b019b300 con 0x7f13b0101770 2024-06-15T01:12:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.770+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b019b850 con 0x7f13b0101770 2024-06-15T01:12:53.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.771+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0109160 con 0x7f13b0101770 2024-06-15T01:12:53.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.771+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f139c007d20 con 0x7f13b0101770 2024-06-15T01:12:53.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.771+0000 7f13adffb700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 0x7f13a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:53.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.771+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f139c08a1b0 con 0x7f13b0101770 2024-06-15T01:12:53.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.771+0000 7f13b4e71700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 0x7f13a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:53.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.772+0000 7f13b4e71700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 0x7f13a006d440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006010 tx=0x7f13a401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:53.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.776+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f139c0551a0 con 0x7f13b0101770 2024-06-15T01:12:53.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.998+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13b019bbc0 con 0x7f13a006af80 2024-06-15T01:12:53.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:53.999+0000 7f13adffb700 1 -- 172.21.15.137:0/1505950281 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f13b019bbc0 con 0x7f13a006af80 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.001+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 msgr2=0x7f13a006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.001+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 0x7f13a006d440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006010 tx=0x7f13a401a040 comp rx=0 tx=0).stop 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.001+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 msgr2=0x7f13b0196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.001+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c8f0 tx=0x7f139c00ccb0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 shutdown_connections 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13a006af80 0x7f13a006d440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13b0100dc0 0x7f13b0196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 --2- 172.21.15.137:0/1505950281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b0101770 0x7f13b0196bb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:53.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 >> 172.21.15.137:0/1505950281 conn(0x7f13b00fc650 msgr2=0x7f13b00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:53.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 shutdown_connections 2024-06-15T01:12:53.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.002+0000 7f13b70d5700 1 -- 172.21.15.137:0/1505950281 wait complete. 2024-06-15T01:12:54.004 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 -- 172.21.15.137:0/2318800273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff618105a40 msgr2=0x7ff618105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/2318800273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff618105a40 0x7ff618105e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff608009910 tx=0x7ff608009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 -- 172.21.15.137:0/2318800273 shutdown_connections 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/2318800273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6180ffa40 0x7ff6180ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/2318800273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff618105a40 0x7ff618105e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 -- 172.21.15.137:0/2318800273 >> 172.21.15.137:0/2318800273 conn(0x7ff6180fb570 msgr2=0x7ff6180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 -- 172.21.15.137:0/2318800273 shutdown_connections 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.125+0000 7ff61d1c0700 1 -- 172.21.15.137:0/2318800273 wait complete. 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 Processor -- start 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 -- start start 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff618105a40 0x7ff618196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618197290 con 0x7ff6180ffa40 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61d1c0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819b020 con 0x7ff618105a40 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47452/0 (socket says 172.21.15.137:47452) 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 -- 172.21.15.137:0/208002593 learned_addr learned my addr 172.21.15.137:0/208002593 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff61659c700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff618105a40 0x7ff618196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 -- 172.21.15.137:0/208002593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff618105a40 msgr2=0x7ff618196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff618105a40 0x7ff618196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.126+0000 7ff616d9d700 1 -- 172.21.15.137:0/208002593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff6180ffa40 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff616d9d700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff60800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60801d070 con 0x7ff6180ffa40 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61819b2a0 con 0x7ff6180ffa40 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff608005940 con 0x7ff6180ffa40 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60800fd90 con 0x7ff6180ffa40 2024-06-15T01:12:54.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.127+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61819b790 con 0x7ff6180ffa40 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.128+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618109050 con 0x7ff6180ffa40 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.129+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff608003a60 con 0x7ff6180ffa40 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.129+0000 7ff5fffff700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 0x7ff60006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.129+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff60808b370 con 0x7ff6180ffa40 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.129+0000 7ff61659c700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 0x7ff60006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.130+0000 7ff61659c700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 0x7ff60006d440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff618197c90 tx=0x7ff60c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.134+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff608056480 con 0x7ff6180ffa40 2024-06-15T01:12:54.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:54 smithi137 ceph-mon[112295]: pgmap v171: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:54.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:54 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:12:54.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:54 smithi137 ceph-mon[112295]: osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480] boot 2024-06-15T01:12:54.368 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:54 smithi137 ceph-mon[112295]: osdmap e63: 6 total, 6 up, 6 in 2024-06-15T01:12:54.369 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:54 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:12:54.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.375+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff61802cd30 con 0x7ff60006af80 2024-06-15T01:12:54.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.377+0000 7ff5fffff700 1 -- 172.21.15.137:0/208002593 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff61802cd30 con 0x7ff60006af80 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 msgr2=0x7ff60006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 0x7ff60006d440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff618197c90 tx=0x7ff60c00b410 comp rx=0 tx=0).stop 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 msgr2=0x7ff618196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff60800bd70 comp rx=0 tx=0).stop 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 shutdown_connections 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff60006af80 0x7ff60006d440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6180ffa40 0x7ff618196670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 --2- 172.21.15.137:0/208002593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff618105a40 0x7ff618196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 >> 172.21.15.137:0/208002593 conn(0x7ff6180fb570 msgr2=0x7ff618068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:54.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 shutdown_connections 2024-06-15T01:12:54.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.379+0000 7ff61d1c0700 1 -- 172.21.15.137:0/208002593 wait complete. 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 -- 172.21.15.137:0/1669869439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 msgr2=0x7f8f940ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 --2- 172.21.15.137:0/1669869439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940ffed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009910 tx=0x7f8f88009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 -- 172.21.15.137:0/1669869439 shutdown_connections 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 --2- 172.21.15.137:0/1669869439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940ffed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 --2- 172.21.15.137:0/1669869439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 0x7f8f94105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 -- 172.21.15.137:0/1669869439 >> 172.21.15.137:0/1669869439 conn(0x7f8f940fb540 msgr2=0x7f8f940fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 -- 172.21.15.137:0/1669869439 shutdown_connections 2024-06-15T01:12:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.486+0000 7f8f99146700 1 -- 172.21.15.137:0/1669869439 wait complete. 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 Processor -- start 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 -- start start 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 0x7f8f940fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940fc9a0 con 0x7f8f940ffa50 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f99146700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940fcb10 con 0x7f8f94105a50 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f92d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f92d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47472/0 (socket says 172.21.15.137:47472) 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.487+0000 7f8f92d9d700 1 -- 172.21.15.137:0/1982516450 learned_addr learned my addr 172.21.15.137:0/1982516450 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f9259c700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 0x7f8f940fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f92d9d700 1 -- 172.21.15.137:0/1982516450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 msgr2=0x7f8f940fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f92d9d700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 0x7f8f940fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f92d9d700 1 -- 172.21.15.137:0/1982516450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880095f0 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f92d9d700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d8d0 tx=0x7f8f8400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f840098a0 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f941a2820 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f84010460 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f84009ea0 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.488+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f941a2c30 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.489+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94108ff0 con 0x7f8f940ffa50 2024-06-15T01:12:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.490+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8f84009a00 con 0x7f8f940ffa50 2024-06-15T01:12:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.490+0000 7f8f7bfff700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 0x7f8f7c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.491+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8f8408a0a0 con 0x7f8f940ffa50 2024-06-15T01:12:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.491+0000 7f8f9259c700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 0x7f8f7c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.491+0000 7f8f9259c700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 0x7f8f7c06d440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b5c0 tx=0x7f8f8801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.495+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8f84055090 con 0x7f8f940ffa50 2024-06-15T01:12:54.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:54 smithi142 ceph-mon[82136]: pgmap v171: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:54.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:54 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:12:54.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:54 smithi142 ceph-mon[82136]: osd.2 [v2:172.21.15.137:6818/3034759480,v1:172.21.15.137:6819/3034759480] boot 2024-06-15T01:12:54.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:54 smithi142 ceph-mon[82136]: osdmap e63: 6 total, 6 up, 6 in 2024-06-15T01:12:54.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:54 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-15T01:12:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.710+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f9402cce0 con 0x7f8f7c06af80 2024-06-15T01:12:54.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.720+0000 7f8f7bfff700 1 -- 172.21.15.137:0/1982516450 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8f9402cce0 con 0x7f8f7c06af80 2024-06-15T01:12:54.711 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:12:54.711 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (3m) 3s ago 14m 28.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (14m) 3s ago 14m 8238k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (13m) 92s ago 13m 7768k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (100s) 3s ago 14m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (95s) 92s ago 13m 37.6M - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (3m) 3s ago 13m 81.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (8m) 3s ago 8m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (8m) 3s ago 8m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (8m) 92s ago 8m 15.4M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (8m) 92s ago 8m 18.7M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (5m) 3s ago 15m 496M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (4m) 92s ago 12m 438M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (2m) 3s ago 15m 43.6M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (112s) 92s ago 12m 28.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (4m) 3s ago 14m 15.7M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:12:54.712 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (3m) 92s ago 13m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (76s) 3s ago 12m 93.2M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (42s) 3s ago 11m 94.6M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (9s) 3s ago 11m 14.5M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (10m) 92s ago 10m 267M 2240M 18.2.1 5be31c24972a 722a17c0092e 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (10m) 92s ago 10m 241M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (9m) 92s ago 9m 223M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:12:54.713 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (3m) 3s ago 13m 73.4M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 msgr2=0x7f8f7c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 0x7f8f7c06d440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b5c0 tx=0x7f8f8801a040 comp rx=0 tx=0).stop 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 msgr2=0x7f8f940fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d8d0 tx=0x7f8f8400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 shutdown_connections 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f7c06af80 0x7f8f7c06d440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f940ffa50 0x7f8f940fbe90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 --2- 172.21.15.137:0/1982516450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f94105a50 0x7f8f940fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.722+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 >> 172.21.15.137:0/1982516450 conn(0x7f8f940fb540 msgr2=0x7f8f94069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:54.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.723+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 shutdown_connections 2024-06-15T01:12:54.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.723+0000 7f8f99146700 1 -- 172.21.15.137:0/1982516450 wait complete. 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 -- 172.21.15.137:0/2187985618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ffdb0 msgr2=0x7f0fec103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/2187985618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec103e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009910 tx=0x7f0fdc009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 -- 172.21.15.137:0/2187985618 shutdown_connections 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/2187985618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec103e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/2187985618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ff400 0x7f0fec0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 -- 172.21.15.137:0/2187985618 >> 172.21.15.137:0/2187985618 conn(0x7f0fec075e30 msgr2=0x7f0fec0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:54.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 -- 172.21.15.137:0/2187985618 shutdown_connections 2024-06-15T01:12:54.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.830+0000 7f0ff44a7700 1 -- 172.21.15.137:0/2187985618 wait complete. 2024-06-15T01:12:54.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 Processor -- start 2024-06-15T01:12:54.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 -- start start 2024-06-15T01:12:54.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec19f200 con 0x7f0fec0ff400 2024-06-15T01:12:54.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff44a7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec1986b0 con 0x7f0fec0ffdb0 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47488/0 (socket says 172.21.15.137:47488) 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 -- 172.21.15.137:0/672928948 learned_addr learned my addr 172.21.15.137:0/672928948 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff1a42700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 -- 172.21.15.137:0/672928948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ffdb0 msgr2=0x7f0fec19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 -- 172.21.15.137:0/672928948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0095f0 con 0x7f0fec0ff400 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.831+0000 7f0ff2243700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800ea30 tx=0x7f0fe800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.832+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe800cca0 con 0x7f0fec0ff400 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.832+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fe800ce00 con 0x7f0fec0ff400 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.832+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec198990 con 0x7f0fec0ff400 2024-06-15T01:12:54.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.832+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe80156d0 con 0x7f0fec0ff400 2024-06-15T01:12:54.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.832+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec198eb0 con 0x7f0fec0ff400 2024-06-15T01:12:54.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.833+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec107800 con 0x7f0fec0ff400 2024-06-15T01:12:54.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.833+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0fe80163e0 con 0x7f0fec0ff400 2024-06-15T01:12:54.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.833+0000 7f0fe37fe700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 0x7f0fd806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:54.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.834+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0fe808a580 con 0x7f0fec0ff400 2024-06-15T01:12:54.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.834+0000 7f0ff1a42700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 0x7f0fd806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:54.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.834+0000 7f0ff1a42700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 0x7f0fd806d440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009fd0 tx=0x7f0fdc005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:54.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:54.838+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0fe8059080 con 0x7f0fec0ff400 2024-06-15T01:12:55.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.140+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0fec04ea90 con 0x7f0fec0ff400 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.140+0000 7f0fe37fe700 1 -- 172.21.15.137:0/672928948 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f0fe8058c10 con 0x7f0fec0ff400 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:12:55.132 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 3 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 7 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:12:55.133 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 msgr2=0x7f0fd806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 0x7f0fd806d440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009fd0 tx=0x7f0fdc005a90 comp rx=0 tx=0).stop 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 msgr2=0x7f0fec19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800ea30 tx=0x7f0fe800ed40 comp rx=0 tx=0).stop 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 shutdown_connections 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0fd806af80 0x7f0fd806d440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fec0ff400 0x7f0fec19e630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 --2- 172.21.15.137:0/672928948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0ffdb0 0x7f0fec19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 >> 172.21.15.137:0/672928948 conn(0x7f0fec075e30 msgr2=0x7f0fec103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 shutdown_connections 2024-06-15T01:12:55.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.143+0000 7f0ff44a7700 1 -- 172.21.15.137:0/672928948 wait complete. 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 -- 172.21.15.137:0/2556693529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 msgr2=0x7f1f200ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 --2- 172.21.15.137:0/2556693529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f200ffec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009910 tx=0x7f1f1c009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 -- 172.21.15.137:0/2556693529 shutdown_connections 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 --2- 172.21.15.137:0/2556693529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f200ffec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 --2- 172.21.15.137:0/2556693529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 0x7f1f20105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 -- 172.21.15.137:0/2556693529 >> 172.21.15.137:0/2556693529 conn(0x7f1f200fb550 msgr2=0x7f1f200fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 -- 172.21.15.137:0/2556693529 shutdown_connections 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.249+0000 7f1f282c0700 1 -- 172.21.15.137:0/2556693529 wait complete. 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 Processor -- start 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 -- start start 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 0x7f1f20197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20198140 con 0x7f1f200ffa40 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f282c0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f201982b0 con 0x7f1f20105a40 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f2605c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.250+0000 7f1f2605c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47502/0 (socket says 172.21.15.137:47502) 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2605c700 1 -- 172.21.15.137:0/430290437 learned_addr learned my addr 172.21.15.137:0/430290437 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:55.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2585b700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 0x7f1f20197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2605c700 1 -- 172.21.15.137:0/430290437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 msgr2=0x7f1f20197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2605c700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 0x7f1f20197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2605c700 1 -- 172.21.15.137:0/430290437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0095f0 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f2605c700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c8f0 tx=0x7f1f1000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f10004a00 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f10005020 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.251+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f201a0590 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.252+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f10007c40 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.252+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f201a0ae0 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.253+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20107fd0 con 0x7f1f200ffa40 2024-06-15T01:12:55.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.253+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f1f1000f450 con 0x7f1f200ffa40 2024-06-15T01:12:55.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.253+0000 7f1f177fe700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 0x7f1f0c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.254+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1f10013070 con 0x7f1f200ffa40 2024-06-15T01:12:55.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.254+0000 7f1f2585b700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 0x7f1f0c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.254+0000 7f1f2585b700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 0x7f1f0c06d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f20069660 tx=0x7f1f1c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.260+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1f10055000 con 0x7f1f200ffa40 2024-06-15T01:12:55.401 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:55 smithi137 ceph-mon[112295]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:55.401 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:55 smithi137 ceph-mon[112295]: osdmap e64: 6 total, 6 up, 6 in 2024-06-15T01:12:55.401 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/672928948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.502+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f2002cc70 con 0x7f1f200ffa40 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.502+0000 7f1f177fe700 1 -- 172.21.15.137:0/430290437 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f1f10058620 con 0x7f1f200ffa40 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:55.494 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:12:55.495 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:12:55.496 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:55.497 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 msgr2=0x7f1f0c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 0x7f1f0c06d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f20069660 tx=0x7f1f1c01a040 comp rx=0 tx=0).stop 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 msgr2=0x7f1f20197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c8f0 tx=0x7f1f1000cc00 comp rx=0 tx=0).stop 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 shutdown_connections 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1f0c06af80 0x7f1f0c06d440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f200ffa40 0x7f1f20197520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 --2- 172.21.15.137:0/430290437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f20105a40 0x7f1f20197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.504+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 >> 172.21.15.137:0/430290437 conn(0x7f1f200fb550 msgr2=0x7f1f200fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.505+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 shutdown_connections 2024-06-15T01:12:55.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.505+0000 7f1f282c0700 1 -- 172.21.15.137:0/430290437 wait complete. 2024-06-15T01:12:55.499 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:12:55.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:55 smithi142 ceph-mon[82136]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:55.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:55 smithi142 ceph-mon[82136]: osdmap e64: 6 total, 6 up, 6 in 2024-06-15T01:12:55.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/672928948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:55.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1170797170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 msgr2=0x7f5fe0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1170797170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0105e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009910 tx=0x7f5fdc009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1170797170 shutdown_connections 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1170797170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 0x7f5fe00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1170797170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0105e20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1170797170 >> 172.21.15.137:0/1170797170 conn(0x7f5fe00fb570 msgr2=0x7f5fe00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1170797170 shutdown_connections 2024-06-15T01:12:55.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.609+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1170797170 wait complete. 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 Processor -- start 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 -- start start 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 0x7f5fe0195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01961a0 con 0x7f5fe0105a40 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe7eb9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe0199f30 con 0x7f5fe00ffa40 2024-06-15T01:12:55.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47524/0 (socket says 172.21.15.137:47524) 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 -- 172.21.15.137:0/1080642710 learned_addr learned my addr 172.21.15.137:0/1080642710 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5c55700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 0x7f5fe0195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 -- 172.21.15.137:0/1080642710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 msgr2=0x7f5fe0195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 0x7f5fe0195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 -- 172.21.15.137:0/1080642710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0095f0 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.610+0000 7f5fe5454700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0007910 tx=0x7f5fd0007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.611+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fd0012070 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.611+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fd000d810 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.611+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe019a1b0 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.611+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fd00053d0 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.611+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe019a700 con 0x7f5fe0105a40 2024-06-15T01:12:55.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.612+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe004ea90 con 0x7f5fe0105a40 2024-06-15T01:12:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.612+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5fd0016070 con 0x7f5fe0105a40 2024-06-15T01:12:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.613+0000 7f5fd6ffd700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 0x7f5fcc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.613+0000 7f5fe5c55700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 0x7f5fcc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.613+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5fd008b460 con 0x7f5fe0105a40 2024-06-15T01:12:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.613+0000 7f5fe5c55700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 0x7f5fcc06d440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00b5c0 tx=0x7f5fdc005fb0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.617+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5fd0056450 con 0x7f5fe0105a40 2024-06-15T01:12:55.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.826+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fe019ab20 con 0x7f5fcc06af80 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.828+0000 7f5fd6ffd700 1 -- 172.21.15.137:0/1080642710 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5fe019ab20 con 0x7f5fcc06af80 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:12:55.819 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "9/23 daemons upgraded", 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading osd daemons", 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:12:55.820 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:12:55.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 msgr2=0x7f5fcc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 0x7f5fcc06d440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00b5c0 tx=0x7f5fdc005fb0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 msgr2=0x7f5fe0195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0007910 tx=0x7f5fd0007cd0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 shutdown_connections 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fcc06af80 0x7f5fcc06d440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe00ffa40 0x7f5fe0195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 --2- 172.21.15.137:0/1080642710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fe0105a40 0x7f5fe0195ac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 >> 172.21.15.137:0/1080642710 conn(0x7f5fe00fb570 msgr2=0x7f5fe00fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 shutdown_connections 2024-06-15T01:12:55.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.830+0000 7f5fe7eb9700 1 -- 172.21.15.137:0/1080642710 wait complete. 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 -- 172.21.15.137:0/3201592066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4101750 msgr2=0x7f48d41057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 --2- 172.21.15.137:0/3201592066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4101750 0x7f48d41057a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009910 tx=0x7f48c0009c20 comp rx=0 tx=0).stop 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 -- 172.21.15.137:0/3201592066 shutdown_connections 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 --2- 172.21.15.137:0/3201592066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4101750 0x7f48d41057a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 --2- 172.21.15.137:0/3201592066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4100da0 0x7f48d4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 -- 172.21.15.137:0/3201592066 >> 172.21.15.137:0/3201592066 conn(0x7f48d40fc650 msgr2=0x7f48d40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 -- 172.21.15.137:0/3201592066 shutdown_connections 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 -- 172.21.15.137:0/3201592066 wait complete. 2024-06-15T01:12:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.934+0000 7f48dadca700 1 Processor -- start 2024-06-15T01:12:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48dadca700 1 -- start start 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48dadca700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48dadca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4101750 0x7f48d4196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48dadca700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4197290 con 0x7f48d4100da0 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48dadca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d419b020 con 0x7f48d4101750 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47538/0 (socket says 172.21.15.137:47538) 2024-06-15T01:12:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 -- 172.21.15.137:0/548324618 learned_addr learned my addr 172.21.15.137:0/548324618 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d3fff700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4101750 0x7f48d4196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 -- 172.21.15.137:0/548324618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4101750 msgr2=0x7f48d4196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4101750 0x7f48d4196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 -- 172.21.15.137:0/548324618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00095f0 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d8b66700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c8a0 tx=0x7f48c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.935+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c800cea0 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.936+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d419b300 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.936+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48c8004830 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.936+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c8005500 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.936+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d419b850 con 0x7f48d4100da0 2024-06-15T01:12:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.937+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d41091a0 con 0x7f48d4100da0 2024-06-15T01:12:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.937+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f48c80049a0 con 0x7f48d4100da0 2024-06-15T01:12:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.937+0000 7f48d1ffb700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 0x7f48c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:12:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.938+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f48c8089ee0 con 0x7f48d4100da0 2024-06-15T01:12:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.938+0000 7f48d3fff700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 0x7f48c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:12:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.938+0000 7f48d3fff700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 0x7f48c406d440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b5c0 tx=0x7f48c0005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:12:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:55.942+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f48c8054ed0 con 0x7f48d4100da0 2024-06-15T01:12:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.211+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f48d419bbe0 con 0x7f48d4100da0 2024-06-15T01:12:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.211+0000 7f48d1ffb700 1 -- 172.21.15.137:0/548324618 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1408 (secure 0 0 0) 0x7f48c801d090 con 0x7f48d4100da0 2024-06-15T01:12:56.203 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN Degraded data redundancy: 40/327 objects degraded (12.232%), 12 pgs degraded; 4 mgr modules have recently crashed 2024-06-15T01:12:56.203 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 40/327 objects degraded (12.232%), 12 pgs degraded 2024-06-15T01:12:56.203 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-06-15T01:12:56.203 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:12:56.204 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:12:56.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:12:56.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:12:56.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 msgr2=0x7f48c406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 0x7f48c406d440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b5c0 tx=0x7f48c0005fd0 comp rx=0 tx=0).stop 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 msgr2=0x7f48d4196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c8a0 tx=0x7f48c800cc60 comp rx=0 tx=0).stop 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 shutdown_connections 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f48c406af80 0x7f48c406d440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48d4100da0 0x7f48d4196670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 --2- 172.21.15.137:0/548324618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48d4101750 0x7f48d4196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 >> 172.21.15.137:0/548324618 conn(0x7f48d40fc650 msgr2=0x7f48d40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 shutdown_connections 2024-06-15T01:12:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:12:56.214+0000 7f48dadca700 1 -- 172.21.15.137:0/548324618 wait complete. 2024-06-15T01:12:56.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:56.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: pgmap v173: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:56.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:12:56.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/430290437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:12:56.472 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:56 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/548324618' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: pgmap v173: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/327 objects degraded (12.232%) 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:12:56.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:12:56.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/430290437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:12:56.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:56 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/548324618' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:12:57.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:57 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:12:57.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:57 smithi142 ceph-mon[82136]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-06-15T01:12:57.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:57 smithi142 ceph-mon[82136]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:57.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:57 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:12:57.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:57 smithi137 ceph-mon[112295]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-06-15T01:12:57.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:57 smithi137 ceph-mon[112295]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:12:58.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:58 smithi137 ceph-mon[112295]: pgmap v175: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 32/327 objects degraded (9.786%) 2024-06-15T01:12:58.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:12:58 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 32/327 objects degraded (9.786%), 10 pgs degraded (PG_DEGRADED) 2024-06-15T01:12:58.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:58 smithi142 ceph-mon[82136]: pgmap v175: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 32/327 objects degraded (9.786%) 2024-06-15T01:12:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:12:58 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 32/327 objects degraded (9.786%), 10 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:00.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:00 smithi142 ceph-mon[82136]: pgmap v176: 65 pgs: 5 peering, 7 active+undersized, 6 active+undersized+degraded, 47 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18/327 objects degraded (5.505%) 2024-06-15T01:13:00.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:00 smithi137 ceph-mon[112295]: pgmap v176: 65 pgs: 5 peering, 7 active+undersized, 6 active+undersized+degraded, 47 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18/327 objects degraded (5.505%) 2024-06-15T01:13:01.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:01 smithi142 ceph-mon[82136]: pgmap v177: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:01.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:01 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/327 objects degraded (5.505%), 6 pgs degraded) 2024-06-15T01:13:01.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:01 smithi137 ceph-mon[112295]: pgmap v177: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:01.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:01 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/327 objects degraded (5.505%), 6 pgs degraded) 2024-06-15T01:13:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:04 smithi137 ceph-mon[112295]: pgmap v178: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:04.266 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:04 smithi142 ceph-mon[82136]: pgmap v178: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:04.266 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:04.266 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:06.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:06 smithi142 ceph-mon[82136]: pgmap v179: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:06.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:06 smithi137 ceph-mon[112295]: pgmap v179: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:08.292 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:08 smithi137 ceph-mon[112295]: pgmap v180: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:08.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:08 smithi142 ceph-mon[82136]: pgmap v180: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:09.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:09 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:09.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:09 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:10.297 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:10 smithi142 ceph-mon[82136]: pgmap v181: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:10.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:10 smithi137 ceph-mon[112295]: pgmap v181: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:11.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:11 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:13:11.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:11 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: Upgrade: osd.3 is safe to restart 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: pgmap v182: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-15T01:13:12.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:12.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-06-15T01:13:12.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: Upgrade: osd.3 is safe to restart 2024-06-15T01:13:12.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: pgmap v182: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:12.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:12.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-15T01:13:12.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:13.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:13 smithi142 ceph-mon[82136]: Upgrade: Updating osd.3 2024-06-15T01:13:13.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:13 smithi142 ceph-mon[82136]: Deploying daemon osd.3 on smithi142 2024-06-15T01:13:13.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:13 smithi137 ceph-mon[112295]: Upgrade: Updating osd.3 2024-06-15T01:13:13.345 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:13 smithi137 ceph-mon[112295]: Deploying daemon osd.3 on smithi142 2024-06-15T01:13:14.286 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:14 smithi142 ceph-mon[82136]: pgmap v183: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:14.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:14 smithi137 ceph-mon[112295]: pgmap v183: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:14.899 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:14 smithi142 systemd[1]: Stopping Ceph osd.3 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:13:15.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:15 smithi142 ceph-mon[82136]: osd.3 marked itself down and dead 2024-06-15T01:13:15.312 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:14 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[43427]: 2024-06-15T01:13:14.907+0000 7f91771fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:13:15.313 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:14 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[43427]: 2024-06-15T01:13:14.907+0000 7f91771fd700 -1 osd.3 64 *** Got signal Terminated *** 2024-06-15T01:13:15.313 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:14 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[43427]: 2024-06-15T01:13:14.907+0000 7f91771fd700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:13:15.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:15 smithi137 ceph-mon[112295]: osd.3 marked itself down and dead 2024-06-15T01:13:15.690 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:15 smithi142 podman[89984]: 2024-06-15 01:13:15.438792668 +0000 UTC m=+0.642286788 container died 722a17c0092ea514fd62b8f6344af4e09bf26a3bd135fe3f82add3fc9a5c93dd (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-06-15T01:13:16.043 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:15 smithi142 podman[89984]: 2024-06-15 01:13:15.807851557 +0000 UTC m=+1.011345705 container cleanup 722a17c0092ea514fd62b8f6344af4e09bf26a3bd135fe3f82add3fc9a5c93dd (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-06-15T01:13:16.043 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:15 smithi142 bash[89984]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3 2024-06-15T01:13:16.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:16 smithi142 ceph-mon[82136]: pgmap v184: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:16.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:16 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:13:16.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:16 smithi142 ceph-mon[82136]: osdmap e65: 6 total, 5 up, 6 in 2024-06-15T01:13:16.313 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[89998]: 2024-06-15 01:13:16.100831107 +0000 UTC m=+0.658708926 container remove 722a17c0092ea514fd62b8f6344af4e09bf26a3bd135fe3f82add3fc9a5c93dd (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0) 2024-06-15T01:13:16.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:16 smithi137 ceph-mon[112295]: pgmap v184: 65 pgs: 65 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:16.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:16 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:13:16.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:16 smithi137 ceph-mon[112295]: osdmap e65: 6 total, 5 up, 6 in 2024-06-15T01:13:16.736 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.330597651 +0000 UTC m=+0.017807243 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:16.736 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.443353557 +0000 UTC m=+0.130563131 container create 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-15T01:13:16.736 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.671543869 +0000 UTC m=+0.358753459 container init 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:13:16.736 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.675007782 +0000 UTC m=+0.362217365 container start 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, io.buildah.version=1.35.3) 2024-06-15T01:13:16.736 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.745843786 +0000 UTC m=+0.433053360 container attach 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-15T01:13:17.052 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:16 smithi142 podman[90087]: 2024-06-15 01:13:16.927043028 +0000 UTC m=+0.614252601 container died 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:13:17.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:17 smithi142 ceph-mon[82136]: osdmap e66: 6 total, 5 up, 6 in 2024-06-15T01:13:17.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:17 smithi137 ceph-mon[112295]: osdmap e66: 6 total, 5 up, 6 in 2024-06-15T01:13:17.812 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 podman[90110]: 2024-06-15 01:13:17.466088365 +0000 UTC m=+0.533562785 container remove 14eef2e55d22c59f117f2fc54809004c78c6ed40633b3572e69edac7e66fbf63 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-15T01:13:17.812 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-15T01:13:17.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 systemd[1]: Stopped Ceph osd.3 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:13:17.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.3.service: Consumed 1min 15.418s CPU time. 2024-06-15T01:13:17.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 systemd[1]: Starting Ceph osd.3 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:13:18.168 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-mon[82136]: pgmap v187: 65 pgs: 16 stale+active+clean, 49 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:18.169 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:18.169 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:18.169 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 podman[90214]: 2024-06-15 01:13:17.813148708 +0000 UTC m=+0.017365595 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:18.169 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:17 smithi142 podman[90214]: 2024-06-15 01:13:17.967529672 +0000 UTC m=+0.171746546 container create 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-15T01:13:18.169 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 podman[90214]: 2024-06-15 01:13:18.178280285 +0000 UTC m=+0.382497170 container init 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:13:18.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:18 smithi137 ceph-mon[112295]: pgmap v187: 65 pgs: 16 stale+active+clean, 49 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:18.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:18.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:18.540 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 podman[90214]: 2024-06-15 01:13:18.181618414 +0000 UTC m=+0.385835291 container start 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-15T01:13:18.540 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 podman[90214]: 2024-06-15 01:13:18.261514161 +0000 UTC m=+0.465731045 container attach 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:13:18.812 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-15T01:13:18.812 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate[90229]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 bash[90214]: --> ceph-volume raw activate successful for osd ID: 3 2024-06-15T01:13:18.813 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:18 smithi142 podman[90214]: 2024-06-15 01:13:18.602164391 +0000 UTC m=+0.806381268 container died 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:13:19.180 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:19 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 2/327 objects degraded (0.612%), 1 pg degraded (PG_DEGRADED) 2024-06-15T01:13:19.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:19 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 2/327 objects degraded (0.612%), 1 pg degraded (PG_DEGRADED) 2024-06-15T01:13:19.529 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 podman[90361]: 2024-06-15 01:13:19.190449639 +0000 UTC m=+0.582610925 container remove 0bbcaaf0e41fc434e890fa367be554cffdafc25e6fba5b07c62999bb19d81fd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-activate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-15T01:13:19.529 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 podman[90420]: 2024-06-15 01:13:19.305664442 +0000 UTC m=+0.017129424 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:19.529 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 podman[90420]: 2024-06-15 01:13:19.41630561 +0000 UTC m=+0.127770586 container create 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-06-15T01:13:20.062 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 podman[90420]: 2024-06-15 01:13:19.643945059 +0000 UTC m=+0.355410051 container init 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-15T01:13:20.062 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 podman[90420]: 2024-06-15 01:13:19.647595055 +0000 UTC m=+0.359060031 container start 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=) 2024-06-15T01:13:20.062 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 ceph-osd[90439]: -- 172.21.15.142:0/4102841032 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557fa346d1e0 con 0x557fa340bc00 2024-06-15T01:13:20.063 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 bash[90420]: 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe 2024-06-15T01:13:20.063 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:19 smithi142 systemd[1]: Started Ceph osd.3 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:13:20.384 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:20 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:13:20.184+0000 7fcfccaca640 -1 Falling back to public interface 2024-06-15T01:13:20.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:20 smithi142 ceph-mon[82136]: pgmap v188: 65 pgs: 3 active+undersized, 15 stale+active+clean, 1 active+undersized+degraded, 46 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2/327 objects degraded (0.612%) 2024-06-15T01:13:20.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:20.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:20.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:20 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:20.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:20 smithi137 ceph-mon[112295]: pgmap v188: 65 pgs: 3 active+undersized, 15 stale+active+clean, 1 active+undersized+degraded, 46 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2/327 objects degraded (0.612%) 2024-06-15T01:13:20.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:20.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:20.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:20 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:22.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:22 smithi137 ceph-mon[112295]: pgmap v189: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:22.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:22 smithi142 ceph-mon[82136]: pgmap v189: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:23.543 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.543 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.543 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:23 smithi137 ceph-mon[112295]: pgmap v190: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:23.543 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.543 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:23 smithi142 ceph-mon[82136]: pgmap v190: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:23.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:23.656 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:24.969 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:24 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:13:24.804+0000 7fcfccaca640 -1 osd.3 0 read_superblock omap replica is missing. 2024-06-15T01:13:25.538 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:25 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:13:25.296+0000 7fcfccaca640 -1 osd.3 64 log_to_monitors true 2024-06-15T01:13:25.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:25 smithi142 ceph-mon[82136]: pgmap v191: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:25.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:25 smithi142 ceph-mon[82136]: from='osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:13:25.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:25 smithi142 ceph-mon[82136]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:13:26.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:25 smithi137 ceph-mon[112295]: pgmap v191: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:26.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:25 smithi137 ceph-mon[112295]: from='osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:13:26.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:25 smithi137 ceph-mon[112295]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-15T01:13:26.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 -- 172.21.15.137:0/1473707602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6710105a40 msgr2=0x7f6710105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 --2- 172.21.15.137:0/1473707602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6710105a40 0x7f6710105e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6704009910 tx=0x7f6704009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 -- 172.21.15.137:0/1473707602 shutdown_connections 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 --2- 172.21.15.137:0/1473707602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67100ffbd0 0x7f6710100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 --2- 172.21.15.137:0/1473707602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6710105a40 0x7f6710105e20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 -- 172.21.15.137:0/1473707602 >> 172.21.15.137:0/1473707602 conn(0x7f67100755a0 msgr2=0x7f67100759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 -- 172.21.15.137:0/1473707602 shutdown_connections 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.340+0000 7f6716b23700 1 -- 172.21.15.137:0/1473707602 wait complete. 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 Processor -- start 2024-06-15T01:13:26.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 -- start start 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6710105a40 0x7f67101979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710198000 con 0x7f67100ffbd0 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f6716b23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710198170 con 0x7f6710105a40 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36352/0 (socket says 172.21.15.137:36352) 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 -- 172.21.15.137:0/2498949046 learned_addr learned my addr 172.21.15.137:0/2498949046 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f670ffff700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6710105a40 0x7f67101979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 -- 172.21.15.137:0/2498949046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6710105a40 msgr2=0x7f67101979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6710105a40 0x7f67101979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.341+0000 7f67148bf700 1 -- 172.21.15.137:0/2498949046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040095f0 con 0x7f67100ffbd0 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f67148bf700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670400b5c0 tx=0x7f670400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670401d070 con 0x7f67100ffbd0 2024-06-15T01:13:26.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6704005e00 con 0x7f67100ffbd0 2024-06-15T01:13:26.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670400fe90 con 0x7f67100ffbd0 2024-06-15T01:13:26.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67101a0510 con 0x7f67100ffbd0 2024-06-15T01:13:26.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.342+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67101a08c0 con 0x7f67100ffbd0 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.343+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6710108040 con 0x7f67100ffbd0 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.344+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f670400f6d0 con 0x7f67100ffbd0 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.344+0000 7f670dffb700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 0x7f670006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.344+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f670408b3d0 con 0x7f67100ffbd0 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.344+0000 7f670ffff700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 0x7f670006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.345+0000 7f670ffff700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 0x7f670006d440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6710068ce0 tx=0x7f66fc0073d0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.349+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f67040564e0 con 0x7f67100ffbd0 2024-06-15T01:13:26.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.557+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f671002cce0 con 0x7f670006af80 2024-06-15T01:13:26.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.559+0000 7f670dffb700 1 -- 172.21.15.137:0/2498949046 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f671002cce0 con 0x7f670006af80 2024-06-15T01:13:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 msgr2=0x7f670006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 0x7f670006d440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6710068ce0 tx=0x7f66fc0073d0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 msgr2=0x7f67101974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670400b5c0 tx=0x7f670400bd70 comp rx=0 tx=0).stop 2024-06-15T01:13:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 shutdown_connections 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f670006af80 0x7f670006d440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67100ffbd0 0x7f67101974a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 --2- 172.21.15.137:0/2498949046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6710105a40 0x7f67101979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 >> 172.21.15.137:0/2498949046 conn(0x7f67100755a0 msgr2=0x7f67100fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 shutdown_connections 2024-06-15T01:13:26.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.561+0000 7f6716b23700 1 -- 172.21.15.137:0/2498949046 wait complete. 2024-06-15T01:13:26.563 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 -- 172.21.15.137:0/3082781870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 msgr2=0x7f939c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 --2- 172.21.15.137:0/3082781870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c1011a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9390009910 tx=0x7f9390009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 -- 172.21.15.137:0/3082781870 shutdown_connections 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 --2- 172.21.15.137:0/3082781870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 --2- 172.21.15.137:0/3082781870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c1011a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 -- 172.21.15.137:0/3082781870 >> 172.21.15.137:0/3082781870 conn(0x7f939c0fc650 msgr2=0x7f939c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 -- 172.21.15.137:0/3082781870 shutdown_connections 2024-06-15T01:13:26.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.667+0000 7f93a3678700 1 -- 172.21.15.137:0/3082781870 wait complete. 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 Processor -- start 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 -- start start 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c197260 con 0x7f939c100dc0 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a3678700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c199f90 con 0x7f939c101770 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a1414700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.668+0000 7f93a1414700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36370/0 (socket says 172.21.15.137:36370) 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a1414700 1 -- 172.21.15.137:0/3970954496 learned_addr learned my addr 172.21.15.137:0/3970954496 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:26.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a0c13700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a1414700 1 -- 172.21.15.137:0/3970954496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 msgr2=0x7f939c196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a1414700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a1414700 1 -- 172.21.15.137:0/3970954496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900095f0 con 0x7f939c100dc0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a0c13700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a1414700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f939000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f939001d070 con 0x7f939c100dc0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c19a210 con 0x7f939c100dc0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9390005dc0 con 0x7f939c100dc0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9390003940 con 0x7f939c100dc0 2024-06-15T01:13:26.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.669+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c19a6a0 con 0x7f939c100dc0 2024-06-15T01:13:26.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.670+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c109160 con 0x7f939c100dc0 2024-06-15T01:13:26.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.671+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f9390004ae0 con 0x7f939c100dc0 2024-06-15T01:13:26.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.671+0000 7f938e7fc700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 0x7f938806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.671+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f939008b790 con 0x7f939c100dc0 2024-06-15T01:13:26.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.671+0000 7f93a0c13700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 0x7f938806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.672+0000 7f93a0c13700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 0x7f938806d440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f939c197cb0 tx=0x7f9398009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.675+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9390056780 con 0x7f939c100dc0 2024-06-15T01:13:26.812 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:13:26.531+0000 7fcfc085b640 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:13:26.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-15T01:13:26.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: osdmap e67: 6 total, 5 up, 6 in 2024-06-15T01:13:26.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:26.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:26 smithi142 ceph-mon[82136]: from='osd.3 ' entity='osd.3' 2024-06-15T01:13:26.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.887+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f939c19a970 con 0x7f938806af80 2024-06-15T01:13:26.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.889+0000 7f938e7fc700 1 -- 172.21.15.137:0/3970954496 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f939c19a970 con 0x7f938806af80 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 msgr2=0x7f938806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 0x7f938806d440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f939c197cb0 tx=0x7f9398009260 comp rx=0 tx=0).stop 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 msgr2=0x7f939c196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f939000bde0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 shutdown_connections 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f938806af80 0x7f938806d440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f939c100dc0 0x7f939c196690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 --2- 172.21.15.137:0/3970954496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f939c101770 0x7f939c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 >> 172.21.15.137:0/3970954496 conn(0x7f939c0fc650 msgr2=0x7f939c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:26.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 shutdown_connections 2024-06-15T01:13:26.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.891+0000 7f93a3678700 1 -- 172.21.15.137:0/3970954496 wait complete. 2024-06-15T01:13:26.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-15T01:13:26.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: osdmap e67: 6 total, 5 up, 6 in 2024-06-15T01:13:26.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:26.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:26.971 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:26 smithi137 ceph-mon[112295]: from='osd.3 ' entity='osd.3' 2024-06-15T01:13:26.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 -- 172.21.15.137:0/757497238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768101c90 msgr2=0x7f9768102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 --2- 172.21.15.137:0/757497238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768101c90 0x7f9768102070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9758009910 tx=0x7f9758009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:26.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 -- 172.21.15.137:0/757497238 shutdown_connections 2024-06-15T01:13:26.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 --2- 172.21.15.137:0/757497238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768102640 0x7f976810ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 --2- 172.21.15.137:0/757497238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768101c90 0x7f9768102070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 -- 172.21.15.137:0/757497238 >> 172.21.15.137:0/757497238 conn(0x7f97680fb550 msgr2=0x7f97680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:26.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 -- 172.21.15.137:0/757497238 shutdown_connections 2024-06-15T01:13:26.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.996+0000 7f976c838700 1 -- 172.21.15.137:0/757497238 wait complete. 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 Processor -- start 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 -- start start 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 0x7f976819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976819d130 con 0x7f9768102640 2024-06-15T01:13:26.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976c838700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681965e0 con 0x7f9768101c90 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36396/0 (socket says 172.21.15.137:36396) 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 -- 172.21.15.137:0/2297006764 learned_addr learned my addr 172.21.15.137:0/2297006764 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 -- 172.21.15.137:0/2297006764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 msgr2=0x7f976819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976659c700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 0x7f976819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 0x7f976819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 -- 172.21.15.137:0/2297006764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580095f0 con 0x7f9768102640 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f976659c700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 0x7f976819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.997+0000 7f9765d9b700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b480 tx=0x7f975c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975c0118c0 con 0x7f9768102640 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f975c010380 con 0x7f9768102640 2024-06-15T01:13:26.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97681968c0 con 0x7f9768102640 2024-06-15T01:13:26.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975c00f510 con 0x7f9768102640 2024-06-15T01:13:26.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9768196e10 con 0x7f9768102640 2024-06-15T01:13:26.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.998+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9768108230 con 0x7f9768102640 2024-06-15T01:13:26.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.999+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f975c0104f0 con 0x7f9768102640 2024-06-15T01:13:26.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.999+0000 7f97577fe700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 0x7f975006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:26.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:26.999+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f975c089e00 con 0x7f9768102640 2024-06-15T01:13:26.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.000+0000 7f976659c700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 0x7f975006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:26.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.000+0000 7f976659c700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 0x7f975006d440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9758006010 tx=0x7f975800b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:26.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.004+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f975c058880 con 0x7f9768102640 2024-06-15T01:13:27.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.207+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9768197110 con 0x7f975006af80 2024-06-15T01:13:27.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.216+0000 7f97577fe700 1 -- 172.21.15.137:0/2297006764 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9768197110 con 0x7f975006af80 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (4m) 36s ago 14m 28.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (14m) 36s ago 14m 8238k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (13m) 4s ago 13m 7922k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (2m) 36s ago 14m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (2m) 4s ago 13m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (3m) 36s ago 14m 81.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (8m) 36s ago 8m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (9m) 36s ago 9m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (9m) 4s ago 9m 16.2M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:13:27.208 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (8m) 4s ago 8m 19.5M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (5m) 36s ago 16m 496M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (4m) 4s ago 13m 439M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (2m) 36s ago 16m 43.6M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (2m) 4s ago 13m 38.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (4m) 36s ago 14m 15.7M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (4m) 4s ago 13m 17.2M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (108s) 36s ago 12m 93.2M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (75s) 36s ago 12m 94.6M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (41s) 36s ago 11m 14.5M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (7s) 4s ago 11m 12.0M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (10m) 4s ago 10m 240M 2240M 18.2.1 5be31c24972a d98ba9149167 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (10m) 4s ago 10m 223M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:13:27.209 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (4m) 36s ago 14m 73.4M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:13:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 msgr2=0x7f975006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 0x7f975006d440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9758006010 tx=0x7f975800b540 comp rx=0 tx=0).stop 2024-06-15T01:13:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 msgr2=0x7f976819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b480 tx=0x7f975c00b840 comp rx=0 tx=0).stop 2024-06-15T01:13:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 shutdown_connections 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f975006af80 0x7f975006d440 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9768101c90 0x7f976819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 --2- 172.21.15.137:0/2297006764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9768102640 0x7f976819caa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 >> 172.21.15.137:0/2297006764 conn(0x7f97680fb550 msgr2=0x7f9768105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 shutdown_connections 2024-06-15T01:13:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.219+0000 7f976c838700 1 -- 172.21.15.137:0/2297006764 wait complete. 2024-06-15T01:13:27.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.321+0000 7f5297ac0700 1 -- 172.21.15.137:0/3819638255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290101750 msgr2=0x7f52901057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.321+0000 7f5297ac0700 1 --2- 172.21.15.137:0/3819638255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290101750 0x7f52901057a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5280009910 tx=0x7f5280009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:27.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.321+0000 7f5297ac0700 1 -- 172.21.15.137:0/3819638255 shutdown_connections 2024-06-15T01:13:27.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.321+0000 7f5297ac0700 1 --2- 172.21.15.137:0/3819638255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290101750 0x7f52901057a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.321+0000 7f5297ac0700 1 --2- 172.21.15.137:0/3819638255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290100da0 0x7f5290101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.322+0000 7f5297ac0700 1 -- 172.21.15.137:0/3819638255 >> 172.21.15.137:0/3819638255 conn(0x7f52900fc650 msgr2=0x7f52900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.322+0000 7f5297ac0700 1 -- 172.21.15.137:0/3819638255 shutdown_connections 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.322+0000 7f5297ac0700 1 -- 172.21.15.137:0/3819638255 wait complete. 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.322+0000 7f5297ac0700 1 Processor -- start 2024-06-15T01:13:27.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.322+0000 7f5297ac0700 1 -- start start 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5297ac0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5297ac0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290101750 0x7f529019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5297ac0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529019d130 con 0x7f5290100da0 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5297ac0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52901965e0 con 0x7f5290101750 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36404/0 (socket says 172.21.15.137:36404) 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 -- 172.21.15.137:0/2721191893 learned_addr learned my addr 172.21.15.137:0/2721191893 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529505b700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290101750 0x7f529019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 -- 172.21.15.137:0/2721191893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290101750 msgr2=0x7f529019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290101750 0x7f529019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 -- 172.21.15.137:0/2721191893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800095f0 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f529585c700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b4b0 tx=0x7f528c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528c011800 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f528c011e00 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528c010e60 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.323+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52901968c0 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.324+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5290196dc0 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.324+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52901090d0 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.325+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f528c011960 con 0x7f5290100da0 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.325+0000 7f5286ffd700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 0x7f527c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.326+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f528c089c50 con 0x7f5290100da0 2024-06-15T01:13:27.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.326+0000 7f529505b700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 0x7f527c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.326+0000 7f529505b700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 0x7f527c06d440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52900fcfc0 tx=0x7f528001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:27.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.330+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f528c058750 con 0x7f5290100da0 2024-06-15T01:13:27.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.614+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f52901970f0 con 0x7f5290100da0 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.614+0000 7f5286ffd700 1 -- 172.21.15.137:0/2721191893 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f528c0582e0 con 0x7f5290100da0 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:13:27.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 8 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:13:27.607 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:13:27.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 msgr2=0x7f527c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 0x7f527c06d440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52900fcfc0 tx=0x7f528001a040 comp rx=0 tx=0).stop 2024-06-15T01:13:27.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 msgr2=0x7f529019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b4b0 tx=0x7f528c00b7c0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 shutdown_connections 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f527c06af80 0x7f527c06d440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5290100da0 0x7f529019c560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 --2- 172.21.15.137:0/2721191893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5290101750 0x7f529019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 >> 172.21.15.137:0/2721191893 conn(0x7f52900fc650 msgr2=0x7f52900fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 shutdown_connections 2024-06-15T01:13:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.617+0000 7f5297ac0700 1 -- 172.21.15.137:0/2721191893 wait complete. 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 -- 172.21.15.137:0/2040453366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 msgr2=0x7fc638100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 --2- 172.21.15.137:0/2040453366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc638100050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009910 tx=0x7fc62c009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 -- 172.21.15.137:0/2040453366 shutdown_connections 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 --2- 172.21.15.137:0/2040453366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc638100050 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 --2- 172.21.15.137:0/2040453366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 0x7fc638105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 -- 172.21.15.137:0/2040453366 >> 172.21.15.137:0/2040453366 conn(0x7fc6380755a0 msgr2=0x7fc6380759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 -- 172.21.15.137:0/2040453366 shutdown_connections 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.721+0000 7fc63f808700 1 -- 172.21.15.137:0/2040453366 wait complete. 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 Processor -- start 2024-06-15T01:13:27.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 -- start start 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 0x7fc638197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc638198100 con 0x7fc6380ffbd0 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63f808700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc638198270 con 0x7fc638105a40 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36412/0 (socket says 172.21.15.137:36412) 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 -- 172.21.15.137:0/306684626 learned_addr learned my addr 172.21.15.137:0/306684626 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63cda3700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 0x7fc638197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 -- 172.21.15.137:0/306684626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 msgr2=0x7fc638197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 0x7fc638197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 -- 172.21.15.137:0/306684626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0095f0 con 0x7fc6380ffbd0 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.722+0000 7fc63d5a4700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c8f0 tx=0x7fc63400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:27.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.723+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc634004a00 con 0x7fc6380ffbd0 2024-06-15T01:13:27.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.723+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6381a0550 con 0x7fc6380ffbd0 2024-06-15T01:13:27.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.723+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc634005020 con 0x7fc6380ffbd0 2024-06-15T01:13:27.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.723+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc634007c40 con 0x7fc6380ffbd0 2024-06-15T01:13:27.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.723+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6381a0a70 con 0x7fc6380ffbd0 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.724+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc638107fd0 con 0x7fc6380ffbd0 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.724+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc634005190 con 0x7fc6380ffbd0 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.725+0000 7fc62a7fc700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 0x7fc62406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.725+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc634013070 con 0x7fc6380ffbd0 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.725+0000 7fc63cda3700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 0x7fc62406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:27.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.725+0000 7fc63cda3700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 0x7fc62406d440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc638069040 tx=0x7fc62c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:27.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.729+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc634054f40 con 0x7fc6380ffbd0 2024-06-15T01:13:27.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.962+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc63802cc70 con 0x7fc6380ffbd0 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: pgmap v193: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 67/327 objects degraded (20.489%), 19 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:27.954 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:13:27.955 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228] boot 2024-06-15T01:13:27.955 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: osdmap e68: 6 total, 6 up, 6 in 2024-06-15T01:13:27.955 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:13:27.955 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:27 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2721191893' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:27.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.963+0000 7fc62a7fc700 1 -- 172.21.15.137:0/306684626 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fc634058560 con 0x7fc6380ffbd0 2024-06-15T01:13:27.955 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:13:27.955 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:13:27.955 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:13:27.956 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:13:27.957 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:13:27.958 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:13:27.959 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:13:27.959 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:13:27.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 msgr2=0x7fc62406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 0x7fc62406d440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc638069040 tx=0x7fc62c01a040 comp rx=0 tx=0).stop 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 msgr2=0x7fc6381974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c8f0 tx=0x7fc63400cc00 comp rx=0 tx=0).stop 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 shutdown_connections 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc62406af80 0x7fc62406d440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6380ffbd0 0x7fc6381974e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 --2- 172.21.15.137:0/306684626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc638105a40 0x7fc638197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 >> 172.21.15.137:0/306684626 conn(0x7fc6380755a0 msgr2=0x7fc6380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 shutdown_connections 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:27.965+0000 7fc63f808700 1 -- 172.21.15.137:0/306684626 wait complete. 2024-06-15T01:13:27.960 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:13:28.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 -- 172.21.15.137:0/2893130078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8068670 msgr2=0x7fe2b8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 --2- 172.21.15.137:0/2893130078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8068670 0x7fe2b8068a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009910 tx=0x7fe2b4009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 -- 172.21.15.137:0/2893130078 shutdown_connections 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 --2- 172.21.15.137:0/2893130078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8069020 0x7fe2b8105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 --2- 172.21.15.137:0/2893130078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8068670 0x7fe2b8068a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 -- 172.21.15.137:0/2893130078 >> 172.21.15.137:0/2893130078 conn(0x7fe2b80755a0 msgr2=0x7fe2b80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 -- 172.21.15.137:0/2893130078 shutdown_connections 2024-06-15T01:13:28.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.071+0000 7fe2bf904700 1 -- 172.21.15.137:0/2893130078 wait complete. 2024-06-15T01:13:28.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: pgmap v193: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 67/327 objects degraded (20.489%), 19 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: osd.3 [v2:172.21.15.142:6800/3361419228,v1:172.21.15.142:6801/3361419228] boot 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: osdmap e68: 6 total, 6 up, 6 in 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-15T01:13:28.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:27 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2721191893' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 Processor -- start 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 -- start start 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8068670 0x7fe2b80ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8100720 con 0x7fe2b8069020 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bf904700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8100860 con 0x7fe2b8068670 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36420/0 (socket says 172.21.15.137:36420) 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 -- 172.21.15.137:0/3065904573 learned_addr learned my addr 172.21.15.137:0/3065904573 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bd6a0700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8068670 0x7fe2b80ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 -- 172.21.15.137:0/3065904573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8068670 msgr2=0x7fe2b80ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8068670 0x7fe2b80ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 -- 172.21.15.137:0/3065904573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40095f0 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.072+0000 7fe2bce9f700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e970 tx=0x7fe2a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a8004a00 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2a800c7c0 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81a2840 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a8005510 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a2c50 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.073+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8109130 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.074+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe2a800c930 con 0x7fe2b8069020 2024-06-15T01:13:28.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.074+0000 7fe2ae7fc700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 0x7fe2a406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.075+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe2a8014070 con 0x7fe2b8069020 2024-06-15T01:13:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.075+0000 7fe2bd6a0700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 0x7fe2a406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.075+0000 7fe2bd6a0700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 0x7fe2a406d440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b4005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.079+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe2a8055440 con 0x7fe2b8069020 2024-06-15T01:13:28.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.285+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b802cda0 con 0x7fe2a406af80 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.287+0000 7fe2ae7fc700 1 -- 172.21.15.137:0/3065904573 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe2b802cda0 con 0x7fe2a406af80 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:13:28.278 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:13:28.279 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "10/23 daemons upgraded", 2024-06-15T01:13:28.279 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading osd daemons", 2024-06-15T01:13:28.279 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:13:28.279 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 msgr2=0x7fe2a406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 0x7fe2a406d440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b4005940 comp rx=0 tx=0).stop 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 msgr2=0x7fe2b81001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e970 tx=0x7fe2a800ec80 comp rx=0 tx=0).stop 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 shutdown_connections 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe2a406af80 0x7fe2a406d440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2b8068670 0x7fe2b80ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 --2- 172.21.15.137:0/3065904573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8069020 0x7fe2b81001e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 >> 172.21.15.137:0/3065904573 conn(0x7fe2b80755a0 msgr2=0x7fe2b80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:28.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 shutdown_connections 2024-06-15T01:13:28.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.289+0000 7fe2bf904700 1 -- 172.21.15.137:0/3065904573 wait complete. 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- 172.21.15.137:0/1143354199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 msgr2=0x7f15c010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/1143354199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c010ab40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009910 tx=0x7f15bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- 172.21.15.137:0/1143354199 shutdown_connections 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/1143354199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c010ab40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/1143354199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 0x7f15c0102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- 172.21.15.137:0/1143354199 >> 172.21.15.137:0/1143354199 conn(0x7f15c00fb560 msgr2=0x7f15c00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- 172.21.15.137:0/1143354199 shutdown_connections 2024-06-15T01:13:28.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- 172.21.15.137:0/1143354199 wait complete. 2024-06-15T01:13:28.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 Processor -- start 2024-06-15T01:13:28.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- start start 2024-06-15T01:13:28.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 0x7f15c01966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0197290 con 0x7f15c0102650 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.393+0000 7f15c7a6d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0199fc0 con 0x7f15c0101d30 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36446/0 (socket says 172.21.15.137:36446) 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 -- 172.21.15.137:0/3546833765 learned_addr learned my addr 172.21.15.137:0/3546833765 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5809700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 0x7f15c01966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 -- 172.21.15.137:0/3546833765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 msgr2=0x7f15c01966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 0x7f15c01966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 -- 172.21.15.137:0/3546833765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0095f0 con 0x7f15c0102650 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c5008700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00bd70 tx=0x7f15bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15bc01d070 con 0x7f15c0102650 2024-06-15T01:13:28.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15bc00f460 con 0x7f15c0102650 2024-06-15T01:13:28.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15bc021640 con 0x7f15c0102650 2024-06-15T01:13:28.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c019a240 con 0x7f15c0102650 2024-06-15T01:13:28.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.394+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c019a730 con 0x7f15c0102650 2024-06-15T01:13:28.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.395+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c0108240 con 0x7f15c0102650 2024-06-15T01:13:28.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.396+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f15bc00f9b0 con 0x7f15c0102650 2024-06-15T01:13:28.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.396+0000 7f15b6ffd700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 0x7f15ac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:28.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.396+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f15bc08b530 con 0x7f15c0102650 2024-06-15T01:13:28.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.396+0000 7f15c5809700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 0x7f15ac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:28.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.397+0000 7f15c5809700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 0x7f15ac06d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005950 tx=0x7f15b0005cf0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:28.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.400+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f15bc056480 con 0x7f15c0102650 2024-06-15T01:13:28.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.699+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f15c004ea90 con 0x7f15c0102650 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.700+0000 7f15b6ffd700 1 -- 172.21.15.137:0/3546833765 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1795 (secure 0 0 0) 0x7f15bc059aa0 con 0x7f15c0102650 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN Degraded data redundancy: 67/327 objects degraded (20.489%), 19 pgs degraded; 4 mgr modules have recently crashed 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 67/327 objects degraded (20.489%), 19 pgs degraded 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-06-15T01:13:28.691 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.11 is active+undersized+degraded, acting [4,1] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:13:28.692 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:13:28.693 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:13:28.693 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:13:28.693 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:13:28.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 msgr2=0x7f15ac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 0x7f15ac06d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005950 tx=0x7f15b0005cf0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 msgr2=0x7f15c0196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00bd70 tx=0x7f15bc00be50 comp rx=0 tx=0).stop 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 shutdown_connections 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15ac06af80 0x7f15ac06d440 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0101d30 0x7f15c01966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 --2- 172.21.15.137:0/3546833765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102650 0x7f15c0196c00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 >> 172.21.15.137:0/3546833765 conn(0x7f15c00fb560 msgr2=0x7f15c0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 shutdown_connections 2024-06-15T01:13:28.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:28.702+0000 7f15c7a6d700 1 -- 172.21.15.137:0/3546833765 wait complete. 2024-06-15T01:13:28.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:28 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/306684626' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:13:28.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:28 smithi137 ceph-mon[112295]: osdmap e69: 6 total, 6 up, 6 in 2024-06-15T01:13:28.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:28 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3546833765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:13:29.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:28 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/306684626' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:13:29.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:28 smithi142 ceph-mon[82136]: osdmap e69: 6 total, 6 up, 6 in 2024-06-15T01:13:29.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:28 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3546833765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:13:30.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:29 smithi142 ceph-mon[82136]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:30.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:29 smithi142 ceph-mon[82136]: pgmap v196: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:30.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:29 smithi137 ceph-mon[112295]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:30.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:29 smithi137 ceph-mon[112295]: pgmap v196: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 67/327 objects degraded (20.489%) 2024-06-15T01:13:32.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:31 smithi142 ceph-mon[82136]: pgmap v197: 65 pgs: 8 active+undersized, 11 active+undersized+degraded, 46 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/327 objects degraded (13.150%) 2024-06-15T01:13:32.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:31 smithi137 ceph-mon[112295]: pgmap v197: 65 pgs: 8 active+undersized, 11 active+undersized+degraded, 46 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/327 objects degraded (13.150%) 2024-06-15T01:13:33.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:32 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 43/327 objects degraded (13.150%), 11 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:33.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:32 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 43/327 objects degraded (13.150%), 11 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:33.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:33 smithi137 ceph-mon[112295]: pgmap v198: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:33.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:33 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/327 objects degraded (13.150%), 11 pgs degraded) 2024-06-15T01:13:33.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:33.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:34.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:33 smithi142 ceph-mon[82136]: pgmap v198: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:34.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:33 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/327 objects degraded (13.150%), 11 pgs degraded) 2024-06-15T01:13:34.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:34.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:35.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:35 smithi142 ceph-mon[82136]: pgmap v199: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:36.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:35 smithi137 ceph-mon[112295]: pgmap v199: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:38.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:37 smithi142 ceph-mon[82136]: pgmap v200: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:38.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:37 smithi137 ceph-mon[112295]: pgmap v200: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:39.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:39.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:40.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:40 smithi142 ceph-mon[82136]: pgmap v201: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:40.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:40 smithi137 ceph-mon[112295]: pgmap v201: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:42.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:42 smithi137 ceph-mon[112295]: pgmap v202: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:42.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:42 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:42.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:42 smithi142 ceph-mon[82136]: pgmap v202: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:42.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:42 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:43 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:43 smithi142 ceph-mon[82136]: Upgrade: osd.4 is safe to restart 2024-06-15T01:13:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:43 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:43 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-15T01:13:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:43 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:43.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:43 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-06-15T01:13:43.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:43 smithi137 ceph-mon[112295]: Upgrade: osd.4 is safe to restart 2024-06-15T01:13:43.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:43 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:43.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:43 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-15T01:13:43.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:43 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:44.123 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:44 smithi142 ceph-mon[82136]: pgmap v203: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:44.123 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:44 smithi142 ceph-mon[82136]: Upgrade: Updating osd.4 2024-06-15T01:13:44.123 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:44 smithi142 ceph-mon[82136]: Deploying daemon osd.4 on smithi142 2024-06-15T01:13:44.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:44 smithi137 ceph-mon[112295]: pgmap v203: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:44.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:44 smithi137 ceph-mon[112295]: Upgrade: Updating osd.4 2024-06-15T01:13:44.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:44 smithi137 ceph-mon[112295]: Deploying daemon osd.4 on smithi142 2024-06-15T01:13:45.976 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:45 smithi142 systemd[1]: Stopping Ceph osd.4 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:13:46.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:46 smithi142 ceph-mon[82136]: pgmap v204: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:46.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:46 smithi142 ceph-mon[82136]: osd.4 marked itself down and dead 2024-06-15T01:13:46.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[50119]: 2024-06-15T01:13:45.985+0000 7f95b34cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:13:46.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[50119]: 2024-06-15T01:13:45.985+0000 7f95b34cd700 -1 osd.4 69 *** Got signal Terminated *** 2024-06-15T01:13:46.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[50119]: 2024-06-15T01:13:45.985+0000 7f95b34cd700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:13:46.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:46 smithi137 ceph-mon[112295]: pgmap v204: 65 pgs: 65 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:46.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:46 smithi137 ceph-mon[112295]: osd.4 marked itself down and dead 2024-06-15T01:13:46.812 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:46 smithi142 podman[95018]: 2024-06-15 01:13:46.445229648 +0000 UTC m=+0.544375297 container died d98ba91491672cd60cbefc3637c1390cb185dafa3ec8c0bb8a62318e56110c1e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-15T01:13:47.152 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:47 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:13:47.152 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:47 smithi142 ceph-mon[82136]: osdmap e70: 6 total, 5 up, 6 in 2024-06-15T01:13:47.153 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:46 smithi142 podman[95018]: 2024-06-15 01:13:46.860375642 +0000 UTC m=+0.959521288 container cleanup d98ba91491672cd60cbefc3637c1390cb185dafa3ec8c0bb8a62318e56110c1e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-06-15T01:13:47.153 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:46 smithi142 bash[95018]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4 2024-06-15T01:13:47.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:47 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:13:47.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:47 smithi137 ceph-mon[112295]: osdmap e70: 6 total, 5 up, 6 in 2024-06-15T01:13:47.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95035]: 2024-06-15 01:13:47.162157264 +0000 UTC m=+0.718129953 container remove d98ba91491672cd60cbefc3637c1390cb185dafa3ec8c0bb8a62318e56110c1e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=HEAD) 2024-06-15T01:13:47.888 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95122]: 2024-06-15 01:13:47.532303885 +0000 UTC m=+0.017180867 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:47.889 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95122]: 2024-06-15 01:13:47.671397213 +0000 UTC m=+0.156274183 container create 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-15T01:13:47.889 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95122]: 2024-06-15 01:13:47.898743227 +0000 UTC m=+0.383620210 container init 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:13:48.154 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95122]: 2024-06-15 01:13:47.902017808 +0000 UTC m=+0.386894784 container start 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-15T01:13:48.154 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:47 smithi142 podman[95122]: 2024-06-15 01:13:47.956987791 +0000 UTC m=+0.441864766 container attach 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-15T01:13:48.154 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:48 smithi142 podman[95122]: 2024-06-15 01:13:48.162431439 +0000 UTC m=+0.647308423 container died 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-06-15T01:13:48.154 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: pgmap v206: 65 pgs: 6 stale+active+clean, 59 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:48.154 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: osdmap e71: 6 total, 5 up, 6 in 2024-06-15T01:13:48.154 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 2, 0, 5]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 5]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 2]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [1, 4]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 5]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:48.155 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: pgmap v206: 65 pgs: 6 stale+active+clean, 59 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: osdmap e71: 6 total, 5 up, 6 in 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 2, 0, 5]}]: dispatch 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 5]}]: dispatch 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]: dispatch 2024-06-15T01:13:48.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 2]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [1, 4]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 5]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:48.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:13:49.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:48 smithi142 podman[95144]: 2024-06-15 01:13:48.936491837 +0000 UTC m=+0.767574104 container remove 611fcfb8cd1de536d9d2134dcc9cf1b28f39b49f9db20b24b2773d891183564d (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:13:49.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:48 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-15T01:13:49.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:48 smithi142 systemd[1]: Stopped Ceph osd.4 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:13:49.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:48 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.4.service: Consumed 1min 6.337s CPU time. 2024-06-15T01:13:49.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 systemd[1]: Starting Ceph osd.4 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 2, 0, 5]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 5]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 2]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [1, 4]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 5]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-06-15T01:13:49.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:49 smithi142 ceph-mon[82136]: osdmap e72: 6 total, 5 up, 6 in 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [3, 2, 0, 5]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 5]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 2]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 2]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2]}]': finished 2024-06-15T01:13:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [1, 4]}]': finished 2024-06-15T01:13:49.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 5]}]': finished 2024-06-15T01:13:49.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-06-15T01:13:49.471 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:49 smithi137 ceph-mon[112295]: osdmap e72: 6 total, 5 up, 6 in 2024-06-15T01:13:49.629 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 podman[95245]: 2024-06-15 01:13:49.281410898 +0000 UTC m=+0.017245245 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:49.630 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 podman[95245]: 2024-06-15 01:13:49.396022001 +0000 UTC m=+0.131856340 container create d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, io.buildah.version=1.35.3, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-15T01:13:49.630 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 podman[95245]: 2024-06-15 01:13:49.63993273 +0000 UTC m=+0.375767079 container init d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:13:49.994 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 podman[95245]: 2024-06-15 01:13:49.643403669 +0000 UTC m=+0.379238020 container start d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-15T01:13:49.994 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:49 smithi142 podman[95245]: 2024-06-15 01:13:49.739948252 +0000 UTC m=+0.475782613 container attach d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:13:50.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-mon[82136]: pgmap v209: 65 pgs: 2 active+undersized, 5 stale+active+clean, 58 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:50.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-mon[82136]: osdmap e73: 6 total, 5 up, 6 in 2024-06-15T01:13:50.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-15T01:13:50.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-15T01:13:50.312 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate[95259]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 bash[95245]: --> ceph-volume raw activate successful for osd ID: 4 2024-06-15T01:13:50.313 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 podman[95245]: 2024-06-15 01:13:50.058719065 +0000 UTC m=+0.794553407 container died d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, ceph=True) 2024-06-15T01:13:50.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:50 smithi137 ceph-mon[112295]: pgmap v209: 65 pgs: 2 active+undersized, 5 stale+active+clean, 58 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:13:50.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:50 smithi137 ceph-mon[112295]: osdmap e73: 6 total, 5 up, 6 in 2024-06-15T01:13:50.935 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 podman[95375]: 2024-06-15 01:13:50.727715344 +0000 UTC m=+0.663335386 container remove d5ccb707bfc101ae82a489fe98e2abadf54310b1349bc27ea44718d85ec334ae (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-activate, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-15T01:13:51.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:51 smithi142 ceph-mon[82136]: osdmap e74: 6 total, 5 up, 6 in 2024-06-15T01:13:51.213 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 podman[95435]: 2024-06-15 01:13:50.848975387 +0000 UTC m=+0.018861743 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:13:51.213 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:50 smithi142 podman[95435]: 2024-06-15 01:13:50.978684482 +0000 UTC m=+0.148570830 container create f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-15T01:13:51.213 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 podman[95435]: 2024-06-15 01:13:51.222769705 +0000 UTC m=+0.392656077 container init f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux ) 2024-06-15T01:13:51.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:51 smithi137 ceph-mon[112295]: osdmap e74: 6 total, 5 up, 6 in 2024-06-15T01:13:51.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 podman[95435]: 2024-06-15 01:13:51.226747882 +0000 UTC m=+0.396634234 container start f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.35.3) 2024-06-15T01:13:51.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 ceph-osd[95455]: -- 172.21.15.142:0/1353998836 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a29ebc91e0 con 0x55a29f948000 2024-06-15T01:13:51.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 bash[95435]: f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 2024-06-15T01:13:51.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 systemd[1]: Started Ceph osd.4 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:13:52.062 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:51 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:13:51.769+0000 7f2c9a3aa640 -1 Falling back to public interface 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: pgmap v212: 65 pgs: 2 peering, 2 remapped+peering, 2 unknown, 12 active+undersized, 1 stale+active+clean, 8 active+undersized+degraded, 38 active+clean; 235 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/300 objects degraded (7.667%) 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 23/300 objects degraded (7.667%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: osdmap e75: 6 total, 5 up, 6 in 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:52.370 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:52.371 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: pgmap v212: 65 pgs: 2 peering, 2 remapped+peering, 2 unknown, 12 active+undersized, 1 stale+active+clean, 8 active+undersized+degraded, 38 active+clean; 235 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/300 objects degraded (7.667%) 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 23/300 objects degraded (7.667%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: osdmap e75: 6 total, 5 up, 6 in 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:52.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:53.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:53 smithi142 ceph-mon[82136]: osdmap e76: 6 total, 5 up, 6 in 2024-06-15T01:13:53.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:53 smithi142 ceph-mon[82136]: osdmap e77: 6 total, 5 up, 6 in 2024-06-15T01:13:53.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:53 smithi137 ceph-mon[112295]: osdmap e76: 6 total, 5 up, 6 in 2024-06-15T01:13:53.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:53 smithi137 ceph-mon[112295]: osdmap e77: 6 total, 5 up, 6 in 2024-06-15T01:13:54.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:54 smithi137 ceph-mon[112295]: pgmap v216: 65 pgs: 2 active+clean+remapped, 1 active+undersized+degraded+remapped, 2 peering, 1 remapped+peering, 1 unknown, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 66 MiB/s wr, 686 op/s; 37/315 objects degraded (11.746%); 9/315 objects misplaced (2.857%); 6.7 MiB/s, 1 objects/s recovering 2024-06-15T01:13:54.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:54 smithi137 ceph-mon[112295]: osdmap e78: 6 total, 5 up, 6 in 2024-06-15T01:13:54.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:54 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:54.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:54 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:54.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:54 smithi142 ceph-mon[82136]: pgmap v216: 65 pgs: 2 active+clean+remapped, 1 active+undersized+degraded+remapped, 2 peering, 1 remapped+peering, 1 unknown, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 66 MiB/s wr, 686 op/s; 37/315 objects degraded (11.746%); 9/315 objects misplaced (2.857%); 6.7 MiB/s, 1 objects/s recovering 2024-06-15T01:13:54.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:54 smithi142 ceph-mon[82136]: osdmap e78: 6 total, 5 up, 6 in 2024-06-15T01:13:54.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:54 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:54.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:54 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:55.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:55 smithi142 ceph-mon[82136]: osdmap e79: 6 total, 5 up, 6 in 2024-06-15T01:13:55.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:55.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:55.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:55 smithi142 ceph-mon[82136]: pgmap v219: 65 pgs: 7 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.9 MiB/s wr, 157 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 5.2 MiB/s, 213 objects/s recovering 2024-06-15T01:13:55.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:55 smithi137 ceph-mon[112295]: osdmap e79: 6 total, 5 up, 6 in 2024-06-15T01:13:55.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:55.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:55.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:55 smithi137 ceph-mon[112295]: pgmap v219: 65 pgs: 7 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.9 MiB/s wr, 157 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 5.2 MiB/s, 213 objects/s recovering 2024-06-15T01:13:56.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:56 smithi142 ceph-mon[82136]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-15T01:13:56.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:56 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:13:56.337+0000 7f2c9a3aa640 -1 osd.4 0 read_superblock omap replica is missing. 2024-06-15T01:13:56.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:56 smithi137 ceph-mon[112295]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-15T01:13:57.062 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:56 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:13:56.836+0000 7f2c9a3aa640 -1 osd.4 69 log_to_monitors true 2024-06-15T01:13:57.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:57 smithi142 ceph-mon[82136]: pgmap v220: 65 pgs: 7 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1.5 MiB/s wr, 123 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 4.0 MiB/s, 167 objects/s recovering 2024-06-15T01:13:57.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:57 smithi142 ceph-mon[82136]: from='osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:13:57.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:57 smithi142 ceph-mon[82136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:13:57.488 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:57 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 37/327 objects degraded (11.315%), 11 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:57.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:57 smithi137 ceph-mon[112295]: pgmap v220: 65 pgs: 7 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 32 active+clean; 262 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1.5 MiB/s wr, 123 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 4.0 MiB/s, 167 objects/s recovering 2024-06-15T01:13:57.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:57 smithi137 ceph-mon[112295]: from='osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:13:57.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:57 smithi137 ceph-mon[112295]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-15T01:13:57.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:57 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 37/327 objects degraded (11.315%), 11 pgs degraded (PG_DEGRADED) 2024-06-15T01:13:58.562 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:13:58.260+0000 7f2c8e13b640 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:13:58.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-15T01:13:58.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: osdmap e80: 6 total, 5 up, 6 in 2024-06-15T01:13:58.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:13:58.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-06-15T01:13:58.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:58 smithi142 ceph-mon[82136]: from='osd.4 ' entity='osd.4' 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: osdmap e80: 6 total, 5 up, 6 in 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:13:58.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-06-15T01:13:58.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:58 smithi137 ceph-mon[112295]: from='osd.4 ' entity='osd.4' 2024-06-15T01:13:58.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 -- 172.21.15.137:0/2578570266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570105a40 msgr2=0x7fa570105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:58.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 --2- 172.21.15.137:0/2578570266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570105a40 0x7fa570105e20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa560009910 tx=0x7fa560009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:58.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 -- 172.21.15.137:0/2578570266 shutdown_connections 2024-06-15T01:13:58.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 --2- 172.21.15.137:0/2578570266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5700ffbd0 0x7fa570100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:58.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 --2- 172.21.15.137:0/2578570266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570105a40 0x7fa570105e20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:58.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 -- 172.21.15.137:0/2578570266 >> 172.21.15.137:0/2578570266 conn(0x7fa5700755a0 msgr2=0x7fa5700759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:58.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 -- 172.21.15.137:0/2578570266 shutdown_connections 2024-06-15T01:13:58.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.820+0000 7fa575d1f700 1 -- 172.21.15.137:0/2578570266 wait complete. 2024-06-15T01:13:58.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 Processor -- start 2024-06-15T01:13:58.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 -- start start 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa570105a40 0x7fa5701979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570198000 con 0x7fa5700ffbd0 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa575d1f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa570198170 con 0x7fa570105a40 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.821+0000 7fa56f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36320/0 (socket says 172.21.15.137:36320) 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 -- 172.21.15.137:0/235859421 learned_addr learned my addr 172.21.15.137:0/235859421 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56effd700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa570105a40 0x7fa5701979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 -- 172.21.15.137:0/235859421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa570105a40 msgr2=0x7fa5701979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa570105a40 0x7fa5701979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 -- 172.21.15.137:0/235859421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600095f0 con 0x7fa5700ffbd0 2024-06-15T01:13:58.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56f7fe700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5600092e0 tx=0x7fa56000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:58.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56001d070 con 0x7fa5700ffbd0 2024-06-15T01:13:58.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa560005e00 con 0x7fa5700ffbd0 2024-06-15T01:13:58.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56000fe90 con 0x7fa5700ffbd0 2024-06-15T01:13:58.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5701a0510 con 0x7fa5700ffbd0 2024-06-15T01:13:58.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.822+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701a0980 con 0x7fa5700ffbd0 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.823+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa570108040 con 0x7fa5700ffbd0 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.824+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa56000f6d0 con 0x7fa5700ffbd0 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.824+0000 7fa56cff9700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 0x7fa55806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.825+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa56008b5e0 con 0x7fa5700ffbd0 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.825+0000 7fa56effd700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 0x7fa55806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:58.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.825+0000 7fa56effd700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 0x7fa55806d440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa570068ce0 tx=0x7fa56400b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:58.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:58.829+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa560056390 con 0x7fa5700ffbd0 2024-06-15T01:13:59.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.046+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa57002cce0 con 0x7fa55806af80 2024-06-15T01:13:59.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.047+0000 7fa56cff9700 1 -- 172.21.15.137:0/235859421 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa57002cce0 con 0x7fa55806af80 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 msgr2=0x7fa55806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 0x7fa55806d440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa570068ce0 tx=0x7fa56400b410 comp rx=0 tx=0).stop 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 msgr2=0x7fa5701974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5600092e0 tx=0x7fa56000bd70 comp rx=0 tx=0).stop 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 shutdown_connections 2024-06-15T01:13:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa55806af80 0x7fa55806d440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffbd0 0x7fa5701974a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 --2- 172.21.15.137:0/235859421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa570105a40 0x7fa5701979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 >> 172.21.15.137:0/235859421 conn(0x7fa5700755a0 msgr2=0x7fa5700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.049+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 shutdown_connections 2024-06-15T01:13:59.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.050+0000 7fa575d1f700 1 -- 172.21.15.137:0/235859421 wait complete. 2024-06-15T01:13:59.051 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:13:59.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.155+0000 7fdcee20c700 1 -- 172.21.15.137:0/1596296076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce81003b0 msgr2=0x7fdce8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.155+0000 7fdcee20c700 1 --2- 172.21.15.137:0/1596296076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce81003b0 0x7fdce8100830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009910 tx=0x7fdcd8009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:59.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 -- 172.21.15.137:0/1596296076 shutdown_connections 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 --2- 172.21.15.137:0/1596296076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce81003b0 0x7fdce8100830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 --2- 172.21.15.137:0/1596296076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce8105c30 0x7fdce8106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 -- 172.21.15.137:0/1596296076 >> 172.21.15.137:0/1596296076 conn(0x7fdce8074d80 msgr2=0x7fdce8075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 -- 172.21.15.137:0/1596296076 shutdown_connections 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 -- 172.21.15.137:0/1596296076 wait complete. 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 Processor -- start 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.156+0000 7fdcee20c700 1 -- start start 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdcee20c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 0x7fdce81956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdcee20c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdcee20c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8196270 con 0x7fdce8105c30 2024-06-15T01:13:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdcee20c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8198fa0 con 0x7fdce81003b0 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36340/0 (socket says 172.21.15.137:36340) 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 -- 172.21.15.137:0/3200969155 learned_addr learned my addr 172.21.15.137:0/3200969155 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce77fe700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 0x7fdce81956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 -- 172.21.15.137:0/3200969155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 msgr2=0x7fdce81956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 0x7fdce81956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 -- 172.21.15.137:0/3200969155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80095f0 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce77fe700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 0x7fdce81956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce6ffd700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.157+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd801d070 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.158+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce8199220 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.158+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcd8005dc0 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.158+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd8003940 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.158+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce81996b0 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.159+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce81080e0 con 0x7fdce8105c30 2024-06-15T01:13:59.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.159+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fdcd8004ae0 con 0x7fdce8105c30 2024-06-15T01:13:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.159+0000 7fdce4ff9700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 0x7fdcd006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.160+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fdcd808b930 con 0x7fdce8105c30 2024-06-15T01:13:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.160+0000 7fdce77fe700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 0x7fdcd006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.160+0000 7fdce77fe700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 0x7fdcd006d440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc011490 tx=0x7fdcdc009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.164+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdcd80566e0 con 0x7fdce8105c30 2024-06-15T01:13:59.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.384+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdce8199980 con 0x7fdcd006af80 2024-06-15T01:13:59.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.385+0000 7fdce4ff9700 1 -- 172.21.15.137:0/3200969155 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdce8199980 con 0x7fdcd006af80 2024-06-15T01:13:59.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.387+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 msgr2=0x7fdcd006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.387+0000 7fdcee20c700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 0x7fdcd006d440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc011490 tx=0x7fdcdc009260 comp rx=0 tx=0).stop 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.387+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 msgr2=0x7fdce8195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.387+0000 7fdcee20c700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd800bde0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 shutdown_connections 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdcd006af80 0x7fdcd006d440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdce81003b0 0x7fdce81956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 --2- 172.21.15.137:0/3200969155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdce8105c30 0x7fdce8195be0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 >> 172.21.15.137:0/3200969155 conn(0x7fdce8074d80 msgr2=0x7fdce80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 shutdown_connections 2024-06-15T01:13:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.388+0000 7fdcee20c700 1 -- 172.21.15.137:0/3200969155 wait complete. 2024-06-15T01:13:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- 172.21.15.137:0/1972642865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac104200 msgr2=0x7f15ac1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 --2- 172.21.15.137:0/1972642865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac104200 0x7f15ac1045e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1594009910 tx=0x7f1594009c20 comp rx=0 tx=0).stop 2024-06-15T01:13:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- 172.21.15.137:0/1972642865 shutdown_connections 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 --2- 172.21.15.137:0/1972642865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac0ff580 0x7f15ac0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 --2- 172.21.15.137:0/1972642865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac104200 0x7f15ac1045e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- 172.21.15.137:0/1972642865 >> 172.21.15.137:0/1972642865 conn(0x7f15ac075dd0 msgr2=0x7f15ac0761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- 172.21.15.137:0/1972642865 shutdown_connections 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- 172.21.15.137:0/1972642865 wait complete. 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 Processor -- start 2024-06-15T01:13:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.493+0000 7f15b2796700 1 -- start start 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15b2796700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15b2796700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac104200 0x7f15ac196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15b2796700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac197140 con 0x7f15ac0ff580 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15b2796700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac197280 con 0x7f15ac104200 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36360/0 (socket says 172.21.15.137:36360) 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 -- 172.21.15.137:0/2255773736 learned_addr learned my addr 172.21.15.137:0/2255773736 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15ab7fe700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac104200 0x7f15ac196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 -- 172.21.15.137:0/2255773736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac104200 msgr2=0x7f15ac196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac104200 0x7f15ac196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 -- 172.21.15.137:0/2255773736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940095f0 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15abfff700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f159400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.494+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f159401d070 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.495+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac19b070 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.495+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f159400f460 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.495+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1594021640 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.495+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac19b5e0 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.496+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac108f10 con 0x7f15ac0ff580 2024-06-15T01:13:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.496+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f159400fa20 con 0x7f15ac0ff580 2024-06-15T01:13:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.497+0000 7f15a97fa700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 0x7f159806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.497+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f159408b6f0 con 0x7f15ac0ff580 2024-06-15T01:13:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.497+0000 7f15ab7fe700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 0x7f159806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.497+0000 7f15ab7fe700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 0x7f159806d440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f159c005ea0 tx=0x7f159c005e30 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.501+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f15940564a0 con 0x7f15ac0ff580 2024-06-15T01:13:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.699+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15ac02cd30 con 0x7f159806af80 2024-06-15T01:13:59.690 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: pgmap v222: 65 pgs: 9 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 30 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1.2 MiB/s wr, 94 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 2.4 MiB/s, 365 objects/s recovering 2024-06-15T01:13:59.690 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:59.691 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:13:59.691 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813] boot 2024-06-15T01:13:59.691 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: osdmap e81: 6 total, 6 up, 6 in 2024-06-15T01:13:59.691 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:13:59 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.709+0000 7f15a97fa700 1 -- 172.21.15.137:0/2255773736 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f15ac02cd30 con 0x7f159806af80 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (4m) 68s ago 15m 28.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (15m) 68s ago 15m 8238k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (14m) 5s ago 14m 7968k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (2m) 68s ago 15m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (2m) 5s ago 14m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:13:59.700 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (4m) 68s ago 14m 81.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (9m) 68s ago 9m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (9m) 68s ago 9m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (9m) 5s ago 9m 16.5M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (9m) 5s ago 9m 19.7M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (6m) 68s ago 16m 496M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (5m) 5s ago 14m 439M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (3m) 68s ago 16m 43.6M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (2m) 5s ago 13m 40.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (5m) 68s ago 15m 15.7M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (5m) 5s ago 14m 17.4M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (2m) 68s ago 13m 93.2M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (107s) 68s ago 12m 94.6M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:13:59.701 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (74s) 68s ago 12m 14.5M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:13:59.702 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (40s) 5s ago 11m 116M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:13:59.702 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (8s) 5s ago 11m 12.1M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:13:59.702 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (11m) 5s ago 11m 231M 2240M 18.2.1 5be31c24972a 14b42611cb52 2024-06-15T01:13:59.702 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (4m) 68s ago 14m 73.4M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 msgr2=0x7f159806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 0x7f159806d440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f159c005ea0 tx=0x7f159c005e30 comp rx=0 tx=0).stop 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 msgr2=0x7f15ac1965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f159400be20 comp rx=0 tx=0).stop 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 shutdown_connections 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f159806af80 0x7f159806d440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac0ff580 0x7f15ac1965b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 --2- 172.21.15.137:0/2255773736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15ac104200 0x7f15ac196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 >> 172.21.15.137:0/2255773736 conn(0x7f15ac075dd0 msgr2=0x7f15ac106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 shutdown_connections 2024-06-15T01:13:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.711+0000 7f15b2796700 1 -- 172.21.15.137:0/2255773736 wait complete. 2024-06-15T01:13:59.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 -- 172.21.15.137:0/616459268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 msgr2=0x7f2be8105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 --2- 172.21.15.137:0/616459268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8105e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b0a0 tx=0x7f2bd800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 -- 172.21.15.137:0/616459268 shutdown_connections 2024-06-15T01:13:59.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 --2- 172.21.15.137:0/616459268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 0x7f2be80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 --2- 172.21.15.137:0/616459268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8105e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 -- 172.21.15.137:0/616459268 >> 172.21.15.137:0/616459268 conn(0x7f2be80fb540 msgr2=0x7f2be80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:13:59.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 -- 172.21.15.137:0/616459268 shutdown_connections 2024-06-15T01:13:59.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.814+0000 7f2bed319700 1 -- 172.21.15.137:0/616459268 wait complete. 2024-06-15T01:13:59.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 Processor -- start 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 -- start start 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 0x7f2be8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8197290 con 0x7f2be8105a50 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2bed319700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be819b020 con 0x7f2be80ffa50 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2be67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2be67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36372/0 (socket says 172.21.15.137:36372) 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2be67fc700 1 -- 172.21.15.137:0/36091553 learned_addr learned my addr 172.21.15.137:0/36091553 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.815+0000 7f2be6ffd700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 0x7f2be8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2be67fc700 1 -- 172.21.15.137:0/36091553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 msgr2=0x7f2be8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:13:59.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2be67fc700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 0x7f2be8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2be67fc700 1 -- 172.21.15.137:0/36091553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009d00 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2be67fc700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d8d0 tx=0x7f2bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd0009940 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be819b300 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bd0010460 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd000f5f0 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.816+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be819b850 con 0x7f2be8105a50 2024-06-15T01:13:59.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.817+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be8109060 con 0x7f2be8105a50 2024-06-15T01:13:59.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.817+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2bd00105d0 con 0x7f2be8105a50 2024-06-15T01:13:59.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.818+0000 7f2bdffff700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 0x7f2bd406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:13:59.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.818+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f2bd008a4c0 con 0x7f2be8105a50 2024-06-15T01:13:59.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.818+0000 7f2be6ffd700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 0x7f2bd406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:13:59.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.818+0000 7f2be6ffd700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 0x7f2bd406d440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80062a0 tx=0x7f2bd80061f0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: pgmap v222: 65 pgs: 9 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 30 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1.2 MiB/s wr, 94 op/s; 37/327 objects degraded (11.315%); 17/327 objects misplaced (5.199%); 2.4 MiB/s, 365 objects/s recovering 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: osd.4 [v2:172.21.15.142:6808/464434813,v1:172.21.15.142:6809/464434813] boot 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: osdmap e81: 6 total, 6 up, 6 in 2024-06-15T01:13:59.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:13:59 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-15T01:13:59.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:13:59.822+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2bd0055210 con 0x7f2be8105a50 2024-06-15T01:14:00.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.097+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2be819ba70 con 0x7f2be8105a50 2024-06-15T01:14:00.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.097+0000 7f2bdffff700 1 -- 172.21.15.137:0/36091553 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f2bd0010880 con 0x7f2be8105a50 2024-06-15T01:14:00.088 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 5 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:14:00.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:00.090 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:14:00.090 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-06-15T01:14:00.090 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 9 2024-06-15T01:14:00.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:14:00.090 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.099+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 msgr2=0x7f2bd406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 0x7f2bd406d440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80062a0 tx=0x7f2bd80061f0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 msgr2=0x7f2be8196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d8d0 tx=0x7f2bd000dc90 comp rx=0 tx=0).stop 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 shutdown_connections 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2bd406af80 0x7f2bd406d440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be80ffa50 0x7f2be8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 --2- 172.21.15.137:0/36091553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8105a50 0x7f2be8196bb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 >> 172.21.15.137:0/36091553 conn(0x7f2be80fb540 msgr2=0x7f2be8069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 shutdown_connections 2024-06-15T01:14:00.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.100+0000 7f2bed319700 1 -- 172.21.15.137:0/36091553 wait complete. 2024-06-15T01:14:00.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.205+0000 7f4145a81700 1 -- 172.21.15.137:0/156826354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 msgr2=0x7f41400ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.205+0000 7f4145a81700 1 --2- 172.21.15.137:0/156826354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41400ff910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4130009910 tx=0x7f4130009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.205+0000 7f4145a81700 1 -- 172.21.15.137:0/156826354 shutdown_connections 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 --2- 172.21.15.137:0/156826354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41400ffee0 0x7f4140103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 --2- 172.21.15.137:0/156826354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41400ff910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 -- 172.21.15.137:0/156826354 >> 172.21.15.137:0/156826354 conn(0x7f4140074df0 msgr2=0x7f4140075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 -- 172.21.15.137:0/156826354 shutdown_connections 2024-06-15T01:14:00.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 -- 172.21.15.137:0/156826354 wait complete. 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 Processor -- start 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.206+0000 7f4145a81700 1 -- start start 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f4145a81700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f4145a81700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41400ffee0 0x7f4140197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f4145a81700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140198040 con 0x7f41400ff530 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f4145a81700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401981b0 con 0x7f41400ffee0 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36408/0 (socket says 172.21.15.137:36408) 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 -- 172.21.15.137:0/2200455908 learned_addr learned my addr 172.21.15.137:0/2200455908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 -- 172.21.15.137:0/2200455908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41400ffee0 msgr2=0x7f4140197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41400ffee0 0x7f4140197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 -- 172.21.15.137:0/2200455908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300095f0 con 0x7f41400ff530 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413f7fe700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4130006010 tx=0x7f413000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f413001d070 con 0x7f41400ff530 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f413000f460 con 0x7f41400ff530 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41401a0550 con 0x7f41400ff530 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.207+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4130021640 con 0x7f41400ff530 2024-06-15T01:14:00.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.208+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a09c0 con 0x7f41400ff530 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.209+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4140107820 con 0x7f41400ff530 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.209+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f413000f9e0 con 0x7f41400ff530 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.209+0000 7f413cff9700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 0x7f412806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.210+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f413008b770 con 0x7f41400ff530 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.210+0000 7f413effd700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 0x7f412806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.210+0000 7f413effd700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 0x7f412806d440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f414010b080 tx=0x7f4134005d20 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.214+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4130056520 con 0x7f41400ff530 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.472+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f414002cd60 con 0x7f41400ff530 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.472+0000 7f413cff9700 1 -- 172.21.15.137:0/2200455908 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f4130059b40 con 0x7f41400ff530 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stdout:e21 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:14:00.463 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 21 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:10:39.500783+0000 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:14:00.464 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 47 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=14480,1=14496} 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:14:00.465 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:14480} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.137:6826/183082556,v1:172.21.15.137:6827/183082556] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:14496} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.137:6828/937501599,v1:172.21.15.137:6829/937501599] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:00.466 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 msgr2=0x7f412806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 0x7f412806d440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f414010b080 tx=0x7f4134005d20 comp rx=0 tx=0).stop 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 msgr2=0x7f41401974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4130006010 tx=0x7f413000be20 comp rx=0 tx=0).stop 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 shutdown_connections 2024-06-15T01:14:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f412806af80 0x7f412806d440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41400ff530 0x7f41401974e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 --2- 172.21.15.137:0/2200455908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41400ffee0 0x7f4140197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 >> 172.21.15.137:0/2200455908 conn(0x7f4140074df0 msgr2=0x7f41400fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 shutdown_connections 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.474+0000 7f4145a81700 1 -- 172.21.15.137:0/2200455908 wait complete. 2024-06-15T01:14:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 -- 172.21.15.137:0/3790060592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 msgr2=0x7f344c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 --2- 172.21.15.137:0/3790060592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c1057c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3440009910 tx=0x7f3440009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 -- 172.21.15.137:0/3790060592 shutdown_connections 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 --2- 172.21.15.137:0/3790060592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c1057c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 --2- 172.21.15.137:0/3790060592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 0x7f344c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 -- 172.21.15.137:0/3790060592 >> 172.21.15.137:0/3790060592 conn(0x7f344c0fc670 msgr2=0x7f344c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 -- 172.21.15.137:0/3790060592 shutdown_connections 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.581+0000 7f3452170700 1 -- 172.21.15.137:0/3790060592 wait complete. 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 Processor -- start 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 -- start start 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 0x7f344c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c073b10 con 0x7f344c101770 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f3452170700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c073c50 con 0x7f344c100dc0 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f344affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f344affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36432/0 (socket says 172.21.15.137:36432) 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f344affd700 1 -- 172.21.15.137:0/2929449058 learned_addr learned my addr 172.21.15.137:0/2929449058 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:00.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.582+0000 7f344b7fe700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 0x7f344c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f344affd700 1 -- 172.21.15.137:0/2929449058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 msgr2=0x7f344c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f344affd700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 0x7f344c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f344affd700 1 -- 172.21.15.137:0/2929449058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400095f0 con 0x7f344c101770 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f344affd700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34400092e0 tx=0x7f344000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f344001d070 con 0x7f344c101770 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f344000f460 con 0x7f344c101770 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c073ed0 con 0x7f344c101770 2024-06-15T01:14:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3440021620 con 0x7f344c101770 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.583+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c1a2b50 con 0x7f344c101770 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.584+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c1091c0 con 0x7f344c101770 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.585+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f344000f5d0 con 0x7f344c101770 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.585+0000 7f3448ff9700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 0x7f343406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.585+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6454+0+0 (secure 0 0 0) 0x7f344008b5a0 con 0x7f344c101770 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.585+0000 7f344b7fe700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 0x7f343406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.586+0000 7f344b7fe700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 0x7f343406d440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f343c005950 tx=0x7f343c00a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.589+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3440056600 con 0x7f344c101770 2024-06-15T01:14:00.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:00 smithi137 ceph-mon[112295]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:00.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:00 smithi137 ceph-mon[112295]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:00.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:00 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/36091553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:00.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:00 smithi137 ceph-mon[112295]: osdmap e82: 6 total, 6 up, 6 in 2024-06-15T01:14:00.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:00 smithi142 ceph-mon[82136]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:00.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:00 smithi142 ceph-mon[82136]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:00.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:00 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/36091553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:00.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:00 smithi142 ceph-mon[82136]: osdmap e82: 6 total, 6 up, 6 in 2024-06-15T01:14:00.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.802+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f344c02cd50 con 0x7f343406af80 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.803+0000 7f3448ff9700 1 -- 172.21.15.137:0/2929449058 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f344c02cd50 con 0x7f343406af80 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:14:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "11/23 daemons upgraded", 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading osd daemons", 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:14:00.795 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:14:00.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 msgr2=0x7f343406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 0x7f343406d440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f343c005950 tx=0x7f343c00a300 comp rx=0 tx=0).stop 2024-06-15T01:14:00.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 msgr2=0x7f344c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34400092e0 tx=0x7f344000bd70 comp rx=0 tx=0).stop 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 shutdown_connections 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f343406af80 0x7f343406d440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f344c100dc0 0x7f344c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 --2- 172.21.15.137:0/2929449058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c101770 0x7f344c0735d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 >> 172.21.15.137:0/2929449058 conn(0x7f344c0fc670 msgr2=0x7f344c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 shutdown_connections 2024-06-15T01:14:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.805+0000 7f3452170700 1 -- 172.21.15.137:0/2929449058 wait complete. 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- 172.21.15.137:0/1712618032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 msgr2=0x7fe7a41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 --2- 172.21.15.137:0/1712618032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a41011a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe794009910 tx=0x7fe794009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- 172.21.15.137:0/1712618032 shutdown_connections 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 --2- 172.21.15.137:0/1712618032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 0x7fe7a41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 --2- 172.21.15.137:0/1712618032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a41011a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- 172.21.15.137:0/1712618032 >> 172.21.15.137:0/1712618032 conn(0x7fe7a40fc650 msgr2=0x7fe7a40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- 172.21.15.137:0/1712618032 shutdown_connections 2024-06-15T01:14:00.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- 172.21.15.137:0/1712618032 wait complete. 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 Processor -- start 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.909+0000 7fe7a8974700 1 -- start start 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a8974700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a8974700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 0x7fe7a4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a8974700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4197260 con 0x7fe7a4100dc0 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a8974700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4199f90 con 0x7fe7a4101770 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36456/0 (socket says 172.21.15.137:36456) 2024-06-15T01:14:00.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 -- 172.21.15.137:0/2910212037 learned_addr learned my addr 172.21.15.137:0/2910212037 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a1d9b700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 0x7fe7a4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 -- 172.21.15.137:0/2910212037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 msgr2=0x7fe7a4196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 0x7fe7a4196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 -- 172.21.15.137:0/2910212037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940095f0 con 0x7fe7a4100dc0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7a259c700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b5c0 tx=0x7fe79400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.910+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79401d070 con 0x7fe7a4100dc0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.911+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a419a210 con 0x7fe7a4100dc0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.911+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe79400f460 con 0x7fe7a4100dc0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.911+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe794021640 con 0x7fe7a4100dc0 2024-06-15T01:14:00.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.911+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a419a700 con 0x7fe7a4100dc0 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.912+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a4109160 con 0x7fe7a4100dc0 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.912+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe79400fa20 con 0x7fe7a4100dc0 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.912+0000 7fe7937fe700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 0x7fe78c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.913+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6454+0+0 (secure 0 0 0) 0x7fe79408b520 con 0x7fe7a4100dc0 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.913+0000 7fe7a1d9b700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 0x7fe78c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:00.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.913+0000 7fe7a1d9b700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 0x7fe78c06d440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4197cb0 tx=0x7fe79800a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:00.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:00.917+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe7940563e0 con 0x7fe7a4100dc0 2024-06-15T01:14:01.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.188+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7a4197af0 con 0x7fe7a4100dc0 2024-06-15T01:14:01.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.189+0000 7fe7937fe700 1 -- 172.21.15.137:0/2910212037 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1353 (secure 0 0 0) 0x7fe794059a00 con 0x7fe7a4100dc0 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN Degraded data redundancy: 37/327 objects degraded (11.315%), 11 pgs degraded; 4 mgr modules have recently crashed 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 37/327 objects degraded (11.315%), 11 pgs degraded 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-06-15T01:14:01.180 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-06-15T01:14:01.181 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:14:01.181 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:14:01.181 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:14:01.181 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:14:01.181 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 msgr2=0x7fe78c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 0x7fe78c06d440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4197cb0 tx=0x7fe79800a300 comp rx=0 tx=0).stop 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 msgr2=0x7fe7a4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b5c0 tx=0x7fe79400bd70 comp rx=0 tx=0).stop 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 shutdown_connections 2024-06-15T01:14:01.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe78c06af80 0x7fe78c06d440 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:01.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7a4100dc0 0x7fe7a4196690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:01.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 --2- 172.21.15.137:0/2910212037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a4101770 0x7fe7a4196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:01.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 >> 172.21.15.137:0/2910212037 conn(0x7fe7a40fc650 msgr2=0x7fe7a40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:01.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 shutdown_connections 2024-06-15T01:14:01.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:01.191+0000 7fe7a8974700 1 -- 172.21.15.137:0/2910212037 wait complete. 2024-06-15T01:14:01.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:01 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2200455908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:14:01.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:01 smithi137 ceph-mon[112295]: pgmap v225: 65 pgs: 6 peering, 9 active+clean+remapped, 12 active+undersized, 8 active+undersized+degraded, 30 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/327 objects degraded (7.951%); 17/327 objects misplaced (5.199%); 0 B/s, 237 objects/s recovering 2024-06-15T01:14:01.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:01 smithi137 ceph-mon[112295]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:01.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:01 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2910212037' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:14:01.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:01 smithi137 ceph-mon[112295]: osdmap e83: 6 total, 6 up, 6 in 2024-06-15T01:14:01.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:01 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2200455908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:14:01.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:01 smithi142 ceph-mon[82136]: pgmap v225: 65 pgs: 6 peering, 9 active+clean+remapped, 12 active+undersized, 8 active+undersized+degraded, 30 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/327 objects degraded (7.951%); 17/327 objects misplaced (5.199%); 0 B/s, 237 objects/s recovering 2024-06-15T01:14:01.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:01 smithi142 ceph-mon[82136]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:01.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:01 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2910212037' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:14:01.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:01 smithi142 ceph-mon[82136]: osdmap e83: 6 total, 6 up, 6 in 2024-06-15T01:14:02.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:02 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 26/327 objects degraded (7.951%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:02.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:02 smithi137 ceph-mon[112295]: osdmap e84: 6 total, 6 up, 6 in 2024-06-15T01:14:02.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:02 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 26/327 objects degraded (7.951%), 8 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:02.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:02 smithi142 ceph-mon[82136]: osdmap e84: 6 total, 6 up, 6 in 2024-06-15T01:14:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:04 smithi137 ceph-mon[112295]: pgmap v228: 65 pgs: 6 peering, 5 active+clean+remapped, 5 active+undersized, 4 active+undersized+degraded, 45 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/327 objects degraded (3.058%); 13/327 objects misplaced (3.976%) 2024-06-15T01:14:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:04.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:04 smithi137 ceph-mon[112295]: osdmap e85: 6 total, 6 up, 6 in 2024-06-15T01:14:04.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:04 smithi142 ceph-mon[82136]: pgmap v228: 65 pgs: 6 peering, 5 active+clean+remapped, 5 active+undersized, 4 active+undersized+degraded, 45 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10/327 objects degraded (3.058%); 13/327 objects misplaced (3.976%) 2024-06-15T01:14:04.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:04.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:04.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:04 smithi142 ceph-mon[82136]: osdmap e85: 6 total, 6 up, 6 in 2024-06-15T01:14:05.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:05 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/327 objects degraded (3.058%), 4 pgs degraded) 2024-06-15T01:14:05.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:05 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/327 objects degraded (3.058%), 4 pgs degraded) 2024-06-15T01:14:06.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:06 smithi142 ceph-mon[82136]: pgmap v230: 65 pgs: 6 peering, 59 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 161 objects/s recovering 2024-06-15T01:14:06.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:06 smithi137 ceph-mon[112295]: pgmap v230: 65 pgs: 6 peering, 59 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 161 objects/s recovering 2024-06-15T01:14:08.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:08 smithi142 ceph-mon[82136]: pgmap v231: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 119 objects/s recovering 2024-06-15T01:14:08.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:08 smithi137 ceph-mon[112295]: pgmap v231: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 119 objects/s recovering 2024-06-15T01:14:10.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:10 smithi142 ceph-mon[82136]: pgmap v232: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 96 objects/s recovering 2024-06-15T01:14:10.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:10 smithi137 ceph-mon[112295]: pgmap v232: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 96 objects/s recovering 2024-06-15T01:14:12.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:12 smithi142 ceph-mon[82136]: pgmap v233: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 86 objects/s recovering 2024-06-15T01:14:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:12 smithi137 ceph-mon[112295]: pgmap v233: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 86 objects/s recovering 2024-06-15T01:14:13.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:13 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:14:13.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:13 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:13.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:13 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:14:13.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:13 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:14.332 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:14 smithi142 ceph-mon[82136]: pgmap v234: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 71 objects/s recovering 2024-06-15T01:14:14.332 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:14 smithi142 ceph-mon[82136]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:14:14.333 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:14 smithi142 ceph-mon[82136]: Upgrade: osd.5 is safe to restart 2024-06-15T01:14:14.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:14 smithi137 ceph-mon[112295]: pgmap v234: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 71 objects/s recovering 2024-06-15T01:14:14.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:14 smithi137 ceph-mon[112295]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-06-15T01:14:14.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:14 smithi137 ceph-mon[112295]: Upgrade: osd.5 is safe to restart 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: Upgrade: Updating osd.5 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: Deploying daemon osd.5 on smithi142 2024-06-15T01:14:15.641 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:15 smithi142 ceph-mon[82136]: pgmap v235: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 63 objects/s recovering 2024-06-15T01:14:15.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: Upgrade: Updating osd.5 2024-06-15T01:14:15.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:15.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-15T01:14:15.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:15.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: Deploying daemon osd.5 on smithi142 2024-06-15T01:14:15.721 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:15 smithi137 ceph-mon[112295]: pgmap v235: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 63 objects/s recovering 2024-06-15T01:14:17.553 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:17 smithi142 systemd[1]: Stopping Ceph osd.5 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:14:17.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:17 smithi142 ceph-mon[82136]: pgmap v236: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:17.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:17 smithi142 ceph-mon[82136]: osd.5 marked itself down and dead 2024-06-15T01:14:17.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:17 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[56598]: 2024-06-15T01:14:17.562+0000 7fb1d7c98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:14:17.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:17 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[56598]: 2024-06-15T01:14:17.562+0000 7fb1d7c98700 -1 osd.5 85 *** Got signal Terminated *** 2024-06-15T01:14:17.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:17 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[56598]: 2024-06-15T01:14:17.562+0000 7fb1d7c98700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:14:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:17 smithi137 ceph-mon[112295]: pgmap v236: 65 pgs: 65 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:18.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:17 smithi137 ceph-mon[112295]: osd.5 marked itself down and dead 2024-06-15T01:14:18.435 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:18 smithi142 podman[99861]: 2024-06-15 01:14:18.175018029 +0000 UTC m=+0.715856102 container died 14b42611cb52a9a8d1c51d74f3015b71d9d91a3512ac0ac308a7a09c6395fb36 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-06-15T01:14:18.736 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:18 smithi142 podman[99861]: 2024-06-15 01:14:18.537086689 +0000 UTC m=+1.077924762 container cleanup 14b42611cb52a9a8d1c51d74f3015b71d9d91a3512ac0ac308a7a09c6395fb36 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-15T01:14:18.736 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:18 smithi142 bash[99861]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5 2024-06-15T01:14:19.043 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:18 smithi137 ceph-mon[112295]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:14:19.043 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:18 smithi137 ceph-mon[112295]: osdmap e86: 6 total, 5 up, 6 in 2024-06-15T01:14:19.044 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:19.044 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:19.061 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:18 smithi142 podman[99877]: 2024-06-15 01:14:18.838349694 +0000 UTC m=+0.659601260 container remove 14b42611cb52a9a8d1c51d74f3015b71d9d91a3512ac0ac308a7a09c6395fb36 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux ) 2024-06-15T01:14:19.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:18 smithi142 ceph-mon[82136]: Health check failed: 1 osds down (OSD_DOWN) 2024-06-15T01:14:19.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:18 smithi142 ceph-mon[82136]: osdmap e86: 6 total, 5 up, 6 in 2024-06-15T01:14:19.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:19.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:19.414 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.066949925 +0000 UTC m=+0.017495681 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:14:19.415 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.181071204 +0000 UTC m=+0.131617003 container create ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-15T01:14:19.415 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.425295371 +0000 UTC m=+0.375841127 container init ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-06-15T01:14:19.671 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.428789023 +0000 UTC m=+0.379334788 container start ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-15T01:14:19.671 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.508420569 +0000 UTC m=+0.458966353 container attach ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True) 2024-06-15T01:14:20.018 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:19 smithi142 ceph-mon[82136]: pgmap v238: 65 pgs: 15 stale+active+clean, 50 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:20.018 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:19 smithi142 ceph-mon[82136]: osdmap e87: 6 total, 5 up, 6 in 2024-06-15T01:14:20.018 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:19 smithi142 podman[99968]: 2024-06-15 01:14:19.68163819 +0000 UTC m=+0.632183949 container died ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-15T01:14:20.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:19 smithi137 ceph-mon[112295]: pgmap v238: 65 pgs: 15 stale+active+clean, 50 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:20.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:19 smithi137 ceph-mon[112295]: osdmap e87: 6 total, 5 up, 6 in 2024-06-15T01:14:20.272 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 podman[99991]: 2024-06-15 01:14:20.26997803 +0000 UTC m=+0.581997696 container remove ad436baa359251787fa3593496989cb9f5d5401a982840e4c5024077e130f8e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-15T01:14:20.562 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.5.service: Deactivated successfully. 2024-06-15T01:14:20.562 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 systemd[1]: Stopped Ceph osd.5 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:14:20.562 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.5.service: Consumed 1min 8.447s CPU time. 2024-06-15T01:14:20.562 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 systemd[1]: Starting Ceph osd.5 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:14:20.955 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:20 smithi142 ceph-mon[82136]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-15T01:14:20.955 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 podman[100093]: 2024-06-15 01:14:20.602804756 +0000 UTC m=+0.017162014 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:14:20.955 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 podman[100093]: 2024-06-15 01:14:20.712534105 +0000 UTC m=+0.126891353 container create c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:14:20.955 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 podman[100093]: 2024-06-15 01:14:20.965421885 +0000 UTC m=+0.379779152 container init c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:14:21.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:20 smithi137 ceph-mon[112295]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-15T01:14:21.313 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:20 smithi142 podman[100093]: 2024-06-15 01:14:20.969275874 +0000 UTC m=+0.383633129 container start c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-15T01:14:21.314 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 podman[100093]: 2024-06-15 01:14:21.056535921 +0000 UTC m=+0.470893188 container attach c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-15T01:14:21.706 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate[100108]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 bash[100093]: --> ceph-volume raw activate successful for osd ID: 5 2024-06-15T01:14:21.707 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 podman[100093]: 2024-06-15 01:14:21.38631797 +0000 UTC m=+0.800675226 container died c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-15T01:14:21.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:21 smithi142 ceph-mon[82136]: pgmap v240: 65 pgs: 3 active+undersized, 4 peering, 12 stale+active+clean, 46 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:22.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:21 smithi137 ceph-mon[112295]: pgmap v240: 65 pgs: 3 active+undersized, 4 peering, 12 stale+active+clean, 46 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:22.273 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:21 smithi142 podman[100240]: 2024-06-15 01:14:21.976948651 +0000 UTC m=+0.580086156 container remove c341f4f5fcd9299324243957d4d449fecddb160e347f2c2331e41279d7f01688 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:14:22.273 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 podman[100300]: 2024-06-15 01:14:22.186137389 +0000 UTC m=+0.112862121 container create ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-15T01:14:22.273 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 podman[100300]: 2024-06-15 01:14:22.090772547 +0000 UTC m=+0.017497275 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:14:22.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 podman[100300]: 2024-06-15 01:14:22.422135447 +0000 UTC m=+0.348860171 container init ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:14:22.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 podman[100300]: 2024-06-15 01:14:22.425520424 +0000 UTC m=+0.352245144 container start ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3) 2024-06-15T01:14:22.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 ceph-osd[100319]: -- 172.21.15.142:0/3786064599 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ea6e631e0 con 0x563ea6e01c00 2024-06-15T01:14:22.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 bash[100300]: ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 2024-06-15T01:14:22.812 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 systemd[1]: Started Ceph osd.5 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:14:23.278 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:22 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:14:22.965+0000 7f57e7237640 -1 Falling back to public interface 2024-06-15T01:14:23.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:23.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:23.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:23 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:23.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:23 smithi142 ceph-mon[82136]: pgmap v241: 65 pgs: 14 active+undersized, 4 peering, 13 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 45/327 objects degraded (13.761%) 2024-06-15T01:14:23.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:23.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:23.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:23 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:23.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:23 smithi137 ceph-mon[112295]: pgmap v241: 65 pgs: 14 active+undersized, 4 peering, 13 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 45/327 objects degraded (13.761%) 2024-06-15T01:14:24.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:24 smithi142 ceph-mon[82136]: Health check failed: Degraded data redundancy: 45/327 objects degraded (13.761%), 13 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:24.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:24 smithi137 ceph-mon[112295]: Health check failed: Degraded data redundancy: 45/327 objects degraded (13.761%), 13 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:25.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:25 smithi142 ceph-mon[82136]: pgmap v242: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:25.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:25.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:25 smithi137 ceph-mon[112295]: pgmap v242: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:26.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:26 smithi142 ceph-mon[82136]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-15T01:14:26.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:26.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:26.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:26 smithi137 ceph-mon[112295]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-15T01:14:26.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:26.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:27.518 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:27 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:14:27.440+0000 7f57e7237640 -1 osd.5 0 read_superblock omap replica is missing. 2024-06-15T01:14:27.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:27 smithi142 ceph-mon[82136]: pgmap v243: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:27.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:27 smithi137 ceph-mon[112295]: pgmap v243: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:28.311 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:27 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:14:27.908+0000 7f57e7237640 -1 osd.5 85 log_to_monitors true 2024-06-15T01:14:28.933 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:28 smithi142 ceph-mon[82136]: from='osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:14:28.933 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:28 smithi142 ceph-mon[82136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:14:28.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:28 smithi137 ceph-mon[112295]: from='osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:14:28.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:28 smithi137 ceph-mon[112295]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-15T01:14:29.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:14:29.155+0000 7f9ac2041640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:14:29.777 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-15T01:14:29.777 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: osdmap e88: 6 total, 5 up, 6 in 2024-06-15T01:14:29.777 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:14:29.777 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: pgmap v245: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.778 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all osd 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-15T01:14:29.779 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: osdmap e89: 6 total, 5 up, 6 in 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-06-15T01:14:29.780 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:29 smithi137 ceph-mon[112295]: from='osd.5 ' entity='osd.5' 2024-06-15T01:14:29.811 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:14:29.422+0000 7f57dafc8640 -1 osd.5 85 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-15T01:14:29.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-15T01:14:29.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: osdmap e88: 6 total, 5 up, 6 in 2024-06-15T01:14:29.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:14:29.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-06-15T01:14:29.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: pgmap v245: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all osd 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-06-15T01:14:29.814 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: osdmap e89: 6 total, 5 up, 6 in 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-06-15T01:14:29.815 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:29 smithi142 ceph-mon[82136]: from='osd.5 ' entity='osd.5' 2024-06-15T01:14:31.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 -- 172.21.15.137:0/3417068735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb00ffbd0 msgr2=0x7f3fb0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 --2- 172.21.15.137:0/3417068735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0100050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009910 tx=0x7f3fa0009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:31.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 -- 172.21.15.137:0/3417068735 shutdown_connections 2024-06-15T01:14:31.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 --2- 172.21.15.137:0/3417068735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0100050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 --2- 172.21.15.137:0/3417068735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb0105a40 0x7f3fb0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 -- 172.21.15.137:0/3417068735 >> 172.21.15.137:0/3417068735 conn(0x7f3fb00755a0 msgr2=0x7f3fb00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 -- 172.21.15.137:0/3417068735 shutdown_connections 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.300+0000 7f3fb79de700 1 -- 172.21.15.137:0/3417068735 wait complete. 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 Processor -- start 2024-06-15T01:14:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 -- start start 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0199520 con 0x7f3fb0105a40 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb79de700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb019d330 con 0x7f3fb00ffbd0 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb4f79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb4f79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58718/0 (socket says 172.21.15.137:58718) 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb4f79700 1 -- 172.21.15.137:0/2969034579 learned_addr learned my addr 172.21.15.137:0/2969034579 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb577a700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb4f79700 1 -- 172.21.15.137:0/2969034579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb00ffbd0 msgr2=0x7f3fb0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.301+0000 7f3fb4f79700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fb4f79700 1 -- 172.21.15.137:0/2969034579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00095f0 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fb4f79700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa001d070 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fa000f460 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb019d5b0 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa0021640 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.302+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb019daa0 con 0x7f3fb0105a40 2024-06-15T01:14:31.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.303+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb004ea90 con 0x7f3fb0105a40 2024-06-15T01:14:31.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.303+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3fa000fa60 con 0x7f3fb0105a40 2024-06-15T01:14:31.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.304+0000 7f3fa67fc700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 0x7f3f9c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.304+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3fa008b6c0 con 0x7f3fb0105a40 2024-06-15T01:14:31.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.304+0000 7f3fb577a700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 0x7f3f9c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.304+0000 7f3fb577a700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 0x7f3f9c06d440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fac006fd0 tx=0x7f3fac008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:31.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.308+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3fa0059f80 con 0x7f3fb0105a40 2024-06-15T01:14:31.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3864353360,v1:172.21.15.137:6828/3864353360] up:boot 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/850529376,v1:172.21.15.137:6829/850529376] up:boot 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146] boot 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: osdmap e90: 6 total, 6 up, 6 in 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:14:31.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:31 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:31.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3864353360,v1:172.21.15.137:6828/3864353360] up:boot 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/850529376,v1:172.21.15.137:6829/850529376] up:boot 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: osd.5 [v2:172.21.15.142:6816/1909901146,v1:172.21.15.142:6817/1909901146] boot 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: osdmap e90: 6 total, 6 up, 6 in 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.nsntjq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:14:31.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:31 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:31.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.542+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fb019dd80 con 0x7f3f9c06af80 2024-06-15T01:14:31.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.543+0000 7f3fa67fc700 1 -- 172.21.15.137:0/2969034579 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3fb019dd80 con 0x7f3f9c06af80 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 msgr2=0x7f3f9c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 0x7f3f9c06d440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fac006fd0 tx=0x7f3fac008040 comp rx=0 tx=0).stop 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 msgr2=0x7f3fb0198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa000be20 comp rx=0 tx=0).stop 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 shutdown_connections 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f9c06af80 0x7f3f9c06d440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb00ffbd0 0x7f3fb0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 --2- 172.21.15.137:0/2969034579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0105a40 0x7f3fb0198e90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 >> 172.21.15.137:0/2969034579 conn(0x7f3fb00755a0 msgr2=0x7f3fb00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:31.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 shutdown_connections 2024-06-15T01:14:31.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.545+0000 7f3fb79de700 1 -- 172.21.15.137:0/2969034579 wait complete. 2024-06-15T01:14:31.547 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:14:31.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/2474658816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb640ffa70 msgr2=0x7fcb640ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/2474658816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb640ffa70 0x7fcb640ffef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009910 tx=0x7fcb60009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/2474658816 shutdown_connections 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/2474658816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb640ffa70 0x7fcb640ffef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/2474658816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb64105a70 0x7fcb64105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/2474658816 >> 172.21.15.137:0/2474658816 conn(0x7fcb640fb560 msgr2=0x7fcb640fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/2474658816 shutdown_connections 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.652+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/2474658816 wait complete. 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 Processor -- start 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 -- start start 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb640ffa70 0x7fcb64195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64196250 con 0x7fcb64105a70 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6c5fa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64198f80 con 0x7fcb640ffa70 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58728/0 (socket says 172.21.15.137:58728) 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 -- 172.21.15.137:0/3692906269 learned_addr learned my addr 172.21.15.137:0/3692906269 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb6a396700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb640ffa70 0x7fcb64195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 -- 172.21.15.137:0/3692906269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb640ffa70 msgr2=0x7fcb64195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb640ffa70 0x7fcb64195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.653+0000 7fcb69b95700 1 -- 172.21.15.137:0/3692906269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600095f0 con 0x7fcb64105a70 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb69b95700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006010 tx=0x7fcb6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb6001d070 con 0x7fcb64105a70 2024-06-15T01:14:31.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb6000f460 con 0x7fcb64105a70 2024-06-15T01:14:31.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb60021640 con 0x7fcb64105a70 2024-06-15T01:14:31.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb64199200 con 0x7fcb64105a70 2024-06-15T01:14:31.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.654+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb641996f0 con 0x7fcb64105a70 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.655+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64108090 con 0x7fcb64105a70 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.655+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcb60003a60 con 0x7fcb64105a70 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.656+0000 7fcb5b7fe700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 0x7fcb5006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.656+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcb6008b6a0 con 0x7fcb64105a70 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.656+0000 7fcb6a396700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 0x7fcb5006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.656+0000 7fcb6a396700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 0x7fcb5006d440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006fd0 tx=0x7fcb54008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:31.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.660+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcb600564d0 con 0x7fcb64105a70 2024-06-15T01:14:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.874+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb6402cd30 con 0x7fcb5006af80 2024-06-15T01:14:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.875+0000 7fcb5b7fe700 1 -- 172.21.15.137:0/3692906269 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcb6402cd30 con 0x7fcb5006af80 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 msgr2=0x7fcb5006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 0x7fcb5006d440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb54006fd0 tx=0x7fcb54008040 comp rx=0 tx=0).stop 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 msgr2=0x7fcb64195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006010 tx=0x7fcb6000be20 comp rx=0 tx=0).stop 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 shutdown_connections 2024-06-15T01:14:31.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcb5006af80 0x7fcb5006d440 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb640ffa70 0x7fcb64195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 --2- 172.21.15.137:0/3692906269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb64105a70 0x7fcb64195bc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.877+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 >> 172.21.15.137:0/3692906269 conn(0x7fcb640fb560 msgr2=0x7fcb640fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:31.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.878+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 shutdown_connections 2024-06-15T01:14:31.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:31.878+0000 7fcb6c5fa700 1 -- 172.21.15.137:0/3692906269 wait complete. 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- 172.21.15.137:0/3693149963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c101770 msgr2=0x7f0c8c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 --2- 172.21.15.137:0/3693149963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c101770 0x7f0c8c1057c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009910 tx=0x7f0c7c009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- 172.21.15.137:0/3693149963 shutdown_connections 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 --2- 172.21.15.137:0/3693149963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c101770 0x7f0c8c1057c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 --2- 172.21.15.137:0/3693149963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- 172.21.15.137:0/3693149963 >> 172.21.15.137:0/3693149963 conn(0x7f0c8c0fc650 msgr2=0x7f0c8c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- 172.21.15.137:0/3693149963 shutdown_connections 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- 172.21.15.137:0/3693149963 wait complete. 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 Processor -- start 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.004+0000 7f0c935d0700 1 -- start start 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c935d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c935d0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 0x7f0c8c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c935d0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c199530 con 0x7f0c8c100dc0 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c935d0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c19d340 con 0x7f0c8c101770 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58750/0 (socket says 172.21.15.137:58750) 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 -- 172.21.15.137:0/1422380355 learned_addr learned my addr 172.21.15.137:0/1422380355 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c90b6b700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 0x7f0c8c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 -- 172.21.15.137:0/1422380355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 msgr2=0x7f0c8c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 0x7f0c8c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 -- 172.21.15.137:0/1422380355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0095f0 con 0x7f0c8c100dc0 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c90b6b700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 0x7f0c8c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c9136c700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800b4b0 tx=0x7f0c8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.005+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c88011800 con 0x7f0c8c100dc0 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.006+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c88011e00 con 0x7f0c8c100dc0 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.006+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c88010e50 con 0x7f0c8c100dc0 2024-06-15T01:14:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.006+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c19d620 con 0x7f0c8c100dc0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.006+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c19db20 con 0x7f0c8c100dc0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.006+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c104f40 con 0x7f0c8c100dc0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.007+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0c88010380 con 0x7f0c8c100dc0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.008+0000 7f0c827fc700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 0x7f0c7806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.008+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0c88089f70 con 0x7f0c8c100dc0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.008+0000 7f0c90b6b700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 0x7f0c7806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.008+0000 7f0c90b6b700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 0x7f0c7806d440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:32.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.016+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0c880bd900 con 0x7f0c8c100dc0 2024-06-15T01:14:32.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.242+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c8c19ddf0 con 0x7f0c7806af80 2024-06-15T01:14:32.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.253+0000 7f0c827fc700 1 -- 172.21.15.137:0/1422380355 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0c8c19ddf0 con 0x7f0c7806af80 2024-06-15T01:14:32.243 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (5m) 101s ago 15m 28.3M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (15m) 101s ago 15m 8238k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (14m) 6s ago 14m 8006k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (3m) 101s ago 15m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (3m) 6s ago 14m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (4m) 101s ago 15m 81.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (10m) 101s ago 10m 115M - 18.2.1 5be31c24972a 44b9bfd795f9 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (10m) 101s ago 10m 129M - 18.2.1 5be31c24972a 6d8b29528060 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (10m) 6s ago 10m 16.7M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (10m) 6s ago 10m 19.9M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (6m) 101s ago 17m 496M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:14:32.244 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (5m) 6s ago 14m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (3m) 101s ago 17m 43.6M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (3m) 6s ago 14m 41.5M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (5m) 101s ago 15m 15.7M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (5m) 6s ago 14m 17.2M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (2m) 101s ago 13m 93.2M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (2m) 101s ago 13m 94.6M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (106s) 101s ago 12m 14.5M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (72s) 6s ago 12m 113M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (41s) 6s ago 12m 108M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:14:32.245 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (10s) 6s ago 11m 14.5M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:14:32.246 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (5m) 101s ago 15m 73.4M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:14:32.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 msgr2=0x7f0c7806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 0x7f0c7806d440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c005960 comp rx=0 tx=0).stop 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 msgr2=0x7f0c8c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800b4b0 tx=0x7f0c8800b870 comp rx=0 tx=0).stop 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 shutdown_connections 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c7806af80 0x7f0c7806d440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c8c100dc0 0x7f0c8c198960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 --2- 172.21.15.137:0/1422380355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8c101770 0x7f0c8c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 >> 172.21.15.137:0/1422380355 conn(0x7f0c8c0fc650 msgr2=0x7f0c8c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 shutdown_connections 2024-06-15T01:14:32.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.255+0000 7f0c935d0700 1 -- 172.21.15.137:0/1422380355 wait complete. 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 -- 172.21.15.137:0/3540239672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 msgr2=0x7f293c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 --2- 172.21.15.137:0/3540239672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c105e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f293400b0a0 tx=0x7f293400b3b0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 -- 172.21.15.137:0/3540239672 shutdown_connections 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 --2- 172.21.15.137:0/3540239672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 0x7f293c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 --2- 172.21.15.137:0/3540239672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c105e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 -- 172.21.15.137:0/3540239672 >> 172.21.15.137:0/3540239672 conn(0x7f293c0755a0 msgr2=0x7f293c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:32.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:32 smithi137 ceph-mon[112295]: Upgrade: Updating mds.cephfs.smithi137.nsntjq 2024-06-15T01:14:32.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:32 smithi137 ceph-mon[112295]: Deploying daemon mds.cephfs.smithi137.nsntjq on smithi137 2024-06-15T01:14:32.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:32 smithi137 ceph-mon[112295]: pgmap v248: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:32.359 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:32 smithi137 ceph-mon[112295]: osdmap e91: 6 total, 6 up, 6 in 2024-06-15T01:14:32.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 -- 172.21.15.137:0/3540239672 shutdown_connections 2024-06-15T01:14:32.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.367+0000 7f29436b3700 1 -- 172.21.15.137:0/3540239672 wait complete. 2024-06-15T01:14:32.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 Processor -- start 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 -- start start 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 0x7f293c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c198040 con 0x7f293c105a40 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.368+0000 7f29436b3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c198180 con 0x7f293c0ffbd0 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58768/0 (socket says 172.21.15.137:58768) 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 -- 172.21.15.137:0/3693589200 learned_addr learned my addr 172.21.15.137:0/3693589200 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f294144f700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 0x7f293c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 -- 172.21.15.137:0/3693589200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 msgr2=0x7f293c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 0x7f293c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 -- 172.21.15.137:0/3693589200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934009d00 con 0x7f293c105a40 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f2940c4e700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f293000c8f0 tx=0x7f293000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:32.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2930007a70 con 0x7f293c105a40 2024-06-15T01:14:32.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2930004cd0 con 0x7f293c105a40 2024-06-15T01:14:32.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c1a0550 con 0x7f293c105a40 2024-06-15T01:14:32.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2930005580 con 0x7f293c105a40 2024-06-15T01:14:32.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.369+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c1a0a10 con 0x7f293c105a40 2024-06-15T01:14:32.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.370+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c108040 con 0x7f293c105a40 2024-06-15T01:14:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.371+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2930004750 con 0x7f293c105a40 2024-06-15T01:14:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.371+0000 7f293a7fc700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 0x7f292406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.371+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2930013070 con 0x7f293c105a40 2024-06-15T01:14:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.371+0000 7f294144f700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 0x7f292406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.372+0000 7f294144f700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 0x7f292406d440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2934009850 tx=0x7f29340097c0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.376+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2930054ff0 con 0x7f293c105a40 2024-06-15T01:14:32.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:32 smithi142 ceph-mon[82136]: Upgrade: Updating mds.cephfs.smithi137.nsntjq 2024-06-15T01:14:32.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:32 smithi142 ceph-mon[82136]: Deploying daemon mds.cephfs.smithi137.nsntjq on smithi137 2024-06-15T01:14:32.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:32 smithi142 ceph-mon[82136]: pgmap v248: 65 pgs: 15 active+undersized, 16 active+undersized+degraded, 34 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 57/327 objects degraded (17.431%) 2024-06-15T01:14:32.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:32 smithi142 ceph-mon[82136]: osdmap e91: 6 total, 6 up, 6 in 2024-06-15T01:14:32.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.654+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f293c04ea90 con 0x7f293c105a40 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.654+0000 7f293a7fc700 1 -- 172.21.15.137:0/3693589200 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+709 (secure 0 0 0) 0x7f2930058610 con 0x7f293c105a40 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:14:32.645 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 10 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:14:32.646 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:14:32.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 msgr2=0x7f292406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 0x7f292406d440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2934009850 tx=0x7f29340097c0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 msgr2=0x7f293c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f293000c8f0 tx=0x7f293000ccb0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 shutdown_connections 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f292406af80 0x7f292406d440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f293c0ffbd0 0x7f293c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 --2- 172.21.15.137:0/3693589200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f293c105a40 0x7f293c197a20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 >> 172.21.15.137:0/3693589200 conn(0x7f293c0755a0 msgr2=0x7f293c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 shutdown_connections 2024-06-15T01:14:32.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.657+0000 7f29436b3700 1 -- 172.21.15.137:0/3693589200 wait complete. 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.763+0000 7f4373269700 1 -- 172.21.15.137:0/3925187075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 msgr2=0x7f436c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.763+0000 7f4373269700 1 --2- 172.21.15.137:0/3925187075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c105ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4368009910 tx=0x7f4368009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- 172.21.15.137:0/3925187075 shutdown_connections 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 --2- 172.21.15.137:0/3925187075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 0x7f436c100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 --2- 172.21.15.137:0/3925187075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c105ef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- 172.21.15.137:0/3925187075 >> 172.21.15.137:0/3925187075 conn(0x7f436c074bf0 msgr2=0x7f436c075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- 172.21.15.137:0/3925187075 shutdown_connections 2024-06-15T01:14:32.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- 172.21.15.137:0/3925187075 wait complete. 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 Processor -- start 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- start start 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 0x7f436c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c198130 con 0x7f436c105b10 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.764+0000 7f4373269700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c198270 con 0x7f436c100420 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58790/0 (socket says 172.21.15.137:58790) 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 -- 172.21.15.137:0/2031585464 learned_addr learned my addr 172.21.15.137:0/2031585464 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4371005700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 0x7f436c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 -- 172.21.15.137:0/2031585464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 msgr2=0x7f436c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 0x7f436c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 -- 172.21.15.137:0/2031585464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680095f0 con 0x7f436c105b10 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4370804700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b4b0 tx=0x7f435c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:32.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435c011800 con 0x7f436c105b10 2024-06-15T01:14:32.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f435c011e00 con 0x7f436c105b10 2024-06-15T01:14:32.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c1a0640 con 0x7f436c105b10 2024-06-15T01:14:32.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435c010e50 con 0x7f436c105b10 2024-06-15T01:14:32.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.765+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c1a0b00 con 0x7f436c105b10 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.766+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c108010 con 0x7f436c105b10 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.767+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f435c010380 con 0x7f436c105b10 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.767+0000 7f43627fc700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 0x7f435806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.767+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f435c089f30 con 0x7f436c105b10 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.767+0000 7f4371005700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 0x7f435806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:32.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.768+0000 7f4371005700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 0x7f435806d440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4368006010 tx=0x7f436801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:32.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:32.772+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f435c0bd900 con 0x7f436c105b10 2024-06-15T01:14:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.012+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f436c04ea90 con 0x7f436c105b10 2024-06-15T01:14:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.013+0000 7f43627fc700 1 -- 172.21.15.137:0/2031585464 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f435c054d60 con 0x7f436c105b10 2024-06-15T01:14:33.003 INFO:teuthology.orchestra.run.smithi137.stdout:e23 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:14:33.004 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:14:33.005 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:14:33.006 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:14:33.007 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:33.007 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:33.007 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34304} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3864353360,v1:172.21.15.137:6828/3864353360] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:33.007 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34306} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/850529376,v1:172.21.15.137:6829/850529376] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:14:33.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 msgr2=0x7f435806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 0x7f435806d440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4368006010 tx=0x7f436801a040 comp rx=0 tx=0).stop 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 msgr2=0x7f436c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b4b0 tx=0x7f435c00b870 comp rx=0 tx=0).stop 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 shutdown_connections 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f435806af80 0x7f435806d440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f436c100420 0x7f436c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 --2- 172.21.15.137:0/2031585464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c105b10 0x7f436c197b10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 >> 172.21.15.137:0/2031585464 conn(0x7f436c074bf0 msgr2=0x7f436c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 shutdown_connections 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.015+0000 7f4373269700 1 -- 172.21.15.137:0/2031585464 wait complete. 2024-06-15T01:14:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 -- 172.21.15.137:0/152108845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 msgr2=0x7efef0104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 --2- 172.21.15.137:0/152108845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef0104e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efee0009910 tx=0x7efee0009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 -- 172.21.15.137:0/152108845 shutdown_connections 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 --2- 172.21.15.137:0/152108845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 0x7efef01005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 --2- 172.21.15.137:0/152108845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef0104e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 -- 172.21.15.137:0/152108845 >> 172.21.15.137:0/152108845 conn(0x7efef0075e30 msgr2=0x7efef00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 -- 172.21.15.137:0/152108845 shutdown_connections 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 -- 172.21.15.137:0/152108845 wait complete. 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.120+0000 7efef5709700 1 Processor -- start 2024-06-15T01:14:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 -- start start 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 0x7efef00fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00fbd20 con 0x7efef0104a50 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00fbe60 con 0x7efef0100140 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58802/0 (socket says 172.21.15.137:58802) 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 -- 172.21.15.137:0/170368569 learned_addr learned my addr 172.21.15.137:0/170368569 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeeeffd700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 0x7efef00fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 -- 172.21.15.137:0/170368569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 msgr2=0x7efef00fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 0x7efef00fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 -- 172.21.15.137:0/170368569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00095f0 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efeee7fc700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efee400d8d0 tx=0x7efee400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee4009940 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01a2770 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efee4010460 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.121+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee400f5f0 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.122+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01a2c00 con 0x7efef0104a50 2024-06-15T01:14:33.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.123+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef004ea90 con 0x7efef0104a50 2024-06-15T01:14:33.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.123+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efee4009aa0 con 0x7efef0104a50 2024-06-15T01:14:33.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.123+0000 7efed7fff700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 0x7efed806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.123+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efee408a390 con 0x7efef0104a50 2024-06-15T01:14:33.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.123+0000 7efeeeffd700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 0x7efed806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.124+0000 7efeeeffd700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 0x7efed806d440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efee0006010 tx=0x7efee001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:33.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.128+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efee40551c0 con 0x7efef0104a50 2024-06-15T01:14:33.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.340+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efef0102840 con 0x7efed806af80 2024-06-15T01:14:33.331 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: Health check update: Degraded data redundancy: 57/327 objects degraded (17.431%), 16 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: osdmap e92: 6 total, 6 up, 6 in 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3693589200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:33.332 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:33 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2031585464' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.342+0000 7efed7fff700 1 -- 172.21.15.137:0/170368569 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efef0102840 con 0x7efed806af80 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:14:33.333 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:14:33.334 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:14:33.334 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "12/23 daemons upgraded", 2024-06-15T01:14:33.334 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:14:33.334 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:14:33.334 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 msgr2=0x7efed806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 0x7efed806d440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efee0006010 tx=0x7efee001a040 comp rx=0 tx=0).stop 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 msgr2=0x7efef00fb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efee400d8d0 tx=0x7efee400dc90 comp rx=0 tx=0).stop 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 shutdown_connections 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efed806af80 0x7efed806d440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef0100140 0x7efef00fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 --2- 172.21.15.137:0/170368569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0104a50 0x7efef00fb7e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 >> 172.21.15.137:0/170368569 conn(0x7efef0075e30 msgr2=0x7efef00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 shutdown_connections 2024-06-15T01:14:33.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.344+0000 7efef5709700 1 -- 172.21.15.137:0/170368569 wait complete. 2024-06-15T01:14:33.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.448+0000 7fa52c9a2700 1 -- 172.21.15.137:0/3546634964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 msgr2=0x7fa528105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.448+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/3546634964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa528105870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009910 tx=0x7fa51c009c20 comp rx=0 tx=0).stop 2024-06-15T01:14:33.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 -- 172.21.15.137:0/3546634964 shutdown_connections 2024-06-15T01:14:33.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/3546634964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa528105870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/3546634964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 0x7fa5280ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 -- 172.21.15.137:0/3546634964 >> 172.21.15.137:0/3546634964 conn(0x7fa528074df0 msgr2=0x7fa528075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:33.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 -- 172.21.15.137:0/3546634964 shutdown_connections 2024-06-15T01:14:33.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 -- 172.21.15.137:0/3546634964 wait complete. 2024-06-15T01:14:33.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 Processor -- start 2024-06-15T01:14:33.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.449+0000 7fa52c9a2700 1 -- start start 2024-06-15T01:14:33.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa52c9a2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 0x7fa52819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa52c9a2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa52c9a2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52819d160 con 0x7fa5280ffee0 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa52c9a2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa528196610 con 0x7fa5280ff530 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58812/0 (socket says 172.21.15.137:58812) 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 -- 172.21.15.137:0/269158834 learned_addr learned my addr 172.21.15.137:0/269158834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa52659c700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 0x7fa52819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 -- 172.21.15.137:0/269158834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 msgr2=0x7fa52819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 0x7fa52819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 -- 172.21.15.137:0/269158834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0095f0 con 0x7fa5280ffee0 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa525d9b700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa51c005f50 tx=0x7fa51c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:33.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.450+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51c01d070 con 0x7fa5280ffee0 2024-06-15T01:14:33.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.451+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa51c00f460 con 0x7fa5280ffee0 2024-06-15T01:14:33.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.451+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa528196890 con 0x7fa5280ffee0 2024-06-15T01:14:33.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.451+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51c021640 con 0x7fa5280ffee0 2024-06-15T01:14:33.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.451+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa528196d80 con 0x7fa5280ffee0 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.452+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa528109190 con 0x7fa5280ffee0 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.452+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa51c00f9e0 con 0x7fa5280ffee0 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.453+0000 7fa5177fe700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 0x7fa51006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.453+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa51c08b5b0 con 0x7fa5280ffee0 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.453+0000 7fa52659c700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 0x7fa51006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:14:33.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.453+0000 7fa52659c700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 0x7fa51006d440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa518009a10 tx=0x7fa518009190 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:14:33.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.457+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa51c059e70 con 0x7fa5280ffee0 2024-06-15T01:14:33.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: Health check update: Degraded data redundancy: 57/327 objects degraded (17.431%), 16 pgs degraded (PG_DEGRADED) 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: osdmap e92: 6 total, 6 up, 6 in 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3693589200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:33.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:33 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2031585464' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:14:33.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.734+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa528197060 con 0x7fa5280ffee0 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.734+0000 7fa5177fe700 1 -- 172.21.15.137:0/269158834 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1404 (secure 0 0 0) 0x7fa51c059a00 con 0x7fa5280ffee0 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded; 4 mgr modules have recently crashed 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded 2024-06-15T01:14:33.725 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1a is active+undersized+degraded, acting [1,4] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:14:33.726 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:14:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 msgr2=0x7fa51006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 0x7fa51006d440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa518009a10 tx=0x7fa518009190 comp rx=0 tx=0).stop 2024-06-15T01:14:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 msgr2=0x7fa52819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa51c005f50 tx=0x7fa51c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 shutdown_connections 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa51006af80 0x7fa51006d440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5280ff530 0x7fa52819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 --2- 172.21.15.137:0/269158834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5280ffee0 0x7fa52819cad0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.736+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 >> 172.21.15.137:0/269158834 conn(0x7fa528074df0 msgr2=0x7fa5280fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.737+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 shutdown_connections 2024-06-15T01:14:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:14:33.737+0000 7fa52c9a2700 1 -- 172.21.15.137:0/269158834 wait complete. 2024-06-15T01:14:34.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:34 smithi137 ceph-mon[112295]: pgmap v251: 65 pgs: 6 peering, 11 active+undersized, 8 active+undersized+degraded, 40 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 28/327 objects degraded (8.563%) 2024-06-15T01:14:34.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:34 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:14:34.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:34 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/269158834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:14:34.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:34 smithi142 ceph-mon[82136]: pgmap v251: 65 pgs: 6 peering, 11 active+undersized, 8 active+undersized+degraded, 40 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 28/327 objects degraded (8.563%) 2024-06-15T01:14:34.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:34 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:14:34.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:34 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/269158834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:14:35.462 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:35 smithi142 ceph-mon[82136]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:35.462 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:35 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:35.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:35 smithi137 ceph-mon[112295]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:14:35.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:35 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:36.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:36 smithi142 ceph-mon[82136]: pgmap v252: 65 pgs: 6 peering, 1 active+undersized, 58 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 137 objects/s recovering 2024-06-15T01:14:36.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:36 smithi142 ceph-mon[82136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded) 2024-06-15T01:14:36.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:36 smithi137 ceph-mon[112295]: pgmap v252: 65 pgs: 6 peering, 1 active+undersized, 58 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 137 objects/s recovering 2024-06-15T01:14:36.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:36 smithi137 ceph-mon[112295]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/327 objects degraded (8.563%), 8 pgs degraded) 2024-06-15T01:14:38.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:38 smithi137 ceph-mon[112295]: pgmap v253: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s, 121 objects/s recovering 2024-06-15T01:14:38.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:38 smithi142 ceph-mon[82136]: pgmap v253: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s, 121 objects/s recovering 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: pgmap v254: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 841 KiB/s, 95 objects/s recovering 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:boot 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:39.527 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:39.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: pgmap v254: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 841 KiB/s, 95 objects/s recovering 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:boot 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:39.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:14:42.033 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:41 smithi137 ceph-mon[112295]: pgmap v255: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 726 KiB/s, 82 objects/s recovering 2024-06-15T01:14:42.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:41 smithi142 ceph-mon[82136]: pgmap v255: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 726 KiB/s, 82 objects/s recovering 2024-06-15T01:14:44.303 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: pgmap v256: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 654 KiB/s, 73 objects/s recovering 2024-06-15T01:14:44.303 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.303 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.303 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.303 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.304 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: pgmap v256: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 654 KiB/s, 73 objects/s recovering 2024-06-15T01:14:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:44.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:46.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:46 smithi142 ceph-mon[82136]: pgmap v257: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 561 KiB/s, 63 objects/s recovering 2024-06-15T01:14:46.391 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:46 smithi137 ceph-mon[112295]: pgmap v257: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 561 KiB/s, 63 objects/s recovering 2024-06-15T01:14:48.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: pgmap v258: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.221 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:48.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: pgmap v258: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:14:48.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:48.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: Upgrade: Updating mds.cephfs.smithi137.afplli 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: Deploying daemon mds.cephfs.smithi137.afplli on smithi137 2024-06-15T01:14:49.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:49 smithi137 ceph-mon[112295]: pgmap v259: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:50.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: Upgrade: Updating mds.cephfs.smithi137.afplli 2024-06-15T01:14:50.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:50.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi137.afplli", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:14:50.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:14:50.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: Deploying daemon mds.cephfs.smithi137.afplli on smithi137 2024-06-15T01:14:50.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:49 smithi142 ceph-mon[82136]: pgmap v259: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:51.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:51 smithi142 ceph-mon[82136]: osdmap e93: 6 total, 6 up, 6 in 2024-06-15T01:14:51.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:51 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:14:51.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:51 smithi142 ceph-mon[82136]: pgmap v261: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:51.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:51 smithi137 ceph-mon[112295]: osdmap e93: 6 total, 6 up, 6 in 2024-06-15T01:14:51.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:51 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:14:51.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:51 smithi137 ceph-mon[112295]: pgmap v261: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:53.880 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:53 smithi137 ceph-mon[112295]: pgmap v262: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:54.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:53 smithi142 ceph-mon[82136]: pgmap v262: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:56.172 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: pgmap v263: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:boot 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:56.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:55 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:14:56.211 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: pgmap v263: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:56.211 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:56.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:14:56.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:14:56.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:boot 2024-06-15T01:14:56.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:14:56.212 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:55 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:14:58.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:57 smithi137 ceph-mon[112295]: pgmap v264: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:14:58.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:57 smithi142 ceph-mon[82136]: pgmap v264: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:00.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:59 smithi137 ceph-mon[112295]: pgmap v265: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:00.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:59 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:00.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:14:59 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:59 smithi142 ceph-mon[82136]: pgmap v265: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:59 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:14:59 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:01.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:01 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:01.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:01 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:01.562 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:01 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:01.562 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:01 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:02.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:02 smithi142 ceph-mon[82136]: pgmap v266: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:02.629 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:02 smithi137 ceph-mon[112295]: pgmap v266: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:03.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:03.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:03.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 -- 172.21.15.137:0/3672692758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 msgr2=0x7f3f141020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:03.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3672692758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f141020f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009910 tx=0x7f3efc009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:03.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 -- 172.21.15.137:0/3672692758 shutdown_connections 2024-06-15T01:15:03.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3672692758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 0x7f3f1410ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:03.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3672692758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f141020f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:03.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 -- 172.21.15.137:0/3672692758 >> 172.21.15.137:0/3672692758 conn(0x7f3f140fb540 msgr2=0x7f3f140fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:03.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 -- 172.21.15.137:0/3672692758 shutdown_connections 2024-06-15T01:15:03.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.844+0000 7f3f1973b700 1 -- 172.21.15.137:0/3672692758 wait complete. 2024-06-15T01:15:03.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 Processor -- start 2024-06-15T01:15:03.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 -- start start 2024-06-15T01:15:03.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 0x7f3f1419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1419d160 con 0x7f3f14101d10 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f1973b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14196610 con 0x7f3f14102630 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.845+0000 7f3f12ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58180/0 (socket says 172.21.15.137:58180) 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 -- 172.21.15.137:0/3001914492 learned_addr learned my addr 172.21.15.137:0/3001914492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f127fc700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 0x7f3f1419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 -- 172.21.15.137:0/3001914492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 msgr2=0x7f3f1419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 0x7f3f1419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 -- 172.21.15.137:0/3001914492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0095f0 con 0x7f3f14101d10 2024-06-15T01:15:03.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f12ffd700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006010 tx=0x7f3efc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3efc01d070 con 0x7f3f14101d10 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14196890 con 0x7f3f14101d10 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3efc00f460 con 0x7f3f14101d10 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3efc021640 con 0x7f3f14101d10 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.846+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f14196d80 con 0x7f3f14101d10 2024-06-15T01:15:03.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.847+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f14108220 con 0x7f3f14101d10 2024-06-15T01:15:03.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.848+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3efc00fa60 con 0x7f3f14101d10 2024-06-15T01:15:03.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.848+0000 7f3f0bfff700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 0x7f3f0006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:03.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.848+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3efc08b6c0 con 0x7f3f14101d10 2024-06-15T01:15:03.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.848+0000 7f3f127fc700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 0x7f3f0006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:03.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.849+0000 7f3f127fc700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 0x7f3f0006d440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f140fcd20 tx=0x7f3f04008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:03.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:03.853+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3efc059f80 con 0x7f3f14101d10 2024-06-15T01:15:04.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.063+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f14197060 con 0x7f3f0006af80 2024-06-15T01:15:04.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.065+0000 7f3f0bfff700 1 -- 172.21.15.137:0/3001914492 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3f14197060 con 0x7f3f0006af80 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 msgr2=0x7f3f0006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 0x7f3f0006d440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f140fcd20 tx=0x7f3f04008040 comp rx=0 tx=0).stop 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 msgr2=0x7f3f1419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006010 tx=0x7f3efc00be20 comp rx=0 tx=0).stop 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 shutdown_connections 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f0006af80 0x7f3f0006d440 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f14101d10 0x7f3f1419c590 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 --2- 172.21.15.137:0/3001914492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f14102630 0x7f3f1419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 >> 172.21.15.137:0/3001914492 conn(0x7f3f140fb540 msgr2=0x7f3f14105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 shutdown_connections 2024-06-15T01:15:04.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.067+0000 7f3f1973b700 1 -- 172.21.15.137:0/3001914492 wait complete. 2024-06-15T01:15:04.069 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 -- 172.21.15.137:0/2653307368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b80fef70 msgr2=0x7ff0b80ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2653307368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b80fef70 0x7ff0b80ff3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009910 tx=0x7ff0a8009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 -- 172.21.15.137:0/2653307368 shutdown_connections 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2653307368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b80fef70 0x7ff0b80ff3f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2653307368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b8105b50 0x7ff0b8105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 -- 172.21.15.137:0/2653307368 >> 172.21.15.137:0/2653307368 conn(0x7ff0b8075e30 msgr2=0x7ff0b80fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 -- 172.21.15.137:0/2653307368 shutdown_connections 2024-06-15T01:15:04.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.173+0000 7ff0bd210700 1 -- 172.21.15.137:0/2653307368 wait complete. 2024-06-15T01:15:04.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 Processor -- start 2024-06-15T01:15:04.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 -- start start 2024-06-15T01:15:04.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 0x7ff0b8196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b81972d0 con 0x7ff0b8105b50 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.174+0000 7ff0bd210700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b819a000 con 0x7ff0b80fef70 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58190/0 (socket says 172.21.15.137:58190) 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 -- 172.21.15.137:0/2510161331 learned_addr learned my addr 172.21.15.137:0/2510161331 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b6d9d700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 0x7ff0b8196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 -- 172.21.15.137:0/2510161331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 msgr2=0x7ff0b8196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 0x7ff0b8196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 -- 172.21.15.137:0/2510161331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80095f0 con 0x7ff0b8105b50 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b6d9d700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 0x7ff0b8196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:15:04.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0b659c700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006010 tx=0x7ff0a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a801d070 con 0x7ff0b8105b50 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b819a280 con 0x7ff0b8105b50 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0a8005710 con 0x7ff0b8105b50 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a800fe20 con 0x7ff0b8105b50 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.175+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b819a770 con 0x7ff0b8105b50 2024-06-15T01:15:04.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.177+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b81090d0 con 0x7ff0b8105b50 2024-06-15T01:15:04.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.177+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff0a802b440 con 0x7ff0b8105b50 2024-06-15T01:15:04.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.178+0000 7ff0affff700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 0x7ff0a406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.178+0000 7ff0b6d9d700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 0x7ff0a406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.178+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff0a808b7b0 con 0x7ff0b8105b50 2024-06-15T01:15:04.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.178+0000 7ff0b6d9d700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 0x7ff0a406d440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005950 tx=0x7ff0a000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.182+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff0a8056770 con 0x7ff0b8105b50 2024-06-15T01:15:04.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.396+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0b8197b40 con 0x7ff0a406af80 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: pgmap v267: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:04.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.388 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.388 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.388 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.388 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:04 smithi137 ceph-mon[112295]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:04.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.398+0000 7ff0affff700 1 -- 172.21.15.137:0/2510161331 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff0b8197b40 con 0x7ff0a406af80 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 msgr2=0x7ff0a406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 0x7ff0a406d440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005950 tx=0x7ff0a000b410 comp rx=0 tx=0).stop 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 msgr2=0x7ff0b8196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006010 tx=0x7ff0a800bd70 comp rx=0 tx=0).stop 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 shutdown_connections 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0a406af80 0x7ff0a406d440 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b80fef70 0x7ff0b8196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 --2- 172.21.15.137:0/2510161331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b8105b50 0x7ff0b8196c40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 >> 172.21.15.137:0/2510161331 conn(0x7ff0b8075e30 msgr2=0x7ff0b81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 shutdown_connections 2024-06-15T01:15:04.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.400+0000 7ff0bd210700 1 -- 172.21.15.137:0/2510161331 wait complete. 2024-06-15T01:15:04.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 -- 172.21.15.137:0/2011665408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 msgr2=0x7fde34105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/2011665408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34105e20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde28009910 tx=0x7fde28009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 -- 172.21.15.137:0/2011665408 shutdown_connections 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/2011665408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 0x7fde340ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/2011665408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34105e20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 -- 172.21.15.137:0/2011665408 >> 172.21.15.137:0/2011665408 conn(0x7fde340fb570 msgr2=0x7fde340fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 -- 172.21.15.137:0/2011665408 shutdown_connections 2024-06-15T01:15:04.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.506+0000 7fde3aaf3700 1 -- 172.21.15.137:0/2011665408 wait complete. 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 Processor -- start 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 -- start start 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 0x7fde341974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34198020 con 0x7fde34105a40 2024-06-15T01:15:04.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3aaf3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34198160 con 0x7fde340ffa40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde33fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde33fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58216/0 (socket says 172.21.15.137:58216) 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde33fff700 1 -- 172.21.15.137:0/487073012 learned_addr learned my addr 172.21.15.137:0/487073012 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.507+0000 7fde3888f700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 0x7fde341974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde33fff700 1 -- 172.21.15.137:0/487073012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 msgr2=0x7fde341974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde33fff700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 0x7fde341974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde33fff700 1 -- 172.21.15.137:0/487073012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280095f0 con 0x7fde34105a40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde33fff700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d8d0 tx=0x7fde2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde20009940 con 0x7fde34105a40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde20010460 con 0x7fde34105a40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde341a0530 con 0x7fde34105a40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde2000f5f0 con 0x7fde34105a40 2024-06-15T01:15:04.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.508+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde341a09f0 con 0x7fde34105a40 2024-06-15T01:15:04.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.509+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fde200105d0 con 0x7fde34105a40 2024-06-15T01:15:04.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.510+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3404ea90 con 0x7fde34105a40 2024-06-15T01:15:04.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.510+0000 7fde31ffb700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 0x7fde2406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.510+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fde2008a440 con 0x7fde34105a40 2024-06-15T01:15:04.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.510+0000 7fde3888f700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 0x7fde2406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.510+0000 7fde3888f700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 0x7fde2406d440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde28006010 tx=0x7fde2801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.518+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fde20055210 con 0x7fde34105a40 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: pgmap v267: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:15:04.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:04.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:04 smithi142 ceph-mon[82136]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:04.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.727+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde340fc6f0 con 0x7fde2406af80 2024-06-15T01:15:04.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.736+0000 7fde31ffb700 1 -- 172.21.15.137:0/487073012 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fde340fc6f0 con 0x7fde2406af80 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (5m) 5s ago 16m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (16m) 5s ago 16m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (15m) 39s ago 15m 8006k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (3m) 5s ago 16m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (3m) 39s ago 15m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (5m) 5s ago 15m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (10s) 5s ago 10m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (26s) 5s ago 10m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (10m) 39s ago 10m 16.7M - 18.2.1 5be31c24972a e06209a9ce3e 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (10m) 39s ago 10m 19.9M - 18.2.1 5be31c24972a a906104bdb2c 2024-06-15T01:15:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (7m) 5s ago 17m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (6m) 39s ago 15m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (4m) 5s ago 17m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (4m) 39s ago 15m 41.5M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (6m) 5s ago 16m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 39s ago 15m 17.2M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (3m) 5s ago 14m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (2m) 5s ago 13m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (2m) 5s ago 13m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (105s) 39s ago 12m 113M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (73s) 39s ago 12m 108M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (42s) 39s ago 12m 14.5M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:15:04.728 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (5m) 5s ago 15m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 msgr2=0x7fde2406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 0x7fde2406d440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde28006010 tx=0x7fde2801a040 comp rx=0 tx=0).stop 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 msgr2=0x7fde34197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d8d0 tx=0x7fde2000dc90 comp rx=0 tx=0).stop 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 shutdown_connections 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fde2406af80 0x7fde2406d440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde340ffa40 0x7fde341974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 --2- 172.21.15.137:0/487073012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde34105a40 0x7fde34197a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 >> 172.21.15.137:0/487073012 conn(0x7fde340fb570 msgr2=0x7fde340fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 shutdown_connections 2024-06-15T01:15:04.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.739+0000 7fde3aaf3700 1 -- 172.21.15.137:0/487073012 wait complete. 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 -- 172.21.15.137:0/965677371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4105a40 msgr2=0x7ff3b4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 --2- 172.21.15.137:0/965677371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4105a40 0x7ff3b4105e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009910 tx=0x7ff3b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 -- 172.21.15.137:0/965677371 shutdown_connections 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 --2- 172.21.15.137:0/965677371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b40ffa40 0x7ff3b40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 --2- 172.21.15.137:0/965677371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b4105a40 0x7ff3b4105e20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.848+0000 7ff3bb754700 1 -- 172.21.15.137:0/965677371 >> 172.21.15.137:0/965677371 conn(0x7ff3b40fb550 msgr2=0x7ff3b40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.849+0000 7ff3bb754700 1 -- 172.21.15.137:0/965677371 shutdown_connections 2024-06-15T01:15:04.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.849+0000 7ff3bb754700 1 -- 172.21.15.137:0/965677371 wait complete. 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.849+0000 7ff3bb754700 1 Processor -- start 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.849+0000 7ff3bb754700 1 -- start start 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b4105a40 0x7ff3b4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4197260 con 0x7ff3b40ffa40 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4199f90 con 0x7ff3b4105a40 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58224/0 (socket says 172.21.15.137:58224) 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 -- 172.21.15.137:0/2781640553 learned_addr learned my addr 172.21.15.137:0/2781640553 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b8cef700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b4105a40 0x7ff3b4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 -- 172.21.15.137:0/2781640553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b4105a40 msgr2=0x7ff3b4196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b4105a40 0x7ff3b4196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 -- 172.21.15.137:0/2781640553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00095f0 con 0x7ff3b40ffa40 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3b94f0700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000b5c0 tx=0x7ff3b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b001d070 con 0x7ff3b40ffa40 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b419a210 con 0x7ff3b40ffa40 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3b0005e00 con 0x7ff3b40ffa40 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b000fe90 con 0x7ff3b40ffa40 2024-06-15T01:15:04.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.850+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b419a700 con 0x7ff3b40ffa40 2024-06-15T01:15:04.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.851+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4108040 con 0x7ff3b40ffa40 2024-06-15T01:15:04.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.852+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff3b000f6d0 con 0x7ff3b40ffa40 2024-06-15T01:15:04.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.852+0000 7ff3aa7fc700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 0x7ff3a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:04.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.852+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff3b008b5c0 con 0x7ff3b40ffa40 2024-06-15T01:15:04.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.852+0000 7ff3b8cef700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 0x7ff3a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:04.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.853+0000 7ff3b8cef700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 0x7ff3a006d440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4197cb0 tx=0x7ff3a400b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:04.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:04.856+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3b0056510 con 0x7ff3b40ffa40 2024-06-15T01:15:05.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.125+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3b4107a20 con 0x7ff3b40ffa40 2024-06-15T01:15:05.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.126+0000 7ff3aa7fc700 1 -- 172.21.15.137:0/2781640553 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7ff3b0059b30 con 0x7ff3b40ffa40 2024-06-15T01:15:05.116 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:15:05.116 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:15:05.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 12 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 msgr2=0x7ff3a006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 0x7ff3a006d440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4197cb0 tx=0x7ff3a400b410 comp rx=0 tx=0).stop 2024-06-15T01:15:05.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 msgr2=0x7ff3b4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000b5c0 tx=0x7ff3b000bd70 comp rx=0 tx=0).stop 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 shutdown_connections 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff3a006af80 0x7ff3a006d440 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3b40ffa40 0x7ff3b4196690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 --2- 172.21.15.137:0/2781640553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3b4105a40 0x7ff3b4196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 >> 172.21.15.137:0/2781640553 conn(0x7ff3b40fb550 msgr2=0x7ff3b40fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 shutdown_connections 2024-06-15T01:15:05.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.128+0000 7ff3bb754700 1 -- 172.21.15.137:0/2781640553 wait complete. 2024-06-15T01:15:05.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 -- 172.21.15.137:0/1808378804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 msgr2=0x7ff8f0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/1808378804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f0100050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009910 tx=0x7ff8e0009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 -- 172.21.15.137:0/1808378804 shutdown_connections 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/1808378804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f0100050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/1808378804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 0x7ff8f0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 -- 172.21.15.137:0/1808378804 >> 172.21.15.137:0/1808378804 conn(0x7ff8f00755a0 msgr2=0x7ff8f00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 -- 172.21.15.137:0/1808378804 shutdown_connections 2024-06-15T01:15:05.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.233+0000 7ff8f6a50700 1 -- 172.21.15.137:0/1808378804 wait complete. 2024-06-15T01:15:05.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 Processor -- start 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 -- start start 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 0x7ff8f019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f019d110 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8f6a50700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01965c0 con 0x7ff8f0105a40 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8effff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8effff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58250/0 (socket says 172.21.15.137:58250) 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8effff700 1 -- 172.21.15.137:0/2868776590 learned_addr learned my addr 172.21.15.137:0/2868776590 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.234+0000 7ff8ef7fe700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 0x7ff8f019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8effff700 1 -- 172.21.15.137:0/2868776590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 msgr2=0x7ff8f019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8effff700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 0x7ff8f019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8effff700 1 -- 172.21.15.137:0/2868776590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00095f0 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8effff700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400b4b0 tx=0x7ff8e400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e4011800 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8e4011e00 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f01968a0 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e4010e60 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.235+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f0196df0 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.236+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0109070 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.237+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff8e4011960 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.237+0000 7ff8ed7fa700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 0x7ff8d806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.237+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff8e4089e20 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.237+0000 7ff8ef7fe700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 0x7ff8d806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.238+0000 7ff8ef7fe700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 0x7ff8d806d440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0104eb0 tx=0x7ff8e001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.241+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff8e4058760 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.476+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8f004ea90 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.476+0000 7ff8ed7fa700 1 -- 172.21.15.137:0/2868776590 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1841 (secure 0 0 0) 0x7ff8e4015750 con 0x7ff8f00ffbd0 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:e27 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:15:05.467 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:15:05.468 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:15:05.469 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:24273} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.142:6826/201053718,v1:172.21.15.142:6827/201053718] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/2666411319,v1:172.21.15.142:6825/2666411319] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:05.470 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 msgr2=0x7ff8d806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 0x7ff8d806d440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0104eb0 tx=0x7ff8e001a040 comp rx=0 tx=0).stop 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 msgr2=0x7ff8f019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400b4b0 tx=0x7ff8e400b7c0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 shutdown_connections 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff8d806af80 0x7ff8d806d440 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8f00ffbd0 0x7ff8f019c540 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 --2- 172.21.15.137:0/2868776590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f0105a40 0x7ff8f019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 >> 172.21.15.137:0/2868776590 conn(0x7ff8f00755a0 msgr2=0x7ff8f0068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 shutdown_connections 2024-06-15T01:15:05.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.479+0000 7ff8f6a50700 1 -- 172.21.15.137:0/2868776590 wait complete. 2024-06-15T01:15:05.472 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 -- 172.21.15.137:0/3479722954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e00ffbd0 msgr2=0x7f10e0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 --2- 172.21.15.137:0/3479722954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e00ffbd0 0x7f10e0100050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009910 tx=0x7f10d4009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 -- 172.21.15.137:0/3479722954 shutdown_connections 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 --2- 172.21.15.137:0/3479722954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e00ffbd0 0x7f10e0100050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 --2- 172.21.15.137:0/3479722954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e0105a40 0x7f10e0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 -- 172.21.15.137:0/3479722954 >> 172.21.15.137:0/3479722954 conn(0x7f10e00755a0 msgr2=0x7f10e00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 -- 172.21.15.137:0/3479722954 shutdown_connections 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.584+0000 7f10e828a700 1 -- 172.21.15.137:0/3479722954 wait complete. 2024-06-15T01:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 Processor -- start 2024-06-15T01:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 -- start start 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e00ffbd0 0x7f10e0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0198160 con 0x7f10e0105a40 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e828a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e01982d0 con 0x7f10e00ffbd0 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58268/0 (socket says 172.21.15.137:58268) 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 -- 172.21.15.137:0/1159226323 learned_addr learned my addr 172.21.15.137:0/1159226323 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e6026700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e00ffbd0 0x7f10e0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 -- 172.21.15.137:0/1159226323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e00ffbd0 msgr2=0x7f10e0197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e00ffbd0 0x7f10e0197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.585+0000 7f10e5825700 1 -- 172.21.15.137:0/1159226323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40095f0 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10e5825700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10d40092e0 tx=0x7f10d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d401d070 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10d400f460 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e01a0670 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d4021620 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.586+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e01a0ae0 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.587+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e0108060 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.587+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f10d400fa00 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.588+0000 7f10d37fe700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 0x7f10cc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.588+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f10d408b560 con 0x7f10e0105a40 2024-06-15T01:15:05.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.588+0000 7f10e6026700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 0x7f10cc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.588+0000 7f10e6026700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 0x7f10cc06d440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009a10 tx=0x7f10dc009190 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.592+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10d40563b0 con 0x7f10e0105a40 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: Upgrade: Updating mds.cephfs.smithi142.examgd 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: pgmap v268: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: Deploying daemon mds.cephfs.smithi142.examgd on smithi142 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:05.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2781640553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:05.720 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:05 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2868776590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:15:05.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.804+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10e002cce0 con 0x7f10cc06af80 2024-06-15T01:15:05.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.805+0000 7f10d37fe700 1 -- 172.21.15.137:0/1159226323 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f10e002cce0 con 0x7f10cc06af80 2024-06-15T01:15:05.795 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:15:05.795 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:15:05.795 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "crash" 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "14/23 daemons upgraded", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:15:05.796 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:15:05.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 msgr2=0x7f10cc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 0x7f10cc06d440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009a10 tx=0x7f10dc009190 comp rx=0 tx=0).stop 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 msgr2=0x7f10e0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10d40092e0 tx=0x7f10d400bd70 comp rx=0 tx=0).stop 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 shutdown_connections 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f10cc06af80 0x7f10cc06d440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10e00ffbd0 0x7f10e0197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 --2- 172.21.15.137:0/1159226323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10e0105a40 0x7f10e0197b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.807+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 >> 172.21.15.137:0/1159226323 conn(0x7f10e00755a0 msgr2=0x7f10e00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.808+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 shutdown_connections 2024-06-15T01:15:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.808+0000 7f10e828a700 1 -- 172.21.15.137:0/1159226323 wait complete. 2024-06-15T01:15:05.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- 172.21.15.137:0/1311065343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec105a40 msgr2=0x7fc7ec105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 --2- 172.21.15.137:0/1311065343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec105a40 0x7fc7ec105e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009910 tx=0x7fc7dc009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:05.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- 172.21.15.137:0/1311065343 shutdown_connections 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 --2- 172.21.15.137:0/1311065343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 --2- 172.21.15.137:0/1311065343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec105a40 0x7fc7ec105e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- 172.21.15.137:0/1311065343 >> 172.21.15.137:0/1311065343 conn(0x7fc7ec0fb570 msgr2=0x7fc7ec0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- 172.21.15.137:0/1311065343 shutdown_connections 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- 172.21.15.137:0/1311065343 wait complete. 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 Processor -- start 2024-06-15T01:15:05.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.912+0000 7fc7f269d700 1 -- start start 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7f269d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7f269d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec105a40 0x7fc7ec195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7f269d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec196170 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7f269d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec199f00 con 0x7fc7ec105a40 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58290/0 (socket says 172.21.15.137:58290) 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 -- 172.21.15.137:0/3017313552 learned_addr learned my addr 172.21.15.137:0/3017313552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7eb7fe700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec105a40 0x7fc7ec195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 -- 172.21.15.137:0/3017313552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec105a40 msgr2=0x7fc7ec195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec105a40 0x7fc7ec195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 -- 172.21.15.137:0/3017313552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc0095f0 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.913+0000 7fc7ebfff700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc000c00 tx=0x7fc7dc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.914+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7dc01f070 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.914+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ec19a180 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.914+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7dc005dc0 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.914+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7dc003940 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.914+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ec19a610 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.915+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ec04ea90 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.915+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc7dc003aa0 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.915+0000 7fc7e97fa700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 0x7fc7d806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.915+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc7dc08db50 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.915+0000 7fc7eb7fe700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 0x7fc7d806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.916+0000 7fc7eb7fe700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 0x7fc7d806d440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec196b70 tx=0x7fc7d4009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:05.920+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc7dc058950 con 0x7fc7ec0ffa40 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: Upgrade: Updating mds.cephfs.smithi142.examgd 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.examgd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: pgmap v268: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: Deploying daemon mds.cephfs.smithi142.examgd on smithi142 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2781640553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:05.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:05 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2868776590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:15:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.192+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc7ec19ab20 con 0x7fc7ec0ffa40 2024-06-15T01:15:06.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.193+0000 7fc7e97fa700 1 -- 172.21.15.137:0/3017313552 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fc7dc028330 con 0x7fc7ec0ffa40 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:15:06.184 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:15:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 msgr2=0x7fc7d806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 0x7fc7d806d440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec196b70 tx=0x7fc7d4009260 comp rx=0 tx=0).stop 2024-06-15T01:15:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 msgr2=0x7fc7ec195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc000c00 tx=0x7fc7dc00bde0 comp rx=0 tx=0).stop 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 shutdown_connections 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc7d806af80 0x7fc7d806d440 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7ec0ffa40 0x7fc7ec195550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 --2- 172.21.15.137:0/3017313552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7ec105a40 0x7fc7ec195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 >> 172.21.15.137:0/3017313552 conn(0x7fc7ec0fb570 msgr2=0x7fc7ec0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 shutdown_connections 2024-06-15T01:15:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:06.195+0000 7fc7f269d700 1 -- 172.21.15.137:0/3017313552 wait complete. 2024-06-15T01:15:06.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:06 smithi142 ceph-mon[82136]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:06.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:06 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3017313552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:15:06.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:06 smithi142 ceph-mon[82136]: osdmap e94: 6 total, 6 up, 6 in 2024-06-15T01:15:06.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:06 smithi137 ceph-mon[112295]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:06.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:06 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3017313552' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:15:06.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:06 smithi137 ceph-mon[112295]: osdmap e94: 6 total, 6 up, 6 in 2024-06-15T01:15:07.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:07 smithi137 ceph-mon[112295]: pgmap v270: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:07.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:07 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:15:08.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:07 smithi142 ceph-mon[82136]: pgmap v270: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:08.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:07 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:15:08.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:08 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:15:09.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:08 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:15:09.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:09 smithi142 ceph-mon[82136]: pgmap v271: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:09.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:09 smithi137 ceph-mon[112295]: pgmap v271: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: pgmap v272: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] up:boot 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:15:12.452 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:12 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: pgmap v272: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] up:boot 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:15:12.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:12 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:15:14.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:14 smithi137 ceph-mon[112295]: pgmap v273: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:14.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:14 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:14.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:14 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:14.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:14 smithi142 ceph-mon[82136]: pgmap v273: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:14.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:14 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:14.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:14 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:15.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:15.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:15 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:15.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:15 smithi142 ceph-mon[82136]: pgmap v274: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:15.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:15.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:15 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:15.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:15 smithi137 ceph-mon[112295]: pgmap v274: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: pgmap v275: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.004 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:18.005 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.005 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.005 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.005 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:17 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: pgmap v275: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:18.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:18.220 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:17 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:19.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:19.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: pgmap v276: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: Upgrade: Updating mds.cephfs.smithi142.uczriv 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:19.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:19 smithi137 ceph-mon[112295]: Deploying daemon mds.cephfs.smithi142.uczriv on smithi142 2024-06-15T01:15:20.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: pgmap v276: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:20.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: Upgrade: Updating mds.cephfs.smithi142.uczriv 2024-06-15T01:15:20.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:20.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.uczriv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-15T01:15:20.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:15:20.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:19 smithi142 ceph-mon[82136]: Deploying daemon mds.cephfs.smithi142.uczriv on smithi142 2024-06-15T01:15:21.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:21 smithi142 ceph-mon[82136]: osdmap e95: 6 total, 6 up, 6 in 2024-06-15T01:15:21.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:21 smithi142 ceph-mon[82136]: pgmap v278: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:21.811 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:21 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:15:21.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:21 smithi137 ceph-mon[112295]: osdmap e95: 6 total, 6 up, 6 in 2024-06-15T01:15:21.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:21 smithi137 ceph-mon[112295]: pgmap v278: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:21.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:21 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-06-15T01:15:23.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:23 smithi142 ceph-mon[82136]: pgmap v279: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:23.719 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:23 smithi137 ceph-mon[112295]: pgmap v279: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] up:boot 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:25.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:25 smithi142 ceph-mon[82136]: pgmap v280: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] up:boot 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:25.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:15:25.970 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:25 smithi137 ceph-mon[112295]: pgmap v280: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:28.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:27 smithi142 ceph-mon[82136]: pgmap v281: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:28.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:27 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:28.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:27 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:28.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:27 smithi137 ceph-mon[112295]: pgmap v281: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:28.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:27 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:28.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:27 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:29.169 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:28 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:29.169 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:28 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:29.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:28 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:29.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:28 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:15:29.969 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:29 smithi137 ceph-mon[112295]: pgmap v282: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:30.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:29 smithi142 ceph-mon[82136]: pgmap v282: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:32.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:31 smithi137 ceph-mon[112295]: pgmap v283: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:32.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:31 smithi142 ceph-mon[82136]: pgmap v283: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:34.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:33 smithi137 ceph-mon[112295]: pgmap v284: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:34.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:34.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:33 smithi142 ceph-mon[82136]: pgmap v284: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:34.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:36.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:35 smithi137 ceph-mon[112295]: pgmap v285: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.316+0000 7f72cc391700 1 -- 172.21.15.137:0/907239654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c40ffa60 msgr2=0x7f72c40ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.316+0000 7f72cc391700 1 --2- 172.21.15.137:0/907239654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c40ffa60 0x7f72c40ffee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009910 tx=0x7f72b4009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 -- 172.21.15.137:0/907239654 shutdown_connections 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 --2- 172.21.15.137:0/907239654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c40ffa60 0x7f72c40ffee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 --2- 172.21.15.137:0/907239654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c4105a60 0x7f72c4105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 -- 172.21.15.137:0/907239654 >> 172.21.15.137:0/907239654 conn(0x7f72c40fb570 msgr2=0x7f72c40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 -- 172.21.15.137:0/907239654 shutdown_connections 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 -- 172.21.15.137:0/907239654 wait complete. 2024-06-15T01:15:36.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 Processor -- start 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.317+0000 7f72cc391700 1 -- start start 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72cc391700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 0x7f72c41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72cc391700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72cc391700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4198000 con 0x7f72c4105a60 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72cc391700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4198140 con 0x7f72c40ffa60 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37390/0 (socket says 172.21.15.137:37390) 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 -- 172.21.15.137:0/1101073380 learned_addr learned my addr 172.21.15.137:0/1101073380 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72ca12d700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 0x7f72c41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 -- 172.21.15.137:0/1101073380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 msgr2=0x7f72c41974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 0x7f72c41974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 -- 172.21.15.137:0/1101073380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40095f0 con 0x7f72c4105a60 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72ca12d700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 0x7f72c41974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72c992c700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.318+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b401d070 con 0x7f72c4105a60 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.319+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72b400f460 con 0x7f72c4105a60 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.319+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c41a0510 con 0x7f72c4105a60 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.319+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b4021620 con 0x7f72c4105a60 2024-06-15T01:15:36.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.319+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c41a0920 con 0x7f72c4105a60 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.320+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c4107ff0 con 0x7f72c4105a60 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.320+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f72b400fa40 con 0x7f72c4105a60 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.321+0000 7f72bb7fe700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 0x7f72b006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.321+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f72b408b5a0 con 0x7f72c4105a60 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.321+0000 7f72ca12d700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 0x7f72b006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.321+0000 7f72ca12d700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 0x7f72b006d440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005950 tx=0x7f72c00058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:35 smithi142 ceph-mon[82136]: pgmap v285: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:36.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.325+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f72b40563d0 con 0x7f72c4105a60 2024-06-15T01:15:36.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.534+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c40611d0 con 0x7f72b006af80 2024-06-15T01:15:36.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.535+0000 7f72bb7fe700 1 -- 172.21.15.137:0/1101073380 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72c40611d0 con 0x7f72b006af80 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 msgr2=0x7f72b006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 0x7f72b006d440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005950 tx=0x7f72c00058e0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 msgr2=0x7f72c41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b400be20 comp rx=0 tx=0).stop 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 shutdown_connections 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f72b006af80 0x7f72b006d440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c40ffa60 0x7f72c41974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 --2- 172.21.15.137:0/1101073380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c4105a60 0x7f72c41979e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 >> 172.21.15.137:0/1101073380 conn(0x7f72c40fb570 msgr2=0x7f72c40fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:36.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 shutdown_connections 2024-06-15T01:15:36.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.537+0000 7f72cc391700 1 -- 172.21.15.137:0/1101073380 wait complete. 2024-06-15T01:15:36.537 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 -- 172.21.15.137:0/437185372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 msgr2=0x7fb7dc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/437185372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc1011a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009910 tx=0x7fb7d8009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 -- 172.21.15.137:0/437185372 shutdown_connections 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/437185372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/437185372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc1011a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 -- 172.21.15.137:0/437185372 >> 172.21.15.137:0/437185372 conn(0x7fb7dc0fc650 msgr2=0x7fb7dc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 -- 172.21.15.137:0/437185372 shutdown_connections 2024-06-15T01:15:36.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 -- 172.21.15.137:0/437185372 wait complete. 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.643+0000 7fb7e3db8700 1 Processor -- start 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e3db8700 1 -- start start 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e3db8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e3db8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e3db8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc073b10 con 0x7fb7dc100dc0 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e3db8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc073c50 con 0x7fb7dc101770 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37418/0 (socket says 172.21.15.137:37418) 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 -- 172.21.15.137:0/672081294 learned_addr learned my addr 172.21.15.137:0/672081294 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:36.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1353700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 -- 172.21.15.137:0/672081294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 msgr2=0x7fb7dc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 -- 172.21.15.137:0/672081294 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80095f0 con 0x7fb7dc100dc0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1353700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.644+0000 7fb7e1b54700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80092e0 tx=0x7fb7d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.645+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d801d070 con 0x7fb7dc100dc0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.645+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7d8005e00 con 0x7fb7dc100dc0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.645+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc073ed0 con 0x7fb7dc100dc0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.645+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d800fe60 con 0x7fb7dc100dc0 2024-06-15T01:15:36.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.645+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1a2b90 con 0x7fb7dc100dc0 2024-06-15T01:15:36.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.646+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc109160 con 0x7fb7dc100dc0 2024-06-15T01:15:36.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.647+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb7d800f6d0 con 0x7fb7dc100dc0 2024-06-15T01:15:36.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.647+0000 7fb7d2ffd700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 0x7fb7c806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.647+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb7d808ba00 con 0x7fb7dc100dc0 2024-06-15T01:15:36.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.647+0000 7fb7e1353700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 0x7fb7c806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.648+0000 7fb7e1353700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 0x7fb7c806d440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc074880 tx=0x7fb7cc00b2c0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.651+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb7d8056830 con 0x7fb7dc100dc0 2024-06-15T01:15:36.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.862+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7dc02cce0 con 0x7fb7c806af80 2024-06-15T01:15:36.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.863+0000 7fb7d2ffd700 1 -- 172.21.15.137:0/672081294 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb7dc02cce0 con 0x7fb7c806af80 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.865+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 msgr2=0x7fb7c806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 0x7fb7c806d440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc074880 tx=0x7fb7cc00b2c0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 msgr2=0x7fb7dc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80092e0 tx=0x7fb7d800be20 comp rx=0 tx=0).stop 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 shutdown_connections 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb7c806af80 0x7fb7c806d440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7dc100dc0 0x7fb7dc073090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 --2- 172.21.15.137:0/672081294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7dc101770 0x7fb7dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 >> 172.21.15.137:0/672081294 conn(0x7fb7dc0fc650 msgr2=0x7fb7dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 shutdown_connections 2024-06-15T01:15:36.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.866+0000 7fb7e3db8700 1 -- 172.21.15.137:0/672081294 wait complete. 2024-06-15T01:15:36.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 -- 172.21.15.137:0/1518188090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378101770 msgr2=0x7f53781057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 --2- 172.21.15.137:0/1518188090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378101770 0x7f53781057c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5368009910 tx=0x7f5368009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:36.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 -- 172.21.15.137:0/1518188090 shutdown_connections 2024-06-15T01:15:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 --2- 172.21.15.137:0/1518188090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378101770 0x7f53781057c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 --2- 172.21.15.137:0/1518188090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378100dc0 0x7f53781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 -- 172.21.15.137:0/1518188090 >> 172.21.15.137:0/1518188090 conn(0x7f53780fc650 msgr2=0x7f53780fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 -- 172.21.15.137:0/1518188090 shutdown_connections 2024-06-15T01:15:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.970+0000 7f537d156700 1 -- 172.21.15.137:0/1518188090 wait complete. 2024-06-15T01:15:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 Processor -- start 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 -- start start 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 0x7f537819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537819d1c0 con 0x7f5378100dc0 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f537d156700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378196670 con 0x7f5378101770 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.971+0000 7f5376d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37436/0 (socket says 172.21.15.137:37436) 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 -- 172.21.15.137:0/2928051378 learned_addr learned my addr 172.21.15.137:0/2928051378 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f537659c700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 0x7f537819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 -- 172.21.15.137:0/2928051378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 msgr2=0x7f537819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:36.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 0x7f537819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 -- 172.21.15.137:0/2928051378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680095f0 con 0x7f5378100dc0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f537659c700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 0x7f537819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f5376d9d700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b4b0 tx=0x7f536c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c011800 con 0x7f5378100dc0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f536c011e00 con 0x7f5378100dc0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5378196950 con 0x7f5378100dc0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c010e50 con 0x7f5378100dc0 2024-06-15T01:15:36.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.972+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5378196ea0 con 0x7f5378100dc0 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.973+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53781090f0 con 0x7f5378100dc0 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.974+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f536c010380 con 0x7f5378100dc0 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.974+0000 7f535ffff700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 0x7f536006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.974+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f536c089f00 con 0x7f5378100dc0 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.974+0000 7f537659c700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 0x7f536006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:36.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.975+0000 7f537659c700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 0x7f536006d440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53780fef40 tx=0x7f536801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:36.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:36.979+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f536c058840 con 0x7f5378100dc0 2024-06-15T01:15:37.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.181+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f53781970c0 con 0x7f536006af80 2024-06-15T01:15:37.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.191+0000 7f535ffff700 1 -- 172.21.15.137:0/2928051378 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f53781970c0 con 0x7f536006af80 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (6m) 37s ago 16m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (17m) 37s ago 17m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (15m) 9s ago 15m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (4m) 37s ago 16m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (4m) 9s ago 15m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (5m) 37s ago 16m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (42s) 37s ago 11m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (59s) 37s ago 11m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:15:37.180 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (26s) 9s ago 11m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (13s) 9s ago 11m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (8m) 37s ago 18m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (6m) 9s ago 15m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (4m) 37s ago 18m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (4m) 9s ago 15m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (6m) 37s ago 16m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 9s ago 15m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (3m) 37s ago 14m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (3m) 37s ago 14m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (2m) 37s ago 13m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (2m) 9s ago 13m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:15:37.181 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (106s) 9s ago 13m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:15:37.182 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (75s) 9s ago 12m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:15:37.182 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (6m) 37s ago 16m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:15:37.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 msgr2=0x7f536006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 0x7f536006d440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53780fef40 tx=0x7f536801a040 comp rx=0 tx=0).stop 2024-06-15T01:15:37.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 msgr2=0x7f537819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b4b0 tx=0x7f536c00b870 comp rx=0 tx=0).stop 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 shutdown_connections 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f536006af80 0x7f536006d440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378100dc0 0x7f537819c5f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 --2- 172.21.15.137:0/2928051378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5378101770 0x7f537819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 >> 172.21.15.137:0/2928051378 conn(0x7f53780fc650 msgr2=0x7f53780fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 shutdown_connections 2024-06-15T01:15:37.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.193+0000 7f537d156700 1 -- 172.21.15.137:0/2928051378 wait complete. 2024-06-15T01:15:37.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 -- 172.21.15.137:0/1448121851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 msgr2=0x7f9d60105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 --2- 172.21.15.137:0/1448121851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d60105e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009910 tx=0x7f9d54009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:37.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 -- 172.21.15.137:0/1448121851 shutdown_connections 2024-06-15T01:15:37.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 --2- 172.21.15.137:0/1448121851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d60100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 --2- 172.21.15.137:0/1448121851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d60105e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 -- 172.21.15.137:0/1448121851 >> 172.21.15.137:0/1448121851 conn(0x7f9d600755a0 msgr2=0x7f9d600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:37.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 -- 172.21.15.137:0/1448121851 shutdown_connections 2024-06-15T01:15:37.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.298+0000 7f9d64b63700 1 -- 172.21.15.137:0/1448121851 wait complete. 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 Processor -- start 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 -- start start 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d601952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60195d30 con 0x7f9d60105a40 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d64b63700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60195ea0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d601952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37450/0 (socket says 172.21.15.137:37450) 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5dd9b700 1 -- 172.21.15.137:0/2292423465 learned_addr learned my addr 172.21.15.137:0/2292423465 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5e59c700 1 -- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 msgr2=0x7f9d601957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5e59c700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5e59c700 1 -- 172.21.15.137:0/2292423465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540095f0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.299+0000 7f9d5dd9b700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d5e59c700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d601952b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005b40 tx=0x7f9d5400bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5401d070 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d54005dc0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d540039d0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d60196120 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.300+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d601a2af0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.301+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d60108040 con 0x7f9d600ffbd0 2024-06-15T01:15:37.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.301+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f9d54004b70 con 0x7f9d600ffbd0 2024-06-15T01:15:37.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.302+0000 7f9d4f7fe700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 0x7f9d4806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.302+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9d5408b200 con 0x7f9d600ffbd0 2024-06-15T01:15:37.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.302+0000 7f9d5dd9b700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 0x7f9d4806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.302+0000 7f9d5dd9b700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 0x7f9d4806d440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d60068d70 tx=0x7f9d50009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:37.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.306+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9d54059a60 con 0x7f9d600ffbd0 2024-06-15T01:15:37.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.577+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d601960c0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.578+0000 7f9d4f7fe700 1 -- 172.21.15.137:0/2292423465 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9d540595f0 con 0x7f9d600ffbd0 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:15:37.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:15:37.568 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 msgr2=0x7f9d4806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 0x7f9d4806d440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d60068d70 tx=0x7f9d50009260 comp rx=0 tx=0).stop 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 msgr2=0x7f9d601952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d601952b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005b40 tx=0x7f9d5400bde0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 shutdown_connections 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d4806af80 0x7f9d4806d440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d600ffbd0 0x7f9d601952b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 --2- 172.21.15.137:0/2292423465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d60105a40 0x7f9d601957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 >> 172.21.15.137:0/2292423465 conn(0x7f9d600755a0 msgr2=0x7f9d600fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:37.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 shutdown_connections 2024-06-15T01:15:37.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.580+0000 7f9d64b63700 1 -- 172.21.15.137:0/2292423465 wait complete. 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 -- 172.21.15.137:0/2779438065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 msgr2=0x7f8314105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 --2- 172.21.15.137:0/2779438065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314105790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8304009910 tx=0x7f8304009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 -- 172.21.15.137:0/2779438065 shutdown_connections 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 --2- 172.21.15.137:0/2779438065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314105790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 --2- 172.21.15.137:0/2779438065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 0x7f8314068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 -- 172.21.15.137:0/2779438065 >> 172.21.15.137:0/2779438065 conn(0x7f83140755a0 msgr2=0x7f83140759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:37.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 -- 172.21.15.137:0/2779438065 shutdown_connections 2024-06-15T01:15:37.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.684+0000 7f831996b700 1 -- 172.21.15.137:0/2779438065 wait complete. 2024-06-15T01:15:37.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 Processor -- start 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 -- start start 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 0x7f8314196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8314197260 con 0x7f8314069020 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f831996b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831419aff0 con 0x7f8314068670 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f83127fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f83127fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37468/0 (socket says 172.21.15.137:37468) 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f83127fc700 1 -- 172.21.15.137:0/245358737 learned_addr learned my addr 172.21.15.137:0/245358737 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f8312ffd700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 0x7f8314196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f83127fc700 1 -- 172.21.15.137:0/245358737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 msgr2=0x7f8314196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.685+0000 7f83127fc700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 0x7f8314196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f83127fc700 1 -- 172.21.15.137:0/245358737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040095f0 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f83127fc700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8304000c00 tx=0x7f830400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f830401d070 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f830400bed0 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831419b270 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f830400fd90 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.686+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831419b700 con 0x7f8314069020 2024-06-15T01:15:37.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.687+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83141090a0 con 0x7f8314069020 2024-06-15T01:15:37.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.687+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8304021410 con 0x7f8314069020 2024-06-15T01:15:37.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.688+0000 7f8318969700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 0x7f830006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:37.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.688+0000 7f8312ffd700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 0x7f830006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:37.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.688+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f830408b500 con 0x7f8314069020 2024-06-15T01:15:37.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.688+0000 7f8312ffd700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 0x7f830006d440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009c60 tx=0x7f82fc009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:37.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.692+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8304056310 con 0x7f8314069020 2024-06-15T01:15:37.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.926+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f831419ba30 con 0x7f8314069020 2024-06-15T01:15:37.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.927+0000 7f8318969700 1 -- 172.21.15.137:0/245358737 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f8304059930 con 0x7f8314069020 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:15:37.916 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:15:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:37.919 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:37.919 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:15:37.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 msgr2=0x7f830006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 0x7f830006d440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009c60 tx=0x7f82fc009260 comp rx=0 tx=0).stop 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 msgr2=0x7f8314196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8304000c00 tx=0x7f830400ba70 comp rx=0 tx=0).stop 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 shutdown_connections 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f830006af80 0x7f830006d440 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8314068670 0x7f8314196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 --2- 172.21.15.137:0/245358737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8314069020 0x7f8314196b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 >> 172.21.15.137:0/245358737 conn(0x7f83140755a0 msgr2=0x7f8314105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 shutdown_connections 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:37.929+0000 7f831996b700 1 -- 172.21.15.137:0/245358737 wait complete. 2024-06-15T01:15:37.920 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:15:38.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.034+0000 7fd4af721700 1 -- 172.21.15.137:0/2307088246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8100dc0 msgr2=0x7fd4a81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.034+0000 7fd4af721700 1 --2- 172.21.15.137:0/2307088246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8100dc0 0x7fd4a81011a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009910 tx=0x7fd4a4009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 -- 172.21.15.137:0/2307088246 shutdown_connections 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 --2- 172.21.15.137:0/2307088246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8101770 0x7fd4a81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 --2- 172.21.15.137:0/2307088246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8100dc0 0x7fd4a81011a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 -- 172.21.15.137:0/2307088246 >> 172.21.15.137:0/2307088246 conn(0x7fd4a80fc650 msgr2=0x7fd4a80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 -- 172.21.15.137:0/2307088246 shutdown_connections 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 -- 172.21.15.137:0/2307088246 wait complete. 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 Processor -- start 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.035+0000 7fd4af721700 1 -- start start 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8100dc0 0x7fd4a8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8197290 con 0x7fd4a8101770 2024-06-15T01:15:38.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a819b020 con 0x7fd4a8100dc0 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37488/0 (socket says 172.21.15.137:37488) 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 -- 172.21.15.137:0/4021656124 learned_addr learned my addr 172.21.15.137:0/4021656124 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4ad4bd700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8100dc0 0x7fd4a8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 -- 172.21.15.137:0/4021656124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8100dc0 msgr2=0x7fd4a8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8100dc0 0x7fd4a8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 -- 172.21.15.137:0/4021656124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40095f0 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4accbc700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e8f0 tx=0x7fd49800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd498009790 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd49800c810 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a819b300 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd498005520 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.036+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a819b850 con 0x7fd4a8101770 2024-06-15T01:15:38.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.037+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a8109160 con 0x7fd4a8101770 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.038+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd49800c980 con 0x7fd4a8101770 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.038+0000 7fd49e7fc700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 0x7fd49406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.038+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd498014070 con 0x7fd4a8101770 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.038+0000 7fd4ad4bd700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 0x7fd49406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.039+0000 7fd4ad4bd700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 0x7fd49406d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009fd0 tx=0x7fd4a4005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.042+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd498055460 con 0x7fd4a8101770 2024-06-15T01:15:38.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:37 smithi137 ceph-mon[112295]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:37 smithi137 ceph-mon[112295]: pgmap v286: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:38.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:37 smithi137 ceph-mon[112295]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:37 smithi137 ceph-mon[112295]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:37 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2292423465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:38.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.254+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4a819bc10 con 0x7fd49406af80 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.255+0000 7fd49e7fc700 1 -- 172.21.15.137:0/4021656124 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd4a819bc10 con 0x7fd49406af80 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:15:38.244 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:15:38.245 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:15:38.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 msgr2=0x7fd49406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 0x7fd49406d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009fd0 tx=0x7fd4a4005940 comp rx=0 tx=0).stop 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 msgr2=0x7fd4a8196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e8f0 tx=0x7fd49800ec00 comp rx=0 tx=0).stop 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 shutdown_connections 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd49406af80 0x7fd49406d440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4a8100dc0 0x7fd4a8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 --2- 172.21.15.137:0/4021656124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a8101770 0x7fd4a8196bb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.257+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 >> 172.21.15.137:0/4021656124 conn(0x7fd4a80fc650 msgr2=0x7fd4a80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.258+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 shutdown_connections 2024-06-15T01:15:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.258+0000 7fd4af721700 1 -- 172.21.15.137:0/4021656124 wait complete. 2024-06-15T01:15:38.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:37 smithi142 ceph-mon[82136]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:37 smithi142 ceph-mon[82136]: pgmap v286: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:38.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:37 smithi142 ceph-mon[82136]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:37 smithi142 ceph-mon[82136]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:38.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:37 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2292423465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 -- 172.21.15.137:0/1056384084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 msgr2=0x7fd00c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 --2- 172.21.15.137:0/1056384084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c100050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009910 tx=0x7fcffc009c20 comp rx=0 tx=0).stop 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 -- 172.21.15.137:0/1056384084 shutdown_connections 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 --2- 172.21.15.137:0/1056384084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c100050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 --2- 172.21.15.137:0/1056384084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 0x7fd00c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 -- 172.21.15.137:0/1056384084 >> 172.21.15.137:0/1056384084 conn(0x7fd00c0755a0 msgr2=0x7fd00c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:38.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 -- 172.21.15.137:0/1056384084 shutdown_connections 2024-06-15T01:15:38.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.361+0000 7fd01246a700 1 -- 172.21.15.137:0/1056384084 wait complete. 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 Processor -- start 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 -- start start 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 0x7fd00c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c197f80 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd01246a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1980c0 con 0x7fd00c105a40 2024-06-15T01:15:38.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37496/0 (socket says 172.21.15.137:37496) 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd00bfff700 1 -- 172.21.15.137:0/1805747063 learned_addr learned my addr 172.21.15.137:0/1805747063 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.362+0000 7fd00b7fe700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 0x7fd00c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd00bfff700 1 -- 172.21.15.137:0/1805747063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 msgr2=0x7fd00c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd00bfff700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 0x7fd00c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd00bfff700 1 -- 172.21.15.137:0/1805747063 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0095f0 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd00bfff700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c930 tx=0x7fd00000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd000007ad0 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd000007c30 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd000018630 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c1a0520 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.363+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c1a0990 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.364+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c107fd0 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.364+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd00000f450 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.365+0000 7fd0097fa700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 0x7fcff406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:15:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.365+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd00008a600 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.365+0000 7fd00b7fe700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 0x7fcff406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:15:38.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.365+0000 7fd00b7fe700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 0x7fcff406d440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd00c068cc0 tx=0x7fcffc01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:15:38.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.369+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd0000554c0 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.638+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd00c04ea90 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.638+0000 7fd0097fa700 1 -- 172.21.15.137:0/1805747063 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fd000058ae0 con 0x7fd00c0ffbd0 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:15:38.627 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:15:38.628 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:15:38.628 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:15:38.628 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:15:38.628 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:15:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 msgr2=0x7fcff406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 0x7fcff406d440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd00c068cc0 tx=0x7fcffc01a040 comp rx=0 tx=0).stop 2024-06-15T01:15:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 msgr2=0x7fd00c1974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:15:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c930 tx=0x7fd00000ccf0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 shutdown_connections 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcff406af80 0x7fcff406d440 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd00c0ffbd0 0x7fd00c1974b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 --2- 172.21.15.137:0/1805747063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00c105a40 0x7fd00c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 >> 172.21.15.137:0/1805747063 conn(0x7fd00c0755a0 msgr2=0x7fd00c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 shutdown_connections 2024-06-15T01:15:38.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:15:38.640+0000 7fd01246a700 1 -- 172.21.15.137:0/1805747063 wait complete. 2024-06-15T01:15:39.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:38 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/245358737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:15:39.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:38 smithi137 ceph-mon[112295]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:39.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:38 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1805747063' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:15:39.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:38 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/245358737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:15:39.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:38 smithi142 ceph-mon[82136]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:15:39.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:38 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1805747063' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:15:40.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:39 smithi137 ceph-mon[112295]: pgmap v287: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:40.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:39 smithi142 ceph-mon[82136]: pgmap v287: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:42.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:41 smithi137 ceph-mon[112295]: pgmap v288: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:42.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:41 smithi142 ceph-mon[82136]: pgmap v288: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:44.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:43 smithi137 ceph-mon[112295]: pgmap v289: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:44.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:43 smithi142 ceph-mon[82136]: pgmap v289: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:46.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:45 smithi137 ceph-mon[112295]: pgmap v290: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:46.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:45 smithi142 ceph-mon[82136]: pgmap v290: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:48.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:47 smithi137 ceph-mon[112295]: pgmap v291: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:48.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:47 smithi142 ceph-mon[82136]: pgmap v291: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:49.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:15:50.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:49 smithi137 ceph-mon[112295]: pgmap v292: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:50.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:49 smithi142 ceph-mon[82136]: pgmap v292: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:52.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:51 smithi142 ceph-mon[82136]: pgmap v293: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:52.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:51 smithi137 ceph-mon[112295]: pgmap v293: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:54.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:53 smithi137 ceph-mon[112295]: pgmap v294: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:54.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:53 smithi142 ceph-mon[82136]: pgmap v294: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:56.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:55 smithi137 ceph-mon[112295]: pgmap v295: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:56.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:55 smithi142 ceph-mon[82136]: pgmap v295: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:58.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:57 smithi137 ceph-mon[112295]: pgmap v296: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:15:58.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:57 smithi142 ceph-mon[82136]: pgmap v296: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:15:59 smithi137 ceph-mon[112295]: pgmap v297: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:00.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:15:59 smithi142 ceph-mon[82136]: pgmap v297: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:02.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:01 smithi142 ceph-mon[82136]: pgmap v298: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:02.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:01 smithi137 ceph-mon[112295]: pgmap v298: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:04.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:03 smithi137 ceph-mon[112295]: pgmap v299: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:04.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:04.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:03 smithi142 ceph-mon[82136]: pgmap v299: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:04.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:06.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:05 smithi137 ceph-mon[112295]: pgmap v300: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:06.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:05 smithi142 ceph-mon[82136]: pgmap v300: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:08.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:07 smithi137 ceph-mon[112295]: pgmap v301: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:08.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:07 smithi142 ceph-mon[82136]: pgmap v301: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 -- 172.21.15.137:0/3971494457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4101780 msgr2=0x7f61f41057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 --2- 172.21.15.137:0/3971494457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4101780 0x7f61f41057d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009910 tx=0x7f61e8009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 -- 172.21.15.137:0/3971494457 shutdown_connections 2024-06-15T01:16:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 --2- 172.21.15.137:0/3971494457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4101780 0x7f61f41057d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 --2- 172.21.15.137:0/3971494457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4100dd0 0x7f61f41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 -- 172.21.15.137:0/3971494457 >> 172.21.15.137:0/3971494457 conn(0x7f61f40fc640 msgr2=0x7f61f40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:08.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 -- 172.21.15.137:0/3971494457 shutdown_connections 2024-06-15T01:16:08.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 -- 172.21.15.137:0/3971494457 wait complete. 2024-06-15T01:16:08.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.757+0000 7f61fa353700 1 Processor -- start 2024-06-15T01:16:08.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61fa353700 1 -- start start 2024-06-15T01:16:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61fa353700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61fa353700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 0x7f61f40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61fa353700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4073b10 con 0x7f61f4100dd0 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61fa353700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4073c50 con 0x7f61f4101780 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54276/0 (socket says 172.21.15.137:54276) 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 -- 172.21.15.137:0/1476584811 learned_addr learned my addr 172.21.15.137:0/1476584811 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f37fe700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 0x7f61f40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 -- 172.21.15.137:0/1476584811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 msgr2=0x7f61f40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 0x7f61f40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f3fff700 1 -- 172.21.15.137:0/1476584811 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80095f0 con 0x7f61f4100dd0 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.758+0000 7f61f37fe700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 0x7f61f40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61f3fff700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e920 tx=0x7f61e400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:08.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e4009790 con 0x7f61f4100dd0 2024-06-15T01:16:08.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f41a28d0 con 0x7f61f4100dd0 2024-06-15T01:16:08.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61e400cba0 con 0x7f61f4100dd0 2024-06-15T01:16:08.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e4018650 con 0x7f61f4100dd0 2024-06-15T01:16:08.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.759+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f41a2ce0 con 0x7f61f4100dd0 2024-06-15T01:16:08.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.760+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f41091d0 con 0x7f61f4100dd0 2024-06-15T01:16:08.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.760+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f61e400cd10 con 0x7f61f4100dd0 2024-06-15T01:16:08.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.761+0000 7f61f17fa700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 0x7f61dc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:08.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.761+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f61e4014070 con 0x7f61f4100dd0 2024-06-15T01:16:08.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.761+0000 7f61f37fe700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 0x7f61dc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:08.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.761+0000 7f61f37fe700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 0x7f61dc06d440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009fd0 tx=0x7f61e8005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:08.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.765+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f61e4052900 con 0x7f61f4100dd0 2024-06-15T01:16:08.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.975+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f402ccc0 con 0x7f61dc06af80 2024-06-15T01:16:08.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.976+0000 7f61f17fa700 1 -- 172.21.15.137:0/1476584811 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f61f402ccc0 con 0x7f61dc06af80 2024-06-15T01:16:08.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.978+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 msgr2=0x7f61dc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:08.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.978+0000 7f61fa353700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 0x7f61dc06d440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009fd0 tx=0x7f61e8005940 comp rx=0 tx=0).stop 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.978+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 msgr2=0x7f61f4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.978+0000 7f61fa353700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e920 tx=0x7f61e400ece0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 shutdown_connections 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61dc06af80 0x7f61dc06d440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61f4100dd0 0x7f61f4073090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 --2- 172.21.15.137:0/1476584811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61f4101780 0x7f61f40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 >> 172.21.15.137:0/1476584811 conn(0x7f61f40fc640 msgr2=0x7f61f40fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 shutdown_connections 2024-06-15T01:16:08.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:08.979+0000 7f61fa353700 1 -- 172.21.15.137:0/1476584811 wait complete. 2024-06-15T01:16:08.979 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 -- 172.21.15.137:0/4269478819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a84105a40 msgr2=0x7f7a84105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 --2- 172.21.15.137:0/4269478819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a84105a40 0x7f7a84105e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009910 tx=0x7f7a74009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 -- 172.21.15.137:0/4269478819 shutdown_connections 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 --2- 172.21.15.137:0/4269478819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a840ffbd0 0x7f7a84100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 --2- 172.21.15.137:0/4269478819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a84105a40 0x7f7a84105e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 -- 172.21.15.137:0/4269478819 >> 172.21.15.137:0/4269478819 conn(0x7f7a840755a0 msgr2=0x7f7a840759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:09.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 -- 172.21.15.137:0/4269478819 shutdown_connections 2024-06-15T01:16:09.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.084+0000 7f7a89e70700 1 -- 172.21.15.137:0/4269478819 wait complete. 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 Processor -- start 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 -- start start 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a84105a40 0x7f7a841979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84198000 con 0x7f7a840ffbd0 2024-06-15T01:16:09.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.085+0000 7f7a89e70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84198170 con 0x7f7a84105a40 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54296/0 (socket says 172.21.15.137:54296) 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 -- 172.21.15.137:0/2890787218 learned_addr learned my addr 172.21.15.137:0/2890787218 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a7affd700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a84105a40 0x7f7a841979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 -- 172.21.15.137:0/2890787218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a84105a40 msgr2=0x7f7a841979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a84105a40 0x7f7a841979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 -- 172.21.15.137:0/2890787218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740095f0 con 0x7f7a840ffbd0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a837fe700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b5c0 tx=0x7f7a7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7401d070 con 0x7f7a840ffbd0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a74005e00 con 0x7f7a840ffbd0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7400fe90 con 0x7f7a840ffbd0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.086+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a841a0510 con 0x7f7a840ffbd0 2024-06-15T01:16:09.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.087+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a841a0980 con 0x7f7a840ffbd0 2024-06-15T01:16:09.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.088+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84108040 con 0x7f7a840ffbd0 2024-06-15T01:16:09.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.088+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7a7400f6d0 con 0x7f7a840ffbd0 2024-06-15T01:16:09.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.088+0000 7f7a817fa700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 0x7f7a7006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.089+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7a7408b4e0 con 0x7f7a840ffbd0 2024-06-15T01:16:09.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.089+0000 7f7a7affd700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 0x7f7a7006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.089+0000 7f7a7affd700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 0x7f7a7006d440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a84068ce0 tx=0x7f7a6c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.093+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7a740564b0 con 0x7f7a840ffbd0 2024-06-15T01:16:09.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.303+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a8402cd30 con 0x7f7a7006af80 2024-06-15T01:16:09.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.304+0000 7f7a817fa700 1 -- 172.21.15.137:0/2890787218 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7a8402cd30 con 0x7f7a7006af80 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 msgr2=0x7f7a7006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 0x7f7a7006d440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a84068ce0 tx=0x7f7a6c00b410 comp rx=0 tx=0).stop 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 msgr2=0x7f7a841974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b5c0 tx=0x7f7a7400bd70 comp rx=0 tx=0).stop 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 shutdown_connections 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7006af80 0x7f7a7006d440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffbd0 0x7f7a841974a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 --2- 172.21.15.137:0/2890787218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a84105a40 0x7f7a841979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 >> 172.21.15.137:0/2890787218 conn(0x7f7a840755a0 msgr2=0x7f7a840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:09.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 shutdown_connections 2024-06-15T01:16:09.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.306+0000 7f7a89e70700 1 -- 172.21.15.137:0/2890787218 wait complete. 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1654101824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 msgr2=0x7f60a00ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1654101824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a00ffec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6094009910 tx=0x7f6094009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1654101824 shutdown_connections 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1654101824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a00ffec0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1654101824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 0x7f60a0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1654101824 >> 172.21.15.137:0/1654101824 conn(0x7f60a00fb550 msgr2=0x7f60a00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1654101824 shutdown_connections 2024-06-15T01:16:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.411+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1654101824 wait complete. 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 Processor -- start 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 -- start start 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 0x7f60a0196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01972c0 con 0x7f60a00ffa40 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f60a4fe6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a019b000 con 0x7f60a0105a40 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f609e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f609e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54304/0 (socket says 172.21.15.137:54304) 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f609e59c700 1 -- 172.21.15.137:0/1957016508 learned_addr learned my addr 172.21.15.137:0/1957016508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.412+0000 7f609dd9b700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 0x7f60a0196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f609e59c700 1 -- 172.21.15.137:0/1957016508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 msgr2=0x7f60a0196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f609e59c700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 0x7f60a0196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f609e59c700 1 -- 172.21.15.137:0/1957016508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940095f0 con 0x7f60a00ffa40 2024-06-15T01:16:09.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f609e59c700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f609000d8d0 tx=0x7f609000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60900098a0 con 0x7f60a00ffa40 2024-06-15T01:16:09.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6090010460 con 0x7f60a00ffa40 2024-06-15T01:16:09.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a019b2e0 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6090009ea0 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.413+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a019b830 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.414+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a004ea90 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.415+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f60900105d0 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.415+0000 7f608f7fe700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 0x7f608806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.415+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f609008a1b0 con 0x7f60a00ffa40 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.415+0000 7f609dd9b700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 0x7f608806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.415+0000 7f609dd9b700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 0x7f608806d440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60a0197c70 tx=0x7f609401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.419+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6090055060 con 0x7f60a00ffa40 2024-06-15T01:16:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.618+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60a00008d0 con 0x7f608806af80 2024-06-15T01:16:09.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.628+0000 7f608f7fe700 1 -- 172.21.15.137:0/1957016508 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f60a00008d0 con 0x7f608806af80 2024-06-15T01:16:09.617 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:16:09.617 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (6m) 69s ago 17m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:16:09.617 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (17m) 69s ago 17m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (16m) 42s ago 16m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (4m) 69s ago 17m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (4m) 42s ago 16m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (6m) 69s ago 17m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (75s) 69s ago 11m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (91s) 69s ago 11m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (59s) 42s ago 11m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (45s) 42s ago 11m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (8m) 69s ago 18m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (7m) 42s ago 16m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (5m) 69s ago 18m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (5m) 42s ago 16m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:16:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (7m) 69s ago 17m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 42s ago 16m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (4m) 69s ago 15m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (3m) 69s ago 14m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (3m) 69s ago 14m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (2m) 42s ago 14m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (2m) 42s ago 13m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (107s) 42s ago 13m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:16:09.619 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (6m) 69s ago 16m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 msgr2=0x7f608806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 0x7f608806d440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60a0197c70 tx=0x7f609401a040 comp rx=0 tx=0).stop 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 msgr2=0x7f60a01966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f609000d8d0 tx=0x7f609000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 shutdown_connections 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f608806af80 0x7f608806d440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60a00ffa40 0x7f60a01966a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 --2- 172.21.15.137:0/1957016508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0105a40 0x7f60a0196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 >> 172.21.15.137:0/1957016508 conn(0x7f60a00fb550 msgr2=0x7f60a01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:09.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 shutdown_connections 2024-06-15T01:16:09.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.630+0000 7f60a4fe6700 1 -- 172.21.15.137:0/1957016508 wait complete. 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 -- 172.21.15.137:0/3202420547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 msgr2=0x7f0c2c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 --2- 172.21.15.137:0/3202420547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1011c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009910 tx=0x7f0c24009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 -- 172.21.15.137:0/3202420547 shutdown_connections 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 --2- 172.21.15.137:0/3202420547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 0x7f0c2c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 --2- 172.21.15.137:0/3202420547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1011c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 -- 172.21.15.137:0/3202420547 >> 172.21.15.137:0/3202420547 conn(0x7f0c2c0fc6b0 msgr2=0x7f0c2c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 -- 172.21.15.137:0/3202420547 shutdown_connections 2024-06-15T01:16:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.733+0000 7f0c333e8700 1 -- 172.21.15.137:0/3202420547 wait complete. 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 Processor -- start 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 -- start start 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 0x7f0c2c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1972a0 con 0x7f0c2c100de0 2024-06-15T01:16:09.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c333e8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c199fd0 con 0x7f0c2c101790 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54310/0 (socket says 172.21.15.137:54310) 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 -- 172.21.15.137:0/4097980783 learned_addr learned my addr 172.21.15.137:0/4097980783 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c30983700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 0x7f0c2c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 -- 172.21.15.137:0/4097980783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 msgr2=0x7f0c2c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 0x7f0c2c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.734+0000 7f0c31184700 1 -- 172.21.15.137:0/4097980783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c240095f0 con 0x7f0c2c100de0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c31184700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c2400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2401d070 con 0x7f0c2c100de0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c24005dc0 con 0x7f0c2c100de0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c19a250 con 0x7f0c2c100de0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c240039b0 con 0x7f0c2c100de0 2024-06-15T01:16:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.735+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c19a6e0 con 0x7f0c2c100de0 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.736+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c109180 con 0x7f0c2c100de0 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.736+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0c240048e0 con 0x7f0c2c100de0 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.737+0000 7f0c2a7fc700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 0x7f0c1406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.737+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0c2408b9e0 con 0x7f0c2c100de0 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.737+0000 7f0c30983700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 0x7f0c1406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.737+0000 7f0c30983700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 0x7f0c1406d440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c197cf0 tx=0x7f0c20009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:09.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:09.741+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0c24056810 con 0x7f0c2c100de0 2024-06-15T01:16:10.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.010+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c2c108b60 con 0x7f0c2c100de0 2024-06-15T01:16:10.000 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:09 smithi137 ceph-mon[112295]: pgmap v302: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:10.001 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:09 smithi137 ceph-mon[112295]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.011+0000 7f0c2a7fc700 1 -- 172.21.15.137:0/4097980783 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0c24059e30 con 0x7f0c2c100de0 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:16:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:16:10.002 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 msgr2=0x7f0c1406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 0x7f0c1406d440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c197cf0 tx=0x7f0c20009260 comp rx=0 tx=0).stop 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 msgr2=0x7f0c2c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c2400be90 comp rx=0 tx=0).stop 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 shutdown_connections 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0c1406af80 0x7f0c1406d440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c2c100de0 0x7f0c2c1966d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 --2- 172.21.15.137:0/4097980783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c101790 0x7f0c2c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 >> 172.21.15.137:0/4097980783 conn(0x7f0c2c0fc6b0 msgr2=0x7f0c2c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 shutdown_connections 2024-06-15T01:16:10.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.013+0000 7f0c333e8700 1 -- 172.21.15.137:0/4097980783 wait complete. 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 -- 172.21.15.137:0/1653958877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba40ff260 msgr2=0x7f0ba40ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 --2- 172.21.15.137:0/1653958877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba40ff260 0x7f0ba40ff6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009910 tx=0x7f0b94009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 -- 172.21.15.137:0/1653958877 shutdown_connections 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 --2- 172.21.15.137:0/1653958877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba40ff260 0x7f0ba40ff6e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 --2- 172.21.15.137:0/1653958877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba4106be0 0x7f0ba4106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 -- 172.21.15.137:0/1653958877 >> 172.21.15.137:0/1653958877 conn(0x7f0ba4074ce0 msgr2=0x7f0ba40750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 -- 172.21.15.137:0/1653958877 shutdown_connections 2024-06-15T01:16:10.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.121+0000 7f0babccb700 1 -- 172.21.15.137:0/1653958877 wait complete. 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 Processor -- start 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 -- start start 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba40ff260 0x7f0ba41965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4197200 con 0x7f0ba4106be0 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0babccb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba419af90 con 0x7f0ba40ff260 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0ba9266700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0ba9266700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54318/0 (socket says 172.21.15.137:54318) 2024-06-15T01:16:10.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.122+0000 7f0ba9266700 1 -- 172.21.15.137:0/4050378288 learned_addr learned my addr 172.21.15.137:0/4050378288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0ba9a67700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba40ff260 0x7f0ba41965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0ba9266700 1 -- 172.21.15.137:0/4050378288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba40ff260 msgr2=0x7f0ba41965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0ba9266700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba40ff260 0x7f0ba41965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0ba9266700 1 -- 172.21.15.137:0/4050378288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940095f0 con 0x7f0ba4106be0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0ba9266700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0b940092e0 tx=0x7f0b9400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b9401d070 con 0x7f0ba4106be0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b94005e00 con 0x7f0ba4106be0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b9400fe20 con 0x7f0ba4106be0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba419b210 con 0x7f0ba4106be0 2024-06-15T01:16:10.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.123+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba419b700 con 0x7f0ba4106be0 2024-06-15T01:16:10.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.125+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba4109030 con 0x7f0ba4106be0 2024-06-15T01:16:10.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.125+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0b9400f660 con 0x7f0ba4106be0 2024-06-15T01:16:10.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.125+0000 7f0b9affd700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 0x7f0b9006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.125+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0b9408b610 con 0x7f0ba4106be0 2024-06-15T01:16:10.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.125+0000 7f0ba9a67700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 0x7f0b9006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.126+0000 7f0ba9a67700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 0x7f0b9006d440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005950 tx=0x7f0ba000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.130+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0b94056440 con 0x7f0ba4106be0 2024-06-15T01:16:10.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:09 smithi142 ceph-mon[82136]: pgmap v302: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:10.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:09 smithi142 ceph-mon[82136]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:10.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.366+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ba404ea90 con 0x7f0ba4106be0 2024-06-15T01:16:10.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.366+0000 7f0b9affd700 1 -- 172.21.15.137:0/4050378288 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f0b94059a60 con 0x7f0ba4106be0 2024-06-15T01:16:10.356 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:16:10.356 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:16:10.357 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:16:10.358 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:10.359 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:10.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 msgr2=0x7f0b9006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 0x7f0b9006d440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005950 tx=0x7f0ba000b410 comp rx=0 tx=0).stop 2024-06-15T01:16:10.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 msgr2=0x7f0ba4196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0b940092e0 tx=0x7f0b9400bd70 comp rx=0 tx=0).stop 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 shutdown_connections 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b9006af80 0x7f0b9006d440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba40ff260 0x7f0ba41965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 --2- 172.21.15.137:0/4050378288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba4106be0 0x7f0ba4196b20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 >> 172.21.15.137:0/4050378288 conn(0x7f0ba4074ce0 msgr2=0x7f0ba40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 shutdown_connections 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.369+0000 7f0babccb700 1 -- 172.21.15.137:0/4050378288 wait complete. 2024-06-15T01:16:10.361 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:16:10.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 -- 172.21.15.137:0/4005190729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 msgr2=0x7fd24c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 --2- 172.21.15.137:0/4005190729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c1057c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd238009910 tx=0x7fd238009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:10.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 -- 172.21.15.137:0/4005190729 shutdown_connections 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 --2- 172.21.15.137:0/4005190729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c1057c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 --2- 172.21.15.137:0/4005190729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 0x7fd24c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 -- 172.21.15.137:0/4005190729 >> 172.21.15.137:0/4005190729 conn(0x7fd24c0fc650 msgr2=0x7fd24c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 -- 172.21.15.137:0/4005190729 shutdown_connections 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.475+0000 7fd252bd8700 1 -- 172.21.15.137:0/4005190729 wait complete. 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 Processor -- start 2024-06-15T01:16:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 -- start start 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 0x7fd24c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c1972a0 con 0x7fd24c101770 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd252bd8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c199fd0 con 0x7fd24c100dc0 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54336/0 (socket says 172.21.15.137:54336) 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 -- 172.21.15.137:0/2038751742 learned_addr learned my addr 172.21.15.137:0/2038751742 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd250974700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 0x7fd24c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 -- 172.21.15.137:0/2038751742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 msgr2=0x7fd24c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 0x7fd24c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.476+0000 7fd24bfff700 1 -- 172.21.15.137:0/2038751742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380095f0 con 0x7fd24c101770 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd24bfff700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2380092e0 tx=0x7fd23800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23801d070 con 0x7fd24c101770 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd238005940 con 0x7fd24c101770 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c19a250 con 0x7fd24c101770 2024-06-15T01:16:10.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23800fe50 con 0x7fd24c101770 2024-06-15T01:16:10.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.477+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c19a740 con 0x7fd24c101770 2024-06-15T01:16:10.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.478+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c1091c0 con 0x7fd24c101770 2024-06-15T01:16:10.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.478+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd238003a60 con 0x7fd24c101770 2024-06-15T01:16:10.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.479+0000 7fd249ffb700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 0x7fd23c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.479+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd23808b5b0 con 0x7fd24c101770 2024-06-15T01:16:10.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.479+0000 7fd250974700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 0x7fd23c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.479+0000 7fd250974700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 0x7fd23c06d440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd240005f60 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.483+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd238056460 con 0x7fd24c101770 2024-06-15T01:16:10.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.689+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd24c19aa20 con 0x7fd23c06af80 2024-06-15T01:16:10.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.691+0000 7fd249ffb700 1 -- 172.21.15.137:0/2038751742 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd24c19aa20 con 0x7fd23c06af80 2024-06-15T01:16:10.680 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:16:10.681 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:16:10.682 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:16:10.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.693+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 msgr2=0x7fd23c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 0x7fd23c06d440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd240005f60 comp rx=0 tx=0).stop 2024-06-15T01:16:10.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 msgr2=0x7fd24c196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2380092e0 tx=0x7fd23800be20 comp rx=0 tx=0).stop 2024-06-15T01:16:10.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 shutdown_connections 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd23c06af80 0x7fd23c06d440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd24c100dc0 0x7fd24c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 --2- 172.21.15.137:0/2038751742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd24c101770 0x7fd24c196c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 >> 172.21.15.137:0/2038751742 conn(0x7fd24c0fc650 msgr2=0x7fd24c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 shutdown_connections 2024-06-15T01:16:10.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.694+0000 7fd252bd8700 1 -- 172.21.15.137:0/2038751742 wait complete. 2024-06-15T01:16:10.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 -- 172.21.15.137:0/181527707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 msgr2=0x7f23081057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 --2- 172.21.15.137:0/181527707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f23081057e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22f800b0a0 tx=0x7f22f800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 -- 172.21.15.137:0/181527707 shutdown_connections 2024-06-15T01:16:10.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 --2- 172.21.15.137:0/181527707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f23081057e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 --2- 172.21.15.137:0/181527707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 0x7f23081011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 -- 172.21.15.137:0/181527707 >> 172.21.15.137:0/181527707 conn(0x7f23080fc670 msgr2=0x7f23080fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 -- 172.21.15.137:0/181527707 shutdown_connections 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 -- 172.21.15.137:0/181527707 wait complete. 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.798+0000 7f230ecba700 1 Processor -- start 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ecba700 1 -- start start 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ecba700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 0x7f23081966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ecba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ecba700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23081972a0 con 0x7f2308101790 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ecba700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2308199fd0 con 0x7f2308100de0 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54352/0 (socket says 172.21.15.137:54352) 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 -- 172.21.15.137:0/1577568742 learned_addr learned my addr 172.21.15.137:0/1577568742 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:10.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f230ca56700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 0x7f23081966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 -- 172.21.15.137:0/1577568742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 msgr2=0x7f23081966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 0x7f23081966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 -- 172.21.15.137:0/1577568742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009d00 con 0x7f2308101790 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2307fff700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22f8006130 tx=0x7f22f8007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.799+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f800e040 con 0x7f2308101790 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.800+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230819a250 con 0x7f2308101790 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.800+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22f8003de0 con 0x7f2308101790 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.800+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f801d7d0 con 0x7f2308101790 2024-06-15T01:16:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.800+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230819a740 con 0x7f2308101790 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.801+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23081091e0 con 0x7f2308101790 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.801+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f22f8004770 con 0x7f2308101790 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.801+0000 7f2305ffb700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 0x7f22f406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.802+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f22f808be20 con 0x7f2308101790 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.802+0000 7f230ca56700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 0x7f22f406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.802+0000 7f230ca56700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 0x7f22f406d440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2300007be0 tx=0x7f23000073d0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:10.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:10.806+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f22f8056bb0 con 0x7f2308101790 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.074+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f230804ea90 con 0x7f2308101790 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.074+0000 7f2305ffb700 1 -- 172.21.15.137:0/1577568742 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f22f805a1d0 con 0x7f2308101790 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:16:11.064 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:16:11.065 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:16:11.065 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:16:11.065 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:16:11.065 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:16:11.065 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 msgr2=0x7f22f406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 0x7f22f406d440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2300007be0 tx=0x7f23000073d0 comp rx=0 tx=0).stop 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 msgr2=0x7f2308196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22f8006130 tx=0x7f22f8007b30 comp rx=0 tx=0).stop 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 shutdown_connections 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f22f406af80 0x7f22f406d440 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2308100de0 0x7f23081966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 --2- 172.21.15.137:0/1577568742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2308101790 0x7f2308196c10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 >> 172.21.15.137:0/1577568742 conn(0x7f23080fc670 msgr2=0x7f23080fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 shutdown_connections 2024-06-15T01:16:11.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:11.077+0000 7f230ecba700 1 -- 172.21.15.137:0/1577568742 wait complete. 2024-06-15T01:16:11.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:10 smithi137 ceph-mon[112295]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:11.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:10 smithi137 ceph-mon[112295]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:11.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:10 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4097980783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:16:11.219 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:10 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4050378288' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:16:11.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:10 smithi142 ceph-mon[82136]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:11.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:10 smithi142 ceph-mon[82136]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:11.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:10 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4097980783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:16:11.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:10 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4050378288' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:16:12.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:11 smithi137 ceph-mon[112295]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:12.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:11 smithi137 ceph-mon[112295]: pgmap v303: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:12.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:11 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1577568742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:16:12.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:11 smithi142 ceph-mon[82136]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:12.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:11 smithi142 ceph-mon[82136]: pgmap v303: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:12.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:11 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1577568742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:16:14.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:13 smithi137 ceph-mon[112295]: pgmap v304: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:14.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:13 smithi142 ceph-mon[82136]: pgmap v304: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:16.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:15 smithi137 ceph-mon[112295]: pgmap v305: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:16.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:15 smithi142 ceph-mon[82136]: pgmap v305: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:18.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:17 smithi137 ceph-mon[112295]: pgmap v306: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:18.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:17 smithi142 ceph-mon[82136]: pgmap v306: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:19.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:19.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:20.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:19 smithi137 ceph-mon[112295]: pgmap v307: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:20.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:19 smithi142 ceph-mon[82136]: pgmap v307: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:22.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:21 smithi137 ceph-mon[112295]: pgmap v308: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:22.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:21 smithi142 ceph-mon[82136]: pgmap v308: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:24.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:23 smithi137 ceph-mon[112295]: pgmap v309: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:24.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:23 smithi142 ceph-mon[82136]: pgmap v309: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:26.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:25 smithi142 ceph-mon[82136]: pgmap v310: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:26.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:25 smithi137 ceph-mon[112295]: pgmap v310: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:28.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:27 smithi142 ceph-mon[82136]: pgmap v311: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:28.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:27 smithi137 ceph-mon[112295]: pgmap v311: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:30.257 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:29 smithi137 ceph-mon[112295]: pgmap v312: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:30.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:29 smithi142 ceph-mon[82136]: pgmap v312: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:32.302 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:31 smithi137 ceph-mon[112295]: pgmap v313: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:32.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:31 smithi142 ceph-mon[82136]: pgmap v313: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:34.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:33 smithi137 ceph-mon[112295]: pgmap v314: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:34.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:34.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:33 smithi142 ceph-mon[82136]: pgmap v314: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:34.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:36.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:36 smithi142 ceph-mon[82136]: pgmap v315: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:36.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:36 smithi137 ceph-mon[112295]: pgmap v315: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:38.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:38 smithi142 ceph-mon[82136]: pgmap v316: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:38.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:38 smithi137 ceph-mon[112295]: pgmap v316: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:40.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:40 smithi142 ceph-mon[82136]: pgmap v317: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:40.347 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:40 smithi137 ceph-mon[112295]: pgmap v317: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:41.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 -- 172.21.15.137:0/1976965060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 msgr2=0x7f50b41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 --2- 172.21.15.137:0/1976965060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b41057c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009910 tx=0x7f50a4009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 -- 172.21.15.137:0/1976965060 shutdown_connections 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 --2- 172.21.15.137:0/1976965060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b41057c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 --2- 172.21.15.137:0/1976965060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 0x7f50b41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 -- 172.21.15.137:0/1976965060 >> 172.21.15.137:0/1976965060 conn(0x7f50b40fc650 msgr2=0x7f50b40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 -- 172.21.15.137:0/1976965060 shutdown_connections 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.196+0000 7f50bb859700 1 -- 172.21.15.137:0/1976965060 wait complete. 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 Processor -- start 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 -- start start 2024-06-15T01:16:41.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 0x7f50b41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41972a0 con 0x7f50b4101770 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50bb859700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4199fd0 con 0x7f50b4100dc0 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50b8df4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50b8df4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52748/0 (socket says 172.21.15.137:52748) 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50b8df4700 1 -- 172.21.15.137:0/1279881492 learned_addr learned my addr 172.21.15.137:0/1279881492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.197+0000 7f50b95f5700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 0x7f50b41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50b8df4700 1 -- 172.21.15.137:0/1279881492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 msgr2=0x7f50b41966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50b8df4700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 0x7f50b41966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50b8df4700 1 -- 172.21.15.137:0/1279881492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40095f0 con 0x7f50b4101770 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50b8df4700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f50a40092e0 tx=0x7f50a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a401d070 con 0x7f50b4101770 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50a4005dc0 con 0x7f50b4101770 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b419a250 con 0x7f50b4101770 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a400fea0 con 0x7f50b4101770 2024-06-15T01:16:41.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.198+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b419a740 con 0x7f50b4101770 2024-06-15T01:16:41.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.199+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b41091c0 con 0x7f50b4101770 2024-06-15T01:16:41.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.200+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f50a4003a60 con 0x7f50b4101770 2024-06-15T01:16:41.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.200+0000 7f50aa7fc700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 0x7f50a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.200+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f50a408b490 con 0x7f50b4101770 2024-06-15T01:16:41.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.200+0000 7f50b95f5700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 0x7f50a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.201+0000 7f50b95f5700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 0x7f50a006d440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009550 tx=0x7f50b00091f0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.205+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f50a4056340 con 0x7f50b4101770 2024-06-15T01:16:41.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.416+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50b419aa20 con 0x7f50a006af80 2024-06-15T01:16:41.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.418+0000 7f50aa7fc700 1 -- 172.21.15.137:0/1279881492 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f50b419aa20 con 0x7f50a006af80 2024-06-15T01:16:41.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 msgr2=0x7f50a006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 0x7f50a006d440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009550 tx=0x7f50b00091f0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 msgr2=0x7f50b4196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f50a40092e0 tx=0x7f50a400bd70 comp rx=0 tx=0).stop 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 shutdown_connections 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50a006af80 0x7f50a006d440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100dc0 0x7f50b41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 --2- 172.21.15.137:0/1279881492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50b4101770 0x7f50b4196c10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 >> 172.21.15.137:0/1279881492 conn(0x7f50b40fc650 msgr2=0x7f50b40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 shutdown_connections 2024-06-15T01:16:41.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.420+0000 7f50bb859700 1 -- 172.21.15.137:0/1279881492 wait complete. 2024-06-15T01:16:41.419 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:16:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 -- 172.21.15.137:0/1450918037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 msgr2=0x7f5a5c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/1450918037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c100a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00b0a0 tx=0x7f5a4c00b3b0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 -- 172.21.15.137:0/1450918037 shutdown_connections 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/1450918037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/1450918037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c100a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 -- 172.21.15.137:0/1450918037 >> 172.21.15.137:0/1450918037 conn(0x7f5a5c074d80 msgr2=0x7f5a5c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 -- 172.21.15.137:0/1450918037 shutdown_connections 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 -- 172.21.15.137:0/1450918037 wait complete. 2024-06-15T01:16:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.526+0000 7f5a61bc4700 1 Processor -- start 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a61bc4700 1 -- start start 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a61bc4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a61bc4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a61bc4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1971f0 con 0x7f5a5c100620 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a61bc4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c197360 con 0x7f5a5c100fd0 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52774/0 (socket says 172.21.15.137:52774) 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 -- 172.21.15.137:0/2727125824 learned_addr learned my addr 172.21.15.137:0/2727125824 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5affd700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 -- 172.21.15.137:0/2727125824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 msgr2=0x7f5a5c196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 -- 172.21.15.137:0/2727125824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c009d00 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5affd700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.527+0000 7f5a5b7fe700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c015040 tx=0x7f5a4c007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.528+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c00e040 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.528+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c19a0a0 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.528+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a4c0044e0 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.528+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c01d9e0 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.528+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c19a590 con 0x7f5a5c100620 2024-06-15T01:16:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.529+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c109060 con 0x7f5a5c100620 2024-06-15T01:16:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.529+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5a4c004650 con 0x7f5a5c100620 2024-06-15T01:16:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.529+0000 7f5a58ff9700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 0x7f5a4406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.530+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5a4c08bea0 con 0x7f5a5c100620 2024-06-15T01:16:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.530+0000 7f5a5affd700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 0x7f5a4406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.530+0000 7f5a5affd700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 0x7f5a4406d440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005fd0 tx=0x7f5a50005e30 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.534+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5a4c056cd0 con 0x7f5a5c100620 2024-06-15T01:16:41.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.742+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a5c02ccc0 con 0x7f5a4406af80 2024-06-15T01:16:41.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.743+0000 7f5a58ff9700 1 -- 172.21.15.137:0/2727125824 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a5c02ccc0 con 0x7f5a4406af80 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 msgr2=0x7f5a4406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 0x7f5a4406d440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a50005fd0 tx=0x7f5a50005e30 comp rx=0 tx=0).stop 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 msgr2=0x7f5a5c1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c015040 tx=0x7f5a4c007be0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 shutdown_connections 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5a4406af80 0x7f5a4406d440 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a5c100620 0x7f5a5c1966b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 --2- 172.21.15.137:0/2727125824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c100fd0 0x7f5a5c196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 >> 172.21.15.137:0/2727125824 conn(0x7f5a5c074d80 msgr2=0x7f5a5c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:41.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 shutdown_connections 2024-06-15T01:16:41.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.745+0000 7f5a61bc4700 1 -- 172.21.15.137:0/2727125824 wait complete. 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.848+0000 7f6f84557700 1 -- 172.21.15.137:0/1061503261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c0ffa60 msgr2=0x7f6f7c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.848+0000 7f6f84557700 1 --2- 172.21.15.137:0/1061503261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c0ffee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009910 tx=0x7f6f6c009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- 172.21.15.137:0/1061503261 shutdown_connections 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 --2- 172.21.15.137:0/1061503261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c0ffee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 --2- 172.21.15.137:0/1061503261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c105a60 0x7f6f7c105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- 172.21.15.137:0/1061503261 >> 172.21.15.137:0/1061503261 conn(0x7f6f7c0fb550 msgr2=0x7f6f7c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:41.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- 172.21.15.137:0/1061503261 shutdown_connections 2024-06-15T01:16:41.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- 172.21.15.137:0/1061503261 wait complete. 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 Processor -- start 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- start start 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c197260 con 0x7f6f7c105a60 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.849+0000 7f6f84557700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c199f90 con 0x7f6f7c0ffa60 2024-06-15T01:16:41.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52798/0 (socket says 172.21.15.137:52798) 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 -- 172.21.15.137:0/488176518 learned_addr learned my addr 172.21.15.137:0/488176518 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f822f3700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 -- 172.21.15.137:0/488176518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c0ffa60 msgr2=0x7f6f7c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 -- 172.21.15.137:0/488176518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c0095f0 con 0x7f6f7c105a60 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f81af2700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006010 tx=0x7f6f6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c01d070 con 0x7f6f7c105a60 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f6c00f460 con 0x7f6f7c105a60 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c021620 con 0x7f6f7c105a60 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c19a210 con 0x7f6f7c105a60 2024-06-15T01:16:41.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.850+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c19a6b0 con 0x7f6f7c105a60 2024-06-15T01:16:41.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.851+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c109000 con 0x7f6f7c105a60 2024-06-15T01:16:41.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.852+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6f6c00f5d0 con 0x7f6f7c105a60 2024-06-15T01:16:41.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.853+0000 7f6f737fe700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 0x7f6f6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:41.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.853+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6f6c08b950 con 0x7f6f7c105a60 2024-06-15T01:16:41.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.853+0000 7f6f822f3700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 0x7f6f6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:41.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.854+0000 7f6f822f3700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 0x7f6f6806d440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005950 tx=0x7f6f7800a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:41.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:41.857+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6f6c056780 con 0x7f6f7c105a60 2024-06-15T01:16:42.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.059+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f7c19a990 con 0x7f6f6806af80 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.068+0000 7f6f737fe700 1 -- 172.21.15.137:0/488176518 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6f7c19a990 con 0x7f6f6806af80 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (7m) 102s ago 17m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (18m) 102s ago 18m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (16m) 74s ago 16m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (5m) 102s ago 18m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (5m) 74s ago 16m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (7m) 102s ago 17m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:16:42.058 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (107s) 102s ago 12m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (2m) 102s ago 12m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (91s) 74s ago 12m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (77s) 74s ago 12m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (9m) 102s ago 19m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (8m) 74s ago 16m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (5m) 102s ago 19m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (5m) 74s ago 16m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (7m) 102s ago 17m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 74s ago 16m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (5m) 102s ago 15m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:16:42.059 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (4m) 102s ago 15m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:16:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (3m) 102s ago 15m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:16:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (3m) 74s ago 14m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:16:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (2m) 74s ago 14m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:16:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (2m) 74s ago 13m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:16:42.060 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (7m) 102s ago 17m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 msgr2=0x7f6f6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 0x7f6f6806d440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005950 tx=0x7f6f7800a300 comp rx=0 tx=0).stop 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 msgr2=0x7f6f7c196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006010 tx=0x7f6f6c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 shutdown_connections 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6f6806af80 0x7f6f6806d440 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f7c0ffa60 0x7f6f7c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 --2- 172.21.15.137:0/488176518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f7c105a60 0x7f6f7c196bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 >> 172.21.15.137:0/488176518 conn(0x7f6f7c0fb550 msgr2=0x7f6f7c069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 shutdown_connections 2024-06-15T01:16:42.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.071+0000 7f6f84557700 1 -- 172.21.15.137:0/488176518 wait complete. 2024-06-15T01:16:42.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 -- 172.21.15.137:0/3894574646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ff530 msgr2=0x7fab280ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 --2- 172.21.15.137:0/3894574646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ff530 0x7fab280ff910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab18009910 tx=0x7fab18009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:42.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 -- 172.21.15.137:0/3894574646 shutdown_connections 2024-06-15T01:16:42.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 --2- 172.21.15.137:0/3894574646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ffee0 0x7fab28105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 --2- 172.21.15.137:0/3894574646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ff530 0x7fab280ff910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 -- 172.21.15.137:0/3894574646 >> 172.21.15.137:0/3894574646 conn(0x7fab28074df0 msgr2=0x7fab28075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.164 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:42 smithi137 ceph-mon[112295]: pgmap v318: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:42.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 -- 172.21.15.137:0/3894574646 shutdown_connections 2024-06-15T01:16:42.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 -- 172.21.15.137:0/3894574646 wait complete. 2024-06-15T01:16:42.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.174+0000 7fab2d283700 1 Processor -- start 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab2d283700 1 -- start start 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab2d283700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ff530 0x7fab281004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab2d283700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab2d283700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28100f30 con 0x7fab280ffee0 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab2d283700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28101070 con 0x7fab280ff530 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52818/0 (socket says 172.21.15.137:52818) 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 -- 172.21.15.137:0/340579952 learned_addr learned my addr 172.21.15.137:0/340579952 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab26ffd700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ff530 0x7fab281004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 -- 172.21.15.137:0/340579952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ff530 msgr2=0x7fab281004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ff530 0x7fab281004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.175+0000 7fab1e7fc700 1 -- 172.21.15.137:0/340579952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180095f0 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab1e7fc700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e8f0 tx=0x7fab1000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab10009790 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab1000c810 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab281a2920 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab10005520 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.176+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab281a2c80 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.177+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fab1000c980 con 0x7fab280ffee0 2024-06-15T01:16:42.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.177+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab281091e0 con 0x7fab280ffee0 2024-06-15T01:16:42.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.178+0000 7fab24ff9700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 0x7fab1406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.178+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fab10014070 con 0x7fab280ffee0 2024-06-15T01:16:42.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.178+0000 7fab26ffd700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 0x7fab1406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.178+0000 7fab26ffd700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 0x7fab1406d440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fd0 tx=0x7fab18005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.182+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab10055490 con 0x7fab280ffee0 2024-06-15T01:16:42.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:42 smithi142 ceph-mon[82136]: pgmap v318: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.455+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab28101230 con 0x7fab280ffee0 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.455+0000 7fab24ff9700 1 -- 172.21.15.137:0/340579952 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fab1000cc30 con 0x7fab280ffee0 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:16:42.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:16:42.446 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:16:42.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 msgr2=0x7fab1406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 0x7fab1406d440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fd0 tx=0x7fab18005940 comp rx=0 tx=0).stop 2024-06-15T01:16:42.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 msgr2=0x7fab281009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e8f0 tx=0x7fab1000ec00 comp rx=0 tx=0).stop 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 shutdown_connections 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fab1406af80 0x7fab1406d440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab280ff530 0x7fab281004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 --2- 172.21.15.137:0/340579952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab280ffee0 0x7fab281009f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 >> 172.21.15.137:0/340579952 conn(0x7fab28074df0 msgr2=0x7fab280fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 shutdown_connections 2024-06-15T01:16:42.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.458+0000 7fab2d283700 1 -- 172.21.15.137:0/340579952 wait complete. 2024-06-15T01:16:42.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 -- 172.21.15.137:0/896414018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c100dc0 msgr2=0x7f7b2c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 --2- 172.21.15.137:0/896414018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c1011a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009910 tx=0x7f7b28009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 -- 172.21.15.137:0/896414018 shutdown_connections 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 --2- 172.21.15.137:0/896414018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c101770 0x7f7b2c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 --2- 172.21.15.137:0/896414018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c1011a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 -- 172.21.15.137:0/896414018 >> 172.21.15.137:0/896414018 conn(0x7f7b2c0fc650 msgr2=0x7f7b2c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 -- 172.21.15.137:0/896414018 shutdown_connections 2024-06-15T01:16:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.562+0000 7f7b3307e700 1 -- 172.21.15.137:0/896414018 wait complete. 2024-06-15T01:16:42.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 Processor -- start 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 -- start start 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c197290 con 0x7f7b2c101770 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b3307e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19b020 con 0x7f7b2c100dc0 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52826/0 (socket says 172.21.15.137:52826) 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 -- 172.21.15.137:0/2474969318 learned_addr learned my addr 172.21.15.137:0/2474969318 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b30e1a700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 -- 172.21.15.137:0/2474969318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c100dc0 msgr2=0x7f7b2c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 -- 172.21.15.137:0/2474969318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280095f0 con 0x7f7b2c101770 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.563+0000 7f7b23fff700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c8f0 tx=0x7f7b1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.564+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1c007a50 con 0x7f7b2c101770 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.564+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c19b300 con 0x7f7b2c101770 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.564+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b1c007bb0 con 0x7f7b2c101770 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.564+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1c018630 con 0x7f7b2c101770 2024-06-15T01:16:42.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.564+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c19b850 con 0x7f7b2c101770 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.565+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c109160 con 0x7f7b2c101770 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.565+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7b1c007d20 con 0x7f7b2c101770 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.566+0000 7f7b21ffb700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 0x7f7b1406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.566+0000 7f7b30e1a700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 0x7f7b1406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.566+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7b1c08a370 con 0x7f7b2c101770 2024-06-15T01:16:42.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.566+0000 7f7b30e1a700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 0x7f7b1406d440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b5c0 tx=0x7f7b28005fb0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.570+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7b1c0551a0 con 0x7f7b2c101770 2024-06-15T01:16:42.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.803+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7b2c04ea90 con 0x7f7b2c101770 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.803+0000 7f7b21ffb700 1 -- 172.21.15.137:0/2474969318 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f7b1c0587c0 con 0x7f7b2c101770 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:16:42.793 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:16:42.794 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:16:42.795 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:42.796 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 msgr2=0x7f7b1406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 0x7f7b1406d440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b5c0 tx=0x7f7b28005fb0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 msgr2=0x7f7b2c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00c8f0 tx=0x7f7b1c00ccb0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 shutdown_connections 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b1406af80 0x7f7b1406d440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b2c100dc0 0x7f7b2c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 --2- 172.21.15.137:0/2474969318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b2c101770 0x7f7b2c196bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 >> 172.21.15.137:0/2474969318 conn(0x7f7b2c0fc650 msgr2=0x7f7b2c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 shutdown_connections 2024-06-15T01:16:42.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.806+0000 7f7b3307e700 1 -- 172.21.15.137:0/2474969318 wait complete. 2024-06-15T01:16:42.798 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:16:42.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 -- 172.21.15.137:0/1039122238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4068670 msgr2=0x7f15f4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 --2- 172.21.15.137:0/1039122238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4068670 0x7f15f4068a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009910 tx=0x7f15e4009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:42.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 -- 172.21.15.137:0/1039122238 shutdown_connections 2024-06-15T01:16:42.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 --2- 172.21.15.137:0/1039122238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4069020 0x7f15f4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 --2- 172.21.15.137:0/1039122238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4068670 0x7f15f4068a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 -- 172.21.15.137:0/1039122238 >> 172.21.15.137:0/1039122238 conn(0x7f15f40755a0 msgr2=0x7f15f40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:42.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 -- 172.21.15.137:0/1039122238 shutdown_connections 2024-06-15T01:16:42.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.910+0000 7f15f8eed700 1 -- 172.21.15.137:0/1039122238 wait complete. 2024-06-15T01:16:42.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 Processor -- start 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 -- start start 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 0x7f15f40ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4100720 con 0x7f15f4069020 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f8eed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4100860 con 0x7f15f4068670 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52848/0 (socket says 172.21.15.137:52848) 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 -- 172.21.15.137:0/4054364912 learned_addr learned my addr 172.21.15.137:0/4054364912 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f259c700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 0x7f15f40ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 -- 172.21.15.137:0/4054364912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 msgr2=0x7f15f40ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 0x7f15f40ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15e9d9b700 1 -- 172.21.15.137:0/4054364912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40095f0 con 0x7f15f4069020 2024-06-15T01:16:42.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.911+0000 7f15f259c700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 0x7f15f40ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15e9d9b700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d900 tx=0x7f15dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc009900 con 0x7f15f4069020 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f41a2840 con 0x7f15f4069020 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15dc010460 con 0x7f15f4069020 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc00f5f0 con 0x7f15f4069020 2024-06-15T01:16:42.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.912+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f41a2c50 con 0x7f15f4069020 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.913+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f4109130 con 0x7f15f4069020 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.914+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f15dc0105d0 con 0x7f15f4069020 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.914+0000 7f15ebfff700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 0x7f15e006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.914+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f15dc08a1c0 con 0x7f15f4069020 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.914+0000 7f15f259c700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 0x7f15e006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:42.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.914+0000 7f15f259c700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 0x7f15e006d440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e400b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:42.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:42.918+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f15dc054ff0 con 0x7f15f4069020 2024-06-15T01:16:43.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.133+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15f402cda0 con 0x7f15e006af80 2024-06-15T01:16:43.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:43 smithi137 ceph-mon[112295]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:43 smithi137 ceph-mon[112295]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:43 smithi137 ceph-mon[112295]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:43 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/340579952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:16:43.123 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:43 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2474969318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:16:43.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.134+0000 7f15ebfff700 1 -- 172.21.15.137:0/4054364912 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f15f402cda0 con 0x7f15e006af80 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:16:43.124 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:16:43.125 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:16:43.125 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:16:43.125 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.136+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 msgr2=0x7f15e006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.136+0000 7f15f8eed700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 0x7f15e006d440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e400b540 comp rx=0 tx=0).stop 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.136+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 msgr2=0x7f15f41001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.136+0000 7f15f8eed700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d900 tx=0x7f15dc00dcc0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 shutdown_connections 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f15e006af80 0x7f15e006d440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4068670 0x7f15f40ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 --2- 172.21.15.137:0/4054364912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15f4069020 0x7f15f41001e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 >> 172.21.15.137:0/4054364912 conn(0x7f15f40755a0 msgr2=0x7f15f40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:43.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 shutdown_connections 2024-06-15T01:16:43.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.137+0000 7f15f8eed700 1 -- 172.21.15.137:0/4054364912 wait complete. 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 -- 172.21.15.137:0/3971626108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0100de0 msgr2=0x7f8da01011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 --2- 172.21.15.137:0/3971626108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0100de0 0x7f8da01011c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009910 tx=0x7f8d90009c20 comp rx=0 tx=0).stop 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 -- 172.21.15.137:0/3971626108 shutdown_connections 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 --2- 172.21.15.137:0/3971626108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0101790 0x7f8da01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 --2- 172.21.15.137:0/3971626108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0100de0 0x7f8da01011c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 -- 172.21.15.137:0/3971626108 >> 172.21.15.137:0/3971626108 conn(0x7f8da00fc670 msgr2=0x7f8da00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 -- 172.21.15.137:0/3971626108 shutdown_connections 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.241+0000 7f8da58bb700 1 -- 172.21.15.137:0/3971626108 wait complete. 2024-06-15T01:16:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 Processor -- start 2024-06-15T01:16:43.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 -- start start 2024-06-15T01:16:43.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 0x7f8da0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:43.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:43.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0197200 con 0x7f8da0101790 2024-06-15T01:16:43.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8da58bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0197340 con 0x7f8da0100de0 2024-06-15T01:16:43.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52870/0 (socket says 172.21.15.137:52870) 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d967fc700 1 -- 172.21.15.137:0/4093709362 learned_addr learned my addr 172.21.15.137:0/4093709362 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d9effd700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 0x7f8da0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d967fc700 1 -- 172.21.15.137:0/4093709362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 msgr2=0x7f8da0196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.242+0000 7f8d967fc700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 0x7f8da0196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8d967fc700 1 -- 172.21.15.137:0/4093709362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900095f0 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8d9effd700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 0x7f8da0196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8d967fc700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c960 tx=0x7f8d8800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d88007a70 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d88007bd0 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da019b190 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d88018630 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.243+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da019b6e0 con 0x7f8da0101790 2024-06-15T01:16:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.244+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da0109180 con 0x7f8da0101790 2024-06-15T01:16:43.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.244+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8d8800f9f0 con 0x7f8da0101790 2024-06-15T01:16:43.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.245+0000 7f8da48b9700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 0x7f8d8c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:16:43.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.245+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8d8808a330 con 0x7f8da0101790 2024-06-15T01:16:43.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.245+0000 7f8d9effd700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 0x7f8d8c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:16:43.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.245+0000 7f8d9effd700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 0x7f8d8c06d440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000b5c0 tx=0x7f8d90005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:16:43.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.249+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8d88058c70 con 0x7f8da0101790 2024-06-15T01:16:43.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:43 smithi142 ceph-mon[82136]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:43 smithi142 ceph-mon[82136]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:43 smithi142 ceph-mon[82136]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:43.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:43 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/340579952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:16:43.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:43 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2474969318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.523+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8da004ea90 con 0x7f8da0101790 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.523+0000 7f8da48b9700 1 -- 172.21.15.137:0/4093709362 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f8d88058800 con 0x7f8da0101790 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:16:43.513 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:16:43.514 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:16:43.514 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 msgr2=0x7f8d8c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 0x7f8d8c06d440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000b5c0 tx=0x7f8d90005960 comp rx=0 tx=0).stop 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 msgr2=0x7f8da0196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c960 tx=0x7f8d8800cd20 comp rx=0 tx=0).stop 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 shutdown_connections 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8d8c06af80 0x7f8d8c06d440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da0100de0 0x7f8da0196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 --2- 172.21.15.137:0/4093709362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da0101790 0x7f8da0196bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:16:43.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 >> 172.21.15.137:0/4093709362 conn(0x7f8da00fc670 msgr2=0x7f8da00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:16:43.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 shutdown_connections 2024-06-15T01:16:43.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:16:43.526+0000 7f8da58bb700 1 -- 172.21.15.137:0/4093709362 wait complete. 2024-06-15T01:16:44.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:44 smithi137 ceph-mon[112295]: pgmap v319: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:44.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:44 smithi137 ceph-mon[112295]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:44.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:44 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4093709362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:16:44.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:44 smithi142 ceph-mon[82136]: pgmap v319: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:44.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:44 smithi142 ceph-mon[82136]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:16:44.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:44 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4093709362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:16:46.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:46 smithi142 ceph-mon[82136]: pgmap v320: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:46.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:46 smithi137 ceph-mon[112295]: pgmap v320: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:48.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:48 smithi142 ceph-mon[82136]: pgmap v321: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:48.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:48.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:48 smithi137 ceph-mon[112295]: pgmap v321: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:48.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:16:50.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:50 smithi142 ceph-mon[82136]: pgmap v322: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:50.439 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:50 smithi137 ceph-mon[112295]: pgmap v322: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:52.302 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:52 smithi137 ceph-mon[112295]: pgmap v323: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:52.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:52 smithi142 ceph-mon[82136]: pgmap v323: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:54.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:54 smithi142 ceph-mon[82136]: pgmap v324: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:54.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:54 smithi137 ceph-mon[112295]: pgmap v324: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:56.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:56 smithi142 ceph-mon[82136]: pgmap v325: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:56.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:56 smithi137 ceph-mon[112295]: pgmap v325: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:58.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:16:58 smithi142 ceph-mon[82136]: pgmap v326: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:16:58.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:16:58 smithi137 ceph-mon[112295]: pgmap v326: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:00.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:00 smithi142 ceph-mon[82136]: pgmap v327: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:00.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:00 smithi137 ceph-mon[112295]: pgmap v327: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:02.302 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:02 smithi137 ceph-mon[112295]: pgmap v328: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:02.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:02 smithi142 ceph-mon[82136]: pgmap v328: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:03.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:03.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:04.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:04 smithi142 ceph-mon[82136]: pgmap v329: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:04.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:04 smithi137 ceph-mon[112295]: pgmap v329: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:06.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:06 smithi142 ceph-mon[82136]: pgmap v330: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:06.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:06 smithi137 ceph-mon[112295]: pgmap v330: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:08.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:08 smithi142 ceph-mon[82136]: pgmap v331: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:08.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:08 smithi137 ceph-mon[112295]: pgmap v331: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:10.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:10 smithi142 ceph-mon[82136]: pgmap v332: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:10.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:10 smithi137 ceph-mon[112295]: pgmap v332: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:12.430 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:12 smithi142 ceph-mon[82136]: pgmap v333: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:12.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:12 smithi137 ceph-mon[112295]: pgmap v333: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:13.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 -- 172.21.15.137:0/484793389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4100da0 msgr2=0x7f8aa4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/484793389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4100da0 0x7f8aa4101180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009910 tx=0x7f8a98009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:13.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 -- 172.21.15.137:0/484793389 shutdown_connections 2024-06-15T01:17:13.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/484793389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4101750 0x7f8aa41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/484793389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4100da0 0x7f8aa4101180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 -- 172.21.15.137:0/484793389 >> 172.21.15.137:0/484793389 conn(0x7f8aa40fc650 msgr2=0x7f8aa40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:13.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 -- 172.21.15.137:0/484793389 shutdown_connections 2024-06-15T01:17:13.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.640+0000 7f8aaaca0700 1 -- 172.21.15.137:0/484793389 wait complete. 2024-06-15T01:17:13.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 Processor -- start 2024-06-15T01:17:13.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 -- start start 2024-06-15T01:17:13.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 0x7f8aa419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa419d0d0 con 0x7f8aa4101750 2024-06-15T01:17:13.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aaaca0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4196580 con 0x7f8aa4100da0 2024-06-15T01:17:13.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60516/0 (socket says 172.21.15.137:60516) 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 -- 172.21.15.137:0/2920008997 learned_addr learned my addr 172.21.15.137:0/2920008997 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa8a3c700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 0x7f8aa419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 -- 172.21.15.137:0/2920008997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 msgr2=0x7f8aa419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 0x7f8aa419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa3fff700 1 -- 172.21.15.137:0/2920008997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a980095f0 con 0x7f8aa4101750 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.641+0000 7f8aa8a3c700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 0x7f8aa419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aa3fff700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c8f0 tx=0x7f8a9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a90007a50 con 0x7f8aa4101750 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a90007bb0 con 0x7f8aa4101750 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4196860 con 0x7f8aa4101750 2024-06-15T01:17:13.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a90018630 con 0x7f8aa4101750 2024-06-15T01:17:13.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.642+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4196db0 con 0x7f8aa4101750 2024-06-15T01:17:13.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.643+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa4109140 con 0x7f8aa4101750 2024-06-15T01:17:13.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.643+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8a90007d20 con 0x7f8aa4101750 2024-06-15T01:17:13.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.644+0000 7f8aa1ffb700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 0x7f8a9406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.644+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8a9008a370 con 0x7f8aa4101750 2024-06-15T01:17:13.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.644+0000 7f8aa8a3c700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 0x7f8a9406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.644+0000 7f8aa8a3c700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 0x7f8a9406d440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b5c0 tx=0x7f8a98005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:13.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.648+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8a90058c30 con 0x7f8aa4101750 2024-06-15T01:17:13.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.861+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8aa4197030 con 0x7f8a9406af80 2024-06-15T01:17:13.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.863+0000 7f8aa1ffb700 1 -- 172.21.15.137:0/2920008997 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8aa4197030 con 0x7f8a9406af80 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 msgr2=0x7f8a9406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 0x7f8a9406d440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b5c0 tx=0x7f8a98005960 comp rx=0 tx=0).stop 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 msgr2=0x7f8aa419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c8f0 tx=0x7f8a9000ccb0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 shutdown_connections 2024-06-15T01:17:13.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8a9406af80 0x7f8a9406d440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa4100da0 0x7f8aa419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 --2- 172.21.15.137:0/2920008997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aa4101750 0x7f8aa419ca40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 >> 172.21.15.137:0/2920008997 conn(0x7f8aa40fc650 msgr2=0x7f8aa40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:13.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 shutdown_connections 2024-06-15T01:17:13.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.865+0000 7f8aaaca0700 1 -- 172.21.15.137:0/2920008997 wait complete. 2024-06-15T01:17:13.865 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 -- 172.21.15.137:0/2519751637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c100dc0 msgr2=0x7f5c1c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 --2- 172.21.15.137:0/2519751637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c1011a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009910 tx=0x7f5c04009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 -- 172.21.15.137:0/2519751637 shutdown_connections 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 --2- 172.21.15.137:0/2519751637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c101770 0x7f5c1c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 --2- 172.21.15.137:0/2519751637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c1011a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 -- 172.21.15.137:0/2519751637 >> 172.21.15.137:0/2519751637 conn(0x7f5c1c0fc650 msgr2=0x7f5c1c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 -- 172.21.15.137:0/2519751637 shutdown_connections 2024-06-15T01:17:13.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.970+0000 7f5c22739700 1 -- 172.21.15.137:0/2519751637 wait complete. 2024-06-15T01:17:13.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 Processor -- start 2024-06-15T01:17:13.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 -- start start 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c197290 con 0x7f5c1c101770 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c22739700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19b020 con 0x7f5c1c100dc0 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c1b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c1b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60532/0 (socket says 172.21.15.137:60532) 2024-06-15T01:17:13.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.971+0000 7f5c1b7fe700 1 -- 172.21.15.137:0/2561448868 learned_addr learned my addr 172.21.15.137:0/2561448868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c1bfff700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c1b7fe700 1 -- 172.21.15.137:0/2561448868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c100dc0 msgr2=0x7f5c1c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c1b7fe700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c1b7fe700 1 -- 172.21.15.137:0/2561448868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040095f0 con 0x7f5c1c101770 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c1b7fe700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d8d0 tx=0x7f5c0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c0041f0 con 0x7f5c1c101770 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c19b300 con 0x7f5c1c101770 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c0c007620 con 0x7f5c1c101770 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c005610 con 0x7f5c1c101770 2024-06-15T01:17:13.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.972+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c19b850 con 0x7f5c1c101770 2024-06-15T01:17:13.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.973+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c109160 con 0x7f5c1c101770 2024-06-15T01:17:13.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.974+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5c0c00f690 con 0x7f5c1c101770 2024-06-15T01:17:13.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.974+0000 7f5c197fa700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 0x7f5c0806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:13.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.974+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5c0c08a200 con 0x7f5c1c101770 2024-06-15T01:17:13.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.974+0000 7f5c1bfff700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 0x7f5c0806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:13.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.975+0000 7f5c1bfff700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 0x7f5c0806d440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:13.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:13.978+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c0c055030 con 0x7f5c1c101770 2024-06-15T01:17:14.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.187+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c1c19bb60 con 0x7f5c0806af80 2024-06-15T01:17:14.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.189+0000 7f5c197fa700 1 -- 172.21.15.137:0/2561448868 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c1c19bb60 con 0x7f5c0806af80 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 msgr2=0x7f5c0806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 0x7f5c0806d440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0401a040 comp rx=0 tx=0).stop 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 msgr2=0x7f5c1c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d8d0 tx=0x7f5c0c00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 shutdown_connections 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c0806af80 0x7f5c0806d440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c1c100dc0 0x7f5c1c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 --2- 172.21.15.137:0/2561448868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c1c101770 0x7f5c1c196bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 >> 172.21.15.137:0/2561448868 conn(0x7f5c1c0fc650 msgr2=0x7f5c1c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 shutdown_connections 2024-06-15T01:17:14.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.191+0000 7f5c22739700 1 -- 172.21.15.137:0/2561448868 wait complete. 2024-06-15T01:17:14.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:14 smithi137 ceph-mon[112295]: pgmap v334: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:14.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- 172.21.15.137:0/4120851691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc100dc0 msgr2=0x7f43dc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 --2- 172.21.15.137:0/4120851691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc100dc0 0x7f43dc1011a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009910 tx=0x7f43d8009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- 172.21.15.137:0/4120851691 shutdown_connections 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 --2- 172.21.15.137:0/4120851691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc101770 0x7f43dc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 --2- 172.21.15.137:0/4120851691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc100dc0 0x7f43dc1011a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- 172.21.15.137:0/4120851691 >> 172.21.15.137:0/4120851691 conn(0x7f43dc0fc650 msgr2=0x7f43dc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- 172.21.15.137:0/4120851691 shutdown_connections 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- 172.21.15.137:0/4120851691 wait complete. 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 Processor -- start 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.295+0000 7f43e398f700 1 -- start start 2024-06-15T01:17:14.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e398f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc100dc0 0x7f43dc196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e398f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e398f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc197290 con 0x7f43dc101770 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e398f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19b020 con 0x7f43dc100dc0 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60550/0 (socket says 172.21.15.137:60550) 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 -- 172.21.15.137:0/921671090 learned_addr learned my addr 172.21.15.137:0/921671090 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e172b700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc100dc0 0x7f43dc196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 -- 172.21.15.137:0/921671090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc100dc0 msgr2=0x7f43dc196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc100dc0 0x7f43dc196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 -- 172.21.15.137:0/921671090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d80095f0 con 0x7f43dc101770 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e0f2a700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43d000d900 tx=0x7f43d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d0009900 con 0x7f43dc101770 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43d0010460 con 0x7f43dc101770 2024-06-15T01:17:14.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d000f5f0 con 0x7f43dc101770 2024-06-15T01:17:14.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.296+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc19b300 con 0x7f43dc101770 2024-06-15T01:17:14.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.297+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc19b850 con 0x7f43dc101770 2024-06-15T01:17:14.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.298+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc109160 con 0x7f43dc101770 2024-06-15T01:17:14.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.298+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f43d00105d0 con 0x7f43dc101770 2024-06-15T01:17:14.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.298+0000 7f43ce7fc700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 0x7f43c806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.299+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f43d008a180 con 0x7f43dc101770 2024-06-15T01:17:14.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.299+0000 7f43e172b700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 0x7f43c806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.299+0000 7f43e172b700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 0x7f43c806d440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43d8006010 tx=0x7f43d801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:14.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.303+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f43d0054fb0 con 0x7f43dc101770 2024-06-15T01:17:14.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.503+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43dc19bb60 con 0x7f43c806af80 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.513+0000 7f43ce7fc700 1 -- 172.21.15.137:0/921671090 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f43dc19bb60 con 0x7f43c806af80 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (7m) 2m ago 18m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (18m) 2m ago 18m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (17m) 107s ago 17m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (6m) 2m ago 18m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (5m) 107s ago 17m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (7m) 2m ago 18m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:17:14.502 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (2m) 2m ago 12m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (2m) 2m ago 12m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (2m) 107s ago 12m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (110s) 107s ago 12m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (9m) 2m ago 19m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (8m) 107s ago 17m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (6m) 2m ago 20m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (6m) 107s ago 17m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (8m) 2m ago 18m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 107s ago 17m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (5m) 2m ago 16m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (5m) 2m ago 16m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (4m) 2m ago 15m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:17:14.503 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (3m) 107s ago 15m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:17:14.504 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (3m) 107s ago 14m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:17:14.504 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (2m) 107s ago 14m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:17:14.504 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (8m) 2m ago 17m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:17:14.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 msgr2=0x7f43c806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 0x7f43c806d440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43d8006010 tx=0x7f43d801a040 comp rx=0 tx=0).stop 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 msgr2=0x7f43dc196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43d000d900 tx=0x7f43d000dcc0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 shutdown_connections 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f43c806af80 0x7f43c806d440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43dc100dc0 0x7f43dc196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 --2- 172.21.15.137:0/921671090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43dc101770 0x7f43dc196bb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 >> 172.21.15.137:0/921671090 conn(0x7f43dc0fc650 msgr2=0x7f43dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 shutdown_connections 2024-06-15T01:17:14.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.515+0000 7f43e398f700 1 -- 172.21.15.137:0/921671090 wait complete. 2024-06-15T01:17:14.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:14 smithi142 ceph-mon[82136]: pgmap v334: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.618+0000 7f7e43d43700 1 -- 172.21.15.137:0/1619176292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c100dc0 msgr2=0x7f7e3c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.618+0000 7f7e43d43700 1 --2- 172.21.15.137:0/1619176292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c1011a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009910 tx=0x7f7e38009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 -- 172.21.15.137:0/1619176292 shutdown_connections 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 --2- 172.21.15.137:0/1619176292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c101770 0x7f7e3c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 --2- 172.21.15.137:0/1619176292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c1011a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 -- 172.21.15.137:0/1619176292 >> 172.21.15.137:0/1619176292 conn(0x7f7e3c0fc650 msgr2=0x7f7e3c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 -- 172.21.15.137:0/1619176292 shutdown_connections 2024-06-15T01:17:14.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 -- 172.21.15.137:0/1619176292 wait complete. 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.619+0000 7f7e43d43700 1 Processor -- start 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e43d43700 1 -- start start 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e43d43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e43d43700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e43d43700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c197290 con 0x7f7e3c101770 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e43d43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c19b020 con 0x7f7e3c100dc0 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e41adf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60570/0 (socket says 172.21.15.137:60570) 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 -- 172.21.15.137:0/2735007842 learned_addr learned my addr 172.21.15.137:0/2735007842 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 -- 172.21.15.137:0/2735007842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c100dc0 msgr2=0x7f7e3c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.620+0000 7f7e412de700 1 -- 172.21.15.137:0/2735007842 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380095f0 con 0x7f7e3c101770 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e412de700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00d900 tx=0x7f7e2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2c009900 con 0x7f7e3c101770 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e2c010460 con 0x7f7e3c101770 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c19b300 con 0x7f7e3c101770 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2c00f5f0 con 0x7f7e3c101770 2024-06-15T01:17:14.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.621+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c19b850 con 0x7f7e3c101770 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.622+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c109160 con 0x7f7e3c101770 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.622+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7e2c0105d0 con 0x7f7e3c101770 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.623+0000 7f7e32ffd700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 0x7f7e2806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.623+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7e2c089e60 con 0x7f7e3c101770 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.623+0000 7f7e41adf700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 0x7f7e2806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.623+0000 7f7e41adf700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 0x7f7e2806d440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e3801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:14.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.627+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7e2c054c70 con 0x7f7e3c101770 2024-06-15T01:17:14.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.896+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e3c04ea90 con 0x7f7e3c101770 2024-06-15T01:17:14.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.897+0000 7f7e32ffd700 1 -- 172.21.15.137:0/2735007842 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7e2c020290 con 0x7f7e3c101770 2024-06-15T01:17:14.886 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:17:14.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:17:14.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:17:14.888 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 msgr2=0x7f7e2806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 0x7f7e2806d440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e3801a040 comp rx=0 tx=0).stop 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 msgr2=0x7f7e3c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00d900 tx=0x7f7e2c00dc10 comp rx=0 tx=0).stop 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 shutdown_connections 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7e2806af80 0x7f7e2806d440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c100dc0 0x7f7e3c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 --2- 172.21.15.137:0/2735007842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e3c101770 0x7f7e3c196bb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 >> 172.21.15.137:0/2735007842 conn(0x7f7e3c0fc650 msgr2=0x7f7e3c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 shutdown_connections 2024-06-15T01:17:14.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:14.900+0000 7f7e43d43700 1 -- 172.21.15.137:0/2735007842 wait complete. 2024-06-15T01:17:14.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 -- 172.21.15.137:0/2277765960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c101790 msgr2=0x7f840c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 --2- 172.21.15.137:0/2277765960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c101790 0x7f840c1057e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009910 tx=0x7f83f4009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:14.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 -- 172.21.15.137:0/2277765960 shutdown_connections 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 --2- 172.21.15.137:0/2277765960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c101790 0x7f840c1057e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 --2- 172.21.15.137:0/2277765960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c100de0 0x7f840c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 -- 172.21.15.137:0/2277765960 >> 172.21.15.137:0/2277765960 conn(0x7f840c0fc6b0 msgr2=0x7f840c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.004+0000 7f8411e47700 1 -- 172.21.15.137:0/2277765960 shutdown_connections 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 -- 172.21.15.137:0/2277765960 wait complete. 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 Processor -- start 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 -- start start 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c101790 0x7f840c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c197290 con 0x7f840c100de0 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f8411e47700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19b020 con 0x7f840c101790 2024-06-15T01:17:14.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60588/0 (socket says 172.21.15.137:60588) 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.005+0000 7f840b7fe700 1 -- 172.21.15.137:0/3916716398 learned_addr learned my addr 172.21.15.137:0/3916716398 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f8403fff700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c101790 0x7f840c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f840b7fe700 1 -- 172.21.15.137:0/3916716398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c101790 msgr2=0x7f840c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f840b7fe700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c101790 0x7f840c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f840b7fe700 1 -- 172.21.15.137:0/3916716398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40095f0 con 0x7f840c100de0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f840b7fe700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d8d0 tx=0x7f83fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc0098a0 con 0x7f840c100de0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f83fc010460 con 0x7f840c100de0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc009ea0 con 0x7f840c100de0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c19b300 con 0x7f840c100de0 2024-06-15T01:17:14.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.006+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c19b820 con 0x7f840c100de0 2024-06-15T01:17:14.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.007+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c1091e0 con 0x7f840c100de0 2024-06-15T01:17:14.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.008+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f83fc0105d0 con 0x7f840c100de0 2024-06-15T01:17:14.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.008+0000 7f84097fa700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 0x7f83f806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:14.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.008+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f83fc08a250 con 0x7f840c100de0 2024-06-15T01:17:14.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.008+0000 7f8403fff700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 0x7f83f806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:14.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.008+0000 7f8403fff700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 0x7f83f806d440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:15.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.012+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f83fc055080 con 0x7f840c100de0 2024-06-15T01:17:15.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:15 smithi137 ceph-mon[112295]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:15.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:15 smithi137 ceph-mon[112295]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:15.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:15 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2735007842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:17:15.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.246+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f840c19ba70 con 0x7f840c100de0 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.246+0000 7f84097fa700 1 -- 172.21.15.137:0/3916716398 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f83fc010880 con 0x7f840c100de0 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:17:15.235 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:17:15.236 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:17:15.237 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:15.238 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:15.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 msgr2=0x7f83f806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 0x7f83f806d440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f401a040 comp rx=0 tx=0).stop 2024-06-15T01:17:15.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 msgr2=0x7f840c196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d8d0 tx=0x7f83fc00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 shutdown_connections 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f83f806af80 0x7f83f806d440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100de0 0x7f840c196670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 --2- 172.21.15.137:0/3916716398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f840c101790 0x7f840c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 >> 172.21.15.137:0/3916716398 conn(0x7f840c0fc6b0 msgr2=0x7f840c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 shutdown_connections 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.248+0000 7f8411e47700 1 -- 172.21.15.137:0/3916716398 wait complete. 2024-06-15T01:17:15.240 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:17:15.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 -- 172.21.15.137:0/350315751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 msgr2=0x7f18a8105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/350315751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8105e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1898009910 tx=0x7f1898009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 -- 172.21.15.137:0/350315751 shutdown_connections 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/350315751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 0x7f18a80fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/350315751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8105e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 -- 172.21.15.137:0/350315751 >> 172.21.15.137:0/350315751 conn(0x7f18a80fb590 msgr2=0x7f18a80fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 -- 172.21.15.137:0/350315751 shutdown_connections 2024-06-15T01:17:15.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.352+0000 7f18ae7b6700 1 -- 172.21.15.137:0/350315751 wait complete. 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 Processor -- start 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 -- start start 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 0x7f18a8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8197260 con 0x7f18a8105a80 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.353+0000 7f18ae7b6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a819aff0 con 0x7f18a80ffa80 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60612/0 (socket says 172.21.15.137:60612) 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 -- 172.21.15.137:0/4201047690 learned_addr learned my addr 172.21.15.137:0/4201047690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:15.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a7fff700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 0x7f18a8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 -- 172.21.15.137:0/4201047690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 msgr2=0x7f18a8196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 0x7f18a8196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 -- 172.21.15.137:0/4201047690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980095f0 con 0x7f18a8105a80 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a77fe700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c8a0 tx=0x7f189c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c0078e0 con 0x7f18a8105a80 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a819b2d0 con 0x7f18a8105a80 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f189c00f450 con 0x7f18a8105a80 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189c00e5e0 con 0x7f18a8105a80 2024-06-15T01:17:15.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.354+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a819b820 con 0x7f18a8105a80 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.355+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8109090 con 0x7f18a8105a80 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.356+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f189c00f5c0 con 0x7f18a8105a80 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.356+0000 7f18a57fa700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 0x7f189006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.356+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f189c089fa0 con 0x7f18a8105a80 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.356+0000 7f18a7fff700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 0x7f189006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.357+0000 7f18a7fff700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 0x7f189006d440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1898006010 tx=0x7f189801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:15.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.361+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f189c054dd0 con 0x7f18a8105a80 2024-06-15T01:17:15.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:15 smithi142 ceph-mon[82136]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:15.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:15 smithi142 ceph-mon[82136]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:15.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:15 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2735007842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:17:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.571+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a819bb90 con 0x7f189006af80 2024-06-15T01:17:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.573+0000 7f18a57fa700 1 -- 172.21.15.137:0/4201047690 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f18a819bb90 con 0x7f189006af80 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:17:15.562 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:17:15.563 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:17:15.563 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:17:15.563 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:17:15.563 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 msgr2=0x7f189006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 0x7f189006d440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1898006010 tx=0x7f189801a040 comp rx=0 tx=0).stop 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 msgr2=0x7f18a8196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c8a0 tx=0x7f189c00cbb0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 shutdown_connections 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f189006af80 0x7f189006d440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80ffa80 0x7f18a8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 --2- 172.21.15.137:0/4201047690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18a8105a80 0x7f18a8196b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 >> 172.21.15.137:0/4201047690 conn(0x7f18a80fb590 msgr2=0x7f18a8068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:15.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 shutdown_connections 2024-06-15T01:17:15.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.576+0000 7f18ae7b6700 1 -- 172.21.15.137:0/4201047690 wait complete. 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3513089449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c100dc0 msgr2=0x7fe39c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3513089449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c100dc0 0x7fe39c1011a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe390009910 tx=0x7fe390009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3513089449 shutdown_connections 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3513089449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c101770 0x7fe39c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3513089449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c100dc0 0x7fe39c1011a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3513089449 >> 172.21.15.137:0/3513089449 conn(0x7fe39c0fc650 msgr2=0x7fe39c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3513089449 shutdown_connections 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.681+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3513089449 wait complete. 2024-06-15T01:17:15.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 Processor -- start 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 -- start start 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c100dc0 0x7fe39c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c073b10 con 0x7fe39c101770 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a3aeb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c073c50 con 0x7fe39c100dc0 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a1086700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a1086700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60638/0 (socket says 172.21.15.137:60638) 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.682+0000 7fe3a1086700 1 -- 172.21.15.137:0/3698781807 learned_addr learned my addr 172.21.15.137:0/3698781807 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a1887700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c100dc0 0x7fe39c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a1086700 1 -- 172.21.15.137:0/3698781807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c100dc0 msgr2=0x7fe39c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a1086700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c100dc0 0x7fe39c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a1086700 1 -- 172.21.15.137:0/3698781807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900095f0 con 0x7fe39c101770 2024-06-15T01:17:15.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a1086700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe39800b480 tx=0x7fe39800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe398011800 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe398011e00 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c1a2970 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe398010e30 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.683+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c1a2d80 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.684+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c109160 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.685+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe398011960 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.685+0000 7fe38effd700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 0x7fe38806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.685+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe398089e60 con 0x7fe39c101770 2024-06-15T01:17:15.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.685+0000 7fe3a1887700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 0x7fe38806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:15.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.686+0000 7fe3a1887700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 0x7fe38806d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe39001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:15.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.690+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe398054c90 con 0x7fe39c101770 2024-06-15T01:17:15.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.959+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe39c02cc70 con 0x7fe39c101770 2024-06-15T01:17:15.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.959+0000 7fe38effd700 1 -- 172.21.15.137:0/3698781807 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fe3980582b0 con 0x7fe39c101770 2024-06-15T01:17:15.948 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:17:15.948 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:17:15.949 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 msgr2=0x7fe38806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 0x7fe38806d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe39001a040 comp rx=0 tx=0).stop 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 msgr2=0x7fe39c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe39800b480 tx=0x7fe39800b790 comp rx=0 tx=0).stop 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 shutdown_connections 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe38806af80 0x7fe38806d440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe39c100dc0 0x7fe39c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 --2- 172.21.15.137:0/3698781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe39c101770 0x7fe39c0735d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 >> 172.21.15.137:0/3698781807 conn(0x7fe39c0fc650 msgr2=0x7fe39c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 shutdown_connections 2024-06-15T01:17:15.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:15.962+0000 7fe3a3aeb700 1 -- 172.21.15.137:0/3698781807 wait complete. 2024-06-15T01:17:16.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:16 smithi137 ceph-mon[112295]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:16.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:16 smithi137 ceph-mon[112295]: pgmap v335: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:16.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:16 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3916716398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:17:16.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:16 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3698781807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:17:16.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:16 smithi142 ceph-mon[82136]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:16.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:16 smithi142 ceph-mon[82136]: pgmap v335: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:16.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:16 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3916716398' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:17:16.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:16 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3698781807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:17:17.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:17 smithi137 ceph-mon[112295]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:17.475 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:17 smithi142 ceph-mon[82136]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:18.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:18 smithi137 ceph-mon[112295]: pgmap v336: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:18.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:18.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:18 smithi142 ceph-mon[82136]: pgmap v336: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:18.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:20.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:20 smithi137 ceph-mon[112295]: pgmap v337: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:20.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:20 smithi142 ceph-mon[82136]: pgmap v337: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:22.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:22 smithi137 ceph-mon[112295]: pgmap v338: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:22.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:22 smithi142 ceph-mon[82136]: pgmap v338: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:24.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:24 smithi137 ceph-mon[112295]: pgmap v339: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:24.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:24 smithi142 ceph-mon[82136]: pgmap v339: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:26.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:26 smithi137 ceph-mon[112295]: pgmap v340: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:26.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:26 smithi142 ceph-mon[82136]: pgmap v340: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:28.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:28 smithi137 ceph-mon[112295]: pgmap v341: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:28.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:28 smithi142 ceph-mon[82136]: pgmap v341: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:30.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:30 smithi137 ceph-mon[112295]: pgmap v342: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:30.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:30 smithi142 ceph-mon[82136]: pgmap v342: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:32.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:32 smithi137 ceph-mon[112295]: pgmap v343: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:32.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:32 smithi142 ceph-mon[82136]: pgmap v343: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:33.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:33.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:34.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:34 smithi137 ceph-mon[112295]: pgmap v344: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:34.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:34 smithi142 ceph-mon[82136]: pgmap v344: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:36.362 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:36 smithi142 ceph-mon[82136]: pgmap v345: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:36.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:36 smithi137 ceph-mon[112295]: pgmap v345: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:38.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:38 smithi137 ceph-mon[112295]: pgmap v346: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:38.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:38 smithi142 ceph-mon[82136]: pgmap v346: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:40.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:40 smithi137 ceph-mon[112295]: pgmap v347: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:40.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:40 smithi142 ceph-mon[82136]: pgmap v347: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:42.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:42 smithi137 ceph-mon[112295]: pgmap v348: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:42.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:42 smithi142 ceph-mon[82136]: pgmap v348: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:44.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:44 smithi137 ceph-mon[112295]: pgmap v349: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:44.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:44 smithi142 ceph-mon[82136]: pgmap v349: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 -- 172.21.15.137:0/3399931535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318100dd0 msgr2=0x7f63181011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 --2- 172.21.15.137:0/3399931535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318100dd0 0x7f63181011b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f630c009910 tx=0x7f630c009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 -- 172.21.15.137:0/3399931535 shutdown_connections 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 --2- 172.21.15.137:0/3399931535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318101780 0x7f63181057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 --2- 172.21.15.137:0/3399931535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318100dd0 0x7f63181011b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.079+0000 7f631fbbd700 1 -- 172.21.15.137:0/3399931535 >> 172.21.15.137:0/3399931535 conn(0x7f63180fc640 msgr2=0x7f63180fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 -- 172.21.15.137:0/3399931535 shutdown_connections 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 -- 172.21.15.137:0/3399931535 wait complete. 2024-06-15T01:17:46.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 Processor -- start 2024-06-15T01:17:46.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 -- start start 2024-06-15T01:17:46.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318100dd0 0x7f6318198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318199540 con 0x7f6318101780 2024-06-15T01:17:46.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.080+0000 7f631fbbd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631819d350 con 0x7f6318100dd0 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34942/0 (socket says 172.21.15.137:34942) 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 -- 172.21.15.137:0/1507776027 learned_addr learned my addr 172.21.15.137:0/1507776027 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d959700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318100dd0 0x7f6318198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 -- 172.21.15.137:0/1507776027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318100dd0 msgr2=0x7f6318198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318100dd0 0x7f6318198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 -- 172.21.15.137:0/1507776027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0095f0 con 0x7f6318101780 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631d158700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f631400b480 tx=0x7f631400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6314011800 con 0x7f6318101780 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6314011e00 con 0x7f6318101780 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631819d630 con 0x7f6318101780 2024-06-15T01:17:46.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6314010e30 con 0x7f6318101780 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.081+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631819db80 con 0x7f6318101780 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.082+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318104f50 con 0x7f6318101780 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.083+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6314011960 con 0x7f6318101780 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.083+0000 7f630affd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 0x7f630406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.083+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6314089e80 con 0x7f6318101780 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.083+0000 7f631d959700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 0x7f630406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.084+0000 7f631d959700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 0x7f630406d440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f630c005f50 tx=0x7f630c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.088+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6314058720 con 0x7f6318101780 2024-06-15T01:17:46.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:46 smithi137 ceph-mon[112295]: pgmap v350: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:46.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.302+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f631819de50 con 0x7f630406af80 2024-06-15T01:17:46.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.303+0000 7f630affd700 1 -- 172.21.15.137:0/1507776027 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f631819de50 con 0x7f630406af80 2024-06-15T01:17:46.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 msgr2=0x7f630406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 0x7f630406d440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f630c005f50 tx=0x7f630c01a040 comp rx=0 tx=0).stop 2024-06-15T01:17:46.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 msgr2=0x7f6318198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f631400b480 tx=0x7f631400b790 comp rx=0 tx=0).stop 2024-06-15T01:17:46.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 shutdown_connections 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f630406af80 0x7f630406d440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6318100dd0 0x7f6318198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 --2- 172.21.15.137:0/1507776027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6318101780 0x7f6318198eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 >> 172.21.15.137:0/1507776027 conn(0x7f63180fc640 msgr2=0x7f63180fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 shutdown_connections 2024-06-15T01:17:46.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.305+0000 7f631fbbd700 1 -- 172.21.15.137:0/1507776027 wait complete. 2024-06-15T01:17:46.305 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 -- 172.21.15.137:0/2002692387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 msgr2=0x7fc3f0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2002692387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0105e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009910 tx=0x7fc3e0009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 -- 172.21.15.137:0/2002692387 shutdown_connections 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2002692387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 0x7fc3f0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2002692387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0105e20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 -- 172.21.15.137:0/2002692387 >> 172.21.15.137:0/2002692387 conn(0x7fc3f00755a0 msgr2=0x7fc3f00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 -- 172.21.15.137:0/2002692387 shutdown_connections 2024-06-15T01:17:46.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.410+0000 7fc3f7985700 1 -- 172.21.15.137:0/2002692387 wait complete. 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 Processor -- start 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 -- start start 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 0x7fc3f0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0197290 con 0x7fc3f0105a40 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f7985700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f019b020 con 0x7fc3f00ffbd0 2024-06-15T01:17:46.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34958/0 (socket says 172.21.15.137:34958) 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 -- 172.21.15.137:0/2249355802 learned_addr learned my addr 172.21.15.137:0/2249355802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f5721700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 0x7fc3f0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 -- 172.21.15.137:0/2249355802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 msgr2=0x7fc3f0196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 0x7fc3f0196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.411+0000 7fc3f4f20700 1 -- 172.21.15.137:0/2249355802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00095f0 con 0x7fc3f0105a40 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3f4f20700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d8d0 tx=0x7fc3ec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ec0098a0 con 0x7fc3f0105a40 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3ec010460 con 0x7fc3f0105a40 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ec009ea0 con 0x7fc3f0105a40 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f019b300 con 0x7fc3f0105a40 2024-06-15T01:17:46.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.412+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f019b850 con 0x7fc3f0105a40 2024-06-15T01:17:46.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.413+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f004ea90 con 0x7fc3f0105a40 2024-06-15T01:17:46.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.413+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc3ec0105d0 con 0x7fc3f0105a40 2024-06-15T01:17:46.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.414+0000 7fc3e67fc700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 0x7fc3dc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.414+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc3ec08a250 con 0x7fc3f0105a40 2024-06-15T01:17:46.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.414+0000 7fc3f5721700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 0x7fc3dc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.414+0000 7fc3f5721700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 0x7fc3dc06d440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000b5c0 tx=0x7fc3e001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.418+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc3ec055080 con 0x7fc3f0105a40 2024-06-15T01:17:46.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:46 smithi142 ceph-mon[82136]: pgmap v350: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:46.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.627+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3f019bc60 con 0x7fc3dc06af80 2024-06-15T01:17:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.629+0000 7fc3e67fc700 1 -- 172.21.15.137:0/2249355802 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc3f019bc60 con 0x7fc3dc06af80 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 msgr2=0x7fc3dc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 0x7fc3dc06d440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000b5c0 tx=0x7fc3e001a040 comp rx=0 tx=0).stop 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 msgr2=0x7fc3f0196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d8d0 tx=0x7fc3ec00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 shutdown_connections 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc3dc06af80 0x7fc3dc06d440 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f00ffbd0 0x7fc3f0196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 --2- 172.21.15.137:0/2249355802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f0105a40 0x7fc3f0196bb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 >> 172.21.15.137:0/2249355802 conn(0x7fc3f00755a0 msgr2=0x7fc3f01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 shutdown_connections 2024-06-15T01:17:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.631+0000 7fc3f7985700 1 -- 172.21.15.137:0/2249355802 wait complete. 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- 172.21.15.137:0/1910866274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 msgr2=0x7f6058068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 --2- 172.21.15.137:0/1910866274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058068a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6048009910 tx=0x7f6048009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- 172.21.15.137:0/1910866274 shutdown_connections 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 --2- 172.21.15.137:0/1910866274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 0x7f6058105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 --2- 172.21.15.137:0/1910866274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058068a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- 172.21.15.137:0/1910866274 >> 172.21.15.137:0/1910866274 conn(0x7f60580755a0 msgr2=0x7f60580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- 172.21.15.137:0/1910866274 shutdown_connections 2024-06-15T01:17:46.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- 172.21.15.137:0/1910866274 wait complete. 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 Processor -- start 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.735+0000 7f605f59d700 1 -- start start 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 0x7f6058196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058197290 con 0x7f6058068670 2024-06-15T01:17:46.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605819b020 con 0x7f6058069020 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34980/0 (socket says 172.21.15.137:34980) 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 -- 172.21.15.137:0/2274009385 learned_addr learned my addr 172.21.15.137:0/2274009385 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605cb38700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 0x7f6058196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 -- 172.21.15.137:0/2274009385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 msgr2=0x7f6058196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 0x7f6058196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 -- 172.21.15.137:0/2274009385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480095f0 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605d339700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6048000c00 tx=0x7f604800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604801d070 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6048005dc0 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605819b2a0 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60480039b0 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.736+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605819b730 con 0x7f6058068670 2024-06-15T01:17:46.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.737+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6058109130 con 0x7f6058068670 2024-06-15T01:17:46.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.738+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f60480048e0 con 0x7f6058068670 2024-06-15T01:17:46.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.738+0000 7f604e7fc700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 0x7f604406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:46.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.738+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f604808b980 con 0x7f6058068670 2024-06-15T01:17:46.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.738+0000 7f605cb38700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 0x7f604406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:46.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.739+0000 7f605cb38700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 0x7f604406d440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6058197c90 tx=0x7f6054009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:46.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.743+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f60480567b0 con 0x7f6058068670 2024-06-15T01:17:46.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.941+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f605819bad0 con 0x7f604406af80 2024-06-15T01:17:46.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.951+0000 7f604e7fc700 1 -- 172.21.15.137:0/2274009385 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f605819bad0 con 0x7f604406af80 2024-06-15T01:17:46.940 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:17:46.940 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (8m) 2m ago 18m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:17:46.940 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (19m) 2m ago 19m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:17:46.940 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (18m) 2m ago 18m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (6m) 2m ago 19m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (6m) 2m ago 18m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (8m) 2m ago 18m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (2m) 2m ago 13m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (3m) 2m ago 13m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (2m) 2m ago 13m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (2m) 2m ago 13m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (10m) 2m ago 20m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (9m) 2m ago 17m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (7m) 2m ago 20m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (6m) 2m ago 17m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (8m) 2m ago 19m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:17:46.941 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 2m ago 17m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (6m) 2m ago 17m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (5m) 2m ago 16m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (5m) 2m ago 16m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (4m) 2m ago 15m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (3m) 2m ago 15m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (3m) 2m ago 14m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:17:46.942 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (8m) 2m ago 18m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 msgr2=0x7f604406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 0x7f604406d440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6058197c90 tx=0x7f6054009260 comp rx=0 tx=0).stop 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 msgr2=0x7f6058196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6048000c00 tx=0x7f604800be90 comp rx=0 tx=0).stop 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 shutdown_connections 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f604406af80 0x7f604406d440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6058068670 0x7f6058196670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 --2- 172.21.15.137:0/2274009385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058069020 0x7f6058196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 >> 172.21.15.137:0/2274009385 conn(0x7f60580755a0 msgr2=0x7f60580fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 shutdown_connections 2024-06-15T01:17:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:46.954+0000 7f605f59d700 1 -- 172.21.15.137:0/2274009385 wait complete. 2024-06-15T01:17:47.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 -- 172.21.15.137:0/2300708588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 msgr2=0x7fd458105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 --2- 172.21.15.137:0/2300708588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458105790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd448009910 tx=0x7fd448009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:47.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 -- 172.21.15.137:0/2300708588 shutdown_connections 2024-06-15T01:17:47.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 --2- 172.21.15.137:0/2300708588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458105790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 --2- 172.21.15.137:0/2300708588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 0x7fd458068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 -- 172.21.15.137:0/2300708588 >> 172.21.15.137:0/2300708588 conn(0x7fd4580755a0 msgr2=0x7fd4580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:47.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 -- 172.21.15.137:0/2300708588 shutdown_connections 2024-06-15T01:17:47.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.057+0000 7fd45cec1700 1 -- 172.21.15.137:0/2300708588 wait complete. 2024-06-15T01:17:47.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 Processor -- start 2024-06-15T01:17:47.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 -- start start 2024-06-15T01:17:47.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 0x7fd458196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458197260 con 0x7fd458069020 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45cec1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819aff0 con 0x7fd458068670 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd455d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd455d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35002/0 (socket says 172.21.15.137:35002) 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd455d9b700 1 -- 172.21.15.137:0/3534337158 learned_addr learned my addr 172.21.15.137:0/3534337158 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd45659c700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 0x7fd458196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd455d9b700 1 -- 172.21.15.137:0/3534337158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 msgr2=0x7fd458196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.058+0000 7fd455d9b700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 0x7fd458196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd455d9b700 1 -- 172.21.15.137:0/3534337158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480095f0 con 0x7fd458069020 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd455d9b700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4480092e0 tx=0x7fd44800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd44801d070 con 0x7fd458069020 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd448005dc0 con 0x7fd458069020 2024-06-15T01:17:47.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819b270 con 0x7fd458069020 2024-06-15T01:17:47.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd44800fea0 con 0x7fd458069020 2024-06-15T01:17:47.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.059+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45819b760 con 0x7fd458069020 2024-06-15T01:17:47.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.060+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4581090a0 con 0x7fd458069020 2024-06-15T01:17:47.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.060+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd44802b440 con 0x7fd458069020 2024-06-15T01:17:47.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.061+0000 7fd4477fe700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 0x7fd44006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.061+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd44808b5b0 con 0x7fd458069020 2024-06-15T01:17:47.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.061+0000 7fd45659c700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 0x7fd44006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.061+0000 7fd45659c700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 0x7fd44006d440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd44c007be0 tx=0x7fd44c0073d0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.065+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd448056340 con 0x7fd458069020 2024-06-15T01:17:47.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.341+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd45804ea90 con 0x7fd458069020 2024-06-15T01:17:47.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.341+0000 7fd4477fe700 1 -- 172.21.15.137:0/3534337158 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd448059960 con 0x7fd458069020 2024-06-15T01:17:47.330 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:17:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:17:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:17:47.331 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:17:47.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 msgr2=0x7fd44006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 0x7fd44006d440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd44c007be0 tx=0x7fd44c0073d0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 msgr2=0x7fd458196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4480092e0 tx=0x7fd44800bd70 comp rx=0 tx=0).stop 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 shutdown_connections 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd44006af80 0x7fd44006d440 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458068670 0x7fd458196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 --2- 172.21.15.137:0/3534337158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd458069020 0x7fd458196b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.343+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 >> 172.21.15.137:0/3534337158 conn(0x7fd4580755a0 msgr2=0x7fd458105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.344+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 shutdown_connections 2024-06-15T01:17:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.344+0000 7fd45cec1700 1 -- 172.21.15.137:0/3534337158 wait complete. 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 -- 172.21.15.137:0/2497524902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 msgr2=0x7ffac0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 --2- 172.21.15.137:0/2497524902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac0100050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009910 tx=0x7ffab4009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 -- 172.21.15.137:0/2497524902 shutdown_connections 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 --2- 172.21.15.137:0/2497524902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac0100050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 --2- 172.21.15.137:0/2497524902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 -- 172.21.15.137:0/2497524902 >> 172.21.15.137:0/2497524902 conn(0x7ffac00755a0 msgr2=0x7ffac00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 -- 172.21.15.137:0/2497524902 shutdown_connections 2024-06-15T01:17:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 -- 172.21.15.137:0/2497524902 wait complete. 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.448+0000 7ffac544f700 1 Processor -- start 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 -- start start 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac019f1e0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0198690 con 0x7ffac0105a40 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35006/0 (socket says 172.21.15.137:35006) 2024-06-15T01:17:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 -- 172.21.15.137:0/1476082068 learned_addr learned my addr 172.21.15.137:0/1476082068 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabe7fc700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 -- 172.21.15.137:0/1476082068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 msgr2=0x7ffac019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 -- 172.21.15.137:0/1476082068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40095f0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabe7fc700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffabeffd700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d900 tx=0x7ffab000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab00041f0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffab0004cd0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac0198970 con 0x7ffac00ffbd0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.449+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab00056d0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.450+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac0198ec0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.450+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac004ea90 con 0x7ffac00ffbd0 2024-06-15T01:17:47.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.451+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ffab0004e40 con 0x7ffac00ffbd0 2024-06-15T01:17:47.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.451+0000 7ffaa7fff700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 0x7ffaa806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.452+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ffab008a200 con 0x7ffac00ffbd0 2024-06-15T01:17:47.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.452+0000 7ffabe7fc700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 0x7ffaa806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.452+0000 7ffabe7fc700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 0x7ffaa806d440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b5c0 tx=0x7ffab4005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.456+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffab0058b40 con 0x7ffac00ffbd0 2024-06-15T01:17:47.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.687+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffac01992d0 con 0x7ffac00ffbd0 2024-06-15T01:17:47.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.687+0000 7ffaa7fff700 1 -- 172.21.15.137:0/1476082068 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7ffab0016750 con 0x7ffac00ffbd0 2024-06-15T01:17:47.676 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:17:47.677 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:17:47.678 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:47.679 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:17:47.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.689+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 msgr2=0x7ffaa806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.689+0000 7ffac544f700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 0x7ffaa806d440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b5c0 tx=0x7ffab4005fd0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 msgr2=0x7ffac019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d900 tx=0x7ffab000dcc0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 shutdown_connections 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffaa806af80 0x7ffaa806d440 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac00ffbd0 0x7ffac019e610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 --2- 172.21.15.137:0/1476082068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffac0105a40 0x7ffac019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 >> 172.21.15.137:0/1476082068 conn(0x7ffac00755a0 msgr2=0x7ffac00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 shutdown_connections 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.690+0000 7ffac544f700 1 -- 172.21.15.137:0/1476082068 wait complete. 2024-06-15T01:17:47.681 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:17:47.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 -- 172.21.15.137:0/1011553891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 msgr2=0x7f60cc0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 --2- 172.21.15.137:0/1011553891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc0ffef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009910 tx=0x7f60b4009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:47.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 -- 172.21.15.137:0/1011553891 shutdown_connections 2024-06-15T01:17:47.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 --2- 172.21.15.137:0/1011553891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc0ffef0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 --2- 172.21.15.137:0/1011553891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 0x7f60cc105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 -- 172.21.15.137:0/1011553891 >> 172.21.15.137:0/1011553891 conn(0x7f60cc0fb560 msgr2=0x7f60cc0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:47.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 -- 172.21.15.137:0/1011553891 shutdown_connections 2024-06-15T01:17:47.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.795+0000 7f60d2374700 1 -- 172.21.15.137:0/1011553891 wait complete. 2024-06-15T01:17:47.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 Processor -- start 2024-06-15T01:17:47.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 -- start start 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 0x7f60cc196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc197260 con 0x7f60cc0ffa70 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60d2374700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc19aff0 con 0x7f60cc105a70 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60cbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60cbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35022/0 (socket says 172.21.15.137:35022) 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.796+0000 7f60cbfff700 1 -- 172.21.15.137:0/2562082402 learned_addr learned my addr 172.21.15.137:0/2562082402 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60cb7fe700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 0x7f60cc196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60cbfff700 1 -- 172.21.15.137:0/2562082402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 msgr2=0x7f60cc196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60cbfff700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 0x7f60cc196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60cbfff700 1 -- 172.21.15.137:0/2562082402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40095f0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60cbfff700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00d8d0 tx=0x7f60bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc0098a0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60bc010460 con 0x7f60cc0ffa70 2024-06-15T01:17:47.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc009ea0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc19b2d0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.797+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc19b7f0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.798+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc109010 con 0x7f60cc0ffa70 2024-06-15T01:17:47.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.799+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f60bc0105d0 con 0x7f60cc0ffa70 2024-06-15T01:17:47.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.799+0000 7f60c97fa700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 0x7f60b806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:47.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.799+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f60bc08a250 con 0x7f60cc0ffa70 2024-06-15T01:17:47.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.799+0000 7f60cb7fe700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 0x7f60b806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:47.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.800+0000 7f60cb7fe700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 0x7f60b806d440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b5c0 tx=0x7f60b401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:47.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:47.804+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f60bc055080 con 0x7f60cc0ffa70 2024-06-15T01:17:48.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.017+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60cc19bb20 con 0x7f60b806af80 2024-06-15T01:17:48.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.018+0000 7f60c97fa700 1 -- 172.21.15.137:0/2562082402 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f60cc19bb20 con 0x7f60b806af80 2024-06-15T01:17:48.007 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:17:48.008 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:17:48.008 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:17:48.008 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:17:48.008 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:17:48.008 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:17:48.009 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:17:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:17:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:17:48.010 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:17:48.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 msgr2=0x7f60b806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 0x7f60b806d440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b5c0 tx=0x7f60b401a040 comp rx=0 tx=0).stop 2024-06-15T01:17:48.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 msgr2=0x7f60cc196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00d8d0 tx=0x7f60bc00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 shutdown_connections 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f60b806af80 0x7f60b806d440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc0ffa70 0x7f60cc196640 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 --2- 172.21.15.137:0/2562082402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60cc105a70 0x7f60cc196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.020+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 >> 172.21.15.137:0/2562082402 conn(0x7f60cc0fb560 msgr2=0x7f60cc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.021+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 shutdown_connections 2024-06-15T01:17:48.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.021+0000 7f60d2374700 1 -- 172.21.15.137:0/2562082402 wait complete. 2024-06-15T01:17:48.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.126+0000 7f97b780e700 1 -- 172.21.15.137:0/966828797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 msgr2=0x7f97b00ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.126+0000 7f97b780e700 1 --2- 172.21.15.137:0/966828797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b00ffec0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009910 tx=0x7f97a0009c20 comp rx=0 tx=0).stop 2024-06-15T01:17:48.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.126+0000 7f97b780e700 1 -- 172.21.15.137:0/966828797 shutdown_connections 2024-06-15T01:17:48.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 --2- 172.21.15.137:0/966828797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b00ffec0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 --2- 172.21.15.137:0/966828797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 0x7f97b0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- 172.21.15.137:0/966828797 >> 172.21.15.137:0/966828797 conn(0x7f97b00fb570 msgr2=0x7f97b00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- 172.21.15.137:0/966828797 shutdown_connections 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- 172.21.15.137:0/966828797 wait complete. 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 Processor -- start 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- start start 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 0x7f97b0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01972e0 con 0x7f97b00ffa40 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.127+0000 7f97b780e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b019a010 con 0x7f97b0105a40 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35038/0 (socket says 172.21.15.137:35038) 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 -- 172.21.15.137:0/233148346 learned_addr learned my addr 172.21.15.137:0/233148346 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b4da9700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 0x7f97b0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 -- 172.21.15.137:0/233148346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 msgr2=0x7f97b0196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 0x7f97b0196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 -- 172.21.15.137:0/233148346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00095f0 con 0x7f97b00ffa40 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b55aa700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00d8d0 tx=0x7f97ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ac009940 con 0x7f97b00ffa40 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97ac010460 con 0x7f97b00ffa40 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ac00f5f0 con 0x7f97b00ffa40 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b019a2f0 con 0x7f97b00ffa40 2024-06-15T01:17:48.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.128+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b019a840 con 0x7f97b00ffa40 2024-06-15T01:17:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.129+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b004ea90 con 0x7f97b00ffa40 2024-06-15T01:17:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.130+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f97ac0105d0 con 0x7f97b00ffa40 2024-06-15T01:17:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.130+0000 7f97a67fc700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 0x7f979c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:17:48.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.130+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f97ac08a370 con 0x7f97b00ffa40 2024-06-15T01:17:48.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.130+0000 7f97b4da9700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 0x7f979c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:17:48.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.130+0000 7f97b4da9700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 0x7f979c06d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97b0197d30 tx=0x7f97a001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:17:48.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.134+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f97ac055180 con 0x7f97b00ffa40 2024-06-15T01:17:48.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.402+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f97b0197a70 con 0x7f97b00ffa40 2024-06-15T01:17:48.391 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.391 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.392 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: pgmap v351: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:48.392 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.392 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3534337158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:17:48.392 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1476082068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:17:48.392 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:48.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.402+0000 7f97a67fc700 1 -- 172.21.15.137:0/233148346 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f97ac010880 con 0x7f97b00ffa40 2024-06-15T01:17:48.392 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:17:48.392 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:17:48.393 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.404+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 msgr2=0x7f979c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.404+0000 7f97b780e700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 0x7f979c06d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97b0197d30 tx=0x7f97a001a040 comp rx=0 tx=0).stop 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.404+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 msgr2=0x7f97b0196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.404+0000 7f97b780e700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00d8d0 tx=0x7f97ac00dc90 comp rx=0 tx=0).stop 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 shutdown_connections 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f979c06af80 0x7f979c06d440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b00ffa40 0x7f97b0196710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 --2- 172.21.15.137:0/233148346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97b0105a40 0x7f97b0196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 >> 172.21.15.137:0/233148346 conn(0x7f97b00fb570 msgr2=0x7f97b0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:17:48.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 shutdown_connections 2024-06-15T01:17:48.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:17:48.405+0000 7f97b780e700 1 -- 172.21.15.137:0/233148346 wait complete. 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: pgmap v351: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3534337158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1476082068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:17:48.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:17:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:49 smithi137 ceph-mon[112295]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:49 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/233148346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:17:49.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:49 smithi142 ceph-mon[82136]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:17:49.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:49 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/233148346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:17:50.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:50 smithi137 ceph-mon[112295]: pgmap v352: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:50.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:50 smithi142 ceph-mon[82136]: pgmap v352: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:52.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:52 smithi137 ceph-mon[112295]: pgmap v353: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:52.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:52 smithi142 ceph-mon[82136]: pgmap v353: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:54.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:54 smithi137 ceph-mon[112295]: pgmap v354: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:54.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:54 smithi142 ceph-mon[82136]: pgmap v354: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:56.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:56 smithi137 ceph-mon[112295]: pgmap v355: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:56.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:56 smithi142 ceph-mon[82136]: pgmap v355: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:58.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:17:58 smithi137 ceph-mon[112295]: pgmap v356: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:17:58.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:17:58 smithi142 ceph-mon[82136]: pgmap v356: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:00.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:00 smithi137 ceph-mon[112295]: pgmap v357: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:00.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:00 smithi142 ceph-mon[82136]: pgmap v357: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:02.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:02 smithi137 ceph-mon[112295]: pgmap v358: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:02.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:02 smithi142 ceph-mon[82136]: pgmap v358: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:03.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:03.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:04.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:04 smithi137 ceph-mon[112295]: pgmap v359: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:04.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:04 smithi142 ceph-mon[82136]: pgmap v359: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:06.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:06 smithi137 ceph-mon[112295]: pgmap v360: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:06.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:06 smithi142 ceph-mon[82136]: pgmap v360: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:08.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:08 smithi137 ceph-mon[112295]: pgmap v361: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:08.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:08 smithi142 ceph-mon[82136]: pgmap v361: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:10.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:10 smithi137 ceph-mon[112295]: pgmap v362: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:10.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:10 smithi142 ceph-mon[82136]: pgmap v362: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:12.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:12 smithi137 ceph-mon[112295]: pgmap v363: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:12.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:12 smithi142 ceph-mon[82136]: pgmap v363: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:14.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:14 smithi137 ceph-mon[112295]: pgmap v364: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:14.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:14 smithi142 ceph-mon[82136]: pgmap v364: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:16.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:16 smithi137 ceph-mon[112295]: pgmap v365: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:16.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:16 smithi142 ceph-mon[82136]: pgmap v365: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:18.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:18 smithi142 ceph-mon[82136]: pgmap v366: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:18.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:18.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:18 smithi137 ceph-mon[112295]: pgmap v366: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:18.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 -- 172.21.15.137:0/3986444207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30100dc0 msgr2=0x7f0b301011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3986444207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30100dc0 0x7f0b301011a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009910 tx=0x7f0b20009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 -- 172.21.15.137:0/3986444207 shutdown_connections 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3986444207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30101770 0x7f0b301057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3986444207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30100dc0 0x7f0b301011a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 -- 172.21.15.137:0/3986444207 >> 172.21.15.137:0/3986444207 conn(0x7f0b300fc650 msgr2=0x7f0b300fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 -- 172.21.15.137:0/3986444207 shutdown_connections 2024-06-15T01:18:18.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.523+0000 7f0b35efe700 1 -- 172.21.15.137:0/3986444207 wait complete. 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 Processor -- start 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 -- start start 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30100dc0 0x7f0b30073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30073b10 con 0x7f0b30101770 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b35efe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30073c50 con 0x7f0b30100dc0 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b27fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b27fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39526/0 (socket says 172.21.15.137:39526) 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.524+0000 7f0b27fff700 1 -- 172.21.15.137:0/3007971868 learned_addr learned my addr 172.21.15.137:0/3007971868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b2f7fe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30100dc0 0x7f0b30073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b27fff700 1 -- 172.21.15.137:0/3007971868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30100dc0 msgr2=0x7f0b30073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b27fff700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30100dc0 0x7f0b30073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b27fff700 1 -- 172.21.15.137:0/3007971868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200095f0 con 0x7f0b30101770 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b27fff700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d900 tx=0x7f0b1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b18009900 con 0x7f0b30101770 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b18010460 con 0x7f0b30101770 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b301a2840 con 0x7f0b30101770 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1800f5f0 con 0x7f0b30101770 2024-06-15T01:18:18.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.525+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b301a2c50 con 0x7f0b30101770 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.526+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30109160 con 0x7f0b30101770 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.527+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0b180105d0 con 0x7f0b30101770 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.527+0000 7f0b2d7fa700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 0x7f0b1c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.527+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0b18089e70 con 0x7f0b30101770 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.527+0000 7f0b2f7fe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 0x7f0b1c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.528+0000 7f0b2f7fe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 0x7f0b1c06d440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b20006010 tx=0x7f0b2001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:18.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.532+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0b18054ca0 con 0x7f0b30101770 2024-06-15T01:18:18.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.740+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b3002cda0 con 0x7f0b1c06af80 2024-06-15T01:18:18.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.741+0000 7f0b2d7fa700 1 -- 172.21.15.137:0/3007971868 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0b3002cda0 con 0x7f0b1c06af80 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 msgr2=0x7f0b1c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 0x7f0b1c06d440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b20006010 tx=0x7f0b2001a040 comp rx=0 tx=0).stop 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 msgr2=0x7f0b300735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d900 tx=0x7f0b1800dc10 comp rx=0 tx=0).stop 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 shutdown_connections 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b1c06af80 0x7f0b1c06d440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b30100dc0 0x7f0b30073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 --2- 172.21.15.137:0/3007971868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b30101770 0x7f0b300735d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 >> 172.21.15.137:0/3007971868 conn(0x7f0b300fc650 msgr2=0x7f0b300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 shutdown_connections 2024-06-15T01:18:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.743+0000 7f0b35efe700 1 -- 172.21.15.137:0/3007971868 wait complete. 2024-06-15T01:18:18.743 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:18:18.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.849+0000 7f3dacb85700 1 -- 172.21.15.137:0/2127610707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 msgr2=0x7f3da81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.849+0000 7f3dacb85700 1 --2- 172.21.15.137:0/2127610707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da81011a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009910 tx=0x7f3d98009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 -- 172.21.15.137:0/2127610707 shutdown_connections 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 --2- 172.21.15.137:0/2127610707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 --2- 172.21.15.137:0/2127610707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da81011a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 -- 172.21.15.137:0/2127610707 >> 172.21.15.137:0/2127610707 conn(0x7f3da80fc650 msgr2=0x7f3da80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 -- 172.21.15.137:0/2127610707 shutdown_connections 2024-06-15T01:18:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 -- 172.21.15.137:0/2127610707 wait complete. 2024-06-15T01:18:18.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 Processor -- start 2024-06-15T01:18:18.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.850+0000 7f3dacb85700 1 -- start start 2024-06-15T01:18:18.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3dacb85700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3dacb85700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3dacb85700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8073b10 con 0x7f3da8100dc0 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3dacb85700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8073c50 con 0x7f3da8101770 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39550/0 (socket says 172.21.15.137:39550) 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 -- 172.21.15.137:0/3128194107 learned_addr learned my addr 172.21.15.137:0/3128194107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da5d9b700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 -- 172.21.15.137:0/3128194107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 msgr2=0x7f3da80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 -- 172.21.15.137:0/3128194107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980095f0 con 0x7f3da8100dc0 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da5d9b700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.851+0000 7f3da659c700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:18.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.852+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d9801d070 con 0x7f3da8100dc0 2024-06-15T01:18:18.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.852+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d9800f460 con 0x7f3da8100dc0 2024-06-15T01:18:18.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.852+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d98021620 con 0x7f3da8100dc0 2024-06-15T01:18:18.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.852+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8073ed0 con 0x7f3da8100dc0 2024-06-15T01:18:18.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.852+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da81a2b70 con 0x7f3da8100dc0 2024-06-15T01:18:18.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.853+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da8109160 con 0x7f3da8100dc0 2024-06-15T01:18:18.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.853+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3d9800fa40 con 0x7f3da8100dc0 2024-06-15T01:18:18.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.854+0000 7f3d977fe700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 0x7f3d9006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:18.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.854+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3d9808b4e0 con 0x7f3da8100dc0 2024-06-15T01:18:18.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.854+0000 7f3da5d9b700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 0x7f3d9006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:18.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.854+0000 7f3da5d9b700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 0x7f3d9006d440 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c005950 tx=0x7f3d9c0058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:18.858+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3d98056390 con 0x7f3da8100dc0 2024-06-15T01:18:19.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.068+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3da802cce0 con 0x7f3d9006af80 2024-06-15T01:18:19.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.069+0000 7f3d977fe700 1 -- 172.21.15.137:0/3128194107 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3da802cce0 con 0x7f3d9006af80 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 msgr2=0x7f3d9006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 0x7f3d9006d440 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c005950 tx=0x7f3d9c0058e0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 msgr2=0x7f3da8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d9800be20 comp rx=0 tx=0).stop 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 shutdown_connections 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3d9006af80 0x7f3d9006d440 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3da8100dc0 0x7f3da8073090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 --2- 172.21.15.137:0/3128194107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8101770 0x7f3da80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 >> 172.21.15.137:0/3128194107 conn(0x7f3da80fc650 msgr2=0x7f3da80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 shutdown_connections 2024-06-15T01:18:19.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.071+0000 7f3dacb85700 1 -- 172.21.15.137:0/3128194107 wait complete. 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1174534472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc101d70 msgr2=0x7f9adc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1174534472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc101d70 0x7f9adc102150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009940 tx=0x7f9acc009c50 comp rx=0 tx=0).stop 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1174534472 shutdown_connections 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1174534472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc102690 0x7f9adc10ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1174534472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc101d70 0x7f9adc102150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1174534472 >> 172.21.15.137:0/1174534472 conn(0x7f9adc0fb540 msgr2=0x7f9adc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1174534472 shutdown_connections 2024-06-15T01:18:19.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.175+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1174534472 wait complete. 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 Processor -- start 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 -- start start 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc101d70 0x7f9adc196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc197270 con 0x7f9adc102690 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ae19b7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19b000 con 0x7f9adc101d70 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ad27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ad27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39570/0 (socket says 172.21.15.137:39570) 2024-06-15T01:18:19.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9ad27fc700 1 -- 172.21.15.137:0/1740864925 learned_addr learned my addr 172.21.15.137:0/1740864925 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.176+0000 7f9adaffd700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc101d70 0x7f9adc196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ad27fc700 1 -- 172.21.15.137:0/1740864925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc101d70 msgr2=0x7f9adc196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ad27fc700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc101d70 0x7f9adc196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ad27fc700 1 -- 172.21.15.137:0/1740864925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0095f0 con 0x7f9adc102690 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ad27fc700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c8f0 tx=0x7f9ac400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac4007ad0 con 0x7f9adc102690 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc19b2e0 con 0x7f9adc102690 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.177+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ac400ce90 con 0x7f9adc102690 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.178+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac4018630 con 0x7f9adc102690 2024-06-15T01:18:19.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.178+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc19b830 con 0x7f9adc102690 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.178+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f9ac400f450 con 0x7f9adc102690 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.178+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc108280 con 0x7f9adc102690 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.179+0000 7f9ae09b5700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 0x7f9ac806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.179+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9ac408a330 con 0x7f9adc102690 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.179+0000 7f9adaffd700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 0x7f9ac806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.179+0000 7f9adaffd700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 0x7f9ac806d440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005cd0 tx=0x7f9acc005c00 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.183+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9ac4055160 con 0x7f9adc102690 2024-06-15T01:18:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.383+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9adc19bb70 con 0x7f9ac806af80 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.393+0000 7f9ae09b5700 1 -- 172.21.15.137:0/1740864925 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9adc19bb70 con 0x7f9ac806af80 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (8m) 3m ago 19m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (19m) 3m ago 19m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (18m) 2m ago 18m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:18:19.381 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (7m) 3m ago 19m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (7m) 2m ago 18m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (8m) 3m ago 19m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (3m) 3m ago 13m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (3m) 3m ago 13m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (3m) 2m ago 13m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (2m) 2m ago 13m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (10m) 3m ago 21m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (9m) 2m ago 18m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (7m) 3m ago 21m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (7m) 2m ago 18m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (9m) 3m ago 19m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:18:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (9m) 2m ago 18m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (6m) 3m ago 17m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (6m) 3m ago 17m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (5m) 3m ago 16m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (5m) 2m ago 16m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (4m) 2m ago 15m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (3m) 2m ago 15m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:18:19.383 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (9m) 3m ago 18m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 msgr2=0x7f9ac806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 0x7f9ac806d440 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005cd0 tx=0x7f9acc005c00 comp rx=0 tx=0).stop 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 msgr2=0x7f9adc196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c8f0 tx=0x7f9ac400ccb0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 shutdown_connections 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9ac806af80 0x7f9ac806d440 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9adc101d70 0x7f9adc196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 --2- 172.21.15.137:0/1740864925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9adc102690 0x7f9adc196b90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 >> 172.21.15.137:0/1740864925 conn(0x7f9adc0fb540 msgr2=0x7f9adc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 shutdown_connections 2024-06-15T01:18:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.395+0000 7f9ae19b7700 1 -- 172.21.15.137:0/1740864925 wait complete. 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 -- 172.21.15.137:0/74571306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 msgr2=0x7fb16c105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 --2- 172.21.15.137:0/74571306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c105e50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb160009910 tx=0x7fb160009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 -- 172.21.15.137:0/74571306 shutdown_connections 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 --2- 172.21.15.137:0/74571306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 0x7fb16c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 --2- 172.21.15.137:0/74571306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c105e50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 -- 172.21.15.137:0/74571306 >> 172.21.15.137:0/74571306 conn(0x7fb16c0fb560 msgr2=0x7fb16c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 -- 172.21.15.137:0/74571306 shutdown_connections 2024-06-15T01:18:19.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.498+0000 7fb173548700 1 -- 172.21.15.137:0/74571306 wait complete. 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 Processor -- start 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 -- start start 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 0x7fb16c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c197250 con 0x7fb16c105a70 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb173548700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c199f80 con 0x7fb16c0ffa70 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.499+0000 7fb170ae3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39578/0 (socket says 172.21.15.137:39578) 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 -- 172.21.15.137:0/2497528816 learned_addr learned my addr 172.21.15.137:0/2497528816 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb1712e4700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 0x7fb16c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 -- 172.21.15.137:0/2497528816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 msgr2=0x7fb16c196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 0x7fb16c196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 -- 172.21.15.137:0/2497528816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600095f0 con 0x7fb16c105a70 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb170ae3700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c8f0 tx=0x7fb16800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb168004a00 con 0x7fb16c105a70 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb168005020 con 0x7fb16c105a70 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c19a260 con 0x7fb16c105a70 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb168007c40 con 0x7fb16c105a70 2024-06-15T01:18:19.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.500+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c19a7b0 con 0x7fb16c105a70 2024-06-15T01:18:19.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.501+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c04ea90 con 0x7fb16c105a70 2024-06-15T01:18:19.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.501+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb168005190 con 0x7fb16c105a70 2024-06-15T01:18:19.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.502+0000 7fb15e7fc700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 0x7fb15806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.502+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb168013070 con 0x7fb16c105a70 2024-06-15T01:18:19.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.502+0000 7fb1712e4700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 0x7fb15806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.502+0000 7fb1712e4700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 0x7fb15806d440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb16001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.506+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb168054fd0 con 0x7fb16c105a70 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.776+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb16c19aa80 con 0x7fb16c105a70 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.776+0000 7fb15e7fc700 1 -- 172.21.15.137:0/2497528816 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb1680585f0 con 0x7fb16c105a70 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:18:19.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:18:19.766 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:18:19.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.778+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 msgr2=0x7fb15806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.778+0000 7fb173548700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 0x7fb15806d440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb16001a040 comp rx=0 tx=0).stop 2024-06-15T01:18:19.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.778+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 msgr2=0x7fb16c196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.778+0000 7fb173548700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c8f0 tx=0x7fb16800cc00 comp rx=0 tx=0).stop 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 shutdown_connections 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb15806af80 0x7fb15806d440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb16c0ffa70 0x7fb16c196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 --2- 172.21.15.137:0/2497528816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb16c105a70 0x7fb16c196bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 >> 172.21.15.137:0/2497528816 conn(0x7fb16c0fb560 msgr2=0x7fb16c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 shutdown_connections 2024-06-15T01:18:19.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.779+0000 7fb173548700 1 -- 172.21.15.137:0/2497528816 wait complete. 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 -- 172.21.15.137:0/421760863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 msgr2=0x7fa22c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 --2- 172.21.15.137:0/421760863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c105790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa220009910 tx=0x7fa220009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 -- 172.21.15.137:0/421760863 shutdown_connections 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 --2- 172.21.15.137:0/421760863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c105790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 --2- 172.21.15.137:0/421760863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 -- 172.21.15.137:0/421760863 >> 172.21.15.137:0/421760863 conn(0x7fa22c0755a0 msgr2=0x7fa22c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 -- 172.21.15.137:0/421760863 shutdown_connections 2024-06-15T01:18:19.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.882+0000 7fa231a86700 1 -- 172.21.15.137:0/421760863 wait complete. 2024-06-15T01:18:19.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 Processor -- start 2024-06-15T01:18:19.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 -- start start 2024-06-15T01:18:19.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c100720 con 0x7fa22c069020 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa231a86700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c100860 con 0x7fa22c068670 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa22affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa22affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39586/0 (socket says 172.21.15.137:39586) 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa22affd700 1 -- 172.21.15.137:0/1214367071 learned_addr learned my addr 172.21.15.137:0/1214367071 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.883+0000 7fa22b7fe700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa22affd700 1 -- 172.21.15.137:0/1214367071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 msgr2=0x7fa22c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa22affd700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa22affd700 1 -- 172.21.15.137:0/1214367071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2200095f0 con 0x7fa22c069020 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa22b7fe700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa22affd700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2200092e0 tx=0x7fa22000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa22001d070 con 0x7fa22c069020 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa22000f460 con 0x7fa22c069020 2024-06-15T01:18:19.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c100ae0 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa220021600 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.884+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c1a2b40 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.885+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c1090a0 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.885+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa22000f5d0 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.886+0000 7fa228ff9700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 0x7fa21406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.886+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa22008b660 con 0x7fa22c069020 2024-06-15T01:18:19.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.886+0000 7fa22b7fe700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 0x7fa21406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:19.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.886+0000 7fa22b7fe700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 0x7fa21406d440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa21c005950 tx=0x7fa21c005f50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:19.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:19.890+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa220056490 con 0x7fa22c069020 2024-06-15T01:18:20.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.121+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa22c04ea90 con 0x7fa22c069020 2024-06-15T01:18:20.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.122+0000 7fa228ff9700 1 -- 172.21.15.137:0/1214367071 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fa220018e60 con 0x7fa22c069020 2024-06-15T01:18:20.110 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:18:20.110 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:18:20.110 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:18:20.111 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:18:20.112 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:20.113 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 msgr2=0x7fa21406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 0x7fa21406d440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa21c005950 tx=0x7fa21c005f50 comp rx=0 tx=0).stop 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 msgr2=0x7fa22c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2200092e0 tx=0x7fa22000be20 comp rx=0 tx=0).stop 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 shutdown_connections 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa21406af80 0x7fa21406d440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa22c068670 0x7fa22c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 --2- 172.21.15.137:0/1214367071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa22c069020 0x7fa22c1001e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 >> 172.21.15.137:0/1214367071 conn(0x7fa22c0755a0 msgr2=0x7fa22c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:20.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 shutdown_connections 2024-06-15T01:18:20.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.124+0000 7fa231a86700 1 -- 172.21.15.137:0/1214367071 wait complete. 2024-06-15T01:18:20.115 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:18:20.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 -- 172.21.15.137:0/892823156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c100dd0 msgr2=0x7f3a8c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 --2- 172.21.15.137:0/892823156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c1011b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009910 tx=0x7f3a7c009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 -- 172.21.15.137:0/892823156 shutdown_connections 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 --2- 172.21.15.137:0/892823156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c101780 0x7f3a8c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 --2- 172.21.15.137:0/892823156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c1011b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 -- 172.21.15.137:0/892823156 >> 172.21.15.137:0/892823156 conn(0x7f3a8c0fc640 msgr2=0x7f3a8c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 -- 172.21.15.137:0/892823156 shutdown_connections 2024-06-15T01:18:20.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 -- 172.21.15.137:0/892823156 wait complete. 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.231+0000 7f3a91349700 1 Processor -- start 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a91349700 1 -- start start 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a91349700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a91349700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a91349700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c19f250 con 0x7f3a8c101780 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a91349700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c198700 con 0x7f3a8c100dd0 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39600/0 (socket says 172.21.15.137:39600) 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 -- 172.21.15.137:0/1448563205 learned_addr learned my addr 172.21.15.137:0/1448563205 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a8affd700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 -- 172.21.15.137:0/1448563205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 msgr2=0x7f3a8c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 -- 172.21.15.137:0/1448563205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0095f0 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a8affd700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.232+0000 7f3a83fff700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d8d0 tx=0x7f3a7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.233+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a740041f0 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.233+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c1989e0 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.233+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a74007620 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.233+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a74003b50 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.233+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c198f30 con 0x7f3a8c101780 2024-06-15T01:18:20.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.234+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c104f50 con 0x7f3a8c101780 2024-06-15T01:18:20.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:20 smithi137 ceph-mon[112295]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:20.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:20 smithi137 ceph-mon[112295]: pgmap v367: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:20.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:20 smithi137 ceph-mon[112295]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:20.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:20 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2497528816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:18:20.224 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:20 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1214367071' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:18:20.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.234+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3a74004350 con 0x7f3a8c101780 2024-06-15T01:18:20.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.235+0000 7f3a88ff9700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 0x7f3a7806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.235+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3a74089f50 con 0x7f3a8c101780 2024-06-15T01:18:20.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.235+0000 7f3a8affd700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 0x7f3a7806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.236+0000 7f3a8affd700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 0x7f3a7806d440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b5c0 tx=0x7f3a7c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:20.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.239+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a740bd900 con 0x7f3a8c101780 2024-06-15T01:18:20.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.447+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a8c199230 con 0x7f3a7806af80 2024-06-15T01:18:20.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.449+0000 7f3a88ff9700 1 -- 172.21.15.137:0/1448563205 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3a8c199230 con 0x7f3a7806af80 2024-06-15T01:18:20.437 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:18:20.438 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:18:20.438 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:18:20.438 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:18:20.438 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:18:20.438 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:18:20.439 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:18:20.440 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:18:20.440 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:18:20.440 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:18:20.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 msgr2=0x7f3a7806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 0x7f3a7806d440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b5c0 tx=0x7f3a7c01a040 comp rx=0 tx=0).stop 2024-06-15T01:18:20.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 msgr2=0x7f3a8c19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d8d0 tx=0x7f3a7400dc90 comp rx=0 tx=0).stop 2024-06-15T01:18:20.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 shutdown_connections 2024-06-15T01:18:20.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3a7806af80 0x7f3a7806d440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c100dd0 0x7f3a8c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 --2- 172.21.15.137:0/1448563205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c101780 0x7f3a8c19ebc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 >> 172.21.15.137:0/1448563205 conn(0x7f3a8c0fc640 msgr2=0x7f3a8c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:20.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 shutdown_connections 2024-06-15T01:18:20.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.451+0000 7f3a91349700 1 -- 172.21.15.137:0/1448563205 wait complete. 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 -- 172.21.15.137:0/1919525710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 msgr2=0x7feb341057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 --2- 172.21.15.137:0/1919525710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb341057c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009910 tx=0x7feb1c009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 -- 172.21.15.137:0/1919525710 shutdown_connections 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 --2- 172.21.15.137:0/1919525710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb341057c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 --2- 172.21.15.137:0/1919525710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 0x7feb341011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 -- 172.21.15.137:0/1919525710 >> 172.21.15.137:0/1919525710 conn(0x7feb340fc670 msgr2=0x7feb340fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:20.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 -- 172.21.15.137:0/1919525710 shutdown_connections 2024-06-15T01:18:20.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.555+0000 7feb3928b700 1 -- 172.21.15.137:0/1919525710 wait complete. 2024-06-15T01:18:20.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 Processor -- start 2024-06-15T01:18:20.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 -- start start 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 0x7feb34196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb34197270 con 0x7feb34101770 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb3928b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419b000 con 0x7feb34100dc0 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb327fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb327fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39622/0 (socket says 172.21.15.137:39622) 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb327fc700 1 -- 172.21.15.137:0/546623439 learned_addr learned my addr 172.21.15.137:0/546623439 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.556+0000 7feb32ffd700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 0x7feb34196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb327fc700 1 -- 172.21.15.137:0/546623439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 msgr2=0x7feb34196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb327fc700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 0x7feb34196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb327fc700 1 -- 172.21.15.137:0/546623439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c0095f0 con 0x7feb34101770 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb327fc700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00be10 tx=0x7feb1c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1c01d070 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb1c005920 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3419b280 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1c003940 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.557+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3419b710 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.558+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb341091c0 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.558+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7feb1c003bc0 con 0x7feb34101770 2024-06-15T01:18:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.559+0000 7feb2bfff700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 0x7feb2006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:20.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.559+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7feb1c08b680 con 0x7feb34101770 2024-06-15T01:18:20.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.559+0000 7feb32ffd700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 0x7feb2006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:20.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.559+0000 7feb32ffd700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 0x7feb2006d440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb24009380 tx=0x7feb2400b040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:20.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.563+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feb1c0564b0 con 0x7feb34101770 2024-06-15T01:18:20.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:20 smithi142 ceph-mon[82136]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:20.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:20 smithi142 ceph-mon[82136]: pgmap v367: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:20.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:20 smithi142 ceph-mon[82136]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:20.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:20 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2497528816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:18:20.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:20 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1214367071' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:18:20.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.834+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb3404ea90 con 0x7feb34101770 2024-06-15T01:18:20.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.834+0000 7feb2bfff700 1 -- 172.21.15.137:0/546623439 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7feb1c059ad0 con 0x7feb34101770 2024-06-15T01:18:20.823 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:18:20.823 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:18:20.823 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:18:20.824 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:18:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 msgr2=0x7feb2006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 0x7feb2006d440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb24009380 tx=0x7feb2400b040 comp rx=0 tx=0).stop 2024-06-15T01:18:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 msgr2=0x7feb34196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00be10 tx=0x7feb1c00bef0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 shutdown_connections 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7feb2006af80 0x7feb2006d440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb34100dc0 0x7feb34196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 --2- 172.21.15.137:0/546623439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb34101770 0x7feb34196b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 >> 172.21.15.137:0/546623439 conn(0x7feb340fc670 msgr2=0x7feb340fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 shutdown_connections 2024-06-15T01:18:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:20.837+0000 7feb3928b700 1 -- 172.21.15.137:0/546623439 wait complete. 2024-06-15T01:18:21.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:21 smithi137 ceph-mon[112295]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:21.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:21 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/546623439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:18:21.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:21 smithi142 ceph-mon[82136]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:21.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:21 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/546623439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:18:22.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:22 smithi137 ceph-mon[112295]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:22.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:22 smithi137 ceph-mon[112295]: pgmap v368: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:22.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:22 smithi142 ceph-mon[82136]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:22.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:22 smithi142 ceph-mon[82136]: pgmap v368: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:24.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:24 smithi137 ceph-mon[112295]: pgmap v369: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:24.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:24 smithi142 ceph-mon[82136]: pgmap v369: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:26.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:26 smithi137 ceph-mon[112295]: pgmap v370: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:26.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:26 smithi142 ceph-mon[82136]: pgmap v370: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:28.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:28 smithi137 ceph-mon[112295]: pgmap v371: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:28.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:28 smithi142 ceph-mon[82136]: pgmap v371: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:30.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:30 smithi137 ceph-mon[112295]: pgmap v372: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:30.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:30 smithi142 ceph-mon[82136]: pgmap v372: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:32.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:32 smithi137 ceph-mon[112295]: pgmap v373: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:32.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:32 smithi142 ceph-mon[82136]: pgmap v373: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:33.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:33.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:34.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:34 smithi142 ceph-mon[82136]: pgmap v374: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:34.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:34 smithi137 ceph-mon[112295]: pgmap v374: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:36.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:36 smithi142 ceph-mon[82136]: pgmap v375: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:36.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:36 smithi137 ceph-mon[112295]: pgmap v375: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:38.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:38 smithi142 ceph-mon[82136]: pgmap v376: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:38.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:38 smithi137 ceph-mon[112295]: pgmap v376: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:40.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:40 smithi142 ceph-mon[82136]: pgmap v377: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:40.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:40 smithi137 ceph-mon[112295]: pgmap v377: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:42.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:42 smithi142 ceph-mon[82136]: pgmap v378: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:42.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:42 smithi137 ceph-mon[112295]: pgmap v378: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:44.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:44 smithi142 ceph-mon[82136]: pgmap v379: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:44.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:44 smithi137 ceph-mon[112295]: pgmap v379: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:46.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:46 smithi142 ceph-mon[82136]: pgmap v380: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:46.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:46 smithi137 ceph-mon[112295]: pgmap v380: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:48.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:48 smithi142 ceph-mon[82136]: pgmap v381: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:48.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:48.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:48 smithi137 ceph-mon[112295]: pgmap v381: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:48.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:18:50.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:50 smithi142 ceph-mon[82136]: pgmap v382: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:50.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:50 smithi137 ceph-mon[112295]: pgmap v382: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 -- 172.21.15.137:0/4116485936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 msgr2=0x7f87c4102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 --2- 172.21.15.137:0/4116485936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c4102050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009910 tx=0x7f87b4009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 -- 172.21.15.137:0/4116485936 shutdown_connections 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 --2- 172.21.15.137:0/4116485936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 --2- 172.21.15.137:0/4116485936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c4102050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 -- 172.21.15.137:0/4116485936 >> 172.21.15.137:0/4116485936 conn(0x7f87c40fb570 msgr2=0x7f87c40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 -- 172.21.15.137:0/4116485936 shutdown_connections 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.954+0000 7f87ca517700 1 -- 172.21.15.137:0/4116485936 wait complete. 2024-06-15T01:18:50.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 Processor -- start 2024-06-15T01:18:50.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 -- start start 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c419f2a0 con 0x7f87c4101c70 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87ca517700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c4198750 con 0x7f87c4102620 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39086/0 (socket says 172.21.15.137:39086) 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87c3fff700 1 -- 172.21.15.137:0/1392692745 learned_addr learned my addr 172.21.15.137:0/1392692745 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87c37fe700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.955+0000 7f87c3fff700 1 -- 172.21.15.137:0/1392692745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 msgr2=0x7f87c419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c3fff700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:50.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c3fff700 1 -- 172.21.15.137:0/1392692745 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40095f0 con 0x7f87c4101c70 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c37fe700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c3fff700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006010 tx=0x7f87b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b401d070 con 0x7f87c4101c70 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87b400f460 con 0x7f87c4101c70 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c41989d0 con 0x7f87c4101c70 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b4021640 con 0x7f87c4101c70 2024-06-15T01:18:50.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.956+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c4198ec0 con 0x7f87c4101c70 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.957+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c410a670 con 0x7f87c4101c70 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.957+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f87b400fa60 con 0x7f87c4101c70 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.958+0000 7f87c17fa700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 0x7f87ac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.958+0000 7f87c37fe700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 0x7f87ac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.958+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f87b408b6c0 con 0x7f87c4101c70 2024-06-15T01:18:50.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.959+0000 7f87c37fe700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 0x7f87ac06d440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005fd0 tx=0x7f87b8005dc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:50.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:50.963+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f87b4059f80 con 0x7f87c4101c70 2024-06-15T01:18:51.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.171+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87c41990e0 con 0x7f87ac06af80 2024-06-15T01:18:51.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.172+0000 7f87c17fa700 1 -- 172.21.15.137:0/1392692745 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f87c41990e0 con 0x7f87ac06af80 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 msgr2=0x7f87ac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 0x7f87ac06d440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005fd0 tx=0x7f87b8005dc0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 msgr2=0x7f87c419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006010 tx=0x7f87b400be20 comp rx=0 tx=0).stop 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 shutdown_connections 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f87ac06af80 0x7f87ac06d440 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87c4101c70 0x7f87c419e6d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 --2- 172.21.15.137:0/1392692745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c4102620 0x7f87c419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 >> 172.21.15.137:0/1392692745 conn(0x7f87c40fb570 msgr2=0x7f87c4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 shutdown_connections 2024-06-15T01:18:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.174+0000 7f87ca517700 1 -- 172.21.15.137:0/1392692745 wait complete. 2024-06-15T01:18:51.173 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:18:51.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 -- 172.21.15.137:0/2823174907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4069020 msgr2=0x7f3ba4105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/2823174907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4069020 0x7f3ba4105790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009910 tx=0x7f3b8c009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 -- 172.21.15.137:0/2823174907 shutdown_connections 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/2823174907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4069020 0x7f3ba4105790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/2823174907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4068670 0x7f3ba4068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 -- 172.21.15.137:0/2823174907 >> 172.21.15.137:0/2823174907 conn(0x7f3ba40755a0 msgr2=0x7f3ba40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 -- 172.21.15.137:0/2823174907 shutdown_connections 2024-06-15T01:18:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.280+0000 7f3ba8e14700 1 -- 172.21.15.137:0/2823174907 wait complete. 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 Processor -- start 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 -- start start 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 0x7f3ba4196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4197260 con 0x7f3ba4068670 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba8e14700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419aff0 con 0x7f3ba4069020 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39100/0 (socket says 172.21.15.137:39100) 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 -- 172.21.15.137:0/3662441714 learned_addr learned my addr 172.21.15.137:0/3662441714 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba1d9b700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 0x7f3ba4196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 -- 172.21.15.137:0/3662441714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 msgr2=0x7f3ba4196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 0x7f3ba4196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba259c700 1 -- 172.21.15.137:0/3662441714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0095f0 con 0x7f3ba4068670 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.281+0000 7f3ba1d9b700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 0x7f3ba4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3ba259c700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00ada0 tx=0x7f3b9c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9c00f070 con 0x7f3ba4068670 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b9c007bf0 con 0x7f3ba4068670 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba419b2d0 con 0x7f3ba4068670 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9c005630 con 0x7f3ba4068670 2024-06-15T01:18:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.282+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba419b820 con 0x7f3ba4068670 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.283+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba41090a0 con 0x7f3ba4068670 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.283+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3b9c005790 con 0x7f3ba4068670 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.284+0000 7f3b9b7fe700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 0x7f3b9006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.284+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3b9c089cd0 con 0x7f3ba4068670 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.284+0000 7f3ba1d9b700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 0x7f3b9006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.284+0000 7f3ba1d9b700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 0x7f3b9006d440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0059d0 tx=0x7f3b8c005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.288+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3b9c054b80 con 0x7f3ba4068670 2024-06-15T01:18:51.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.500+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba419bb70 con 0x7f3b9006af80 2024-06-15T01:18:51.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.501+0000 7f3b9b7fe700 1 -- 172.21.15.137:0/3662441714 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ba419bb70 con 0x7f3b9006af80 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 msgr2=0x7f3b9006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 0x7f3b9006d440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0059d0 tx=0x7f3b8c005960 comp rx=0 tx=0).stop 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 msgr2=0x7f3ba4196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00ada0 tx=0x7f3b9c00c330 comp rx=0 tx=0).stop 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 shutdown_connections 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3b9006af80 0x7f3b9006d440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4068670 0x7f3ba4196640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 --2- 172.21.15.137:0/3662441714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4069020 0x7f3ba4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 >> 172.21.15.137:0/3662441714 conn(0x7f3ba40755a0 msgr2=0x7f3ba4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 shutdown_connections 2024-06-15T01:18:51.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.503+0000 7f3ba8e14700 1 -- 172.21.15.137:0/3662441714 wait complete. 2024-06-15T01:18:51.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- 172.21.15.137:0/1502619807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528102620 msgr2=0x7fc52810ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 --2- 172.21.15.137:0/1502619807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528102620 0x7fc52810ab10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc518009910 tx=0x7fc518009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:51.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- 172.21.15.137:0/1502619807 shutdown_connections 2024-06-15T01:18:51.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 --2- 172.21.15.137:0/1502619807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528102620 0x7fc52810ab10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 --2- 172.21.15.137:0/1502619807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528101c70 0x7fc528102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- 172.21.15.137:0/1502619807 >> 172.21.15.137:0/1502619807 conn(0x7fc5280fb570 msgr2=0x7fc5280fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- 172.21.15.137:0/1502619807 shutdown_connections 2024-06-15T01:18:51.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- 172.21.15.137:0/1502619807 wait complete. 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 Processor -- start 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.616+0000 7fc52f29b700 1 -- start start 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528102620 0x7fc528196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528197260 con 0x7fc528101c70 2024-06-15T01:18:51.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528199f90 con 0x7fc528102620 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39130/0 (socket says 172.21.15.137:39130) 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 -- 172.21.15.137:0/4066610336 learned_addr learned my addr 172.21.15.137:0/4066610336 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 -- 172.21.15.137:0/4066610336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528102620 msgr2=0x7fc528196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528102620 0x7fc528196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 -- 172.21.15.137:0/4066610336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180095f0 con 0x7fc528101c70 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52d037700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d8d0 tx=0x7fc52400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc524009940 con 0x7fc528101c70 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc524010460 con 0x7fc528101c70 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52819a270 con 0x7fc528101c70 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52400f5f0 con 0x7fc528101c70 2024-06-15T01:18:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.617+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52819a7c0 con 0x7fc528101c70 2024-06-15T01:18:51.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.618+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528108210 con 0x7fc528101c70 2024-06-15T01:18:51.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.619+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc5240105d0 con 0x7fc528101c70 2024-06-15T01:18:51.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.619+0000 7fc51e7fc700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 0x7fc51406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.619+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc52408a2d0 con 0x7fc528101c70 2024-06-15T01:18:51.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.619+0000 7fc52c836700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 0x7fc51406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.620+0000 7fc52c836700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 0x7fc51406d440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc5180059d0 tx=0x7fc518005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.624+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc524055180 con 0x7fc528101c70 2024-06-15T01:18:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.824+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc52819aaf0 con 0x7fc51406af80 2024-06-15T01:18:51.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.834+0000 7fc51e7fc700 1 -- 172.21.15.137:0/4066610336 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc52819aaf0 con 0x7fc51406af80 2024-06-15T01:18:51.822 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:18:51.822 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (9m) 3m ago 19m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (20m) 3m ago 20m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (19m) 3m ago 19m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (7m) 3m ago 20m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (7m) 3m ago 19m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (9m) 3m ago 19m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (3m) 3m ago 14m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (4m) 3m ago 14m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:18:51.823 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (3m) 3m ago 14m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (3m) 3m ago 14m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (11m) 3m ago 21m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (10m) 3m ago 18m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (8m) 3m ago 21m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (7m) 3m ago 18m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (9m) 3m ago 20m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (9m) 3m ago 19m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:18:51.824 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (7m) 3m ago 18m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (6m) 3m ago 17m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (6m) 3m ago 17m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (5m) 3m ago 16m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (5m) 3m ago 16m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (4m) 3m ago 15m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:18:51.825 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (9m) 3m ago 19m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:18:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 msgr2=0x7fc51406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 0x7fc51406d440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc5180059d0 tx=0x7fc518005960 comp rx=0 tx=0).stop 2024-06-15T01:18:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 msgr2=0x7fc528196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d8d0 tx=0x7fc52400dc90 comp rx=0 tx=0).stop 2024-06-15T01:18:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 shutdown_connections 2024-06-15T01:18:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc51406af80 0x7fc51406d440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528101c70 0x7fc528196690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 --2- 172.21.15.137:0/4066610336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528102620 0x7fc528196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 >> 172.21.15.137:0/4066610336 conn(0x7fc5280fb570 msgr2=0x7fc528105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 shutdown_connections 2024-06-15T01:18:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.836+0000 7fc52f29b700 1 -- 172.21.15.137:0/4066610336 wait complete. 2024-06-15T01:18:51.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 -- 172.21.15.137:0/2551864420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 msgr2=0x7f7580105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 --2- 172.21.15.137:0/2551864420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580105e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7568009910 tx=0x7f7568009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:51.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 -- 172.21.15.137:0/2551864420 shutdown_connections 2024-06-15T01:18:51.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 --2- 172.21.15.137:0/2551864420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 0x7f75800ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 --2- 172.21.15.137:0/2551864420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580105e20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 -- 172.21.15.137:0/2551864420 >> 172.21.15.137:0/2551864420 conn(0x7f75800fb550 msgr2=0x7f75800fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:51.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 -- 172.21.15.137:0/2551864420 shutdown_connections 2024-06-15T01:18:51.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.938+0000 7f7584eef700 1 -- 172.21.15.137:0/2551864420 wait complete. 2024-06-15T01:18:51.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 Processor -- start 2024-06-15T01:18:51.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 -- start start 2024-06-15T01:18:51.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 0x7f7580195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75801961b0 con 0x7f7580105a40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f7584eef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580199f40 con 0x7f75800ffa40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39144/0 (socket says 172.21.15.137:39144) 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 -- 172.21.15.137:0/1634086779 learned_addr learned my addr 172.21.15.137:0/1634086779 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757e59c700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 0x7f7580195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 -- 172.21.15.137:0/1634086779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 msgr2=0x7f7580195590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 0x7f7580195590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.939+0000 7f757dd9b700 1 -- 172.21.15.137:0/1634086779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680095f0 con 0x7f7580105a40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f757dd9b700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f757000e920 tx=0x7f757000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757000c9d0 con 0x7f7580105a40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7570010430 con 0x7f7580105a40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758019a220 con 0x7f7580105a40 2024-06-15T01:18:51.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7570018650 con 0x7f7580105a40 2024-06-15T01:18:51.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.940+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758019a770 con 0x7f7580105a40 2024-06-15T01:18:51.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.941+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580108040 con 0x7f7580105a40 2024-06-15T01:18:51.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.941+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f75700105a0 con 0x7f7580105a40 2024-06-15T01:18:51.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.942+0000 7f75777fe700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 0x7f756c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:51.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.942+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7570014070 con 0x7f7580105a40 2024-06-15T01:18:51.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.942+0000 7f757e59c700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 0x7f756c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:51.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.942+0000 7f757e59c700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 0x7f756c06d440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f756800b6e0 tx=0x7f7568005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:51.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:51.946+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7570052900 con 0x7f7580105a40 2024-06-15T01:18:52.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.217+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f758004ea90 con 0x7f7580105a40 2024-06-15T01:18:52.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.218+0000 7f75777fe700 1 -- 172.21.15.137:0/1634086779 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f757005de60 con 0x7f7580105a40 2024-06-15T01:18:52.206 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:18:52.207 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:18:52.208 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:18:52.208 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:18:52.208 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:18:52.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 msgr2=0x7f756c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 0x7f756c06d440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f756800b6e0 tx=0x7f7568005a90 comp rx=0 tx=0).stop 2024-06-15T01:18:52.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 msgr2=0x7f7580195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f757000e920 tx=0x7f757000ec30 comp rx=0 tx=0).stop 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 shutdown_connections 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756c06af80 0x7f756c06d440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75800ffa40 0x7f7580195590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 --2- 172.21.15.137:0/1634086779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580105a40 0x7f7580195ad0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 >> 172.21.15.137:0/1634086779 conn(0x7f75800fb550 msgr2=0x7f75800fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 shutdown_connections 2024-06-15T01:18:52.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.221+0000 7f7584eef700 1 -- 172.21.15.137:0/1634086779 wait complete. 2024-06-15T01:18:52.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 -- 172.21.15.137:0/1717095953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 msgr2=0x7f30581011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 --2- 172.21.15.137:0/1717095953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f30581011a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f304c009910 tx=0x7f304c009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 -- 172.21.15.137:0/1717095953 shutdown_connections 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 --2- 172.21.15.137:0/1717095953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 0x7f30581057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 --2- 172.21.15.137:0/1717095953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f30581011a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 -- 172.21.15.137:0/1717095953 >> 172.21.15.137:0/1717095953 conn(0x7f30580fc670 msgr2=0x7f30580fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 -- 172.21.15.137:0/1717095953 shutdown_connections 2024-06-15T01:18:52.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.328+0000 7f305e48a700 1 -- 172.21.15.137:0/1717095953 wait complete. 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 Processor -- start 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 -- start start 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 0x7f3058196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30581972e0 con 0x7f3058100dc0 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f305e48a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305819a010 con 0x7f3058101770 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39158/0 (socket says 172.21.15.137:39158) 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 -- 172.21.15.137:0/2160163972 learned_addr learned my addr 172.21.15.137:0/2160163972 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:52.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f30577fe700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 0x7f3058196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 -- 172.21.15.137:0/2160163972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 msgr2=0x7f3058196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 0x7f3058196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.329+0000 7f3057fff700 1 -- 172.21.15.137:0/2160163972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0095f0 con 0x7f3058100dc0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f3057fff700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b5c0 tx=0x7f304c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c01d070 con 0x7f3058100dc0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f304c005e00 con 0x7f3058100dc0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305819a290 con 0x7f3058100dc0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c00fe90 con 0x7f3058100dc0 2024-06-15T01:18:52.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.330+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305819a780 con 0x7f3058100dc0 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.331+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3058109160 con 0x7f3058100dc0 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.331+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f304c00f6d0 con 0x7f3058100dc0 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.332+0000 7f30557fa700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 0x7f304006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.332+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f304c08b5c0 con 0x7f3058100dc0 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.332+0000 7f30577fe700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 0x7f304006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.332+0000 7f30577fe700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 0x7f304006d440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3058197d30 tx=0x7f304800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:52.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.336+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f304c056510 con 0x7f3058100dc0 2024-06-15T01:18:52.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:52 smithi137 ceph-mon[112295]: pgmap v383: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:52.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:52 smithi137 ceph-mon[112295]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:52.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:52 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1634086779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:18:52.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.570+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3058197b40 con 0x7f3058100dc0 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.570+0000 7f30557fa700 1 -- 172.21.15.137:0/2160163972 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f304c059b30 con 0x7f3058100dc0 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:18:52.559 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:18:52.560 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:52.561 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:18:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:52 smithi142 ceph-mon[82136]: pgmap v383: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:52 smithi142 ceph-mon[82136]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:52.563 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:52 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1634086779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:18:52.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 msgr2=0x7f304006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 0x7f304006d440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3058197d30 tx=0x7f304800b410 comp rx=0 tx=0).stop 2024-06-15T01:18:52.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 msgr2=0x7f3058196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b5c0 tx=0x7f304c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 shutdown_connections 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f304006af80 0x7f304006d440 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3058100dc0 0x7f3058196710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 --2- 172.21.15.137:0/2160163972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3058101770 0x7f3058196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.572+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 >> 172.21.15.137:0/2160163972 conn(0x7f30580fc670 msgr2=0x7f30580fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.573+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 shutdown_connections 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.573+0000 7f305e48a700 1 -- 172.21.15.137:0/2160163972 wait complete. 2024-06-15T01:18:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:18:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 -- 172.21.15.137:0/1661348514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4101770 msgr2=0x7f0ed41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1661348514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4101770 0x7f0ed41057c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009910 tx=0x7f0ec8009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 -- 172.21.15.137:0/1661348514 shutdown_connections 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1661348514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4101770 0x7f0ed41057c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1661348514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.677+0000 7f0ed981a700 1 -- 172.21.15.137:0/1661348514 >> 172.21.15.137:0/1661348514 conn(0x7f0ed40fc650 msgr2=0x7f0ed40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.678+0000 7f0ed981a700 1 -- 172.21.15.137:0/1661348514 shutdown_connections 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.678+0000 7f0ed981a700 1 -- 172.21.15.137:0/1661348514 wait complete. 2024-06-15T01:18:52.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.678+0000 7f0ed981a700 1 Processor -- start 2024-06-15T01:18:52.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.678+0000 7f0ed981a700 1 -- start start 2024-06-15T01:18:52.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed981a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed981a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4101770 0x7f0ed4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed981a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed41972a0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed981a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4199fd0 con 0x7f0ed4101770 2024-06-15T01:18:52.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39178/0 (socket says 172.21.15.137:39178) 2024-06-15T01:18:52.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 -- 172.21.15.137:0/1114020799 learned_addr learned my addr 172.21.15.137:0/1114020799 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:52.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed27fc700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4101770 0x7f0ed4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 -- 172.21.15.137:0/1114020799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4101770 msgr2=0x7f0ed4196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4101770 0x7f0ed4196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 -- 172.21.15.137:0/1114020799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80095f0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed2ffd700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d8d0 tx=0x7f0ec400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:52.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec40098a0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ec4010460 con 0x7f0ed4100dc0 2024-06-15T01:18:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.679+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed419a2b0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.680+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec4009ea0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.680+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed419a7d0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.680+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed41091c0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.681+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0ec40105d0 con 0x7f0ed4100dc0 2024-06-15T01:18:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.681+0000 7f0ed8818700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 0x7f0ebc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.682+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0ec408a180 con 0x7f0ed4100dc0 2024-06-15T01:18:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.682+0000 7f0ed27fc700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 0x7f0ebc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.682+0000 7f0ed27fc700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 0x7f0ebc06d440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4197df0 tx=0x7f0ec801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.686+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ec4055030 con 0x7f0ed4100dc0 2024-06-15T01:18:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.894+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ed4197bb0 con 0x7f0ebc06af80 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.896+0000 7f0ed8818700 1 -- 172.21.15.137:0/1114020799 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ed4197bb0 con 0x7f0ebc06af80 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:18:52.884 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:18:52.885 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:18:52.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 msgr2=0x7f0ebc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 0x7f0ebc06d440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4197df0 tx=0x7f0ec801a040 comp rx=0 tx=0).stop 2024-06-15T01:18:52.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 msgr2=0x7f0ed41966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d8d0 tx=0x7f0ec400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 shutdown_connections 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ebc06af80 0x7f0ebc06d440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed4100dc0 0x7f0ed41966d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 --2- 172.21.15.137:0/1114020799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed4101770 0x7f0ed4196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 >> 172.21.15.137:0/1114020799 conn(0x7f0ed40fc650 msgr2=0x7f0ed40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 shutdown_connections 2024-06-15T01:18:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:52.898+0000 7f0ed981a700 1 -- 172.21.15.137:0/1114020799 wait complete. 2024-06-15T01:18:52.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 -- 172.21.15.137:0/3533658545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 msgr2=0x7f67701011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 --2- 172.21.15.137:0/3533658545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f67701011a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6760009910 tx=0x7f6760009c20 comp rx=0 tx=0).stop 2024-06-15T01:18:52.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 -- 172.21.15.137:0/3533658545 shutdown_connections 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 --2- 172.21.15.137:0/3533658545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 0x7f67701057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 --2- 172.21.15.137:0/3533658545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f67701011a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.004+0000 7f6776326700 1 -- 172.21.15.137:0/3533658545 >> 172.21.15.137:0/3533658545 conn(0x7f67700fc650 msgr2=0x7f67700fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.005+0000 7f6776326700 1 -- 172.21.15.137:0/3533658545 shutdown_connections 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.005+0000 7f6776326700 1 -- 172.21.15.137:0/3533658545 wait complete. 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.005+0000 7f6776326700 1 Processor -- start 2024-06-15T01:18:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.005+0000 7f6776326700 1 -- start start 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 0x7f67700735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770073b10 con 0x7f6770100dc0 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770073c50 con 0x7f6770101770 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39186/0 (socket says 172.21.15.137:39186) 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 -- 172.21.15.137:0/713549050 learned_addr learned my addr 172.21.15.137:0/713549050 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676f7fe700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 0x7f67700735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 -- 172.21.15.137:0/713549050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 msgr2=0x7f67700735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 0x7f67700735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 -- 172.21.15.137:0/713549050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600095f0 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676ffff700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6760006010 tx=0x7f676000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f676001d070 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f676000f460 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6760021640 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6770073ed0 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.006+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701a2b90 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.007+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770109160 con 0x7f6770100dc0 2024-06-15T01:18:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.008+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f676000fa60 con 0x7f6770100dc0 2024-06-15T01:18:52.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.008+0000 7f676d7fa700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 0x7f675806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:18:52.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.008+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f676008b670 con 0x7f6770100dc0 2024-06-15T01:18:52.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.008+0000 7f676f7fe700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 0x7f675806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:18:52.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.009+0000 7f676f7fe700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 0x7f675806d440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6770074880 tx=0x7f6764005eb0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:18:53.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.012+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f67600564a0 con 0x7f6770100dc0 2024-06-15T01:18:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.309+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f677004ea90 con 0x7f6770100dc0 2024-06-15T01:18:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.309+0000 7f676d7fa700 1 -- 172.21.15.137:0/713549050 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f6760059ac0 con 0x7f6770100dc0 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:18:53.298 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:18:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 msgr2=0x7f675806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 0x7f675806d440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6770074880 tx=0x7f6764005eb0 comp rx=0 tx=0).stop 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 msgr2=0x7f6770073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6760006010 tx=0x7f676000be20 comp rx=0 tx=0).stop 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 shutdown_connections 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f675806af80 0x7f675806d440 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6770100dc0 0x7f6770073090 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 --2- 172.21.15.137:0/713549050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6770101770 0x7f67700735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 >> 172.21.15.137:0/713549050 conn(0x7f67700fc650 msgr2=0x7f67700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 shutdown_connections 2024-06-15T01:18:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:18:53.311+0000 7f6776326700 1 -- 172.21.15.137:0/713549050 wait complete. 2024-06-15T01:18:53.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:53 smithi142 ceph-mon[82136]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:53.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:53 smithi142 ceph-mon[82136]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:53.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:53 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2160163972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:18:53.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:53 smithi137 ceph-mon[112295]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:53.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:53 smithi137 ceph-mon[112295]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:53.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:53 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2160163972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:18:54.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:54 smithi142 ceph-mon[82136]: pgmap v384: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:54.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:54 smithi142 ceph-mon[82136]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:54.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:54 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/713549050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:18:54.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:54 smithi137 ceph-mon[112295]: pgmap v384: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:54.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:54 smithi137 ceph-mon[112295]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:18:54.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:54 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/713549050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:18:56.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:56 smithi142 ceph-mon[82136]: pgmap v385: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:56.577 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:56 smithi137 ceph-mon[112295]: pgmap v385: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:58.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:18:58 smithi142 ceph-mon[82136]: pgmap v386: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:18:58.718 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:18:58 smithi137 ceph-mon[112295]: pgmap v386: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:00.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:00 smithi142 ceph-mon[82136]: pgmap v387: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:00.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:00 smithi137 ceph-mon[112295]: pgmap v387: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:02.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:02 smithi142 ceph-mon[82136]: pgmap v388: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:02.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:02 smithi137 ceph-mon[112295]: pgmap v388: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:03.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:03.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:04.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:04 smithi142 ceph-mon[82136]: pgmap v389: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:04.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:04 smithi137 ceph-mon[112295]: pgmap v389: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:06.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:06 smithi142 ceph-mon[82136]: pgmap v390: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:06.667 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:06 smithi137 ceph-mon[112295]: pgmap v390: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:08.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:08 smithi142 ceph-mon[82136]: pgmap v391: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:08.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:08 smithi137 ceph-mon[112295]: pgmap v391: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:10.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:10 smithi142 ceph-mon[82136]: pgmap v392: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:10.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:10 smithi137 ceph-mon[112295]: pgmap v392: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:12.559 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:12 smithi142 ceph-mon[82136]: pgmap v393: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:12.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:12 smithi137 ceph-mon[112295]: pgmap v393: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:13.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:13 smithi137 ceph-mon[112295]: pgmap v394: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:13.809 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:13 smithi142 ceph-mon[82136]: pgmap v394: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:16.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:15 smithi137 ceph-mon[112295]: pgmap v395: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:16.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:15 smithi142 ceph-mon[82136]: pgmap v395: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:18.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:17 smithi137 ceph-mon[112295]: pgmap v396: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:18.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:17 smithi142 ceph-mon[82136]: pgmap v396: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:19.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:19.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:20.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:19 smithi137 ceph-mon[112295]: pgmap v397: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:20.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:19 smithi142 ceph-mon[82136]: pgmap v397: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:22.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:21 smithi137 ceph-mon[112295]: pgmap v398: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:22.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:21 smithi142 ceph-mon[82136]: pgmap v398: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 -- 172.21.15.137:0/1691131431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 msgr2=0x7fa3d81033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/1691131431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d81033c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009910 tx=0x7fa3c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 -- 172.21.15.137:0/1691131431 shutdown_connections 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/1691131431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 0x7fa3d81079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/1691131431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d81033c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 -- 172.21.15.137:0/1691131431 >> 172.21.15.137:0/1691131431 conn(0x7fa3d80fe870 msgr2=0x7fa3d8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 -- 172.21.15.137:0/1691131431 shutdown_connections 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.429+0000 7fa3de6fa700 1 -- 172.21.15.137:0/1691131431 wait complete. 2024-06-15T01:19:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 Processor -- start 2024-06-15T01:19:23.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 -- start start 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 0x7fa3d819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f2a0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3de6fa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8198750 con 0x7fa3d8103990 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52344/0 (socket says 172.21.15.137:52344) 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3d7fff700 1 -- 172.21.15.137:0/512925152 learned_addr learned my addr 172.21.15.137:0/512925152 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.430+0000 7fa3d77fe700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 0x7fa3d819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d7fff700 1 -- 172.21.15.137:0/512925152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 msgr2=0x7fa3d819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d7fff700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 0x7fa3d819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d7fff700 1 -- 172.21.15.137:0/512925152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80095f0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d7fff700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80092e0 tx=0x7fa3c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c801d070 con 0x7fa3d8102fe0 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3c8005e00 con 0x7fa3d8102fe0 2024-06-15T01:19:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d81989d0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c800fe90 con 0x7fa3d8102fe0 2024-06-15T01:19:23.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.431+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d8198ec0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.432+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d810b380 con 0x7fa3d8102fe0 2024-06-15T01:19:23.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.432+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa3c800f6d0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.433+0000 7fa3d57fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 0x7fa3c006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.433+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa3c808b5c0 con 0x7fa3d8102fe0 2024-06-15T01:19:23.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.433+0000 7fa3d77fe700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 0x7fa3c006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.433+0000 7fa3d77fe700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 0x7fa3c006d440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8199bc0 tx=0x7fa3cc0073d0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:23.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.437+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa3c8059e80 con 0x7fa3d8102fe0 2024-06-15T01:19:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.651+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d802cce0 con 0x7fa3c006af80 2024-06-15T01:19:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.653+0000 7fa3d57fa700 1 -- 172.21.15.137:0/512925152 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa3d802cce0 con 0x7fa3c006af80 2024-06-15T01:19:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 msgr2=0x7fa3c006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 0x7fa3c006d440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8199bc0 tx=0x7fa3cc0073d0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 msgr2=0x7fa3d819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80092e0 tx=0x7fa3c800bd70 comp rx=0 tx=0).stop 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 shutdown_connections 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3c006af80 0x7fa3c006d440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d8102fe0 0x7fa3d819e6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 --2- 172.21.15.137:0/512925152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3d8103990 0x7fa3d819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 >> 172.21.15.137:0/512925152 conn(0x7fa3d80fe870 msgr2=0x7fa3d80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 shutdown_connections 2024-06-15T01:19:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.655+0000 7fa3de6fa700 1 -- 172.21.15.137:0/512925152 wait complete. 2024-06-15T01:19:23.654 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 -- 172.21.15.137:0/3316594682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64102660 msgr2=0x7fbd6410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 --2- 172.21.15.137:0/3316594682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64102660 0x7fbd6410ab50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd60009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 -- 172.21.15.137:0/3316594682 shutdown_connections 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 --2- 172.21.15.137:0/3316594682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64102660 0x7fbd6410ab50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 --2- 172.21.15.137:0/3316594682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64101d40 0x7fbd64102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 -- 172.21.15.137:0/3316594682 >> 172.21.15.137:0/3316594682 conn(0x7fbd640fb550 msgr2=0x7fbd640fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 -- 172.21.15.137:0/3316594682 shutdown_connections 2024-06-15T01:19:23.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.762+0000 7fbd6b941700 1 -- 172.21.15.137:0/3316594682 wait complete. 2024-06-15T01:19:23.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 Processor -- start 2024-06-15T01:19:23.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 -- start start 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64102660 0x7fbd64196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64197290 con 0x7fbd64101d40 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd6b941700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64199fc0 con 0x7fbd64102660 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd696dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd696dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52362/0 (socket says 172.21.15.137:52362) 2024-06-15T01:19:23.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.763+0000 7fbd696dd700 1 -- 172.21.15.137:0/4281452140 learned_addr learned my addr 172.21.15.137:0/4281452140 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd68edc700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64102660 0x7fbd64196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd696dd700 1 -- 172.21.15.137:0/4281452140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64102660 msgr2=0x7fbd64196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd696dd700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64102660 0x7fbd64196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd696dd700 1 -- 172.21.15.137:0/4281452140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600095f0 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd696dd700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8a0 tx=0x7fbd5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5800cea0 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6419a2a0 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd58004830 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd58005500 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.764+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6419a7c0 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.765+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd64108250 con 0x7fbd64101d40 2024-06-15T01:19:23.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.765+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbd580049a0 con 0x7fbd64101d40 2024-06-15T01:19:23.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.766+0000 7fbd567fc700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 0x7fbd5006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:23.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.766+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbd58089fd0 con 0x7fbd64101d40 2024-06-15T01:19:23.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.766+0000 7fbd68edc700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 0x7fbd5006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:23.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.766+0000 7fbd68edc700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 0x7fbd5006d440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd64197de0 tx=0x7fbd6001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:23.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.770+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbd58054e80 con 0x7fbd64101d40 2024-06-15T01:19:23.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.982+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd6419aa90 con 0x7fbd5006af80 2024-06-15T01:19:23.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.983+0000 7fbd567fc700 1 -- 172.21.15.137:0/4281452140 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbd6419aa90 con 0x7fbd5006af80 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 msgr2=0x7fbd5006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 0x7fbd5006d440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd64197de0 tx=0x7fbd6001a040 comp rx=0 tx=0).stop 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 msgr2=0x7fbd641966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8a0 tx=0x7fbd5800cc60 comp rx=0 tx=0).stop 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 shutdown_connections 2024-06-15T01:19:23.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbd5006af80 0x7fbd5006d440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd64101d40 0x7fbd641966c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 --2- 172.21.15.137:0/4281452140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd64102660 0x7fbd64196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:23.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.985+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 >> 172.21.15.137:0/4281452140 conn(0x7fbd640fb550 msgr2=0x7fbd64105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:23.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.986+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 shutdown_connections 2024-06-15T01:19:23.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:23.986+0000 7fbd6b941700 1 -- 172.21.15.137:0/4281452140 wait complete. 2024-06-15T01:19:24.059 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:23 smithi142 ceph-mon[82136]: pgmap v399: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:24.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 -- 172.21.15.137:0/2435164283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 msgr2=0x7f4c14100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/2435164283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c14100050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009910 tx=0x7f4c04009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:24.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 -- 172.21.15.137:0/2435164283 shutdown_connections 2024-06-15T01:19:24.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/2435164283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c14100050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/2435164283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 0x7f4c14105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 -- 172.21.15.137:0/2435164283 >> 172.21.15.137:0/2435164283 conn(0x7f4c140755a0 msgr2=0x7f4c140759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:24.079 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:23 smithi137 ceph-mon[112295]: pgmap v399: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 -- 172.21.15.137:0/2435164283 shutdown_connections 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.090+0000 7f4c1a99e700 1 -- 172.21.15.137:0/2435164283 wait complete. 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 Processor -- start 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 -- start start 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 0x7f4c141979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14198010 con 0x7f4c140ffbd0 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c1a99e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14198150 con 0x7f4c14105a40 2024-06-15T01:19:24.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52380/0 (socket says 172.21.15.137:52380) 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c13fff700 1 -- 172.21.15.137:0/162301584 learned_addr learned my addr 172.21.15.137:0/162301584 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.091+0000 7f4c137fe700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 0x7f4c141979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c13fff700 1 -- 172.21.15.137:0/162301584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 msgr2=0x7f4c141979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c13fff700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 0x7f4c141979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c13fff700 1 -- 172.21.15.137:0/162301584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040095f0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c13fff700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d8d0 tx=0x7f4c0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c08009940 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c141a0520 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c08010460 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0800f5f0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.092+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c141a09e0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.093+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14107fd0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.094+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4c080105d0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.094+0000 7f4c117fa700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 0x7f4bfc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.094+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4c0808a3e0 con 0x7f4c140ffbd0 2024-06-15T01:19:24.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.094+0000 7f4c137fe700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 0x7f4bfc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.095+0000 7f4c137fe700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 0x7f4bfc06d440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c14068cc0 tx=0x7f4c0401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.098+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4c08055210 con 0x7f4c140ffbd0 2024-06-15T01:19:24.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.299+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c1402cd80 con 0x7f4bfc06af80 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.309+0000 7f4c117fa700 1 -- 172.21.15.137:0/162301584 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4c1402cd80 con 0x7f4bfc06af80 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (9m) 4m ago 20m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (20m) 4m ago 20m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (19m) 3m ago 19m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (8m) 4m ago 20m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (8m) 3m ago 19m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (9m) 4m ago 20m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (4m) 4m ago 14m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:19:24.297 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (4m) 4m ago 15m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (4m) 3m ago 14m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (4m) 3m ago 14m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (11m) 4m ago 22m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (10m) 3m ago 19m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (8m) 4m ago 22m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (8m) 3m ago 19m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (10m) 4m ago 20m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (10m) 3m ago 19m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (7m) 4m ago 18m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (7m) 4m ago 18m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (6m) 4m ago 17m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (6m) 3m ago 17m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (5m) 3m ago 16m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:19:24.298 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (5m) 3m ago 16m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:19:24.299 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (10m) 4m ago 19m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:19:24.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 msgr2=0x7f4bfc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 0x7f4bfc06d440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c14068cc0 tx=0x7f4c0401a040 comp rx=0 tx=0).stop 2024-06-15T01:19:24.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 msgr2=0x7f4c141974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d8d0 tx=0x7f4c0800dc90 comp rx=0 tx=0).stop 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 shutdown_connections 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bfc06af80 0x7f4bfc06d440 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c140ffbd0 0x7f4c141974b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 --2- 172.21.15.137:0/162301584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c14105a40 0x7f4c141979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 >> 172.21.15.137:0/162301584 conn(0x7f4c140755a0 msgr2=0x7f4c140fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 shutdown_connections 2024-06-15T01:19:24.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.311+0000 7f4c1a99e700 1 -- 172.21.15.137:0/162301584 wait complete. 2024-06-15T01:19:24.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 -- 172.21.15.137:0/3305858892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 msgr2=0x7f63e4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/3305858892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e4102070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009910 tx=0x7f63d4009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:24.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 -- 172.21.15.137:0/3305858892 shutdown_connections 2024-06-15T01:19:24.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/3305858892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 0x7f63e410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/3305858892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e4102070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 -- 172.21.15.137:0/3305858892 >> 172.21.15.137:0/3305858892 conn(0x7f63e40fb550 msgr2=0x7f63e40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:24.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 -- 172.21.15.137:0/3305858892 shutdown_connections 2024-06-15T01:19:24.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.414+0000 7f63e8cbe700 1 -- 172.21.15.137:0/3305858892 wait complete. 2024-06-15T01:19:24.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 Processor -- start 2024-06-15T01:19:24.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 -- start start 2024-06-15T01:19:24.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 0x7f63e40ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4100400 con 0x7f63e4101c90 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e8cbe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4100540 con 0x7f63e4102640 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52390/0 (socket says 172.21.15.137:52390) 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e259c700 1 -- 172.21.15.137:0/2477576986 learned_addr learned my addr 172.21.15.137:0/2477576986 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.415+0000 7f63e1d9b700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 0x7f63e40ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63e259c700 1 -- 172.21.15.137:0/2477576986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 msgr2=0x7f63e40ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63e259c700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 0x7f63e40ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63e259c700 1 -- 172.21.15.137:0/2477576986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40095f0 con 0x7f63e4101c90 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63e259c700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006010 tx=0x7f63d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d401d070 con 0x7f63e4101c90 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63d4005710 con 0x7f63e4101c90 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e41007c0 con 0x7f63e4101c90 2024-06-15T01:19:24.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.416+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d400fe20 con 0x7f63e4101c90 2024-06-15T01:19:24.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.417+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e41a2af0 con 0x7f63e4101c90 2024-06-15T01:19:24.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.418+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f63d4003a60 con 0x7f63e4101c90 2024-06-15T01:19:24.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.418+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4108230 con 0x7f63e4101c90 2024-06-15T01:19:24.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.418+0000 7f63d37fe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 0x7f63cc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.418+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f63d408b3b0 con 0x7f63e4101c90 2024-06-15T01:19:24.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.418+0000 7f63e1d9b700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 0x7f63cc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.419+0000 7f63e1d9b700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 0x7f63cc06d440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63e4101170 tx=0x7f63d800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.423+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f63d40561e0 con 0x7f63e4101c90 2024-06-15T01:19:24.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.694+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f63e402cc70 con 0x7f63e4101c90 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.695+0000 7f63d37fe700 1 -- 172.21.15.137:0/2477576986 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f63d4059800 con 0x7f63e4101c90 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:19:24.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:19:24.684 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:19:24.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 msgr2=0x7f63cc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 0x7f63cc06d440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63e4101170 tx=0x7f63d800b410 comp rx=0 tx=0).stop 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 msgr2=0x7f63e40ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006010 tx=0x7f63d400bd70 comp rx=0 tx=0).stop 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 shutdown_connections 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f63cc06af80 0x7f63cc06d440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e4101c90 0x7f63e40ff980 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 --2- 172.21.15.137:0/2477576986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e4102640 0x7f63e40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 >> 172.21.15.137:0/2477576986 conn(0x7f63e40fb550 msgr2=0x7f63e4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 shutdown_connections 2024-06-15T01:19:24.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.697+0000 7f63e8cbe700 1 -- 172.21.15.137:0/2477576986 wait complete. 2024-06-15T01:19:24.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 -- 172.21.15.137:0/3277027926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ff500 msgr2=0x7f7b380ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3277027926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ff500 0x7f7b380ff8e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009910 tx=0x7f7b2c009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 -- 172.21.15.137:0/3277027926 shutdown_connections 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3277027926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ffeb0 0x7f7b38103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3277027926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ff500 0x7f7b380ff8e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 -- 172.21.15.137:0/3277027926 >> 172.21.15.137:0/3277027926 conn(0x7f7b38074d80 msgr2=0x7f7b38075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 -- 172.21.15.137:0/3277027926 shutdown_connections 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.801+0000 7f7b3f189700 1 -- 172.21.15.137:0/3277027926 wait complete. 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 Processor -- start 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 -- start start 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 0x7f7b38196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38197260 con 0x7f7b380ffeb0 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b3f189700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38199f90 con 0x7f7b380ff500 2024-06-15T01:19:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52412/0 (socket says 172.21.15.137:52412) 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.802+0000 7f7b37fff700 1 -- 172.21.15.137:0/3865658781 learned_addr learned my addr 172.21.15.137:0/3865658781 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b3cf25700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 0x7f7b38196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b37fff700 1 -- 172.21.15.137:0/3865658781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 msgr2=0x7f7b38196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b37fff700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 0x7f7b38196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b37fff700 1 -- 172.21.15.137:0/3865658781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0095f0 con 0x7f7b380ffeb0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b3cf25700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 0x7f7b38196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b37fff700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400dd70 tx=0x7f7b2400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2400ad50 con 0x7f7b380ffeb0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b2400e8f0 con 0x7f7b380ffeb0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3819a270 con 0x7f7b380ffeb0 2024-06-15T01:19:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b24016810 con 0x7f7b380ffeb0 2024-06-15T01:19:24.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.803+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3819a7c0 con 0x7f7b380ffeb0 2024-06-15T01:19:24.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.804+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3804ea90 con 0x7f7b380ffeb0 2024-06-15T01:19:24.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.804+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7b2400ea60 con 0x7f7b380ffeb0 2024-06-15T01:19:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.805+0000 7f7b35ffb700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 0x7f7b2806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.805+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7b2408a310 con 0x7f7b380ffeb0 2024-06-15T01:19:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.805+0000 7f7b3cf25700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 0x7f7b2806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.805+0000 7f7b3cf25700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 0x7f7b2806d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c006010 tx=0x7f7b2c00b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:24.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:24.809+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7b24055270 con 0x7f7b380ffeb0 2024-06-15T01:19:24.967 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:24 smithi137 ceph-mon[112295]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:24.967 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:24 smithi137 ceph-mon[112295]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:24.967 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:24 smithi137 ceph-mon[112295]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:24.967 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:24 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2477576986' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:19:25.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.045+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7b3819ab90 con 0x7f7b380ffeb0 2024-06-15T01:19:25.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.045+0000 7f7b35ffb700 1 -- 172.21.15.137:0/3865658781 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f7b24058890 con 0x7f7b380ffeb0 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:19:25.034 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:19:25.035 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:25.036 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:25.037 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:25.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 msgr2=0x7f7b2806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 0x7f7b2806d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c006010 tx=0x7f7b2c00b540 comp rx=0 tx=0).stop 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 msgr2=0x7f7b38196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400dd70 tx=0x7f7b2400a530 comp rx=0 tx=0).stop 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 shutdown_connections 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7b2806af80 0x7f7b2806d440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b380ff500 0x7f7b38196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 --2- 172.21.15.137:0/3865658781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b380ffeb0 0x7f7b38196bd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 >> 172.21.15.137:0/3865658781 conn(0x7f7b38074d80 msgr2=0x7f7b380fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 shutdown_connections 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.048+0000 7f7b3f189700 1 -- 172.21.15.137:0/3865658781 wait complete. 2024-06-15T01:19:25.038 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 -- 172.21.15.137:0/2568757123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b80ffa40 msgr2=0x7fb5b80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2568757123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b80ffa40 0x7fb5b80ffec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009910 tx=0x7fb5a8009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 -- 172.21.15.137:0/2568757123 shutdown_connections 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2568757123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b80ffa40 0x7fb5b80ffec0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2568757123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b8105a40 0x7fb5b8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 -- 172.21.15.137:0/2568757123 >> 172.21.15.137:0/2568757123 conn(0x7fb5b80fb550 msgr2=0x7fb5b80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 -- 172.21.15.137:0/2568757123 shutdown_connections 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.154+0000 7fb5bd767700 1 -- 172.21.15.137:0/2568757123 wait complete. 2024-06-15T01:19:25.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 Processor -- start 2024-06-15T01:19:25.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 -- start start 2024-06-15T01:19:25.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 0x7fb5b8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b8197260 con 0x7fb5b8105a40 2024-06-15T01:19:25.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5bd767700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b819aff0 con 0x7fb5b80ffa40 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5b67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5b67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52436/0 (socket says 172.21.15.137:52436) 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5b67fc700 1 -- 172.21.15.137:0/2215279000 learned_addr learned my addr 172.21.15.137:0/2215279000 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.155+0000 7fb5b6ffd700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 0x7fb5b8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5b67fc700 1 -- 172.21.15.137:0/2215279000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 msgr2=0x7fb5b8196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5b67fc700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 0x7fb5b8196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5b67fc700 1 -- 172.21.15.137:0/2215279000 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80095f0 con 0x7fb5b8105a40 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5b6ffd700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 0x7fb5b8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5b67fc700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a801d070 con 0x7fb5b8105a40 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5a800f460 con 0x7fb5b8105a40 2024-06-15T01:19:25.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a8021620 con 0x7fb5b8105a40 2024-06-15T01:19:25.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b819b270 con 0x7fb5b8105a40 2024-06-15T01:19:25.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.156+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b819b710 con 0x7fb5b8105a40 2024-06-15T01:19:25.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.157+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b8108fe0 con 0x7fb5b8105a40 2024-06-15T01:19:25.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.157+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb5a800fa40 con 0x7fb5b8105a40 2024-06-15T01:19:25.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.158+0000 7fb59ffff700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 0x7fb5a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.158+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb5a808b5b0 con 0x7fb5b8105a40 2024-06-15T01:19:25.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.158+0000 7fb5b6ffd700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 0x7fb5a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.158+0000 7fb5b6ffd700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 0x7fb5a006d440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00bc10 tx=0x7fb5ac00b5f0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:25.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.162+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb5a80563e0 con 0x7fb5b8105a40 2024-06-15T01:19:25.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:24 smithi142 ceph-mon[82136]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:25.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:24 smithi142 ceph-mon[82136]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:25.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:24 smithi142 ceph-mon[82136]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:25.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:24 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2477576986' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:19:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.370+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5b802cd60 con 0x7fb5a006af80 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.372+0000 7fb59ffff700 1 -- 172.21.15.137:0/2215279000 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb5b802cd60 con 0x7fb5a006af80 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:19:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:19:25.361 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:19:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 msgr2=0x7fb5a006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 0x7fb5a006d440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00bc10 tx=0x7fb5ac00b5f0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 msgr2=0x7fb5b8196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8006010 tx=0x7fb5a800be20 comp rx=0 tx=0).stop 2024-06-15T01:19:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 shutdown_connections 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb5a006af80 0x7fb5a006d440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5b80ffa40 0x7fb5b8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 --2- 172.21.15.137:0/2215279000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5b8105a40 0x7fb5b8196b80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.374+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 >> 172.21.15.137:0/2215279000 conn(0x7fb5b80fb550 msgr2=0x7fb5b80690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.375+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 shutdown_connections 2024-06-15T01:19:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.375+0000 7fb5bd767700 1 -- 172.21.15.137:0/2215279000 wait complete. 2024-06-15T01:19:25.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 -- 172.21.15.137:0/3635003689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100360 msgr2=0x7fcb00100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 --2- 172.21.15.137:0/3635003689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100360 0x7fcb00100740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009910 tx=0x7fcafc009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 -- 172.21.15.137:0/3635003689 shutdown_connections 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 --2- 172.21.15.137:0/3635003689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100d10 0x7fcb001046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 --2- 172.21.15.137:0/3635003689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100360 0x7fcb00100740 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 -- 172.21.15.137:0/3635003689 >> 172.21.15.137:0/3635003689 conn(0x7fcb00075e30 msgr2=0x7fcb000fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 -- 172.21.15.137:0/3635003689 shutdown_connections 2024-06-15T01:19:25.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.479+0000 7fcb07675700 1 -- 172.21.15.137:0/3635003689 wait complete. 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 Processor -- start 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 -- start start 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100360 0x7fcb001973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00197f50 con 0x7fcb00100d10 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb07675700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00198090 con 0x7fcb00100360 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb04c10700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb04c10700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52452/0 (socket says 172.21.15.137:52452) 2024-06-15T01:19:25.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb04c10700 1 -- 172.21.15.137:0/1660589445 learned_addr learned my addr 172.21.15.137:0/1660589445 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.480+0000 7fcb05411700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100360 0x7fcb001973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb04c10700 1 -- 172.21.15.137:0/1660589445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100360 msgr2=0x7fcb001973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb04c10700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100360 0x7fcb001973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb04c10700 1 -- 172.21.15.137:0/1660589445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0095f0 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb04c10700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000e920 tx=0x7fcaf000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf000c9d0 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcaf0010430 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf0018650 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb001a0460 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.481+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb00195260 con 0x7fcb00100d10 2024-06-15T01:19:25.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.482+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00108010 con 0x7fcb00100d10 2024-06-15T01:19:25.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.483+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcaf00105a0 con 0x7fcb00100d10 2024-06-15T01:19:25.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.483+0000 7fcaf67fc700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 0x7fcaec071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:25.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.483+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcaf0014070 con 0x7fcb00100d10 2024-06-15T01:19:25.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.483+0000 7fcb05411700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 0x7fcaec071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:25.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.483+0000 7fcb05411700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 0x7fcaec071850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009fd0 tx=0x7fcafc005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:25.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.487+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcaf0052900 con 0x7fcb00100d10 2024-06-15T01:19:25.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.755+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcb0004ea90 con 0x7fcb00100d10 2024-06-15T01:19:25.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.755+0000 7fcaf67fc700 1 -- 172.21.15.137:0/1660589445 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fcaf0055480 con 0x7fcb00100d10 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:19:25.744 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:19:25.745 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:19:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.757+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 msgr2=0x7fcaec071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.757+0000 7fcb07675700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 0x7fcaec071850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009fd0 tx=0x7fcafc005940 comp rx=0 tx=0).stop 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.757+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 msgr2=0x7fcb00197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.757+0000 7fcb07675700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000e920 tx=0x7fcaf000ec30 comp rx=0 tx=0).stop 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.757+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 shutdown_connections 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaec06f390 0x7fcaec071850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb00100360 0x7fcb001973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 --2- 172.21.15.137:0/1660589445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb00100d10 0x7fcb00197930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 >> 172.21.15.137:0/1660589445 conn(0x7fcb00075e30 msgr2=0x7fcb000fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 shutdown_connections 2024-06-15T01:19:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:25.758+0000 7fcb07675700 1 -- 172.21.15.137:0/1660589445 wait complete. 2024-06-15T01:19:26.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:25 smithi137 ceph-mon[112295]: pgmap v400: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:26.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:25 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3865658781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:19:26.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:25 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1660589445' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:19:26.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:25 smithi142 ceph-mon[82136]: pgmap v400: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:26.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:25 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3865658781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:19:26.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:25 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1660589445' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:19:27.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:26 smithi137 ceph-mon[112295]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:27.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:26 smithi142 ceph-mon[82136]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:28.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:27 smithi137 ceph-mon[112295]: pgmap v401: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:28.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:27 smithi142 ceph-mon[82136]: pgmap v401: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:29 smithi137 ceph-mon[112295]: pgmap v402: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:30.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:29 smithi142 ceph-mon[82136]: pgmap v402: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:32.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:31 smithi137 ceph-mon[112295]: pgmap v403: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:32.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:31 smithi142 ceph-mon[82136]: pgmap v403: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:34.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:33 smithi142 ceph-mon[82136]: pgmap v404: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:34.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:34.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:33 smithi137 ceph-mon[112295]: pgmap v404: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:34.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:36.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:35 smithi137 ceph-mon[112295]: pgmap v405: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:36.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:35 smithi142 ceph-mon[82136]: pgmap v405: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:38.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:37 smithi142 ceph-mon[82136]: pgmap v406: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:38.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:37 smithi137 ceph-mon[112295]: pgmap v406: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:40.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:39 smithi137 ceph-mon[112295]: pgmap v407: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:40.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:39 smithi142 ceph-mon[82136]: pgmap v407: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:42.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:41 smithi137 ceph-mon[112295]: pgmap v408: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:42.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:41 smithi142 ceph-mon[82136]: pgmap v408: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:44.059 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:43 smithi142 ceph-mon[82136]: pgmap v409: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:44.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:43 smithi137 ceph-mon[112295]: pgmap v409: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:46.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:45 smithi137 ceph-mon[112295]: pgmap v410: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:46.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:45 smithi142 ceph-mon[82136]: pgmap v410: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:48.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:47 smithi137 ceph-mon[112295]: pgmap v411: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:48.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:47 smithi142 ceph-mon[82136]: pgmap v411: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:49.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:19:50.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:49 smithi137 ceph-mon[112295]: pgmap v412: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:50.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:49 smithi142 ceph-mon[82136]: pgmap v412: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:52.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:51 smithi137 ceph-mon[112295]: pgmap v413: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:52.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:51 smithi142 ceph-mon[82136]: pgmap v413: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:54.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:53 smithi137 ceph-mon[112295]: pgmap v414: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:54.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:53 smithi142 ceph-mon[82136]: pgmap v414: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:55.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 -- 172.21.15.137:0/3156442209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 msgr2=0x7f13740ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:55.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 --2- 172.21.15.137:0/3156442209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f13740ffeb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1360009910 tx=0x7f1360009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 -- 172.21.15.137:0/3156442209 shutdown_connections 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 --2- 172.21.15.137:0/3156442209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f13740ffeb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 --2- 172.21.15.137:0/3156442209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 0x7f1374105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.875+0000 7f137b15f700 1 -- 172.21.15.137:0/3156442209 >> 172.21.15.137:0/3156442209 conn(0x7f13740fb520 msgr2=0x7f13740fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 -- 172.21.15.137:0/3156442209 shutdown_connections 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 -- 172.21.15.137:0/3156442209 wait complete. 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 Processor -- start 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 -- start start 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 0x7f1374196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374197220 con 0x7f13740ffa30 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.876+0000 7f137b15f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137419afb0 con 0x7f1374105a30 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34784/0 (socket says 172.21.15.137:34784) 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 -- 172.21.15.137:0/2693338662 learned_addr learned my addr 172.21.15.137:0/2693338662 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1373fff700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 0x7f1374196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 -- 172.21.15.137:0/2693338662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 msgr2=0x7f1374196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 0x7f1374196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 -- 172.21.15.137:0/2693338662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600095f0 con 0x7f13740ffa30 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1378efb700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f136800e920 tx=0x7f136800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136800c9d0 con 0x7f13740ffa30 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1368010430 con 0x7f13740ffa30 2024-06-15T01:19:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137419b290 con 0x7f13740ffa30 2024-06-15T01:19:55.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.877+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1368018650 con 0x7f13740ffa30 2024-06-15T01:19:55.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.878+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137419b7e0 con 0x7f13740ffa30 2024-06-15T01:19:55.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.878+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374108fd0 con 0x7f13740ffa30 2024-06-15T01:19:55.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.879+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f13680105a0 con 0x7f13740ffa30 2024-06-15T01:19:55.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.879+0000 7f1371ffb700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 0x7f136406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:55.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.880+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1368014070 con 0x7f13740ffa30 2024-06-15T01:19:55.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.880+0000 7f1373fff700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 0x7f136406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:55.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.880+0000 7f1373fff700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 0x7f136406d440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1374197c20 tx=0x7f1360005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:55.884+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1368052900 con 0x7f13740ffa30 2024-06-15T01:19:56.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.095+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f137419bb10 con 0x7f136406af80 2024-06-15T01:19:56.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.096+0000 7f1371ffb700 1 -- 172.21.15.137:0/2693338662 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f137419bb10 con 0x7f136406af80 2024-06-15T01:19:56.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 msgr2=0x7f136406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 0x7f136406d440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1374197c20 tx=0x7f1360005940 comp rx=0 tx=0).stop 2024-06-15T01:19:56.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 msgr2=0x7f1374196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f136800e920 tx=0x7f136800ec30 comp rx=0 tx=0).stop 2024-06-15T01:19:56.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 shutdown_connections 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f136406af80 0x7f136406d440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13740ffa30 0x7f1374196600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 --2- 172.21.15.137:0/2693338662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1374105a30 0x7f1374196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 >> 172.21.15.137:0/2693338662 conn(0x7f13740fb520 msgr2=0x7f1374069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 shutdown_connections 2024-06-15T01:19:56.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.098+0000 7f137b15f700 1 -- 172.21.15.137:0/2693338662 wait complete. 2024-06-15T01:19:56.096 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:19:56.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 -- 172.21.15.137:0/1937699075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 msgr2=0x7f5108105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 --2- 172.21.15.137:0/1937699075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f5108105e20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009910 tx=0x7f50f8009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:56.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 -- 172.21.15.137:0/1937699075 shutdown_connections 2024-06-15T01:19:56.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 --2- 172.21.15.137:0/1937699075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f5108100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 --2- 172.21.15.137:0/1937699075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f5108105e20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.203+0000 7f510fe52700 1 -- 172.21.15.137:0/1937699075 >> 172.21.15.137:0/1937699075 conn(0x7f51080755a0 msgr2=0x7f51080759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.192 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:55 smithi137 ceph-mon[112295]: pgmap v415: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:56.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.204+0000 7f510fe52700 1 -- 172.21.15.137:0/1937699075 shutdown_connections 2024-06-15T01:19:56.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.204+0000 7f510fe52700 1 -- 172.21.15.137:0/1937699075 wait complete. 2024-06-15T01:19:56.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.204+0000 7f510fe52700 1 Processor -- start 2024-06-15T01:19:56.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.204+0000 7f510fe52700 1 -- start start 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510fe52700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f51081953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510fe52700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510fe52700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108195e30 con 0x7f5108105a40 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510fe52700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108195fa0 con 0x7f51080ffbd0 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58596/0 (socket says 172.21.15.137:58596) 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 -- 172.21.15.137:0/4138764206 learned_addr learned my addr 172.21.15.137:0/4138764206 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510dbee700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f51081953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 -- 172.21.15.137:0/4138764206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 msgr2=0x7f51081953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f51081953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 -- 172.21.15.137:0/4138764206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80095f0 con 0x7f5108105a40 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510dbee700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f51081953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:19:56.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f510d3ed700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f510400b4b0 tx=0x7f510400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5104011800 con 0x7f5108105a40 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.205+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5104011e00 con 0x7f5108105a40 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.206+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51081a2a20 con 0x7f5108105a40 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.206+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5104010e50 con 0x7f5108105a40 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.206+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a2d80 con 0x7f5108105a40 2024-06-15T01:19:56.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.207+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108108040 con 0x7f5108105a40 2024-06-15T01:19:56.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.207+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5104010380 con 0x7f5108105a40 2024-06-15T01:19:56.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.208+0000 7f50feffd700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 0x7f50f406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.208+0000 7f510dbee700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 0x7f50f406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.208+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5104089e40 con 0x7f5108105a40 2024-06-15T01:19:56.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.208+0000 7f510dbee700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 0x7f50f406d440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006010 tx=0x7f50f800b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.212+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5104058780 con 0x7f5108105a40 2024-06-15T01:19:56.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:55 smithi142 ceph-mon[82136]: pgmap v415: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:56.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.419+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f510802cce0 con 0x7f50f406af80 2024-06-15T01:19:56.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.420+0000 7f50feffd700 1 -- 172.21.15.137:0/4138764206 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f510802cce0 con 0x7f50f406af80 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 msgr2=0x7f50f406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 0x7f50f406d440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006010 tx=0x7f50f800b540 comp rx=0 tx=0).stop 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 msgr2=0x7f51081958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f510400b4b0 tx=0x7f510400b870 comp rx=0 tx=0).stop 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 shutdown_connections 2024-06-15T01:19:56.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50f406af80 0x7f50f406d440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51080ffbd0 0x7f51081953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 --2- 172.21.15.137:0/4138764206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5108105a40 0x7f51081958f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 >> 172.21.15.137:0/4138764206 conn(0x7f51080755a0 msgr2=0x7f51080fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 shutdown_connections 2024-06-15T01:19:56.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.422+0000 7f510fe52700 1 -- 172.21.15.137:0/4138764206 wait complete. 2024-06-15T01:19:56.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 -- 172.21.15.137:0/3707759189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6360105a40 msgr2=0x7f6360105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 --2- 172.21.15.137:0/3707759189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6360105a40 0x7f6360105e20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6350009910 tx=0x7f6350009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 -- 172.21.15.137:0/3707759189 shutdown_connections 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 --2- 172.21.15.137:0/3707759189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63600ffbd0 0x7f6360100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 --2- 172.21.15.137:0/3707759189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6360105a40 0x7f6360105e20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 -- 172.21.15.137:0/3707759189 >> 172.21.15.137:0/3707759189 conn(0x7f63600755a0 msgr2=0x7f63600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 -- 172.21.15.137:0/3707759189 shutdown_connections 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.527+0000 7f6365a5a700 1 -- 172.21.15.137:0/3707759189 wait complete. 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 Processor -- start 2024-06-15T01:19:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 -- start start 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6360105a40 0x7f6360196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360197260 con 0x7f63600ffbd0 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6365a5a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636019aff0 con 0x7f6360105a40 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f635effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f635effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58604/0 (socket says 172.21.15.137:58604) 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f635effd700 1 -- 172.21.15.137:0/771213202 learned_addr learned my addr 172.21.15.137:0/771213202 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f6357fff700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6360105a40 0x7f6360196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.528+0000 7f635effd700 1 -- 172.21.15.137:0/771213202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6360105a40 msgr2=0x7f6360196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f635effd700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6360105a40 0x7f6360196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f635effd700 1 -- 172.21.15.137:0/771213202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500095f0 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f635effd700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6350006010 tx=0x7f635000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f635001d070 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f635000f460 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636019b270 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6350021640 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.529+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636019b760 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.530+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6360109050 con 0x7f63600ffbd0 2024-06-15T01:19:56.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.530+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f635000fa20 con 0x7f63600ffbd0 2024-06-15T01:19:56.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.531+0000 7f6364a58700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 0x7f634c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.531+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f635008b550 con 0x7f63600ffbd0 2024-06-15T01:19:56.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.531+0000 7f6357fff700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 0x7f634c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.531+0000 7f6357fff700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 0x7f634c06d440 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6348005fd0 tx=0x7f6348005f60 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.535+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6350056400 con 0x7f63600ffbd0 2024-06-15T01:19:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.734+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f636019ba40 con 0x7f634c06af80 2024-06-15T01:19:56.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.744+0000 7f6364a58700 1 -- 172.21.15.137:0/771213202 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f636019ba40 con 0x7f634c06af80 2024-06-15T01:19:56.732 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:19:56.732 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (10m) 4m ago 21m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:19:56.732 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (21m) 4m ago 21m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:19:56.732 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (20m) 4m ago 20m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (8m) 4m ago 21m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (8m) 4m ago 20m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (10m) 4m ago 20m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (5m) 4m ago 15m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (5m) 4m ago 15m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (4m) 4m ago 15m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (4m) 4m ago 15m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (12m) 4m ago 22m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (11m) 4m ago 20m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (9m) 4m ago 22m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (8m) 4m ago 19m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (11m) 4m ago 21m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:19:56.733 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (11m) 4m ago 20m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (8m) 4m ago 19m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (7m) 4m ago 18m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (7m) 4m ago 18m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (6m) 4m ago 17m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (6m) 4m ago 17m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (5m) 4m ago 17m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:19:56.734 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (10m) 4m ago 20m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 msgr2=0x7f634c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 0x7f634c06d440 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6348005fd0 tx=0x7f6348005f60 comp rx=0 tx=0).stop 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 msgr2=0x7f6360196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6350006010 tx=0x7f635000be20 comp rx=0 tx=0).stop 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 shutdown_connections 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f634c06af80 0x7f634c06d440 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63600ffbd0 0x7f6360196640 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 --2- 172.21.15.137:0/771213202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6360105a40 0x7f6360196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 >> 172.21.15.137:0/771213202 conn(0x7f63600755a0 msgr2=0x7f6360068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 shutdown_connections 2024-06-15T01:19:56.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.746+0000 7f6365a5a700 1 -- 172.21.15.137:0/771213202 wait complete. 2024-06-15T01:19:56.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 -- 172.21.15.137:0/2749144257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c100da0 msgr2=0x7fe81c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 --2- 172.21.15.137:0/2749144257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c100da0 0x7fe81c101180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009910 tx=0x7fe80c009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:56.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 -- 172.21.15.137:0/2749144257 shutdown_connections 2024-06-15T01:19:56.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 --2- 172.21.15.137:0/2749144257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c101750 0x7fe81c1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 --2- 172.21.15.137:0/2749144257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c100da0 0x7fe81c101180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 -- 172.21.15.137:0/2749144257 >> 172.21.15.137:0/2749144257 conn(0x7fe81c0fc650 msgr2=0x7fe81c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:56.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 -- 172.21.15.137:0/2749144257 shutdown_connections 2024-06-15T01:19:56.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.849+0000 7fe822431700 1 -- 172.21.15.137:0/2749144257 wait complete. 2024-06-15T01:19:56.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 Processor -- start 2024-06-15T01:19:56.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 -- start start 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c100da0 0x7fe81c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c19d0d0 con 0x7fe81c101750 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe822431700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c196580 con 0x7fe81c100da0 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.850+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58616/0 (socket says 172.21.15.137:58616) 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 -- 172.21.15.137:0/1154225868 learned_addr learned my addr 172.21.15.137:0/1154225868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81bfff700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c100da0 0x7fe81c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 -- 172.21.15.137:0/1154225868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c100da0 msgr2=0x7fe81c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c100da0 0x7fe81c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 -- 172.21.15.137:0/1154225868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0095f0 con 0x7fe81c101750 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe81b7fe700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe81000c8f0 tx=0x7fe81000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8100078a0 con 0x7fe81c101750 2024-06-15T01:19:56.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe81000f450 con 0x7fe81c101750 2024-06-15T01:19:56.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe810007ea0 con 0x7fe81c101750 2024-06-15T01:19:56.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c196860 con 0x7fe81c101750 2024-06-15T01:19:56.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.851+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c196d60 con 0x7fe81c101750 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.852+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c109140 con 0x7fe81c101750 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.853+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe81000f5c0 con 0x7fe81c101750 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.853+0000 7fe8197fa700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 0x7fe80406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.853+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe81008a290 con 0x7fe81c101750 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.853+0000 7fe81bfff700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 0x7fe80406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.854+0000 7fe81bfff700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 0x7fe80406d440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006010 tx=0x7fe80c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:56.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:56.858+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe810058b50 con 0x7fe81c101750 2024-06-15T01:19:57.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.128+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe81c197030 con 0x7fe81c101750 2024-06-15T01:19:57.116 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:56 smithi137 ceph-mon[112295]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.129+0000 7fe8197fa700 1 -- 172.21.15.137:0/1154225868 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe8100586e0 con 0x7fe81c101750 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:57.117 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:19:57.118 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 msgr2=0x7fe80406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 0x7fe80406d440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006010 tx=0x7fe80c01a040 comp rx=0 tx=0).stop 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 msgr2=0x7fe81c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe81000c8f0 tx=0x7fe81000cc00 comp rx=0 tx=0).stop 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 shutdown_connections 2024-06-15T01:19:57.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80406af80 0x7fe80406d440 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe81c100da0 0x7fe81c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 --2- 172.21.15.137:0/1154225868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe81c101750 0x7fe81c19ca40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 >> 172.21.15.137:0/1154225868 conn(0x7fe81c0fc650 msgr2=0x7fe81c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 shutdown_connections 2024-06-15T01:19:57.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.131+0000 7fe822431700 1 -- 172.21.15.137:0/1154225868 wait complete. 2024-06-15T01:19:57.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 -- 172.21.15.137:0/57861557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320100140 msgr2=0x7fc3201005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 --2- 172.21.15.137:0/57861557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320100140 0x7fc3201005c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc310009910 tx=0x7fc310009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 -- 172.21.15.137:0/57861557 shutdown_connections 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 --2- 172.21.15.137:0/57861557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320100140 0x7fc3201005c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 --2- 172.21.15.137:0/57861557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320104a50 0x7fc320104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 -- 172.21.15.137:0/57861557 >> 172.21.15.137:0/57861557 conn(0x7fc320075e30 msgr2=0x7fc3200fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 -- 172.21.15.137:0/57861557 shutdown_connections 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.237+0000 7fc3287d9700 1 -- 172.21.15.137:0/57861557 wait complete. 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 Processor -- start 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 -- start start 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320100140 0x7fc3201954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201960d0 con 0x7fc320104a50 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc3287d9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320199e60 con 0x7fc320100140 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58632/0 (socket says 172.21.15.137:58632) 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 -- 172.21.15.137:0/3429149335 learned_addr learned my addr 172.21.15.137:0/3429149335 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc326575700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320100140 0x7fc3201954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 -- 172.21.15.137:0/3429149335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320100140 msgr2=0x7fc3201954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320100140 0x7fc3201954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 -- 172.21.15.137:0/3429149335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100095f0 con 0x7fc320104a50 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.238+0000 7fc325d74700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc310006010 tx=0x7fc31000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.239+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc31001d070 con 0x7fc320104a50 2024-06-15T01:19:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.239+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc31000f460 con 0x7fc320104a50 2024-06-15T01:19:57.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.239+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32019a0e0 con 0x7fc320104a50 2024-06-15T01:19:57.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.239+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc310021640 con 0x7fc320104a50 2024-06-15T01:19:57.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.239+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32019a650 con 0x7fc320104a50 2024-06-15T01:19:57.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.240+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc320107f70 con 0x7fc320104a50 2024-06-15T01:19:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.240+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc31000fa60 con 0x7fc320104a50 2024-06-15T01:19:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.241+0000 7fc3177fe700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 0x7fc30c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.241+0000 7fc326575700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 0x7fc30c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.241+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc31008b5f0 con 0x7fc320104a50 2024-06-15T01:19:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.241+0000 7fc326575700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 0x7fc30c06d440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005950 tx=0x7fc31c005cf0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.245+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc3100564a0 con 0x7fc320104a50 2024-06-15T01:19:57.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:56 smithi142 ceph-mon[82136]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.479+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc32004ea90 con 0x7fc320104a50 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.480+0000 7fc3177fe700 1 -- 172.21.15.137:0/3429149335 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fc310059ac0 con 0x7fc320104a50 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:19:57.471 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:19:57.472 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:19:57.473 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:19:57.474 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:57.475 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:57.475 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:57.475 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:19:57.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 msgr2=0x7fc30c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 0x7fc30c06d440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005950 tx=0x7fc31c005cf0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 msgr2=0x7fc3201959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc310006010 tx=0x7fc31000be20 comp rx=0 tx=0).stop 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 shutdown_connections 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc30c06af80 0x7fc30c06d440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc320100140 0x7fc3201954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 --2- 172.21.15.137:0/3429149335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc320104a50 0x7fc3201959f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 >> 172.21.15.137:0/3429149335 conn(0x7fc320075e30 msgr2=0x7fc320103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 shutdown_connections 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.482+0000 7fc3287d9700 1 -- 172.21.15.137:0/3429149335 wait complete. 2024-06-15T01:19:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:19:57.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 -- 172.21.15.137:0/2103022537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 msgr2=0x7feffc0ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 --2- 172.21.15.137:0/2103022537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc0ffeb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefec009910 tx=0x7fefec009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:57.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 -- 172.21.15.137:0/2103022537 shutdown_connections 2024-06-15T01:19:57.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 --2- 172.21.15.137:0/2103022537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc0ffeb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 --2- 172.21.15.137:0/2103022537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feffc105a30 0x7feffc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 -- 172.21.15.137:0/2103022537 >> 172.21.15.137:0/2103022537 conn(0x7feffc0fb540 msgr2=0x7feffc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 -- 172.21.15.137:0/2103022537 shutdown_connections 2024-06-15T01:19:57.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.587+0000 7ff003457700 1 -- 172.21.15.137:0/2103022537 wait complete. 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 Processor -- start 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 -- start start 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feffc105a30 0x7feffc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc19d0d0 con 0x7feffc0ffa30 2024-06-15T01:19:57.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff003457700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc196580 con 0x7feffc105a30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58644/0 (socket says 172.21.15.137:58644) 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 -- 172.21.15.137:0/160689068 learned_addr learned my addr 172.21.15.137:0/160689068 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 -- 172.21.15.137:0/160689068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feffc105a30 msgr2=0x7feffc19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feffc105a30 0x7feffc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.588+0000 7ff0011f3700 1 -- 172.21.15.137:0/160689068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0095f0 con 0x7feffc0ffa30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7ff0011f3700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feff800b4b0 tx=0x7feff800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff8011800 con 0x7feffc0ffa30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feff8011e00 con 0x7feffc0ffa30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc196860 con 0x7feffc0ffa30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff8010e50 con 0x7feffc0ffa30 2024-06-15T01:19:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.589+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc196db0 con 0x7feffc0ffa30 2024-06-15T01:19:57.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.590+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc109060 con 0x7feffc0ffa30 2024-06-15T01:19:57.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.590+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7feff8010380 con 0x7feffc0ffa30 2024-06-15T01:19:57.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.591+0000 7feff27fc700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 0x7fefe806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.591+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7feff8089f00 con 0x7feffc0ffa30 2024-06-15T01:19:57.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.591+0000 7ff0009f2700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 0x7fefe806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.591+0000 7ff0009f2700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 0x7fefe806d440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefec00b5c0 tx=0x7fefec005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.595+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feff8058840 con 0x7feffc0ffa30 2024-06-15T01:19:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.805+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feffc1970b0 con 0x7fefe806af80 2024-06-15T01:19:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.807+0000 7feff27fc700 1 -- 172.21.15.137:0/160689068 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feffc1970b0 con 0x7fefe806af80 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:19:57.795 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:19:57.796 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:19:57.796 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:19:57.796 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:19:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 msgr2=0x7fefe806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 0x7fefe806d440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefec00b5c0 tx=0x7fefec005960 comp rx=0 tx=0).stop 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 msgr2=0x7feffc19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feff800b4b0 tx=0x7feff800b870 comp rx=0 tx=0).stop 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 shutdown_connections 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fefe806af80 0x7fefe806d440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feffc0ffa30 0x7feffc19c500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 --2- 172.21.15.137:0/160689068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feffc105a30 0x7feffc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 >> 172.21.15.137:0/160689068 conn(0x7feffc0fb540 msgr2=0x7feffc069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 shutdown_connections 2024-06-15T01:19:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.809+0000 7ff003457700 1 -- 172.21.15.137:0/160689068 wait complete. 2024-06-15T01:19:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 -- 172.21.15.137:0/1946696915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 msgr2=0x7feddc1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 --2- 172.21.15.137:0/1946696915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc1057d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009910 tx=0x7fedd0009c20 comp rx=0 tx=0).stop 2024-06-15T01:19:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 -- 172.21.15.137:0/1946696915 shutdown_connections 2024-06-15T01:19:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 --2- 172.21.15.137:0/1946696915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc1057d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 --2- 172.21.15.137:0/1946696915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 0x7feddc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 -- 172.21.15.137:0/1946696915 >> 172.21.15.137:0/1946696915 conn(0x7feddc0fc640 msgr2=0x7feddc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 -- 172.21.15.137:0/1946696915 shutdown_connections 2024-06-15T01:19:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.913+0000 7fede23c1700 1 -- 172.21.15.137:0/1946696915 wait complete. 2024-06-15T01:19:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 Processor -- start 2024-06-15T01:19:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 -- start start 2024-06-15T01:19:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 0x7feddc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc073b10 con 0x7feddc101780 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.914+0000 7fede23c1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc073c50 con 0x7feddc100dd0 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58654/0 (socket says 172.21.15.137:58654) 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 -- 172.21.15.137:0/2927006619 learned_addr learned my addr 172.21.15.137:0/2927006619 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddbfff700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 0x7feddc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 -- 172.21.15.137:0/2927006619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 msgr2=0x7feddc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 0x7feddc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 -- 172.21.15.137:0/2927006619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd00095f0 con 0x7feddc101780 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7feddb7fe700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fedd000bda0 tx=0x7fedd000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedd001d070 con 0x7feddc101780 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fedd000f460 con 0x7feddc101780 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedd0021640 con 0x7feddc101780 2024-06-15T01:19:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc073ed0 con 0x7feddc101780 2024-06-15T01:19:57.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.915+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc1a2c00 con 0x7feddc101780 2024-06-15T01:19:57.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.916+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc1091d0 con 0x7feddc101780 2024-06-15T01:19:57.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.917+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fedd000fa20 con 0x7feddc101780 2024-06-15T01:19:57.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.917+0000 7fedd97fa700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 0x7fedc406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:19:57.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.917+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fedd008b490 con 0x7feddc101780 2024-06-15T01:19:57.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.917+0000 7feddbfff700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 0x7fedc406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:19:57.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.917+0000 7feddbfff700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 0x7fedc406d440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fedcc005950 tx=0x7fedcc0058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:19:57.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:57.921+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fedd0056340 con 0x7feddc101780 2024-06-15T01:19:58.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.199+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feddc04ea90 con 0x7feddc101780 2024-06-15T01:19:58.187 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:57 smithi137 ceph-mon[112295]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:58.187 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:57 smithi137 ceph-mon[112295]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:58.187 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:57 smithi137 ceph-mon[112295]: pgmap v416: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:58.187 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:57 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1154225868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:19:58.188 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:57 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3429149335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.199+0000 7fedd97fa700 1 -- 172.21.15.137:0/2927006619 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fedd0059960 con 0x7feddc101780 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:19:58.188 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.201+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 msgr2=0x7fedc406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.201+0000 7fede23c1700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 0x7fedc406d440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fedcc005950 tx=0x7fedcc0058e0 comp rx=0 tx=0).stop 2024-06-15T01:19:58.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.201+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 msgr2=0x7feddc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.201+0000 7fede23c1700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fedd000bda0 tx=0x7fedd000be80 comp rx=0 tx=0).stop 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 shutdown_connections 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fedc406af80 0x7fedc406d440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc100dd0 0x7feddc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 --2- 172.21.15.137:0/2927006619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feddc101780 0x7feddc0735d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 >> 172.21.15.137:0/2927006619 conn(0x7feddc0fc640 msgr2=0x7feddc0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 shutdown_connections 2024-06-15T01:19:58.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:19:58.202+0000 7fede23c1700 1 -- 172.21.15.137:0/2927006619 wait complete. 2024-06-15T01:19:58.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:57 smithi142 ceph-mon[82136]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:58.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:57 smithi142 ceph-mon[82136]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:58.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:57 smithi142 ceph-mon[82136]: pgmap v416: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:19:58.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:57 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1154225868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:19:58.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:57 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3429149335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:19:59.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:58 smithi137 ceph-mon[112295]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:59.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:58 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2927006619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:19:59.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:58 smithi142 ceph-mon[82136]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:19:59.353 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:58 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2927006619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:19:59 smithi137 ceph-mon[112295]: pgmap v417: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : [WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:20:00.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:20:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:20:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:20:00.218 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:19:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:20:00.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:19:59 smithi142 ceph-mon[82136]: pgmap v417: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: fs cephfs is degraded 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: [WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:20:01.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:00 smithi137 ceph-mon[112295]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: fs cephfs is degraded 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: [WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:20:01.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:20:01.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:20:01.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:00 smithi142 ceph-mon[82136]: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:20:02.165 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:01 smithi137 ceph-mon[112295]: pgmap v418: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:02.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:01 smithi142 ceph-mon[82136]: pgmap v418: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:04.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:03 smithi137 ceph-mon[112295]: pgmap v419: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:04.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:04.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:03 smithi142 ceph-mon[82136]: pgmap v419: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:04.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:06.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:05 smithi137 ceph-mon[112295]: pgmap v420: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:06.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:05 smithi142 ceph-mon[82136]: pgmap v420: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:08.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:07 smithi137 ceph-mon[112295]: pgmap v421: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:08.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:07 smithi142 ceph-mon[82136]: pgmap v421: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:10.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:09 smithi137 ceph-mon[112295]: pgmap v422: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:10.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:09 smithi142 ceph-mon[82136]: pgmap v422: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:12.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:11 smithi137 ceph-mon[112295]: pgmap v423: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:12.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:11 smithi142 ceph-mon[82136]: pgmap v423: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:14.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:13 smithi137 ceph-mon[112295]: pgmap v424: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:14.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:13 smithi142 ceph-mon[82136]: pgmap v424: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:16.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:15 smithi137 ceph-mon[112295]: pgmap v425: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:16.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:15 smithi142 ceph-mon[82136]: pgmap v425: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:18.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:17 smithi137 ceph-mon[112295]: pgmap v426: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:18.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:17 smithi142 ceph-mon[82136]: pgmap v426: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:19.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:19.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:20.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:19 smithi137 ceph-mon[112295]: pgmap v427: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:20.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:19 smithi142 ceph-mon[82136]: pgmap v427: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:22.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:21 smithi137 ceph-mon[112295]: pgmap v428: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:22.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:21 smithi142 ceph-mon[82136]: pgmap v428: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:24.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:23 smithi137 ceph-mon[112295]: pgmap v429: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:24.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:23 smithi142 ceph-mon[82136]: pgmap v429: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:26.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:25 smithi137 ceph-mon[112295]: pgmap v430: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:26.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:25 smithi142 ceph-mon[82136]: pgmap v430: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:28.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:27 smithi137 ceph-mon[112295]: pgmap v431: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:28.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 -- 172.21.15.137:0/2837833852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 msgr2=0x7f6db40ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 --2- 172.21.15.137:0/2837833852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db40ffd20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009910 tx=0x7f6d9c009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 -- 172.21.15.137:0/2837833852 shutdown_connections 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 --2- 172.21.15.137:0/2837833852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db40ffd20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 --2- 172.21.15.137:0/2837833852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 0x7f6db4105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 -- 172.21.15.137:0/2837833852 >> 172.21.15.137:0/2837833852 conn(0x7f6db40fb390 msgr2=0x7f6db40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 -- 172.21.15.137:0/2837833852 shutdown_connections 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.318+0000 7f6db359e700 1 -- 172.21.15.137:0/2837833852 wait complete. 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 Processor -- start 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 -- start start 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 0x7f6db4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41992a0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db359e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db419bfd0 con 0x7f6db41058a0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36126/0 (socket says 172.21.15.137:36126) 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.319+0000 7f6db259c700 1 -- 172.21.15.137:0/1642339009 learned_addr learned my addr 172.21.15.137:0/1642339009 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6dabfff700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 0x7f6db4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db259c700 1 -- 172.21.15.137:0/1642339009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 msgr2=0x7f6db4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db259c700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 0x7f6db4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db259c700 1 -- 172.21.15.137:0/1642339009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0095f0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db259c700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d8d0 tx=0x7f6da400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da40098a0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6da4010460 con 0x7f6db40ff8a0 2024-06-15T01:20:28.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db419c2b0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da4009ea0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.320+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db419c7d0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:27 smithi142 ceph-mon[82136]: pgmap v431: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.321+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db404f2e0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.322+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6da40105d0 con 0x7f6db40ff8a0 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.322+0000 7f6dab7fe700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 0x7f6da006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.322+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6da408a250 con 0x7f6db40ff8a0 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.322+0000 7f6dabfff700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 0x7f6da006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.323+0000 7f6dabfff700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 0x7f6da006d440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b5c0 tx=0x7f6d9c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.327+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6da4055080 con 0x7f6db40ff8a0 2024-06-15T01:20:28.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.534+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6db419cba0 con 0x7f6da006af80 2024-06-15T01:20:28.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.535+0000 7f6dab7fe700 1 -- 172.21.15.137:0/1642339009 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6db419cba0 con 0x7f6da006af80 2024-06-15T01:20:28.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 msgr2=0x7f6da006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 0x7f6da006d440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b5c0 tx=0x7f6d9c01a040 comp rx=0 tx=0).stop 2024-06-15T01:20:28.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 msgr2=0x7f6db41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d8d0 tx=0x7f6da400dbe0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 shutdown_connections 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6da006af80 0x7f6da006d440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db40ff8a0 0x7f6db41986d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 --2- 172.21.15.137:0/1642339009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6db41058a0 0x7f6db4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 >> 172.21.15.137:0/1642339009 conn(0x7f6db40fb390 msgr2=0x7f6db4068f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 shutdown_connections 2024-06-15T01:20:28.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.537+0000 7f6db359e700 1 -- 172.21.15.137:0/1642339009 wait complete. 2024-06-15T01:20:28.536 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:20:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 -- 172.21.15.137:0/3064623186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 msgr2=0x7fe94c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3064623186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c1057c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009910 tx=0x7fe93c009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 -- 172.21.15.137:0/3064623186 shutdown_connections 2024-06-15T01:20:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3064623186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c1057c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3064623186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 -- 172.21.15.137:0/3064623186 >> 172.21.15.137:0/3064623186 conn(0x7fe94c0fc650 msgr2=0x7fe94c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:28.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 -- 172.21.15.137:0/3064623186 shutdown_connections 2024-06-15T01:20:28.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.642+0000 7fe9537e1700 1 -- 172.21.15.137:0/3064623186 wait complete. 2024-06-15T01:20:28.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 Processor -- start 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 -- start start 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c197210 con 0x7fe94c101770 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe9537e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c197380 con 0x7fe94c100dc0 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36144/0 (socket says 172.21.15.137:36144) 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 -- 172.21.15.137:0/3381560869 learned_addr learned my addr 172.21.15.137:0/3381560869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe95157d700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 -- 172.21.15.137:0/3381560869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 msgr2=0x7fe94c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe950d7c700 1 -- 172.21.15.137:0/3381560869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0095f0 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.643+0000 7fe95157d700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe950d7c700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00bda0 tx=0x7fe93c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93c01d070 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe93c00f460 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c19a0c0 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93c021640 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.644+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c19a5b0 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.645+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c1091c0 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.645+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe93c00f9b0 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.646+0000 7fe9427fc700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.646+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe93c08b540 con 0x7fe94c101770 2024-06-15T01:20:28.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.646+0000 7fe95157d700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.646+0000 7fe95157d700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe948005fd0 tx=0x7fe948005e20 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.650+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe93c0563f0 con 0x7fe94c101770 2024-06-15T01:20:28.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.858+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe94c02cd30 con 0x7fe93806af80 2024-06-15T01:20:28.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.859+0000 7fe9427fc700 1 -- 172.21.15.137:0/3381560869 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe94c02cd30 con 0x7fe93806af80 2024-06-15T01:20:28.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 msgr2=0x7fe93806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe948005fd0 tx=0x7fe948005e20 comp rx=0 tx=0).stop 2024-06-15T01:20:28.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 msgr2=0x7fe94c196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00bda0 tx=0x7fe93c00be80 comp rx=0 tx=0).stop 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 shutdown_connections 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 --2- 172.21.15.137:0/3381560869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 >> 172.21.15.137:0/3381560869 conn(0x7fe94c0fc650 msgr2=0x7fe94c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:28.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 shutdown_connections 2024-06-15T01:20:28.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.861+0000 7fe9537e1700 1 -- 172.21.15.137:0/3381560869 wait complete. 2024-06-15T01:20:28.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 -- 172.21.15.137:0/3934757985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc100dc0 msgr2=0x7fe9bc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 --2- 172.21.15.137:0/3934757985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc1011a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009910 tx=0x7fe9b0009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:28.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 -- 172.21.15.137:0/3934757985 shutdown_connections 2024-06-15T01:20:28.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 --2- 172.21.15.137:0/3934757985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc101770 0x7fe9bc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 --2- 172.21.15.137:0/3934757985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc1011a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 -- 172.21.15.137:0/3934757985 >> 172.21.15.137:0/3934757985 conn(0x7fe9bc0fc650 msgr2=0x7fe9bc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:28.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 -- 172.21.15.137:0/3934757985 shutdown_connections 2024-06-15T01:20:28.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.965+0000 7fe9c2631700 1 -- 172.21.15.137:0/3934757985 wait complete. 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.966+0000 7fe9c2631700 1 Processor -- start 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.966+0000 7fe9c2631700 1 -- start start 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9c2631700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9c2631700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9c2631700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc197290 con 0x7fe9bc101770 2024-06-15T01:20:28.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9c2631700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc19b020 con 0x7fe9bc100dc0 2024-06-15T01:20:28.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36162/0 (socket says 172.21.15.137:36162) 2024-06-15T01:20:28.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 -- 172.21.15.137:0/986449835 learned_addr learned my addr 172.21.15.137:0/986449835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:28.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bbfff700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 -- 172.21.15.137:0/986449835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 msgr2=0x7fe9bc196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 -- 172.21.15.137:0/986449835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00095f0 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bbfff700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.967+0000 7fe9bb7fe700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00d900 tx=0x7fe9ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.968+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ac009900 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.968+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9ac010460 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.968+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ac00f5f0 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.968+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc19b300 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.968+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc19b850 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.969+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc109160 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.969+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe9ac00fac0 con 0x7fe9bc101770 2024-06-15T01:20:28.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.970+0000 7fe9b97fa700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 0x7fe9a406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:28.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.970+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe9ac08a180 con 0x7fe9bc101770 2024-06-15T01:20:28.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.970+0000 7fe9bbfff700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 0x7fe9a406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:28.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.970+0000 7fe9bbfff700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 0x7fe9a406d440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b5c0 tx=0x7fe9b001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:28.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:28.975+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe9ac054fb0 con 0x7fe9bc101770 2024-06-15T01:20:29.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.173+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9bc19bb60 con 0x7fe9a406af80 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.183+0000 7fe9b97fa700 1 -- 172.21.15.137:0/986449835 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe9bc19bb60 con 0x7fe9a406af80 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (11m) 5m ago 21m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (21m) 5m ago 21m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (20m) 5m ago 20m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (9m) 5m ago 21m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (9m) 5m ago 20m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (10m) 5m ago 21m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (5m) 5m ago 16m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (5m) 5m ago 16m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:20:29.171 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (5m) 5m ago 16m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (5m) 5m ago 15m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (12m) 5m ago 23m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (11m) 5m ago 20m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (9m) 5m ago 23m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (9m) 5m ago 20m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (11m) 5m ago 21m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (11m) 5m ago 20m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:20:29.172 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (8m) 5m ago 19m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (8m) 5m ago 19m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (7m) 5m ago 18m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (7m) 5m ago 18m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (6m) 5m ago 17m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (6m) 5m ago 17m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:20:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (11m) 5m ago 21m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 msgr2=0x7fe9a406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 0x7fe9a406d440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b5c0 tx=0x7fe9b001a040 comp rx=0 tx=0).stop 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 msgr2=0x7fe9bc196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00d900 tx=0x7fe9ac00dcc0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 shutdown_connections 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe9a406af80 0x7fe9a406d440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9bc100dc0 0x7fe9bc196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 --2- 172.21.15.137:0/986449835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9bc101770 0x7fe9bc196bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 >> 172.21.15.137:0/986449835 conn(0x7fe9bc0fc650 msgr2=0x7fe9bc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 shutdown_connections 2024-06-15T01:20:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.185+0000 7fe9c2631700 1 -- 172.21.15.137:0/986449835 wait complete. 2024-06-15T01:20:29.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 -- 172.21.15.137:0/767992928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c105a40 msgr2=0x7f237c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 --2- 172.21.15.137:0/767992928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c105a40 0x7f237c105e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2364009910 tx=0x7f2364009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:29.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 -- 172.21.15.137:0/767992928 shutdown_connections 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 --2- 172.21.15.137:0/767992928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c0ffa40 0x7f237c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 --2- 172.21.15.137:0/767992928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c105a40 0x7f237c105e20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 -- 172.21.15.137:0/767992928 >> 172.21.15.137:0/767992928 conn(0x7f237c0fb570 msgr2=0x7f237c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 -- 172.21.15.137:0/767992928 shutdown_connections 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.288+0000 7f238277a700 1 -- 172.21.15.137:0/767992928 wait complete. 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 Processor -- start 2024-06-15T01:20:29.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 -- start start 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c105a40 0x7f237c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c19d0f0 con 0x7f237c0ffa40 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f238277a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1965a0 con 0x7f237c105a40 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f237bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f237bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36176/0 (socket says 172.21.15.137:36176) 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f237bfff700 1 -- 172.21.15.137:0/534495235 learned_addr learned my addr 172.21.15.137:0/534495235 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.289+0000 7f237b7fe700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c105a40 0x7f237c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f237bfff700 1 -- 172.21.15.137:0/534495235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c105a40 msgr2=0x7f237c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f237bfff700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c105a40 0x7f237c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f237bfff700 1 -- 172.21.15.137:0/534495235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640095f0 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f237bfff700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236401d070 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f236400f460 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c196820 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2364021640 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.290+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c196d10 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.291+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c109050 con 0x7f237c0ffa40 2024-06-15T01:20:29.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.291+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f236400f9e0 con 0x7f237c0ffa40 2024-06-15T01:20:29.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.292+0000 7f23797fa700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 0x7f236806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.292+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f236408b750 con 0x7f237c0ffa40 2024-06-15T01:20:29.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.292+0000 7f237b7fe700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 0x7f236806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.292+0000 7f237b7fe700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 0x7f236806d440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f236c006fd0 tx=0x7f236c008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:29.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.296+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2364059fb0 con 0x7f237c0ffa40 2024-06-15T01:20:29.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.565+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f237c108a30 con 0x7f237c0ffa40 2024-06-15T01:20:29.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.566+0000 7f23797fa700 1 -- 172.21.15.137:0/534495235 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2364059b40 con 0x7f237c0ffa40 2024-06-15T01:20:29.554 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:20:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:20:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:20:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:20:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:20:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:20:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:20:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:20:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:20:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:20:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:20:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:20:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:20:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:20:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:20:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:20:29.557 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 msgr2=0x7f236806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 0x7f236806d440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f236c006fd0 tx=0x7f236c008040 comp rx=0 tx=0).stop 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 msgr2=0x7f237c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400be20 comp rx=0 tx=0).stop 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 shutdown_connections 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f236806af80 0x7f236806d440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c0ffa40 0x7f237c19c520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 --2- 172.21.15.137:0/534495235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f237c105a40 0x7f237c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 >> 172.21.15.137:0/534495235 conn(0x7f237c0fb570 msgr2=0x7f237c069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:29.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 shutdown_connections 2024-06-15T01:20:29.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.568+0000 7f238277a700 1 -- 172.21.15.137:0/534495235 wait complete. 2024-06-15T01:20:29.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 -- 172.21.15.137:0/1072334294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7578105a40 msgr2=0x7f7578105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 --2- 172.21.15.137:0/1072334294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7578105a40 0x7f7578105e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7574009910 tx=0x7f7574009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:29.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 -- 172.21.15.137:0/1072334294 shutdown_connections 2024-06-15T01:20:29.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 --2- 172.21.15.137:0/1072334294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780ffa40 0x7f75780ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 --2- 172.21.15.137:0/1072334294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7578105a40 0x7f7578105e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 -- 172.21.15.137:0/1072334294 >> 172.21.15.137:0/1072334294 conn(0x7f75780fb550 msgr2=0x7f75780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 -- 172.21.15.137:0/1072334294 shutdown_connections 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 -- 172.21.15.137:0/1072334294 wait complete. 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.672+0000 7f757f752700 1 Processor -- start 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757f752700 1 -- start start 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757f752700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757f752700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 0x7f75780fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757f752700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780fc9c0 con 0x7f75780ffa40 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757f752700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780fcb30 con 0x7f7578105a40 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36190/0 (socket says 172.21.15.137:36190) 2024-06-15T01:20:29.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 -- 172.21.15.137:0/2592963768 learned_addr learned my addr 172.21.15.137:0/2592963768 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757cced700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 0x7f75780fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 -- 172.21.15.137:0/2592963768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 msgr2=0x7f75780fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 0x7f75780fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 -- 172.21.15.137:0/2592963768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740095f0 con 0x7f75780ffa40 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757cced700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 0x7f75780fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f757d4ee700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7574006010 tx=0x7f757400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.673+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757401d070 con 0x7f75780ffa40 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.674+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f757400f460 con 0x7f75780ffa40 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.674+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75781a2850 con 0x7f75780ffa40 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.674+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7574021620 con 0x7f75780ffa40 2024-06-15T01:20:29.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.674+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781a2c00 con 0x7f75780ffa40 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.674+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578109050 con 0x7f75780ffa40 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.675+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f757400fa40 con 0x7f75780ffa40 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.675+0000 7f756e7fc700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 0x7f756406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.676+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f757408b5b0 con 0x7f75780ffa40 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.676+0000 7f757cced700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 0x7f756406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:29.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.676+0000 7f757cced700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 0x7f756406d440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7568005950 tx=0x7f75680058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:29.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.680+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f75740563e0 con 0x7f75780ffa40 2024-06-15T01:20:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.913+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f757804ea90 con 0x7f75780ffa40 2024-06-15T01:20:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.914+0000 7f756e7fc700 1 -- 172.21.15.137:0/2592963768 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f757400f5d0 con 0x7f75780ffa40 2024-06-15T01:20:29.901 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:20:29.902 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:20:29.903 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:20:29.904 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:20:29.905 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 msgr2=0x7f756406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 0x7f756406d440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7568005950 tx=0x7f75680058e0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 msgr2=0x7f75780fbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7574006010 tx=0x7f757400be20 comp rx=0 tx=0).stop 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 shutdown_connections 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f756406af80 0x7f756406d440 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75780ffa40 0x7f75780fbeb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 --2- 172.21.15.137:0/2592963768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7578105a40 0x7f75780fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 >> 172.21.15.137:0/2592963768 conn(0x7f75780fb550 msgr2=0x7f7578069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.916+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 shutdown_connections 2024-06-15T01:20:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:29.917+0000 7f757f752700 1 -- 172.21.15.137:0/2592963768 wait complete. 2024-06-15T01:20:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 -- 172.21.15.137:0/3702787634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 msgr2=0x7fc420100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3702787634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc420100050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc408009910 tx=0x7fc408009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 -- 172.21.15.137:0/3702787634 shutdown_connections 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3702787634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc420100050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3702787634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc420105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 -- 172.21.15.137:0/3702787634 >> 172.21.15.137:0/3702787634 conn(0x7fc4200755a0 msgr2=0x7fc4200759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 -- 172.21.15.137:0/3702787634 shutdown_connections 2024-06-15T01:20:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.022+0000 7fc425dfe700 1 -- 172.21.15.137:0/3702787634 wait complete. 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 Processor -- start 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 -- start start 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc42019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42019d170 con 0x7fc4200ffbd0 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc425dfe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420196620 con 0x7fc420105a40 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc41f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc41f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36212/0 (socket says 172.21.15.137:36212) 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc41f7fe700 1 -- 172.21.15.137:0/3500141206 learned_addr learned my addr 172.21.15.137:0/3500141206 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc417fff700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc42019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc41f7fe700 1 -- 172.21.15.137:0/3500141206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 msgr2=0x7fc42019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.023+0000 7fc41f7fe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc42019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc41f7fe700 1 -- 172.21.15.137:0/3500141206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080095f0 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc417fff700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc42019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc41f7fe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b4b0 tx=0x7fc41000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc410011800 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc410011e00 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc420196900 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc410010e50 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.024+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc420196e50 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.025+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42004ea90 con 0x7fc4200ffbd0 2024-06-15T01:20:30.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.025+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc410010380 con 0x7fc4200ffbd0 2024-06-15T01:20:30.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.026+0000 7fc41d7fa700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 0x7fc40c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.026+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc410089ec0 con 0x7fc4200ffbd0 2024-06-15T01:20:30.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.026+0000 7fc417fff700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 0x7fc40c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.026+0000 7fc417fff700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 0x7fc40c06d440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc420104e40 tx=0x7fc40801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:30.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.030+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc410058800 con 0x7fc4200ffbd0 2024-06-15T01:20:30.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: pgmap v432: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/534495235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:20:30.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:29 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2592963768' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:20:30.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.241+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4201971d0 con 0x7fc40c06af80 2024-06-15T01:20:30.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.242+0000 7fc41d7fa700 1 -- 172.21.15.137:0/3500141206 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc4201971d0 con 0x7fc40c06af80 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:20:30.230 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:20:30.231 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:20:30.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 msgr2=0x7fc40c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 0x7fc40c06d440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc420104e40 tx=0x7fc40801a040 comp rx=0 tx=0).stop 2024-06-15T01:20:30.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 msgr2=0x7fc42019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b4b0 tx=0x7fc41000b870 comp rx=0 tx=0).stop 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 shutdown_connections 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc40c06af80 0x7fc40c06d440 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4200ffbd0 0x7fc42019c5a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 --2- 172.21.15.137:0/3500141206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc420105a40 0x7fc42019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.244+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 >> 172.21.15.137:0/3500141206 conn(0x7fc4200755a0 msgr2=0x7fc420103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.245+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 shutdown_connections 2024-06-15T01:20:30.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.245+0000 7fc425dfe700 1 -- 172.21.15.137:0/3500141206 wait complete. 2024-06-15T01:20:30.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: pgmap v432: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:30.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:30.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/534495235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:20:30.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:29 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2592963768' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 -- 172.21.15.137:0/429672247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d40ffbd0 msgr2=0x7f89d4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/429672247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d40ffbd0 0x7f89d4100050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009910 tx=0x7f89c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 -- 172.21.15.137:0/429672247 shutdown_connections 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/429672247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d40ffbd0 0x7f89d4100050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/429672247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4105a40 0x7f89d4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 -- 172.21.15.137:0/429672247 >> 172.21.15.137:0/429672247 conn(0x7f89d40755a0 msgr2=0x7f89d40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 -- 172.21.15.137:0/429672247 shutdown_connections 2024-06-15T01:20:30.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 -- 172.21.15.137:0/429672247 wait complete. 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.349+0000 7f89d8cd2700 1 Processor -- start 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d8cd2700 1 -- start start 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d8cd2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d40ffbd0 0x7f89d4195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d8cd2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d8cd2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4196160 con 0x7f89d4105a40 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d8cd2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4199ef0 con 0x7f89d40ffbd0 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36222/0 (socket says 172.21.15.137:36222) 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 -- 172.21.15.137:0/2786356926 learned_addr learned my addr 172.21.15.137:0/2786356926 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:20:30.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d259c700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d40ffbd0 0x7f89d4195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 -- 172.21.15.137:0/2786356926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d40ffbd0 msgr2=0x7f89d4195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d40ffbd0 0x7f89d4195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 -- 172.21.15.137:0/2786356926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80095f0 con 0x7f89d4105a40 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.350+0000 7f89d1d9b700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.351+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c801d070 con 0x7f89d4105a40 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.351+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89c800f460 con 0x7f89d4105a40 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.351+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d419a170 con 0x7f89d4105a40 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.351+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c8021620 con 0x7f89d4105a40 2024-06-15T01:20:30.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.351+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d419a660 con 0x7f89d4105a40 2024-06-15T01:20:30.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.352+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4107fd0 con 0x7f89d4105a40 2024-06-15T01:20:30.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.352+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f89c800fa40 con 0x7f89d4105a40 2024-06-15T01:20:30.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.353+0000 7f89c37fe700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 0x7f89bc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:20:30.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.353+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f89c808b5a0 con 0x7f89d4105a40 2024-06-15T01:20:30.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.353+0000 7f89d259c700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 0x7f89bc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:20:30.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.353+0000 7f89d259c700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 0x7f89bc06d440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005950 tx=0x7f89c40058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:20:30.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.357+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f89c80563d0 con 0x7f89d4105a40 2024-06-15T01:20:30.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.627+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f89d419aa00 con 0x7f89d4105a40 2024-06-15T01:20:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.627+0000 7f89c37fe700 1 -- 172.21.15.137:0/2786356926 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f89c80599f0 con 0x7f89d4105a40 2024-06-15T01:20:30.615 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:20:30.615 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:20:30.615 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:20:30.616 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:20:30.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 msgr2=0x7f89bc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 0x7f89bc06d440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005950 tx=0x7f89c40058e0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 msgr2=0x7f89d4195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:20:30.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800be20 comp rx=0 tx=0).stop 2024-06-15T01:20:30.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 shutdown_connections 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f89bc06af80 0x7f89bc06d440 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d40ffbd0 0x7f89d4195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.629+0000 7f89d8cd2700 1 --2- 172.21.15.137:0/2786356926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4105a40 0x7f89d4195a80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.630+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 >> 172.21.15.137:0/2786356926 conn(0x7f89d40755a0 msgr2=0x7f89d40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.630+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 shutdown_connections 2024-06-15T01:20:30.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:20:30.630+0000 7f89d8cd2700 1 -- 172.21.15.137:0/2786356926 wait complete. 2024-06-15T01:20:31.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:30 smithi137 ceph-mon[112295]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:31.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:30 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2786356926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:20:31.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:30 smithi142 ceph-mon[82136]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:20:31.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:30 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2786356926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:20:32.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:31 smithi137 ceph-mon[112295]: pgmap v433: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:32.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:31 smithi142 ceph-mon[82136]: pgmap v433: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:34.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:33 smithi137 ceph-mon[112295]: pgmap v434: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:34.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:34.222 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:33 smithi142 ceph-mon[82136]: pgmap v434: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:34.223 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:36.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:35 smithi142 ceph-mon[82136]: pgmap v435: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:36.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:35 smithi137 ceph-mon[112295]: pgmap v435: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:38.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:37 smithi142 ceph-mon[82136]: pgmap v436: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:38.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:37 smithi137 ceph-mon[112295]: pgmap v436: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:40.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:39 smithi142 ceph-mon[82136]: pgmap v437: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:40.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:39 smithi137 ceph-mon[112295]: pgmap v437: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:42.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:41 smithi137 ceph-mon[112295]: pgmap v438: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:42.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:41 smithi142 ceph-mon[82136]: pgmap v438: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:44.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:43 smithi137 ceph-mon[112295]: pgmap v439: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:44.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:43 smithi142 ceph-mon[82136]: pgmap v439: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:46.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:45 smithi142 ceph-mon[82136]: pgmap v440: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:46.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:45 smithi137 ceph-mon[112295]: pgmap v440: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:48.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:48 smithi142 ceph-mon[82136]: pgmap v441: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:48.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:48 smithi137 ceph-mon[112295]: pgmap v441: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:20:50.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:50 smithi142 ceph-mon[82136]: pgmap v442: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:50.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:50 smithi137 ceph-mon[112295]: pgmap v442: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:52.269 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:52 smithi137 ceph-mon[112295]: pgmap v443: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:52.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:52 smithi142 ceph-mon[82136]: pgmap v443: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:54.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:54 smithi137 ceph-mon[112295]: pgmap v444: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:54.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:54 smithi142 ceph-mon[82136]: pgmap v444: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:56.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:56 smithi142 ceph-mon[82136]: pgmap v445: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:56.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:56 smithi137 ceph-mon[112295]: pgmap v445: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:58.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:20:58 smithi142 ceph-mon[82136]: pgmap v446: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:20:58.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:20:58 smithi137 ceph-mon[112295]: pgmap v446: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:00.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:00 smithi142 ceph-mon[82136]: pgmap v447: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:00.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:00 smithi137 ceph-mon[112295]: pgmap v447: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 -- 172.21.15.137:0/1545395687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 msgr2=0x7f0a14105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 --2- 172.21.15.137:0/1545395687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a14105e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009910 tx=0x7f0a04009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 -- 172.21.15.137:0/1545395687 shutdown_connections 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 --2- 172.21.15.137:0/1545395687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 0x7f0a140ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 --2- 172.21.15.137:0/1545395687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a14105e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 -- 172.21.15.137:0/1545395687 >> 172.21.15.137:0/1545395687 conn(0x7f0a140fb530 msgr2=0x7f0a140fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 -- 172.21.15.137:0/1545395687 shutdown_connections 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.746+0000 7f0a19038700 1 -- 172.21.15.137:0/1545395687 wait complete. 2024-06-15T01:21:00.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.747+0000 7f0a19038700 1 Processor -- start 2024-06-15T01:21:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.747+0000 7f0a19038700 1 -- start start 2024-06-15T01:21:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a19038700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 0x7f0a140fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a19038700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a19038700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140fc920 con 0x7f0a14105a20 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a19038700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140fca60 con 0x7f0a140ffa20 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37858/0 (socket says 172.21.15.137:37858) 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 -- 172.21.15.137:0/2515950044 learned_addr learned my addr 172.21.15.137:0/2515950044 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a12d9d700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 0x7f0a140fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 -- 172.21.15.137:0/2515950044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 msgr2=0x7f0a140fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 0x7f0a140fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 -- 172.21.15.137:0/2515950044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040095f0 con 0x7f0a14105a20 2024-06-15T01:21:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a1259c700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b4b0 tx=0x7f09fc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09fc011800 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a141a28a0 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09fc011e00 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.748+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09fc010e60 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.749+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a141a2cb0 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.749+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14109030 con 0x7f0a14105a20 2024-06-15T01:21:00.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.750+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f09fc011960 con 0x7f0a14105a20 2024-06-15T01:21:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.750+0000 7f0a0bfff700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 0x7f0a0006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.750+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f09fc089ea0 con 0x7f0a14105a20 2024-06-15T01:21:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.751+0000 7f0a12d9d700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 0x7f0a0006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.751+0000 7f0a12d9d700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 0x7f0a0006d440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:00.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.755+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f09fc054cd0 con 0x7f0a14105a20 2024-06-15T01:21:00.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.966+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a1402cd80 con 0x7f0a0006af80 2024-06-15T01:21:00.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.967+0000 7f0a0bfff700 1 -- 172.21.15.137:0/2515950044 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0a1402cd80 con 0x7f0a0006af80 2024-06-15T01:21:00.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 msgr2=0x7f0a0006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:00.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 0x7f0a0006d440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0401a040 comp rx=0 tx=0).stop 2024-06-15T01:21:00.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 msgr2=0x7f0a140fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:00.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b4b0 tx=0x7f09fc00b7c0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 shutdown_connections 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0a0006af80 0x7f0a0006d440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a140ffa20 0x7f0a140fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 --2- 172.21.15.137:0/2515950044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a14105a20 0x7f0a140fc3e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 >> 172.21.15.137:0/2515950044 conn(0x7f0a140fb530 msgr2=0x7f0a14068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.969+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 shutdown_connections 2024-06-15T01:21:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:00.970+0000 7f0a19038700 1 -- 172.21.15.137:0/2515950044 wait complete. 2024-06-15T01:21:00.967 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:21:01.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1737630585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44100dc0 msgr2=0x7fbb441011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1737630585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44100dc0 0x7fbb441011a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009910 tx=0x7fbb34009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:01.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1737630585 shutdown_connections 2024-06-15T01:21:01.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1737630585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44101770 0x7fbb441057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1737630585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44100dc0 0x7fbb441011a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1737630585 >> 172.21.15.137:0/1737630585 conn(0x7fbb440fc650 msgr2=0x7fbb440fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.075+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1737630585 shutdown_connections 2024-06-15T01:21:01.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.076+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1737630585 wait complete. 2024-06-15T01:21:01.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.076+0000 7fbb48eaa700 1 Processor -- start 2024-06-15T01:21:01.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.076+0000 7fbb48eaa700 1 -- start start 2024-06-15T01:21:01.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb48eaa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44100dc0 0x7fbb44073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb48eaa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb48eaa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44073b10 con 0x7fbb44101770 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb48eaa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44073c50 con 0x7fbb44100dc0 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37866/0 (socket says 172.21.15.137:37866) 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 -- 172.21.15.137:0/1750284595 learned_addr learned my addr 172.21.15.137:0/1750284595 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb4259c700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44100dc0 0x7fbb44073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 -- 172.21.15.137:0/1750284595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44100dc0 msgr2=0x7fbb44073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44100dc0 0x7fbb44073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 -- 172.21.15.137:0/1750284595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340095f0 con 0x7fbb44101770 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb41d9b700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c960 tx=0x7fbb3800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb38007a70 con 0x7fbb44101770 2024-06-15T01:21:01.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb38007bd0 con 0x7fbb44101770 2024-06-15T01:21:01.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.077+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb441a2840 con 0x7fbb44101770 2024-06-15T01:21:01.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.078+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb38018630 con 0x7fbb44101770 2024-06-15T01:21:01.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.078+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb441a2ba0 con 0x7fbb44101770 2024-06-15T01:21:01.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.079+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb44109160 con 0x7fbb44101770 2024-06-15T01:21:01.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.079+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbb3800f9f0 con 0x7fbb44101770 2024-06-15T01:21:01.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.079+0000 7fbb337fe700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 0x7fbb2c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.080+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbb3808a2e0 con 0x7fbb44101770 2024-06-15T01:21:01.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.080+0000 7fbb4259c700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 0x7fbb2c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.080+0000 7fbb4259c700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 0x7fbb2c06d440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb3401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.084+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb38058c20 con 0x7fbb44101770 2024-06-15T01:21:01.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.294+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4402cce0 con 0x7fbb2c06af80 2024-06-15T01:21:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.295+0000 7fbb337fe700 1 -- 172.21.15.137:0/1750284595 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbb4402cce0 con 0x7fbb2c06af80 2024-06-15T01:21:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 msgr2=0x7fbb2c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 0x7fbb2c06d440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb3401a040 comp rx=0 tx=0).stop 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 msgr2=0x7fbb440735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c960 tx=0x7fbb3800cd20 comp rx=0 tx=0).stop 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 shutdown_connections 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb2c06af80 0x7fbb2c06d440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb44100dc0 0x7fbb44073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 --2- 172.21.15.137:0/1750284595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb44101770 0x7fbb440735d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 >> 172.21.15.137:0/1750284595 conn(0x7fbb440fc650 msgr2=0x7fbb440fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 shutdown_connections 2024-06-15T01:21:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.297+0000 7fbb48eaa700 1 -- 172.21.15.137:0/1750284595 wait complete. 2024-06-15T01:21:01.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 -- 172.21.15.137:0/1915402250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100fd0 msgr2=0x7fcab81046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 --2- 172.21.15.137:0/1915402250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100fd0 0x7fcab81046e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009910 tx=0x7fcaac009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 -- 172.21.15.137:0/1915402250 shutdown_connections 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 --2- 172.21.15.137:0/1915402250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100fd0 0x7fcab81046e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 --2- 172.21.15.137:0/1915402250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100620 0x7fcab8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 -- 172.21.15.137:0/1915402250 >> 172.21.15.137:0/1915402250 conn(0x7fcab8074d80 msgr2=0x7fcab8075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 -- 172.21.15.137:0/1915402250 shutdown_connections 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 -- 172.21.15.137:0/1915402250 wait complete. 2024-06-15T01:21:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.401+0000 7fcabdd30700 1 Processor -- start 2024-06-15T01:21:01.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcabdd30700 1 -- start start 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcabdd30700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcabdd30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100fd0 0x7fcab8101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcabdd30700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8102200 con 0x7fcab8100620 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcabdd30700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8102340 con 0x7fcab8100fd0 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37880/0 (socket says 172.21.15.137:37880) 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 -- 172.21.15.137:0/429536742 learned_addr learned my addr 172.21.15.137:0/429536742 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab6ffd700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100fd0 0x7fcab8101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 -- 172.21.15.137:0/429536742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100fd0 msgr2=0x7fcab8101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100fd0 0x7fcab8101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 -- 172.21.15.137:0/429536742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0095f0 con 0x7fcab8100620 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.402+0000 7fcab77fe700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80060b0 tx=0x7fcaa800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.403+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa800dda0 con 0x7fcab8100620 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.403+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcaa800f040 con 0x7fcab8100620 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.403+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab81a2810 con 0x7fcab8100620 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.403+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa80146f0 con 0x7fcab8100620 2024-06-15T01:21:01.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.403+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab81a2c20 con 0x7fcab8100620 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.404+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab8108fd0 con 0x7fcab8100620 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.404+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcaa8015400 con 0x7fcab8100620 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.405+0000 7fcab4ff9700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 0x7fcaa006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.405+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcaa8089e70 con 0x7fcab8100620 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.405+0000 7fcab6ffd700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 0x7fcaa006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.405+0000 7fcab6ffd700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 0x7fcaa006d440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b5c0 tx=0x7fcaac005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.409+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcaa8054d50 con 0x7fcab8100620 2024-06-15T01:21:01.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.609+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcab802cde0 con 0x7fcaa006af80 2024-06-15T01:21:01.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.618+0000 7fcab4ff9700 1 -- 172.21.15.137:0/429536742 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcab802cde0 con 0x7fcaa006af80 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (11m) 6m ago 22m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (22m) 6m ago 22m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (21m) 5m ago 21m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (9m) 6m ago 22m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (9m) 5m ago 21m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (11m) 6m ago 21m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (6m) 6m ago 16m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:21:01.606 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (6m) 6m ago 16m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (5m) 5m ago 16m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (5m) 5m ago 16m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (13m) 6m ago 23m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (12m) 5m ago 21m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (10m) 6m ago 23m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (9m) 5m ago 21m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (12m) 6m ago 22m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (12m) 5m ago 21m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (9m) 6m ago 20m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (8m) 6m ago 19m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (8m) 6m ago 19m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (7m) 5m ago 18m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (7m) 5m ago 18m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:21:01.607 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (6m) 5m ago 18m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (11m) 6m ago 21m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 msgr2=0x7fcaa006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 0x7fcaa006d440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b5c0 tx=0x7fcaac005fd0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 msgr2=0x7fcab8101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80060b0 tx=0x7fcaa800d3b0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 shutdown_connections 2024-06-15T01:21:01.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcaa006af80 0x7fcaa006d440 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab8100620 0x7fcab8101780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 --2- 172.21.15.137:0/429536742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8100fd0 0x7fcab8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.620+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 >> 172.21.15.137:0/429536742 conn(0x7fcab8074d80 msgr2=0x7fcab8103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.621+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 shutdown_connections 2024-06-15T01:21:01.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.621+0000 7fcabdd30700 1 -- 172.21.15.137:0/429536742 wait complete. 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 -- 172.21.15.137:0/2398016102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 msgr2=0x7f0504105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 --2- 172.21.15.137:0/2398016102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f0504105e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009910 tx=0x7f04f4009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 -- 172.21.15.137:0/2398016102 shutdown_connections 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 --2- 172.21.15.137:0/2398016102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f05040ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 --2- 172.21.15.137:0/2398016102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f0504105e30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 -- 172.21.15.137:0/2398016102 >> 172.21.15.137:0/2398016102 conn(0x7f05040fb540 msgr2=0x7f05040fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 -- 172.21.15.137:0/2398016102 shutdown_connections 2024-06-15T01:21:01.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.723+0000 7f050889f700 1 -- 172.21.15.137:0/2398016102 wait complete. 2024-06-15T01:21:01.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 Processor -- start 2024-06-15T01:21:01.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 -- start start 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f050419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050419f200 con 0x7f0504105a50 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050889f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05041986b0 con 0x7f05040ffa50 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f04fbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f04fbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37910/0 (socket says 172.21.15.137:37910) 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f04fbfff700 1 -- 172.21.15.137:0/1770674243 learned_addr learned my addr 172.21.15.137:0/1770674243 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.724+0000 7f050259c700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f050419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fbfff700 1 -- 172.21.15.137:0/1770674243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 msgr2=0x7f050419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fbfff700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f050419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fbfff700 1 -- 172.21.15.137:0/1770674243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40095f0 con 0x7f0504105a50 2024-06-15T01:21:01.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f050259c700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f050419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fbfff700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00d8d0 tx=0x7f04ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04ec009940 con 0x7f0504105a50 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0504198990 con 0x7f0504105a50 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04ec010460 con 0x7f0504105a50 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04ec00f5f0 con 0x7f0504105a50 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.725+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0504198ee0 con 0x7f0504105a50 2024-06-15T01:21:01.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.726+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050404ea90 con 0x7f0504105a50 2024-06-15T01:21:01.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.727+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f04ec0105d0 con 0x7f0504105a50 2024-06-15T01:21:01.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.727+0000 7f04fb7fe700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 0x7f04f006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:01.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.727+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f04ec08a3e0 con 0x7f0504105a50 2024-06-15T01:21:01.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.727+0000 7f050259c700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 0x7f04f006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:01.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.728+0000 7f050259c700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 0x7f04f006d440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04f400b5c0 tx=0x7f04f4005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:01.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:01.732+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f04ec058ca0 con 0x7f0504105a50 2024-06-15T01:21:01.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.001+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0504198b20 con 0x7f0504105a50 2024-06-15T01:21:01.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.002+0000 7f04fb7fe700 1 -- 172.21.15.137:0/1770674243 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0504198b20 con 0x7f0504105a50 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:21:01.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:21:01.991 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 msgr2=0x7f04f006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 0x7f04f006d440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04f400b5c0 tx=0x7f04f4005960 comp rx=0 tx=0).stop 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 msgr2=0x7f050419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00d8d0 tx=0x7f04ec00dc90 comp rx=0 tx=0).stop 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 shutdown_connections 2024-06-15T01:21:01.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04f006af80 0x7f04f006d440 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05040ffa50 0x7f050419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 --2- 172.21.15.137:0/1770674243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0504105a50 0x7f050419eb70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:01.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 >> 172.21.15.137:0/1770674243 conn(0x7f05040fb540 msgr2=0x7f0504068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:01.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 shutdown_connections 2024-06-15T01:21:01.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.004+0000 7f050889f700 1 -- 172.21.15.137:0/1770674243 wait complete. 2024-06-15T01:21:02.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- 172.21.15.137:0/4179507588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0101770 msgr2=0x7fbcb01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 --2- 172.21.15.137:0/4179507588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0101770 0x7fbcb01057c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009910 tx=0x7fbca4009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- 172.21.15.137:0/4179507588 shutdown_connections 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 --2- 172.21.15.137:0/4179507588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0101770 0x7fbcb01057c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 --2- 172.21.15.137:0/4179507588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- 172.21.15.137:0/4179507588 >> 172.21.15.137:0/4179507588 conn(0x7fbcb00fc650 msgr2=0x7fbcb00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- 172.21.15.137:0/4179507588 shutdown_connections 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- 172.21.15.137:0/4179507588 wait complete. 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 Processor -- start 2024-06-15T01:21:02.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.110+0000 7fbcb632b700 1 -- start start 2024-06-15T01:21:02.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcb632b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcb632b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0101770 0x7fbcb0196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcb632b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb01972a0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcb632b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb0199fd0 con 0x7fbcb0101770 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37928/0 (socket says 172.21.15.137:37928) 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 -- 172.21.15.137:0/196007210 learned_addr learned my addr 172.21.15.137:0/196007210 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaf7fe700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0101770 0x7fbcb0196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 -- 172.21.15.137:0/196007210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0101770 msgr2=0x7fbcb0196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0101770 0x7fbcb0196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 -- 172.21.15.137:0/196007210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40095f0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcaffff700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbca000d8d0 tx=0x7fbca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca0009940 con 0x7fbcb0100dc0 2024-06-15T01:21:02.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbca0010460 con 0x7fbcb0100dc0 2024-06-15T01:21:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb019a2b0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.111+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca000f5f0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.112+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb019a7d0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.112+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb01091c0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.113+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbca00105d0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.113+0000 7fbcad7fa700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 0x7fbc9806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.113+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbca008a3e0 con 0x7fbcb0100dc0 2024-06-15T01:21:02.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.114+0000 7fbcaf7fe700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 0x7fbc9806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.114+0000 7fbcaf7fe700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 0x7fbc9806d440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0197df0 tx=0x7fbca401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.118+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbca0055210 con 0x7fbcb0100dc0 2024-06-15T01:21:02.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:02 smithi137 ceph-mon[112295]: pgmap v448: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:02.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:02 smithi137 ceph-mon[112295]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:02.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:02 smithi137 ceph-mon[112295]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:02.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:02 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1770674243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:21:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:02 smithi142 ceph-mon[82136]: pgmap v448: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:02 smithi142 ceph-mon[82136]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:02 smithi142 ceph-mon[82136]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:02 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1770674243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:21:02.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.353+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbcb004ea90 con 0x7fbcb0100dc0 2024-06-15T01:21:02.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.353+0000 7fbcad7fa700 1 -- 172.21.15.137:0/196007210 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fbca0010880 con 0x7fbcb0100dc0 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:21:02.341 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:21:02.342 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:02.343 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:02.344 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:02.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 msgr2=0x7fbc9806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 0x7fbc9806d440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0197df0 tx=0x7fbca401a040 comp rx=0 tx=0).stop 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 msgr2=0x7fbcb01966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbca000d8d0 tx=0x7fbca000dc90 comp rx=0 tx=0).stop 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 shutdown_connections 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc9806af80 0x7fbc9806d440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcb0100dc0 0x7fbcb01966d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 --2- 172.21.15.137:0/196007210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb0101770 0x7fbcb0196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 >> 172.21.15.137:0/196007210 conn(0x7fbcb00fc650 msgr2=0x7fbcb00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 shutdown_connections 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.355+0000 7fbcb632b700 1 -- 172.21.15.137:0/196007210 wait complete. 2024-06-15T01:21:02.345 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 -- 172.21.15.137:0/2291241450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc0ffb70 msgr2=0x7fc2cc0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 --2- 172.21.15.137:0/2291241450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc0fffd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009910 tx=0x7fc2bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 -- 172.21.15.137:0/2291241450 shutdown_connections 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 --2- 172.21.15.137:0/2291241450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc0fffd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 --2- 172.21.15.137:0/2291241450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc105a30 0x7fc2cc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 -- 172.21.15.137:0/2291241450 >> 172.21.15.137:0/2291241450 conn(0x7fc2cc0754f0 msgr2=0x7fc2cc075900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 -- 172.21.15.137:0/2291241450 shutdown_connections 2024-06-15T01:21:02.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.461+0000 7fc2d2333700 1 -- 172.21.15.137:0/2291241450 wait complete. 2024-06-15T01:21:02.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 Processor -- start 2024-06-15T01:21:02.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 -- start start 2024-06-15T01:21:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc19d0e0 con 0x7fc2cc105a30 2024-06-15T01:21:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2d2333700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc196590 con 0x7fc2cc0ffb70 2024-06-15T01:21:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37956/0 (socket says 172.21.15.137:37956) 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 -- 172.21.15.137:0/3741360134 learned_addr learned my addr 172.21.15.137:0/3741360134 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cbfff700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 -- 172.21.15.137:0/3741360134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc0ffb70 msgr2=0x7fc2cc19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.462+0000 7fc2cb7fe700 1 -- 172.21.15.137:0/3741360134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0095f0 con 0x7fc2cc105a30 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2cb7fe700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006010 tx=0x7fc2bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2bc01d070 con 0x7fc2cc105a30 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc2bc00f460 con 0x7fc2cc105a30 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc196810 con 0x7fc2cc105a30 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2bc021640 con 0x7fc2cc105a30 2024-06-15T01:21:02.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.463+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc196d00 con 0x7fc2cc105a30 2024-06-15T01:21:02.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.464+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2cc108ff0 con 0x7fc2cc105a30 2024-06-15T01:21:02.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.465+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc2bc00fa60 con 0x7fc2cc105a30 2024-06-15T01:21:02.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.465+0000 7fc2c97fa700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 0x7fc2b406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.465+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc2bc08b6c0 con 0x7fc2cc105a30 2024-06-15T01:21:02.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.465+0000 7fc2cbfff700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 0x7fc2b406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.466+0000 7fc2cbfff700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 0x7fc2b406d440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006fd0 tx=0x7fc2c0008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.470+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc2bc059f80 con 0x7fc2cc105a30 2024-06-15T01:21:02.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.679+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2cc02ccf0 con 0x7fc2b406af80 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.680+0000 7fc2c97fa700 1 -- 172.21.15.137:0/3741360134 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc2cc02ccf0 con 0x7fc2b406af80 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:21:02.668 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:21:02.669 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 msgr2=0x7fc2b406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 0x7fc2b406d440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0006fd0 tx=0x7fc2c0008040 comp rx=0 tx=0).stop 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 msgr2=0x7fc2cc19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006010 tx=0x7fc2bc00be20 comp rx=0 tx=0).stop 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 shutdown_connections 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc2b406af80 0x7fc2b406d440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2cc0ffb70 0x7fc2cc19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 --2- 172.21.15.137:0/3741360134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc105a30 0x7fc2cc19ca50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 >> 172.21.15.137:0/3741360134 conn(0x7fc2cc0754f0 msgr2=0x7fc2cc068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:02.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 shutdown_connections 2024-06-15T01:21:02.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.683+0000 7fc2d2333700 1 -- 172.21.15.137:0/3741360134 wait complete. 2024-06-15T01:21:02.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 -- 172.21.15.137:0/4136545366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 msgr2=0x7f627c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 --2- 172.21.15.137:0/4136545366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c100050 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6270009910 tx=0x7f6270009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 -- 172.21.15.137:0/4136545366 shutdown_connections 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 --2- 172.21.15.137:0/4136545366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c100050 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 --2- 172.21.15.137:0/4136545366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f627c105a40 0x7f627c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 -- 172.21.15.137:0/4136545366 >> 172.21.15.137:0/4136545366 conn(0x7f627c0755a0 msgr2=0x7f627c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 -- 172.21.15.137:0/4136545366 shutdown_connections 2024-06-15T01:21:02.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.790+0000 7f62832d2700 1 -- 172.21.15.137:0/4136545366 wait complete. 2024-06-15T01:21:02.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 Processor -- start 2024-06-15T01:21:02.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 -- start start 2024-06-15T01:21:02.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f627c105a40 0x7f627c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c102090 con 0x7f627c0ffbd0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f62832d2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1021d0 con 0x7f627c105a40 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37976/0 (socket says 172.21.15.137:37976) 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 -- 172.21.15.137:0/2063981945 learned_addr learned my addr 172.21.15.137:0/2063981945 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 -- 172.21.15.137:0/2063981945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f627c105a40 msgr2=0x7f627c101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f627c105a40 0x7f627c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 -- 172.21.15.137:0/2063981945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700095f0 con 0x7f627c0ffbd0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.791+0000 7f628106e700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6274007e90 tx=0x7f627400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.792+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627400de60 con 0x7f627c0ffbd0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.792+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f627400f040 con 0x7f627c0ffbd0 2024-06-15T01:21:02.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.792+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6274004520 con 0x7f627c0ffbd0 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.792+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c1a2910 con 0x7f627c0ffbd0 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.792+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c1a2d20 con 0x7f627c0ffbd0 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.793+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c108fe0 con 0x7f627c0ffbd0 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.793+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6274004680 con 0x7f627c0ffbd0 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.793+0000 7f627a7fc700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 0x7f626406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:02.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.794+0000 7f628086d700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 0x7f626406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:02.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.794+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6274089e60 con 0x7f627c0ffbd0 2024-06-15T01:21:02.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.794+0000 7f628086d700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 0x7f626406d440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f627000b5c0 tx=0x7f6270005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:02.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:02.798+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6274054d10 con 0x7f627c0ffbd0 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.071+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f627c04ea90 con 0x7f627c0ffbd0 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.071+0000 7f627a7fc700 1 -- 172.21.15.137:0/2063981945 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f6274019020 con 0x7f627c0ffbd0 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:21:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:21:03.060 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:21:03.060 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:21:03.060 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:21:03.060 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 msgr2=0x7f626406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 0x7f626406d440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f627000b5c0 tx=0x7f6270005960 comp rx=0 tx=0).stop 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 msgr2=0x7f627c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6274007e90 tx=0x7f627400d3b0 comp rx=0 tx=0).stop 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 shutdown_connections 2024-06-15T01:21:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f626406af80 0x7f626406d440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:03.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f627c0ffbd0 0x7f627c101610 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:03.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 --2- 172.21.15.137:0/2063981945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f627c105a40 0x7f627c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:03.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 >> 172.21.15.137:0/2063981945 conn(0x7f627c0755a0 msgr2=0x7f627c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:03.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 shutdown_connections 2024-06-15T01:21:03.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:03.073+0000 7f62832d2700 1 -- 172.21.15.137:0/2063981945 wait complete. 2024-06-15T01:21:03.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:03 smithi142 ceph-mon[82136]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:03.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:03 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/196007210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:21:03.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:03.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:03 smithi137 ceph-mon[112295]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:03.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:03 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/196007210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:21:03.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:04.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:04 smithi142 ceph-mon[82136]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:04.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:04 smithi142 ceph-mon[82136]: pgmap v449: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:04.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:04 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2063981945' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:21:04.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:04 smithi137 ceph-mon[112295]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:04.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:04 smithi137 ceph-mon[112295]: pgmap v449: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:04.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:04 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2063981945' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:21:06.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:06 smithi142 ceph-mon[82136]: pgmap v450: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:06.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:06 smithi137 ceph-mon[112295]: pgmap v450: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:08.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:08 smithi142 ceph-mon[82136]: pgmap v451: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:08.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:08 smithi137 ceph-mon[112295]: pgmap v451: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:10.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:10 smithi142 ceph-mon[82136]: pgmap v452: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:10.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:10 smithi137 ceph-mon[112295]: pgmap v452: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:12.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:12 smithi137 ceph-mon[112295]: pgmap v453: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:12 smithi142 ceph-mon[82136]: pgmap v453: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:14.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:14 smithi142 ceph-mon[82136]: pgmap v454: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:14.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:14 smithi137 ceph-mon[112295]: pgmap v454: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:16.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:16 smithi142 ceph-mon[82136]: pgmap v455: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:16.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:16 smithi137 ceph-mon[112295]: pgmap v455: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:18.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:18 smithi137 ceph-mon[112295]: pgmap v456: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:18.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:18 smithi142 ceph-mon[82136]: pgmap v456: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:20.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:20 smithi142 ceph-mon[82136]: pgmap v457: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:20.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:20 smithi137 ceph-mon[112295]: pgmap v457: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:22.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:22 smithi137 ceph-mon[112295]: pgmap v458: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:22.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:22 smithi142 ceph-mon[82136]: pgmap v458: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:23.716 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:23 smithi137 ceph-mon[112295]: pgmap v459: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:23.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:23 smithi142 ceph-mon[82136]: pgmap v459: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:26.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:25 smithi137 ceph-mon[112295]: pgmap v460: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:26.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:25 smithi142 ceph-mon[82136]: pgmap v460: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:28.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:27 smithi137 ceph-mon[112295]: pgmap v461: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:28.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:27 smithi142 ceph-mon[82136]: pgmap v461: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:30.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:29 smithi142 ceph-mon[82136]: pgmap v462: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:30.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:29 smithi137 ceph-mon[112295]: pgmap v462: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:32.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:31 smithi137 ceph-mon[112295]: pgmap v463: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:32.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:31 smithi142 ceph-mon[82136]: pgmap v463: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 -- 172.21.15.137:0/742913842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd68105a40 msgr2=0x7efd68105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 --2- 172.21.15.137:0/742913842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd68105a40 0x7efd68105e20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd58009910 tx=0x7efd58009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 -- 172.21.15.137:0/742913842 shutdown_connections 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 --2- 172.21.15.137:0/742913842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd680ffa40 0x7efd680ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 --2- 172.21.15.137:0/742913842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd68105a40 0x7efd68105e20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 -- 172.21.15.137:0/742913842 >> 172.21.15.137:0/742913842 conn(0x7efd680fb550 msgr2=0x7efd680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 -- 172.21.15.137:0/742913842 shutdown_connections 2024-06-15T01:21:33.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.187+0000 7efd6cf65700 1 -- 172.21.15.137:0/742913842 wait complete. 2024-06-15T01:21:33.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 Processor -- start 2024-06-15T01:21:33.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 -- start start 2024-06-15T01:21:33.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68105a40 0x7efd68197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd68198040 con 0x7efd680ffa40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6cf65700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd681981b0 con 0x7efd68105a40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45624/0 (socket says 172.21.15.137:45624) 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 -- 172.21.15.137:0/2516993216 learned_addr learned my addr 172.21.15.137:0/2516993216 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd5ffff700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68105a40 0x7efd68197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 -- 172.21.15.137:0/2516993216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68105a40 msgr2=0x7efd68197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68105a40 0x7efd68197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.188+0000 7efd6659c700 1 -- 172.21.15.137:0/2516993216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580095f0 con 0x7efd680ffa40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd6659c700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd58006010 tx=0x7efd5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd5801d070 con 0x7efd680ffa40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd681a0550 con 0x7efd680ffa40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd58005710 con 0x7efd680ffa40 2024-06-15T01:21:33.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd5800fe20 con 0x7efd680ffa40 2024-06-15T01:21:33.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.189+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd681a09c0 con 0x7efd680ffa40 2024-06-15T01:21:33.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.190+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd68108040 con 0x7efd680ffa40 2024-06-15T01:21:33.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.190+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efd58003a60 con 0x7efd680ffa40 2024-06-15T01:21:33.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.191+0000 7efd5f7fe700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 0x7efd5406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.191+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efd5808b560 con 0x7efd680ffa40 2024-06-15T01:21:33.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.191+0000 7efd5ffff700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 0x7efd5406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.191+0000 7efd5ffff700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 0x7efd5406d440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd68069680 tx=0x7efd5000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.195+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efd58056390 con 0x7efd680ffa40 2024-06-15T01:21:33.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.407+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd680611d0 con 0x7efd5406af80 2024-06-15T01:21:33.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.408+0000 7efd5f7fe700 1 -- 172.21.15.137:0/2516993216 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efd680611d0 con 0x7efd5406af80 2024-06-15T01:21:33.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 msgr2=0x7efd5406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 0x7efd5406d440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd68069680 tx=0x7efd5000b410 comp rx=0 tx=0).stop 2024-06-15T01:21:33.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 msgr2=0x7efd681974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd58006010 tx=0x7efd5800bd70 comp rx=0 tx=0).stop 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 shutdown_connections 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5406af80 0x7efd5406d440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd680ffa40 0x7efd681974e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 --2- 172.21.15.137:0/2516993216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd68105a40 0x7efd68197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 >> 172.21.15.137:0/2516993216 conn(0x7efd680fb550 msgr2=0x7efd680fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 shutdown_connections 2024-06-15T01:21:33.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.411+0000 7efd6cf65700 1 -- 172.21.15.137:0/2516993216 wait complete. 2024-06-15T01:21:33.408 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:21:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 -- 172.21.15.137:0/3864500985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 msgr2=0x7f46cc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/3864500985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc1011a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b0a0 tx=0x7f46bc00b3b0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 -- 172.21.15.137:0/3864500985 shutdown_connections 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/3864500985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 0x7f46cc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/3864500985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc1011a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 -- 172.21.15.137:0/3864500985 >> 172.21.15.137:0/3864500985 conn(0x7f46cc0fc650 msgr2=0x7f46cc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 -- 172.21.15.137:0/3864500985 shutdown_connections 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 -- 172.21.15.137:0/3864500985 wait complete. 2024-06-15T01:21:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.517+0000 7f46d1f3e700 1 Processor -- start 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46d1f3e700 1 -- start start 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46d1f3e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46d1f3e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 0x7f46cc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46d1f3e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc073b10 con 0x7f46cc100dc0 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46d1f3e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc073c50 con 0x7f46cc101770 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45646/0 (socket says 172.21.15.137:45646) 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 -- 172.21.15.137:0/1727141895 learned_addr learned my addr 172.21.15.137:0/1727141895 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46caffd700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 0x7f46cc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 -- 172.21.15.137:0/1727141895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 msgr2=0x7f46cc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 0x7f46cc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 -- 172.21.15.137:0/1727141895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc009d00 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.518+0000 7f46cb7fe700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f80 tx=0x7f46bc0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.519+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc00e040 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.519+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46bc00be10 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.519+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc073ed0 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.519+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc0043b0 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.519+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc1a2b70 con 0x7f46cc100dc0 2024-06-15T01:21:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.520+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc109160 con 0x7f46cc100dc0 2024-06-15T01:21:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.520+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f46bc019070 con 0x7f46cc100dc0 2024-06-15T01:21:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.521+0000 7f46c8ff9700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 0x7f46b406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.521+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f46bc08ba80 con 0x7f46cc100dc0 2024-06-15T01:21:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.521+0000 7f46caffd700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 0x7f46b406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.521+0000 7f46caffd700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 0x7f46b406d440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f46cc074880 tx=0x7f46c0009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.525+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f46bc0568b0 con 0x7f46cc100dc0 2024-06-15T01:21:33.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.735+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46cc02cd50 con 0x7f46b406af80 2024-06-15T01:21:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.737+0000 7f46c8ff9700 1 -- 172.21.15.137:0/1727141895 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46cc02cd50 con 0x7f46b406af80 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 msgr2=0x7f46b406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 0x7f46b406d440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f46cc074880 tx=0x7f46c0009260 comp rx=0 tx=0).stop 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 msgr2=0x7f46cc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f80 tx=0x7f46bc0096e0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 shutdown_connections 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46b406af80 0x7f46b406d440 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46cc100dc0 0x7f46cc073090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 --2- 172.21.15.137:0/1727141895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46cc101770 0x7f46cc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 >> 172.21.15.137:0/1727141895 conn(0x7f46cc0fc650 msgr2=0x7f46cc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 shutdown_connections 2024-06-15T01:21:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.739+0000 7f46d1f3e700 1 -- 172.21.15.137:0/1727141895 wait complete. 2024-06-15T01:21:33.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- 172.21.15.137:0/2076545409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c1026c0 msgr2=0x7fa98c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 --2- 172.21.15.137:0/2076545409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c1026c0 0x7fa98c10abb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa988009910 tx=0x7fa988009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:33.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- 172.21.15.137:0/2076545409 shutdown_connections 2024-06-15T01:21:33.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 --2- 172.21.15.137:0/2076545409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c1026c0 0x7fa98c10abb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 --2- 172.21.15.137:0/2076545409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c101da0 0x7fa98c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- 172.21.15.137:0/2076545409 >> 172.21.15.137:0/2076545409 conn(0x7fa98c0fb530 msgr2=0x7fa98c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- 172.21.15.137:0/2076545409 shutdown_connections 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- 172.21.15.137:0/2076545409 wait complete. 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 Processor -- start 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.843+0000 7fa994744700 1 -- start start 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c1026c0 0x7fa98c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c197270 con 0x7fa98c101da0 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c199fa0 con 0x7fa98c1026c0 2024-06-15T01:21:33.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45652/0 (socket says 172.21.15.137:45652) 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 -- 172.21.15.137:0/1043638874 learned_addr learned my addr 172.21.15.137:0/1043638874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa991cdf700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c1026c0 0x7fa98c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 -- 172.21.15.137:0/1043638874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c1026c0 msgr2=0x7fa98c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c1026c0 0x7fa98c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 -- 172.21.15.137:0/1043638874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880095f0 con 0x7fa98c101da0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9924e0700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e920 tx=0x7fa97c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97c00c9d0 con 0x7fa98c101da0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa97c010430 con 0x7fa98c101da0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c19a280 con 0x7fa98c101da0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97c018650 con 0x7fa98c101da0 2024-06-15T01:21:33.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.844+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c19a7a0 con 0x7fa98c101da0 2024-06-15T01:21:33.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.846+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c1082b0 con 0x7fa98c101da0 2024-06-15T01:21:33.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.846+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa97c0105a0 con 0x7fa98c101da0 2024-06-15T01:21:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.846+0000 7fa9837fe700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 0x7fa97806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.846+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa97c014070 con 0x7fa98c101da0 2024-06-15T01:21:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.847+0000 7fa991cdf700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 0x7fa97806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:33.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.847+0000 7fa991cdf700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 0x7fa97806d440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa988009fd0 tx=0x7fa988005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:33.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:33.851+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa97c052900 con 0x7fa98c101da0 2024-06-15T01:21:34.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.051+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa98c19a990 con 0x7fa97806af80 2024-06-15T01:21:34.038 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:33 smithi137 ceph-mon[112295]: pgmap v464: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:34.038 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:34.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.064+0000 7fa9837fe700 1 -- 172.21.15.137:0/1043638874 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa98c19a990 con 0x7fa97806af80 2024-06-15T01:21:34.050 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:21:34.050 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (12m) 6m ago 22m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (22m) 6m ago 22m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (21m) 6m ago 21m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (10m) 6m ago 22m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (10m) 6m ago 21m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (11m) 6m ago 22m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (6m) 6m ago 17m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (6m) 6m ago 17m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (6m) 6m ago 17m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (6m) 6m ago 17m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (13m) 6m ago 24m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (12m) 6m ago 21m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (10m) 6m ago 24m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:21:34.051 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (10m) 6m ago 21m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (12m) 6m ago 22m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (12m) 6m ago 21m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (9m) 6m ago 20m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (9m) 6m ago 20m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (8m) 6m ago 19m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (8m) 6m ago 19m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (7m) 6m ago 19m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (7m) 6m ago 18m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:21:34.052 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (12m) 6m ago 22m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 msgr2=0x7fa97806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 0x7fa97806d440 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa988009fd0 tx=0x7fa988005940 comp rx=0 tx=0).stop 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 msgr2=0x7fa98c1966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e920 tx=0x7fa97c00ec30 comp rx=0 tx=0).stop 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 shutdown_connections 2024-06-15T01:21:34.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa97806af80 0x7fa97806d440 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa98c101da0 0x7fa98c1966a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 --2- 172.21.15.137:0/1043638874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa98c1026c0 0x7fa98c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 >> 172.21.15.137:0/1043638874 conn(0x7fa98c0fb530 msgr2=0x7fa98c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.066+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 shutdown_connections 2024-06-15T01:21:34.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.067+0000 7fa994744700 1 -- 172.21.15.137:0/1043638874 wait complete. 2024-06-15T01:21:34.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.168+0000 7f4b08b37700 1 -- 172.21.15.137:0/3445477958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 msgr2=0x7f4b041011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.168+0000 7f4b08b37700 1 --2- 172.21.15.137:0/3445477958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b041011a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009910 tx=0x7f4af4009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 -- 172.21.15.137:0/3445477958 shutdown_connections 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 --2- 172.21.15.137:0/3445477958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b041057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 --2- 172.21.15.137:0/3445477958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b041011a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 -- 172.21.15.137:0/3445477958 >> 172.21.15.137:0/3445477958 conn(0x7f4b040fc650 msgr2=0x7f4b040fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 -- 172.21.15.137:0/3445477958 shutdown_connections 2024-06-15T01:21:34.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 -- 172.21.15.137:0/3445477958 wait complete. 2024-06-15T01:21:34.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.169+0000 7f4b08b37700 1 Processor -- start 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 -- start start 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b04196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04197290 con 0x7f4b04100dc0 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0419b020 con 0x7f4b04101770 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45672/0 (socket says 172.21.15.137:45672) 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 -- 172.21.15.137:0/2751437467 learned_addr learned my addr 172.21.15.137:0/2751437467 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b01d9b700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b04196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 -- 172.21.15.137:0/2751437467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 msgr2=0x7f4b04196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b04196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 -- 172.21.15.137:0/2751437467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40095f0 con 0x7f4b04100dc0 2024-06-15T01:21:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b01d9b700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b04196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b0259c700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4af4006010 tx=0x7f4af400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af401d070 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.170+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0419b2a0 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.171+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4af400f460 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.171+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af4021620 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.171+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0419b790 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.171+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b04109160 con 0x7f4b04100dc0 2024-06-15T01:21:34.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.172+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4af400fa40 con 0x7f4b04100dc0 2024-06-15T01:21:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.172+0000 7f4af37fe700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 0x7f4aec06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.172+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4af408b530 con 0x7f4b04100dc0 2024-06-15T01:21:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.172+0000 7f4b01d9b700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 0x7f4aec06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.173+0000 7f4b01d9b700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 0x7f4aec06d440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b04197c90 tx=0x7f4af800b5f0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.177+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4af4056340 con 0x7f4b04100dc0 2024-06-15T01:21:34.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:33 smithi142 ceph-mon[82136]: pgmap v464: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:34.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:34.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.448+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b0419bac0 con 0x7f4b04100dc0 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.448+0000 7f4af37fe700 1 -- 172.21.15.137:0/2751437467 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f4af4059960 con 0x7f4b04100dc0 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:21:34.435 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:21:34.436 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.450+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 msgr2=0x7f4aec06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.450+0000 7f4b08b37700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 0x7f4aec06d440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b04197c90 tx=0x7f4af800b5f0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.450+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 msgr2=0x7f4b04196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.450+0000 7f4b08b37700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4af4006010 tx=0x7f4af400be20 comp rx=0 tx=0).stop 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 shutdown_connections 2024-06-15T01:21:34.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4aec06af80 0x7f4aec06d440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b04100dc0 0x7f4b04196670 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 --2- 172.21.15.137:0/2751437467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b04101770 0x7f4b04196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 >> 172.21.15.137:0/2751437467 conn(0x7f4b040fc650 msgr2=0x7f4b040fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 shutdown_connections 2024-06-15T01:21:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.451+0000 7f4b08b37700 1 -- 172.21.15.137:0/2751437467 wait complete. 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 -- 172.21.15.137:0/3822997629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c100dc0 msgr2=0x7f0b7c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/3822997629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c1011a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009910 tx=0x7f0b70009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 -- 172.21.15.137:0/3822997629 shutdown_connections 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/3822997629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c101770 0x7f0b7c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/3822997629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c1011a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.554+0000 7f0b81f3e700 1 -- 172.21.15.137:0/3822997629 >> 172.21.15.137:0/3822997629 conn(0x7f0b7c0fc650 msgr2=0x7f0b7c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.555+0000 7f0b81f3e700 1 -- 172.21.15.137:0/3822997629 shutdown_connections 2024-06-15T01:21:34.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.555+0000 7f0b81f3e700 1 -- 172.21.15.137:0/3822997629 wait complete. 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.555+0000 7f0b81f3e700 1 Processor -- start 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.555+0000 7f0b81f3e700 1 -- start start 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.555+0000 7f0b81f3e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b81f3e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b81f3e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c073b10 con 0x7f0b7c101770 2024-06-15T01:21:34.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b81f3e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c073c50 con 0x7f0b7c100dc0 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45692/0 (socket says 172.21.15.137:45692) 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 -- 172.21.15.137:0/309515189 learned_addr learned my addr 172.21.15.137:0/309515189 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7b7fe700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 -- 172.21.15.137:0/309515189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c100dc0 msgr2=0x7f0b7c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 -- 172.21.15.137:0/309515189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700095f0 con 0x7f0b7c101770 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b7affd700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00d900 tx=0x7f0b6c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6c009900 con 0x7f0b7c101770 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c1a2840 con 0x7f0b7c101770 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b6c010460 con 0x7f0b7c101770 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6c00f5f0 con 0x7f0b7c101770 2024-06-15T01:21:34.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.556+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c1a2c50 con 0x7f0b7c101770 2024-06-15T01:21:34.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.557+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c109160 con 0x7f0b7c101770 2024-06-15T01:21:34.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.558+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0b6c0105d0 con 0x7f0b7c101770 2024-06-15T01:21:34.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.558+0000 7f0b78ff9700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 0x7f0b6406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.558+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0b6c089e70 con 0x7f0b7c101770 2024-06-15T01:21:34.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.558+0000 7f0b7b7fe700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 0x7f0b6406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.559+0000 7f0b7b7fe700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 0x7f0b6406d440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b5c0 tx=0x7f0b7001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.562+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0b6c054ca0 con 0x7f0b7c101770 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.795+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b7c04ea90 con 0x7f0b7c101770 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.795+0000 7f0b78ff9700 1 -- 172.21.15.137:0/309515189 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f0b6c020750 con 0x7f0b7c101770 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:21:34.782 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:21:34.783 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:21:34.784 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:34.785 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:21:34.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.798+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 msgr2=0x7f0b6406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.798+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 0x7f0b6406d440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000b5c0 tx=0x7f0b7001a040 comp rx=0 tx=0).stop 2024-06-15T01:21:34.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.798+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 msgr2=0x7f0b7c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.798+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00d900 tx=0x7f0b6c00dc10 comp rx=0 tx=0).stop 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.798+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 shutdown_connections 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0b6406af80 0x7f0b6406d440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c100dc0 0x7f0b7c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 --2- 172.21.15.137:0/309515189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b7c101770 0x7f0b7c0735d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 >> 172.21.15.137:0/309515189 conn(0x7f0b7c0fc650 msgr2=0x7f0b7c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 shutdown_connections 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.799+0000 7f0b81f3e700 1 -- 172.21.15.137:0/309515189 wait complete. 2024-06-15T01:21:34.787 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:21:34.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 -- 172.21.15.137:0/500277470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 msgr2=0x7fccb80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 --2- 172.21.15.137:0/500277470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb80ffec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009910 tx=0x7fccb4009c20 comp rx=0 tx=0).stop 2024-06-15T01:21:34.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 -- 172.21.15.137:0/500277470 shutdown_connections 2024-06-15T01:21:34.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 --2- 172.21.15.137:0/500277470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb80ffec0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 --2- 172.21.15.137:0/500277470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 0x7fccb8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 -- 172.21.15.137:0/500277470 >> 172.21.15.137:0/500277470 conn(0x7fccb80fb570 msgr2=0x7fccb80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 -- 172.21.15.137:0/500277470 shutdown_connections 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.905+0000 7fccbf68b700 1 -- 172.21.15.137:0/500277470 wait complete. 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 Processor -- start 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 -- start start 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 0x7fccb8196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81971c0 con 0x7fccb80ffa40 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbf68b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8197300 con 0x7fccb8105a40 2024-06-15T01:21:34.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbd427700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbd427700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45706/0 (socket says 172.21.15.137:45706) 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbd427700 1 -- 172.21.15.137:0/1796465056 learned_addr learned my addr 172.21.15.137:0/1796465056 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.906+0000 7fccbcc26700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 0x7fccb8196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbd427700 1 -- 172.21.15.137:0/1796465056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 msgr2=0x7fccb8196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbd427700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 0x7fccb8196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbd427700 1 -- 172.21.15.137:0/1796465056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb40095f0 con 0x7fccb80ffa40 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbd427700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c930 tx=0x7fcca800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcca8007ad0 con 0x7fccb80ffa40 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcca800ce80 con 0x7fccb80ffa40 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb819b150 con 0x7fccb80ffa40 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcca8018630 con 0x7fccb80ffa40 2024-06-15T01:21:34.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.907+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb819b6a0 con 0x7fccb80ffa40 2024-06-15T01:21:34.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.908+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb8108fe0 con 0x7fccb80ffa40 2024-06-15T01:21:34.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.909+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcca8007c30 con 0x7fccb80ffa40 2024-06-15T01:21:34.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.909+0000 7fccae7fc700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 0x7fcca406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:34.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.909+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcca808a350 con 0x7fccb80ffa40 2024-06-15T01:21:34.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.909+0000 7fccbcc26700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 0x7fcca406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:34.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.910+0000 7fccbcc26700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 0x7fcca406d440 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fccb8197c50 tx=0x7fccb401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:34.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:34.914+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcca8055160 con 0x7fccb80ffa40 2024-06-15T01:21:35.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:34 smithi142 ceph-mon[82136]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:34 smithi142 ceph-mon[82136]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:34 smithi142 ceph-mon[82136]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.059 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:34 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2751437467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:21:35.059 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:34 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/309515189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:21:35.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.124+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccb802cd80 con 0x7fcca406af80 2024-06-15T01:21:35.111 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:34 smithi137 ceph-mon[112295]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.111 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:34 smithi137 ceph-mon[112295]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.111 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:34 smithi137 ceph-mon[112295]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:35.111 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:34 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2751437467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:21:35.112 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:34 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/309515189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:21:35.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.126+0000 7fccae7fc700 1 -- 172.21.15.137:0/1796465056 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fccb802cd80 con 0x7fcca406af80 2024-06-15T01:21:35.112 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:21:35.112 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:21:35.112 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:21:35.113 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:21:35.114 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:21:35.114 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 msgr2=0x7fcca406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 0x7fcca406d440 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fccb8197c50 tx=0x7fccb401a040 comp rx=0 tx=0).stop 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 msgr2=0x7fccb8196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c930 tx=0x7fcca800cc40 comp rx=0 tx=0).stop 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 shutdown_connections 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcca406af80 0x7fcca406d440 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb80ffa40 0x7fccb8196630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 --2- 172.21.15.137:0/1796465056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccb8105a40 0x7fccb8196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 >> 172.21.15.137:0/1796465056 conn(0x7fccb80fb570 msgr2=0x7fccb80690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 shutdown_connections 2024-06-15T01:21:35.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.128+0000 7fccbf68b700 1 -- 172.21.15.137:0/1796465056 wait complete. 2024-06-15T01:21:35.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 -- 172.21.15.137:0/2465498161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 msgr2=0x7f84cc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/2465498161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc1011a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b0a0 tx=0x7f84bc00b3b0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 -- 172.21.15.137:0/2465498161 shutdown_connections 2024-06-15T01:21:35.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/2465498161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/2465498161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc1011a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 -- 172.21.15.137:0/2465498161 >> 172.21.15.137:0/2465498161 conn(0x7f84cc0fc650 msgr2=0x7f84cc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:35.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 -- 172.21.15.137:0/2465498161 shutdown_connections 2024-06-15T01:21:35.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.232+0000 7f84d3f1f700 1 -- 172.21.15.137:0/2465498161 wait complete. 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 Processor -- start 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 -- start start 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1971f0 con 0x7f84cc100dc0 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d3f1f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc197330 con 0x7f84cc101770 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45730/0 (socket says 172.21.15.137:45730) 2024-06-15T01:21:35.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 -- 172.21.15.137:0/100611630 learned_addr learned my addr 172.21.15.137:0/100611630 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d14ba700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 -- 172.21.15.137:0/100611630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 msgr2=0x7f84cc196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d1cbb700 1 -- 172.21.15.137:0/100611630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc009d00 con 0x7f84cc100dc0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.233+0000 7f84d14ba700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84d1cbb700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00d010 tx=0x7f84bc007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84bc00e040 con 0x7f84cc100dc0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84bc003e50 con 0x7f84cc100dc0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84bc01da10 con 0x7f84cc100dc0 2024-06-15T01:21:35.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc19b120 con 0x7f84cc100dc0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.234+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc19b610 con 0x7f84cc100dc0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.235+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc04ea90 con 0x7f84cc100dc0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.235+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f84bc004210 con 0x7f84cc100dc0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.236+0000 7f84caffd700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 0x7f84b406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.236+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f84bc08be70 con 0x7f84cc100dc0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.236+0000 7f84d14ba700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 0x7f84b406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.236+0000 7f84d14ba700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 0x7f84b406d440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fd0 tx=0x7f84c0005e30 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:21:35.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.240+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f84bc056ca0 con 0x7f84cc100dc0 2024-06-15T01:21:35.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.512+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f84cc108d80 con 0x7f84cc100dc0 2024-06-15T01:21:35.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.512+0000 7f84caffd700 1 -- 172.21.15.137:0/100611630 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f84bc05a2c0 con 0x7f84cc100dc0 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:21:35.499 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 msgr2=0x7f84b406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 0x7f84b406d440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fd0 tx=0x7f84c0005e30 comp rx=0 tx=0).stop 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 msgr2=0x7f84cc196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00d010 tx=0x7f84bc007be0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 shutdown_connections 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f84b406af80 0x7f84b406d440 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc100dc0 0x7f84cc196660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 --2- 172.21.15.137:0/100611630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84cc101770 0x7f84cc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 >> 172.21.15.137:0/100611630 conn(0x7f84cc0fc650 msgr2=0x7f84cc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 shutdown_connections 2024-06-15T01:21:35.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:21:35.514+0000 7f84d3f1f700 1 -- 172.21.15.137:0/100611630 wait complete. 2024-06-15T01:21:36.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:35 smithi137 ceph-mon[112295]: pgmap v465: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:36.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:35 smithi137 ceph-mon[112295]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:36.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:35 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/100611630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:21:36.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:35 smithi142 ceph-mon[82136]: pgmap v465: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:36.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:35 smithi142 ceph-mon[82136]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:21:36.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:35 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/100611630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:21:38.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:37 smithi137 ceph-mon[112295]: pgmap v466: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:38.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:37 smithi142 ceph-mon[82136]: pgmap v466: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:40.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:39 smithi137 ceph-mon[112295]: pgmap v467: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:39 smithi142 ceph-mon[82136]: pgmap v467: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:42.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:41 smithi137 ceph-mon[112295]: pgmap v468: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:42.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:41 smithi142 ceph-mon[82136]: pgmap v468: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:44.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:43 smithi137 ceph-mon[112295]: pgmap v469: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:44.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:43 smithi142 ceph-mon[82136]: pgmap v469: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:46.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:45 smithi137 ceph-mon[112295]: pgmap v470: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:46.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:45 smithi142 ceph-mon[82136]: pgmap v470: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:48.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:47 smithi137 ceph-mon[112295]: pgmap v471: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:48.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:47 smithi142 ceph-mon[82136]: pgmap v471: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:49.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:21:50.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:49 smithi137 ceph-mon[112295]: pgmap v472: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:50.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:49 smithi142 ceph-mon[82136]: pgmap v472: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:52.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:51 smithi137 ceph-mon[112295]: pgmap v473: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:52.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:51 smithi142 ceph-mon[82136]: pgmap v473: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:54.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:53 smithi137 ceph-mon[112295]: pgmap v474: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:54.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:53 smithi142 ceph-mon[82136]: pgmap v474: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:56.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:55 smithi137 ceph-mon[112295]: pgmap v475: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:56.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:55 smithi142 ceph-mon[82136]: pgmap v475: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:58.211 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:57 smithi137 ceph-mon[112295]: pgmap v476: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:21:58.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:57 smithi142 ceph-mon[82136]: pgmap v476: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:00.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:21:59 smithi137 ceph-mon[112295]: pgmap v477: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:00.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:21:59 smithi142 ceph-mon[82136]: pgmap v477: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:02.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:01 smithi137 ceph-mon[112295]: pgmap v478: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:01 smithi142 ceph-mon[82136]: pgmap v478: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:04.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:03 smithi137 ceph-mon[112295]: pgmap v479: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:04.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:04.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:03 smithi142 ceph-mon[82136]: pgmap v479: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:04.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:05.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.630+0000 7f7a9954b700 1 -- 172.21.15.137:0/179147619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94101770 msgr2=0x7f7a941057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.630+0000 7f7a9954b700 1 --2- 172.21.15.137:0/179147619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94101770 0x7f7a941057c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009910 tx=0x7f7a84009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:05.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 -- 172.21.15.137:0/179147619 shutdown_connections 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 --2- 172.21.15.137:0/179147619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94101770 0x7f7a941057c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 --2- 172.21.15.137:0/179147619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94100dc0 0x7f7a941011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 -- 172.21.15.137:0/179147619 >> 172.21.15.137:0/179147619 conn(0x7f7a940fc650 msgr2=0x7f7a940fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 -- 172.21.15.137:0/179147619 shutdown_connections 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 -- 172.21.15.137:0/179147619 wait complete. 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 Processor -- start 2024-06-15T01:22:05.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.631+0000 7f7a9954b700 1 -- start start 2024-06-15T01:22:05.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a9954b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a9954b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94101770 0x7f7a940735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a9954b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94073b10 con 0x7f7a94100dc0 2024-06-15T01:22:05.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a9954b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94073c50 con 0x7f7a94101770 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44886/0 (socket says 172.21.15.137:44886) 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 -- 172.21.15.137:0/1710456878 learned_addr learned my addr 172.21.15.137:0/1710456878 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a927fc700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94101770 0x7f7a940735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 -- 172.21.15.137:0/1710456878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94101770 msgr2=0x7f7a940735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94101770 0x7f7a940735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 -- 172.21.15.137:0/1710456878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840095f0 con 0x7f7a94100dc0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.632+0000 7f7a92ffd700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800d8d0 tx=0x7f7a8800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.633+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a880098a0 con 0x7f7a94100dc0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.633+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a941a28a0 con 0x7f7a94100dc0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.633+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a88010460 con 0x7f7a94100dc0 2024-06-15T01:22:05.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.633+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a88009ea0 con 0x7f7a94100dc0 2024-06-15T01:22:05.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.633+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a2cb0 con 0x7f7a94100dc0 2024-06-15T01:22:05.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.634+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a941091c0 con 0x7f7a94100dc0 2024-06-15T01:22:05.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.634+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7a88009a00 con 0x7f7a94100dc0 2024-06-15T01:22:05.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.634+0000 7f7a7bfff700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 0x7f7a7c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.635+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7a8808a260 con 0x7f7a94100dc0 2024-06-15T01:22:05.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.635+0000 7f7a927fc700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 0x7f7a7c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.635+0000 7f7a927fc700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 0x7f7a7c06d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a94074880 tx=0x7f7a8401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:05.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.639+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7a88055090 con 0x7f7a94100dc0 2024-06-15T01:22:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.847+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a94073e10 con 0x7f7a7c06af80 2024-06-15T01:22:05.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.848+0000 7f7a7bfff700 1 -- 172.21.15.137:0/1710456878 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7a94073e10 con 0x7f7a7c06af80 2024-06-15T01:22:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 msgr2=0x7f7a7c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 0x7f7a7c06d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a94074880 tx=0x7f7a8401a040 comp rx=0 tx=0).stop 2024-06-15T01:22:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 msgr2=0x7f7a94073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800d8d0 tx=0x7f7a8800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 shutdown_connections 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7a7c06af80 0x7f7a7c06d440 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a94100dc0 0x7f7a94073090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 --2- 172.21.15.137:0/1710456878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a94101770 0x7f7a940735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 >> 172.21.15.137:0/1710456878 conn(0x7f7a940fc650 msgr2=0x7f7a940fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 shutdown_connections 2024-06-15T01:22:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.850+0000 7f7a9954b700 1 -- 172.21.15.137:0/1710456878 wait complete. 2024-06-15T01:22:05.848 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:22:05.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 -- 172.21.15.137:0/702298856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0100dc0 msgr2=0x7f36b01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/702298856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0100dc0 0x7f36b01011a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009910 tx=0x7f36a4009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 -- 172.21.15.137:0/702298856 shutdown_connections 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/702298856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0101770 0x7f36b01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/702298856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0100dc0 0x7f36b01011a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 -- 172.21.15.137:0/702298856 >> 172.21.15.137:0/702298856 conn(0x7f36b00fc650 msgr2=0x7f36b00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 -- 172.21.15.137:0/702298856 shutdown_connections 2024-06-15T01:22:05.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.957+0000 7f36b6dd4700 1 -- 172.21.15.137:0/702298856 wait complete. 2024-06-15T01:22:05.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 Processor -- start 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 -- start start 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0100dc0 0x7f36b0196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0197290 con 0x7f36b0101770 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b6dd4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b019b020 con 0x7f36b0100dc0 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44898/0 (socket says 172.21.15.137:44898) 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36affff700 1 -- 172.21.15.137:0/3836713043 learned_addr learned my addr 172.21.15.137:0/3836713043 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.958+0000 7f36b4b70700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0100dc0 0x7f36b0196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36affff700 1 -- 172.21.15.137:0/3836713043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0100dc0 msgr2=0x7f36b0196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36affff700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0100dc0 0x7f36b0196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36affff700 1 -- 172.21.15.137:0/3836713043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40095f0 con 0x7f36b0101770 2024-06-15T01:22:05.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36affff700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f369c00d900 tx=0x7f369c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:05.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369c009900 con 0x7f36b0101770 2024-06-15T01:22:05.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f369c010460 con 0x7f36b0101770 2024-06-15T01:22:05.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b019b300 con 0x7f36b0101770 2024-06-15T01:22:05.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369c00f5f0 con 0x7f36b0101770 2024-06-15T01:22:05.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.959+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b019b850 con 0x7f36b0101770 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.960+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0109160 con 0x7f36b0101770 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.960+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f369c0105d0 con 0x7f36b0101770 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.961+0000 7f36adffb700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 0x7f36a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.961+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f369c089e70 con 0x7f36b0101770 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.961+0000 7f36b4b70700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 0x7f36a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:05.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.961+0000 7f36b4b70700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 0x7f36a006d440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:05.965+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f369c054ca0 con 0x7f36b0101770 2024-06-15T01:22:06.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.174+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36b019bb60 con 0x7f36a006af80 2024-06-15T01:22:06.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.175+0000 7f36adffb700 1 -- 172.21.15.137:0/3836713043 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36b019bb60 con 0x7f36a006af80 2024-06-15T01:22:06.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 msgr2=0x7f36a006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 0x7f36a006d440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a401a040 comp rx=0 tx=0).stop 2024-06-15T01:22:06.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 msgr2=0x7f36b0196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f369c00d900 tx=0x7f369c00dc10 comp rx=0 tx=0).stop 2024-06-15T01:22:06.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 shutdown_connections 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f36a006af80 0x7f36a006d440 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36b0100dc0 0x7f36b0196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 --2- 172.21.15.137:0/3836713043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b0101770 0x7f36b0196bb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 >> 172.21.15.137:0/3836713043 conn(0x7f36b00fc650 msgr2=0x7f36b00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 shutdown_connections 2024-06-15T01:22:06.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.177+0000 7f36b6dd4700 1 -- 172.21.15.137:0/3836713043 wait complete. 2024-06-15T01:22:06.165 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:05 smithi137 ceph-mon[112295]: pgmap v480: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.281+0000 7fc0d9c54700 1 -- 172.21.15.137:0/3519312026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 msgr2=0x7fc0d41046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.281+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/3519312026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d41046e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009910 tx=0x7fc0bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- 172.21.15.137:0/3519312026 shutdown_connections 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/3519312026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d41046e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/3519312026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 0x7fc0d4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- 172.21.15.137:0/3519312026 >> 172.21.15.137:0/3519312026 conn(0x7fc0d4074d80 msgr2=0x7fc0d4075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- 172.21.15.137:0/3519312026 shutdown_connections 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- 172.21.15.137:0/3519312026 wait complete. 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 Processor -- start 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- start start 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 0x7fc0d419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d419f260 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.282+0000 7fc0d9c54700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4198710 con 0x7fc0d4100620 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42112/0 (socket says 172.21.15.137:42112) 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 -- 172.21.15.137:0/1326951154 learned_addr learned my addr 172.21.15.137:0/1326951154 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d37fe700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 0x7fc0d419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 -- 172.21.15.137:0/1326951154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 msgr2=0x7fc0d419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 0x7fc0d419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 -- 172.21.15.137:0/1326951154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0095f0 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0cbfff700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0092e0 tx=0x7fc0bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc01d070 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0bc005dc0 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d4198990 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc00fea0 con 0x7fc0d4100fd0 2024-06-15T01:22:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.283+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d4198e80 con 0x7fc0d4100fd0 2024-06-15T01:22:06.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.284+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d404ea90 con 0x7fc0d4100fd0 2024-06-15T01:22:06.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.285+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc0bc003a60 con 0x7fc0d4100fd0 2024-06-15T01:22:06.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.285+0000 7fc0d17fa700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.285+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc0bc08b4d0 con 0x7fc0d4100fd0 2024-06-15T01:22:06.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.285+0000 7fc0d37fe700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.286+0000 7fc0d37fe700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005950 tx=0x7fc0c400b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.289+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc0bc059d90 con 0x7fc0d4100fd0 2024-06-15T01:22:06.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:05 smithi142 ceph-mon[82136]: pgmap v480: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.494+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0d41991b0 con 0x7fc0c006af80 2024-06-15T01:22:06.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.504+0000 7fc0d17fa700 1 -- 172.21.15.137:0/1326951154 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc0d41991b0 con 0x7fc0c006af80 2024-06-15T01:22:06.491 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:22:06.491 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (12m) 7m ago 23m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (23m) 7m ago 23m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (22m) 6m ago 22m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (10m) 7m ago 23m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (10m) 6m ago 22m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (12m) 7m ago 22m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (7m) 7m ago 17m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (7m) 7m ago 17m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (6m) 6m ago 17m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (6m) 6m ago 17m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:22:06.492 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (14m) 7m ago 24m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (13m) 6m ago 22m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (11m) 7m ago 24m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (11m) 6m ago 22m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (13m) 7m ago 23m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (13m) 6m ago 22m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (10m) 7m ago 21m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (9m) 7m ago 20m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (9m) 7m ago 20m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (8m) 6m ago 20m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (8m) 6m ago 19m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:22:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (7m) 6m ago 19m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:22:06.494 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (12m) 7m ago 22m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:22:06.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 msgr2=0x7fc0c006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005950 tx=0x7fc0c400b410 comp rx=0 tx=0).stop 2024-06-15T01:22:06.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 msgr2=0x7fc0d419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0092e0 tx=0x7fc0bc00bd70 comp rx=0 tx=0).stop 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 shutdown_connections 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4100620 0x7fc0d419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 --2- 172.21.15.137:0/1326951154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d4100fd0 0x7fc0d419ebd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 >> 172.21.15.137:0/1326951154 conn(0x7fc0d4074d80 msgr2=0x7fc0d41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.506+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 shutdown_connections 2024-06-15T01:22:06.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.507+0000 7fc0d9c54700 1 -- 172.21.15.137:0/1326951154 wait complete. 2024-06-15T01:22:06.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 -- 172.21.15.137:0/3302636754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f60105a30 msgr2=0x7f8f60105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 --2- 172.21.15.137:0/3302636754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f60105a30 0x7f8f60105e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b0a0 tx=0x7f8f5800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 -- 172.21.15.137:0/3302636754 shutdown_connections 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 --2- 172.21.15.137:0/3302636754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f600ffa30 0x7f8f600ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 --2- 172.21.15.137:0/3302636754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f60105a30 0x7f8f60105e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 -- 172.21.15.137:0/3302636754 >> 172.21.15.137:0/3302636754 conn(0x7f8f600fb540 msgr2=0x7f8f600fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 -- 172.21.15.137:0/3302636754 shutdown_connections 2024-06-15T01:22:06.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.609+0000 7f8f67822700 1 -- 172.21.15.137:0/3302636754 wait complete. 2024-06-15T01:22:06.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 Processor -- start 2024-06-15T01:22:06.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 -- start start 2024-06-15T01:22:06.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60105a30 0x7f8f60196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60197290 con 0x7f8f600ffa30 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f67822700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60199fc0 con 0x7f8f60105a30 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42128/0 (socket says 172.21.15.137:42128) 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 -- 172.21.15.137:0/569617603 learned_addr learned my addr 172.21.15.137:0/569617603 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f64dbd700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60105a30 0x7f8f60196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 -- 172.21.15.137:0/569617603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60105a30 msgr2=0x7f8f60196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60105a30 0x7f8f60196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.610+0000 7f8f655be700 1 -- 172.21.15.137:0/569617603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009d00 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f655be700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000f80 tx=0x7f8f580096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f5800e040 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f5800be10 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6019a240 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f580043d0 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.611+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6019a730 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.612+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60109040 con 0x7f8f600ffa30 2024-06-15T01:22:06.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.612+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8f58019070 con 0x7f8f600ffa30 2024-06-15T01:22:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.613+0000 7f8f5e7fc700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 0x7f8f4806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.613+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8f5808ba20 con 0x7f8f600ffa30 2024-06-15T01:22:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.613+0000 7f8f64dbd700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 0x7f8f4806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.613+0000 7f8f64dbd700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 0x7f8f4806d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f60197ce0 tx=0x7f8f54009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.617+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8f580568d0 con 0x7f8f600ffa30 2024-06-15T01:22:06.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.885+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f60197b30 con 0x7f8f600ffa30 2024-06-15T01:22:06.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.885+0000 7f8f5e7fc700 1 -- 172.21.15.137:0/569617603 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f8f58059ef0 con 0x7f8f600ffa30 2024-06-15T01:22:06.872 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:22:06.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:22:06.874 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:22:06.874 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 msgr2=0x7f8f4806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 0x7f8f4806d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f60197ce0 tx=0x7f8f54009260 comp rx=0 tx=0).stop 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 msgr2=0x7f8f601966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000f80 tx=0x7f8f580096e0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 shutdown_connections 2024-06-15T01:22:06.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f4806af80 0x7f8f4806d440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f600ffa30 0x7f8f601966c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 --2- 172.21.15.137:0/569617603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60105a30 0x7f8f60196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 >> 172.21.15.137:0/569617603 conn(0x7f8f600fb540 msgr2=0x7f8f600690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 shutdown_connections 2024-06-15T01:22:06.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.888+0000 7f8f67822700 1 -- 172.21.15.137:0/569617603 wait complete. 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 -- 172.21.15.137:0/2480318799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 msgr2=0x7efcb0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 --2- 172.21.15.137:0/2480318799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb0100050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcac009910 tx=0x7efcac009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 -- 172.21.15.137:0/2480318799 shutdown_connections 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 --2- 172.21.15.137:0/2480318799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb0100050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 --2- 172.21.15.137:0/2480318799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 0x7efcb0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 -- 172.21.15.137:0/2480318799 >> 172.21.15.137:0/2480318799 conn(0x7efcb00755a0 msgr2=0x7efcb00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 -- 172.21.15.137:0/2480318799 shutdown_connections 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.996+0000 7efcb83b1700 1 -- 172.21.15.137:0/2480318799 wait complete. 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 Processor -- start 2024-06-15T01:22:06.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 -- start start 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 0x7efcb01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0198010 con 0x7efcb00ffbd0 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb83b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0198150 con 0x7efcb0105a40 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42146/0 (socket says 172.21.15.137:42146) 2024-06-15T01:22:06.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 -- 172.21.15.137:0/4242473365 learned_addr learned my addr 172.21.15.137:0/4242473365 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb594c700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 0x7efcb01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 -- 172.21.15.137:0/4242473365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 msgr2=0x7efcb01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 0x7efcb01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.997+0000 7efcb614d700 1 -- 172.21.15.137:0/4242473365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0095f0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efcb614d700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efca400c8a0 tx=0x7efca400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efca400cea0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efca4004830 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb01a0520 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efca4005500 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.998+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb01a09e0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.999+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb0107fd0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:06.999+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efca40049a0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.000+0000 7efca37fe700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 0x7efc9c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:06.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.000+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efca4089fc0 con 0x7efcb00ffbd0 2024-06-15T01:22:06.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.000+0000 7efcb594c700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 0x7efc9c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:06.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.000+0000 7efcb594c700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 0x7efc9c06d440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcac00b5c0 tx=0x7efcac005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:06.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.004+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efca4054e70 con 0x7efcb00ffbd0 2024-06-15T01:22:07.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:06 smithi137 ceph-mon[112295]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:07.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:06 smithi137 ceph-mon[112295]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:07.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:06 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/569617603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:22:07.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.242+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efcb002cd10 con 0x7efcb00ffbd0 2024-06-15T01:22:07.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.242+0000 7efca37fe700 1 -- 172.21.15.137:0/4242473365 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7efca401d070 con 0x7efcb00ffbd0 2024-06-15T01:22:07.229 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:22:07.229 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:22:07.229 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:22:07.230 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:22:07.231 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:07.232 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:07.233 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:07.233 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:07.233 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:07.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 msgr2=0x7efc9c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 0x7efc9c06d440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcac00b5c0 tx=0x7efcac005fd0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 msgr2=0x7efcb01974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efca400c8a0 tx=0x7efca400cc60 comp rx=0 tx=0).stop 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 shutdown_connections 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efc9c06af80 0x7efc9c06d440 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcb00ffbd0 0x7efcb01974b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 --2- 172.21.15.137:0/4242473365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcb0105a40 0x7efcb01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 >> 172.21.15.137:0/4242473365 conn(0x7efcb00755a0 msgr2=0x7efcb00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 shutdown_connections 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.244+0000 7efcb83b1700 1 -- 172.21.15.137:0/4242473365 wait complete. 2024-06-15T01:22:07.234 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:22:07.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:06 smithi142 ceph-mon[82136]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:07.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:06 smithi142 ceph-mon[82136]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:07.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:06 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/569617603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:22:07.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 -- 172.21.15.137:0/3349602925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a80ffbd0 msgr2=0x7f14a8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 --2- 172.21.15.137:0/3349602925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a80ffbd0 0x7f14a8100050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009910 tx=0x7f14a4009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 -- 172.21.15.137:0/3349602925 shutdown_connections 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 --2- 172.21.15.137:0/3349602925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a80ffbd0 0x7f14a8100050 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 --2- 172.21.15.137:0/3349602925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a8105a40 0x7f14a8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 -- 172.21.15.137:0/3349602925 >> 172.21.15.137:0/3349602925 conn(0x7f14a80755a0 msgr2=0x7f14a80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 -- 172.21.15.137:0/3349602925 shutdown_connections 2024-06-15T01:22:07.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.349+0000 7f14af3a4700 1 -- 172.21.15.137:0/3349602925 wait complete. 2024-06-15T01:22:07.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 Processor -- start 2024-06-15T01:22:07.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 -- start start 2024-06-15T01:22:07.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a80ffbd0 0x7f14a8195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8196160 con 0x7f14a8105a40 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14af3a4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8199ef0 con 0x7f14a80ffbd0 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14ac93f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14ac93f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42164/0 (socket says 172.21.15.137:42164) 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14ac93f700 1 -- 172.21.15.137:0/746528520 learned_addr learned my addr 172.21.15.137:0/746528520 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.350+0000 7f14ad140700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a80ffbd0 0x7f14a8195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14ac93f700 1 -- 172.21.15.137:0/746528520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a80ffbd0 msgr2=0x7f14a8195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14ac93f700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a80ffbd0 0x7f14a8195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14ac93f700 1 -- 172.21.15.137:0/746528520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40095f0 con 0x7f14a8105a40 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14ac93f700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006010 tx=0x7f14a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14a401f070 con 0x7f14a8105a40 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14a4005710 con 0x7f14a8105a40 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a819a170 con 0x7f14a8105a40 2024-06-15T01:22:07.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14a400f460 con 0x7f14a8105a40 2024-06-15T01:22:07.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.351+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a819a660 con 0x7f14a8105a40 2024-06-15T01:22:07.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.352+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a804ea90 con 0x7f14a8105a40 2024-06-15T01:22:07.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.352+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f14a400f5c0 con 0x7f14a8105a40 2024-06-15T01:22:07.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.353+0000 7f149a7fc700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 0x7f149406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.353+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f14a408d6c0 con 0x7f14a8105a40 2024-06-15T01:22:07.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.353+0000 7f14ad140700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 0x7f149406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.353+0000 7f14ad140700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 0x7f149406d440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f149c005950 tx=0x7f149c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:07.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.357+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f14a40584f0 con 0x7f14a8105a40 2024-06-15T01:22:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.569+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14a819aa70 con 0x7f149406af80 2024-06-15T01:22:07.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.570+0000 7f149a7fc700 1 -- 172.21.15.137:0/746528520 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f14a819aa70 con 0x7f149406af80 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:22:07.558 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:22:07.559 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:22:07.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.572+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 msgr2=0x7f149406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.572+0000 7f14af3a4700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 0x7f149406d440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f149c005950 tx=0x7f149c00b410 comp rx=0 tx=0).stop 2024-06-15T01:22:07.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.572+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 msgr2=0x7f14a8195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.572+0000 7f14af3a4700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006010 tx=0x7f14a400bd70 comp rx=0 tx=0).stop 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 shutdown_connections 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f149406af80 0x7f149406d440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a80ffbd0 0x7f14a8195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 --2- 172.21.15.137:0/746528520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a8105a40 0x7f14a8195a80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 >> 172.21.15.137:0/746528520 conn(0x7f14a80755a0 msgr2=0x7f14a80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 shutdown_connections 2024-06-15T01:22:07.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.573+0000 7f14af3a4700 1 -- 172.21.15.137:0/746528520 wait complete. 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- 172.21.15.137:0/1477123081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0101770 msgr2=0x7f46e01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 --2- 172.21.15.137:0/1477123081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0101770 0x7f46e01057c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009910 tx=0x7f46d0009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- 172.21.15.137:0/1477123081 shutdown_connections 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 --2- 172.21.15.137:0/1477123081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0101770 0x7f46e01057c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 --2- 172.21.15.137:0/1477123081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0100dc0 0x7f46e01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- 172.21.15.137:0/1477123081 >> 172.21.15.137:0/1477123081 conn(0x7f46e00fc650 msgr2=0x7f46e00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- 172.21.15.137:0/1477123081 shutdown_connections 2024-06-15T01:22:07.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- 172.21.15.137:0/1477123081 wait complete. 2024-06-15T01:22:07.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 Processor -- start 2024-06-15T01:22:07.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.677+0000 7f46e808a700 1 -- start start 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0101770 0x7f46e00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0073b10 con 0x7f46e0100dc0 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0073c50 con 0x7f46e0101770 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42192/0 (socket says 172.21.15.137:42192) 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 -- 172.21.15.137:0/443767473 learned_addr learned my addr 172.21.15.137:0/443767473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5625700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0101770 0x7f46e00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 -- 172.21.15.137:0/443767473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0101770 msgr2=0x7f46e00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0101770 0x7f46e00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 -- 172.21.15.137:0/443767473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00095f0 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e5e26700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c8a0 tx=0x7f46dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46dc00cea0 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46dc004830 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46dc005500 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e01a28a0 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.678+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e01a2cb0 con 0x7f46e0100dc0 2024-06-15T01:22:07.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.679+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e01091c0 con 0x7f46e0100dc0 2024-06-15T01:22:07.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.680+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f46dc0049a0 con 0x7f46e0100dc0 2024-06-15T01:22:07.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.680+0000 7f46d6ffd700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 0x7f46cc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:07.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.680+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f46dc089fd0 con 0x7f46e0100dc0 2024-06-15T01:22:07.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.680+0000 7f46e5625700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 0x7f46cc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:07.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.681+0000 7f46e5625700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 0x7f46cc06d440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46e0074880 tx=0x7f46d001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:07.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.685+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f46dc054e80 con 0x7f46e0100dc0 2024-06-15T01:22:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.959+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f46e00745c0 con 0x7f46e0100dc0 2024-06-15T01:22:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.959+0000 7f46d6ffd700 1 -- 172.21.15.137:0/443767473 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f46e00745c0 con 0x7f46e0100dc0 2024-06-15T01:22:07.946 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:22:07.947 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:22:07.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 msgr2=0x7f46cc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 0x7f46cc06d440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46e0074880 tx=0x7f46d001a040 comp rx=0 tx=0).stop 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 msgr2=0x7f46e0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c8a0 tx=0x7f46dc00cc60 comp rx=0 tx=0).stop 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 shutdown_connections 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46cc06af80 0x7f46cc06d440 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46e0100dc0 0x7f46e0073090 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 --2- 172.21.15.137:0/443767473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e0101770 0x7f46e00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 >> 172.21.15.137:0/443767473 conn(0x7f46e00fc650 msgr2=0x7f46e00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 shutdown_connections 2024-06-15T01:22:07.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:07.961+0000 7f46e808a700 1 -- 172.21.15.137:0/443767473 wait complete. 2024-06-15T01:22:08.209 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:07 smithi142 ceph-mon[82136]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:08.210 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:07 smithi142 ceph-mon[82136]: pgmap v481: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:08.210 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:07 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4242473365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:22:08.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:07 smithi137 ceph-mon[112295]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:08.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:07 smithi137 ceph-mon[112295]: pgmap v481: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:08.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:07 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4242473365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:22:09.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:08 smithi137 ceph-mon[112295]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:09.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:08 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/443767473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:22:09.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:08 smithi142 ceph-mon[82136]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:09.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:08 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/443767473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:22:10.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:09 smithi137 ceph-mon[112295]: pgmap v482: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:10.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:09 smithi142 ceph-mon[82136]: pgmap v482: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:12.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:11 smithi137 ceph-mon[112295]: pgmap v483: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:11 smithi142 ceph-mon[82136]: pgmap v483: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:14.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:13 smithi137 ceph-mon[112295]: pgmap v484: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:14.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:13 smithi142 ceph-mon[82136]: pgmap v484: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:16.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:15 smithi137 ceph-mon[112295]: pgmap v485: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:16.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:15 smithi142 ceph-mon[82136]: pgmap v485: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:18.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:17 smithi137 ceph-mon[112295]: pgmap v486: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:17 smithi142 ceph-mon[82136]: pgmap v486: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:19.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:19.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:20.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:19 smithi137 ceph-mon[112295]: pgmap v487: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:20.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:19 smithi142 ceph-mon[82136]: pgmap v487: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:22.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:21 smithi137 ceph-mon[112295]: pgmap v488: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:22.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:21 smithi142 ceph-mon[82136]: pgmap v488: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:24.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:23 smithi137 ceph-mon[112295]: pgmap v489: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:24.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:23 smithi142 ceph-mon[82136]: pgmap v489: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:26.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:25 smithi137 ceph-mon[112295]: pgmap v490: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:26.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:25 smithi142 ceph-mon[82136]: pgmap v490: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:28.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:27 smithi137 ceph-mon[112295]: pgmap v491: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:28.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:27 smithi142 ceph-mon[82136]: pgmap v491: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:30.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:29 smithi137 ceph-mon[112295]: pgmap v492: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:30.261 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:29 smithi142 ceph-mon[82136]: pgmap v492: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:32.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:31 smithi137 ceph-mon[112295]: pgmap v493: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:32.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:31 smithi142 ceph-mon[82136]: pgmap v493: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:34.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:33 smithi142 ceph-mon[82136]: pgmap v494: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:34.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:34.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:33 smithi137 ceph-mon[112295]: pgmap v494: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:34.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:36.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:35 smithi142 ceph-mon[82136]: pgmap v495: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:36.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:35 smithi137 ceph-mon[112295]: pgmap v495: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:38.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 -- 172.21.15.137:0/639297133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94105a40 msgr2=0x7fcc94105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/639297133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94105a40 0x7fcc94105e20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b0a0 tx=0x7fcc9000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 -- 172.21.15.137:0/639297133 shutdown_connections 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/639297133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc940ffa40 0x7fcc940ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/639297133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94105a40 0x7fcc94105e20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 -- 172.21.15.137:0/639297133 >> 172.21.15.137:0/639297133 conn(0x7fcc940fb550 msgr2=0x7fcc940fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 -- 172.21.15.137:0/639297133 shutdown_connections 2024-06-15T01:22:38.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.078+0000 7fcc9b94b700 1 -- 172.21.15.137:0/639297133 wait complete. 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 Processor -- start 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 -- start start 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc94105a40 0x7fcc94195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941961b0 con 0x7fcc940ffa40 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc9b94b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94199f40 con 0x7fcc94105a40 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc996e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc996e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48522/0 (socket says 172.21.15.137:48522) 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.079+0000 7fcc996e7700 1 -- 172.21.15.137:0/3441928354 learned_addr learned my addr 172.21.15.137:0/3441928354 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc98ee6700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc94105a40 0x7fcc94195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc996e7700 1 -- 172.21.15.137:0/3441928354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc94105a40 msgr2=0x7fcc94195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc996e7700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc94105a40 0x7fcc94195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc996e7700 1 -- 172.21.15.137:0/3441928354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90009d00 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc996e7700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000f80 tx=0x7fcc90009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc9000e040 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc9000be90 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc90004470 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9419a1c0 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.080+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9419a6b0 con 0x7fcc940ffa40 2024-06-15T01:22:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.081+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc94108040 con 0x7fcc940ffa40 2024-06-15T01:22:38.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.081+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcc90019070 con 0x7fcc940ffa40 2024-06-15T01:22:38.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.082+0000 7fcc8a7fc700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 0x7fcc8006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.082+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcc9008bac0 con 0x7fcc940ffa40 2024-06-15T01:22:38.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.082+0000 7fcc98ee6700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 0x7fcc8006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.082+0000 7fcc98ee6700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 0x7fcc8006d440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcc94196bb0 tx=0x7fcc84009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.086+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc900568f0 con 0x7fcc940ffa40 2024-06-15T01:22:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.295+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc9402cd30 con 0x7fcc8006af80 2024-06-15T01:22:38.286 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:37 smithi137 ceph-mon[112295]: pgmap v496: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.296+0000 7fcc8a7fc700 1 -- 172.21.15.137:0/3441928354 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcc9402cd30 con 0x7fcc8006af80 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 msgr2=0x7fcc8006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 0x7fcc8006d440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcc94196bb0 tx=0x7fcc84009260 comp rx=0 tx=0).stop 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 msgr2=0x7fcc94195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000f80 tx=0x7fcc90009790 comp rx=0 tx=0).stop 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 shutdown_connections 2024-06-15T01:22:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc8006af80 0x7fcc8006d440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc940ffa40 0x7fcc94195590 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 --2- 172.21.15.137:0/3441928354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc94105a40 0x7fcc94195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 >> 172.21.15.137:0/3441928354 conn(0x7fcc940fb550 msgr2=0x7fcc940fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 shutdown_connections 2024-06-15T01:22:38.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.298+0000 7fcc9b94b700 1 -- 172.21.15.137:0/3441928354 wait complete. 2024-06-15T01:22:38.295 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:22:38.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:37 smithi142 ceph-mon[82136]: pgmap v496: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 -- 172.21.15.137:0/3391383022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf0105a40 msgr2=0x7faaf0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 --2- 172.21.15.137:0/3391383022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf0105a40 0x7faaf0105e20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faae800b0a0 tx=0x7faae800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 -- 172.21.15.137:0/3391383022 shutdown_connections 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 --2- 172.21.15.137:0/3391383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf00ffa40 0x7faaf00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 --2- 172.21.15.137:0/3391383022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf0105a40 0x7faaf0105e20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 -- 172.21.15.137:0/3391383022 >> 172.21.15.137:0/3391383022 conn(0x7faaf00fb530 msgr2=0x7faaf00fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 -- 172.21.15.137:0/3391383022 shutdown_connections 2024-06-15T01:22:38.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.403+0000 7faaf4acc700 1 -- 172.21.15.137:0/3391383022 wait complete. 2024-06-15T01:22:38.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.404+0000 7faaf4acc700 1 Processor -- start 2024-06-15T01:22:38.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.404+0000 7faaf4acc700 1 -- start start 2024-06-15T01:22:38.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf0105a40 0x7faaf0195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0195d60 con 0x7faaf00ffa40 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0195ed0 con 0x7faaf0105a40 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48544/0 (socket says 172.21.15.137:48544) 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 -- 172.21.15.137:0/1532051173 learned_addr learned my addr 172.21.15.137:0/1532051173 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faae5d9b700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf0105a40 0x7faaf0195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 -- 172.21.15.137:0/1532051173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf0105a40 msgr2=0x7faaf0195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf0105a40 0x7faaf0195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 -- 172.21.15.137:0/1532051173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8009d00 con 0x7faaf00ffa40 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaee59c700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faae800d010 tx=0x7faae8007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae800e040 con 0x7faaf00ffa40 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf0196150 con 0x7faaf00ffa40 2024-06-15T01:22:38.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faae8003de0 con 0x7faaf00ffa40 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae801d6b0 con 0x7faaf00ffa40 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.405+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf01a2bd0 con 0x7faaf00ffa40 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.406+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf0108040 con 0x7faaf00ffa40 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.407+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7faae8004040 con 0x7faaf00ffa40 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.407+0000 7faae7fff700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 0x7faadc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.407+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7faae808ba60 con 0x7faaf00ffa40 2024-06-15T01:22:38.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.407+0000 7faae5d9b700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 0x7faadc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.408+0000 7faae5d9b700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 0x7faadc06d440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faaf0069730 tx=0x7faad800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.411+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faae805a320 con 0x7faaf00ffa40 2024-06-15T01:22:38.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.632+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaf0196150 con 0x7faadc06af80 2024-06-15T01:22:38.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.633+0000 7faae7fff700 1 -- 172.21.15.137:0/1532051173 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faaf0196150 con 0x7faadc06af80 2024-06-15T01:22:38.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 msgr2=0x7faadc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 0x7faadc06d440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faaf0069730 tx=0x7faad800b410 comp rx=0 tx=0).stop 2024-06-15T01:22:38.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 msgr2=0x7faaf01952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faae800d010 tx=0x7faae8007b30 comp rx=0 tx=0).stop 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 shutdown_connections 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faadc06af80 0x7faadc06d440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf00ffa40 0x7faaf01952e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 --2- 172.21.15.137:0/1532051173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaf0105a40 0x7faaf0195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 >> 172.21.15.137:0/1532051173 conn(0x7faaf00fb530 msgr2=0x7faaf00fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 shutdown_connections 2024-06-15T01:22:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.636+0000 7faaf4acc700 1 -- 172.21.15.137:0/1532051173 wait complete. 2024-06-15T01:22:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 -- 172.21.15.137:0/1909351627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ffee0 msgr2=0x7fcc34103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 --2- 172.21.15.137:0/1909351627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ffee0 0x7fcc34103eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009910 tx=0x7fcc28009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 -- 172.21.15.137:0/1909351627 shutdown_connections 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 --2- 172.21.15.137:0/1909351627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ffee0 0x7fcc34103eb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 --2- 172.21.15.137:0/1909351627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ff530 0x7fcc340ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 -- 172.21.15.137:0/1909351627 >> 172.21.15.137:0/1909351627 conn(0x7fcc34074df0 msgr2=0x7fcc34075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 -- 172.21.15.137:0/1909351627 shutdown_connections 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 -- 172.21.15.137:0/1909351627 wait complete. 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.741+0000 7fcc3be11700 1 Processor -- start 2024-06-15T01:22:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 -- start start 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 0x7fcc34195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc341961e0 con 0x7fcc340ff530 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34199f70 con 0x7fcc340ffee0 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48554/0 (socket says 172.21.15.137:48554) 2024-06-15T01:22:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 -- 172.21.15.137:0/2939082948 learned_addr learned my addr 172.21.15.137:0/2939082948 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc393ac700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 0x7fcc34195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 -- 172.21.15.137:0/2939082948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 msgr2=0x7fcc34195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 0x7fcc34195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 -- 172.21.15.137:0/2939082948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280095f0 con 0x7fcc340ff530 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc393ac700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 0x7fcc34195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc39bad700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc240060b0 tx=0x7fcc2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2400dda0 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc2400f040 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3419a250 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.742+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc240146f0 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.743+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3419a7a0 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.743+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc34107790 con 0x7fcc340ff530 2024-06-15T01:22:38.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.744+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcc24015400 con 0x7fcc340ff530 2024-06-15T01:22:38.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.744+0000 7fcc32ffd700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 0x7fcc1c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:38.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.744+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcc24089d30 con 0x7fcc340ff530 2024-06-15T01:22:38.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.744+0000 7fcc393ac700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 0x7fcc1c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:38.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.745+0000 7fcc393ac700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 0x7fcc1c06d440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc34196be0 tx=0x7fcc2801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:38.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.749+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc24054c90 con 0x7fcc340ff530 2024-06-15T01:22:38.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.950+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc3419ab50 con 0x7fcc1c06af80 2024-06-15T01:22:38.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.960+0000 7fcc32ffd700 1 -- 172.21.15.137:0/2939082948 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcc3419ab50 con 0x7fcc1c06af80 2024-06-15T01:22:38.947 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:22:38.947 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (13m) 7m ago 23m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:22:38.947 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (24m) 7m ago 24m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (22m) 7m ago 22m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (11m) 7m ago 24m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (11m) 7m ago 22m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (12m) 7m ago 23m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (7m) 7m ago 18m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (8m) 7m ago 18m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (7m) 7m ago 18m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (7m) 7m ago 18m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (15m) 7m ago 25m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (13m) 7m ago 22m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (11m) 7m ago 25m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:22:38.948 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (11m) 7m ago 22m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (13m) 7m ago 23m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (13m) 7m ago 22m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (11m) 7m ago 21m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (10m) 7m ago 21m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (9m) 7m ago 20m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (9m) 7m ago 20m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (8m) 7m ago 20m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (8m) 7m ago 19m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:22:38.949 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (13m) 7m ago 23m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 msgr2=0x7fcc1c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 0x7fcc1c06d440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc34196be0 tx=0x7fcc2801a040 comp rx=0 tx=0).stop 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 msgr2=0x7fcc341955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc240060b0 tx=0x7fcc2400d3b0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 shutdown_connections 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcc1c06af80 0x7fcc1c06d440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc340ff530 0x7fcc341955c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 --2- 172.21.15.137:0/2939082948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc340ffee0 0x7fcc34195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:38.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 >> 172.21.15.137:0/2939082948 conn(0x7fcc34074df0 msgr2=0x7fcc340fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:38.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 shutdown_connections 2024-06-15T01:22:38.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:38.962+0000 7fcc3be11700 1 -- 172.21.15.137:0/2939082948 wait complete. 2024-06-15T01:22:39.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 -- 172.21.15.137:0/1898638982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc073ae0 msgr2=0x7f04dc10ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 --2- 172.21.15.137:0/1898638982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc073ae0 0x7f04dc10ce60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009910 tx=0x7f04cc009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 -- 172.21.15.137:0/1898638982 shutdown_connections 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 --2- 172.21.15.137:0/1898638982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc073ae0 0x7f04dc10ce60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 --2- 172.21.15.137:0/1898638982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc0731c0 0x7f04dc0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.064+0000 7f04e3c94700 1 -- 172.21.15.137:0/1898638982 >> 172.21.15.137:0/1898638982 conn(0x7f04dc0fc650 msgr2=0x7f04dc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 -- 172.21.15.137:0/1898638982 shutdown_connections 2024-06-15T01:22:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 -- 172.21.15.137:0/1898638982 wait complete. 2024-06-15T01:22:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 Processor -- start 2024-06-15T01:22:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 -- start start 2024-06-15T01:22:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc073ae0 0x7f04dc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc19f270 con 0x7f04dc0731c0 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.065+0000 7f04e3c94700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc198720 con 0x7f04dc073ae0 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48576/0 (socket says 172.21.15.137:48576) 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 -- 172.21.15.137:0/3133917219 learned_addr learned my addr 172.21.15.137:0/3133917219 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e122f700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc073ae0 0x7f04dc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 -- 172.21.15.137:0/3133917219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc073ae0 msgr2=0x7f04dc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc073ae0 0x7f04dc19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 -- 172.21.15.137:0/3133917219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0095f0 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e1a30700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c8a0 tx=0x7f04d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d800cea0 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04d8004830 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc198a00 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d8005500 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.066+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc198f50 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.067+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc10a560 con 0x7f04dc0731c0 2024-06-15T01:22:39.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.068+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f04d80049a0 con 0x7f04dc0731c0 2024-06-15T01:22:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.068+0000 7f04d2ffd700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 0x7f04c806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.068+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f04d808a080 con 0x7f04dc0731c0 2024-06-15T01:22:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.068+0000 7f04e122f700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 0x7f04c806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.069+0000 7f04e122f700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 0x7f04c806d440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b5c0 tx=0x7f04cc005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.072+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f04d8058940 con 0x7f04dc0731c0 2024-06-15T01:22:39.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:38 smithi137 ceph-mon[112295]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:39.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:38 smithi142 ceph-mon[82136]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:39.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.342+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f04dc04ea90 con 0x7f04dc0731c0 2024-06-15T01:22:39.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.342+0000 7f04d2ffd700 1 -- 172.21.15.137:0/3133917219 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f04d80584d0 con 0x7f04dc0731c0 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:22:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:22:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:22:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:22:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:22:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:22:39.331 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 msgr2=0x7f04c806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 0x7f04c806d440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b5c0 tx=0x7f04cc005fd0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 msgr2=0x7f04dc19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c8a0 tx=0x7f04d800cc60 comp rx=0 tx=0).stop 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 shutdown_connections 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04c806af80 0x7f04c806d440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc0731c0 0x7f04dc19e6a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 --2- 172.21.15.137:0/3133917219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04dc073ae0 0x7f04dc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 >> 172.21.15.137:0/3133917219 conn(0x7f04dc0fc650 msgr2=0x7f04dc1076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 shutdown_connections 2024-06-15T01:22:39.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.345+0000 7f04e3c94700 1 -- 172.21.15.137:0/3133917219 wait complete. 2024-06-15T01:22:39.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 -- 172.21.15.137:0/1175451279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75300ff2d0 msgr2=0x7f75300ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 --2- 172.21.15.137:0/1175451279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75300ff2d0 0x7f75300ff750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7518009910 tx=0x7f7518009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 -- 172.21.15.137:0/1175451279 shutdown_connections 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 --2- 172.21.15.137:0/1175451279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75300ff2d0 0x7f75300ff750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 --2- 172.21.15.137:0/1175451279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7530101ad0 0x7f7530101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 -- 172.21.15.137:0/1175451279 >> 172.21.15.137:0/1175451279 conn(0x7f7530074df0 msgr2=0x7f7530075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 -- 172.21.15.137:0/1175451279 shutdown_connections 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.450+0000 7f753626d700 1 -- 172.21.15.137:0/1175451279 wait complete. 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 Processor -- start 2024-06-15T01:22:39.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 -- start start 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75300ff2d0 0x7f753019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753019f2a0 con 0x7f7530101ad0 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f753626d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530198750 con 0x7f75300ff2d0 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48600/0 (socket says 172.21.15.137:48600) 2024-06-15T01:22:39.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 -- 172.21.15.137:0/3993733253 learned_addr learned my addr 172.21.15.137:0/3993733253 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752ffff700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75300ff2d0 0x7f753019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 -- 172.21.15.137:0/3993733253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75300ff2d0 msgr2=0x7f753019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75300ff2d0 0x7f753019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.451+0000 7f752f7fe700 1 -- 172.21.15.137:0/3993733253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180095f0 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f752f7fe700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7518000c00 tx=0x7f751800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f751801d070 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75301989d0 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7518005710 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75180039d0 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.452+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7530198e60 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.453+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753010b1c0 con 0x7f7530101ad0 2024-06-15T01:22:39.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.453+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7518003c50 con 0x7f7530101ad0 2024-06-15T01:22:39.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.454+0000 7f752d7fa700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 0x7f751c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.454+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f751808ba60 con 0x7f7530101ad0 2024-06-15T01:22:39.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.454+0000 7f752ffff700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 0x7f751c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.454+0000 7f752ffff700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 0x7f751c06d440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7520009c60 tx=0x7f7520009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.458+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f751805a2f0 con 0x7f7530101ad0 2024-06-15T01:22:39.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.689+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f753004ea90 con 0x7f7530101ad0 2024-06-15T01:22:39.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.690+0000 7f752d7fa700 1 -- 172.21.15.137:0/3993733253 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f7518059e80 con 0x7f7530101ad0 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:22:39.677 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:22:39.678 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:22:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:39.680 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:39.680 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:39.680 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:22:39.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 msgr2=0x7f751c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 0x7f751c06d440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7520009c60 tx=0x7f7520009260 comp rx=0 tx=0).stop 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 msgr2=0x7f753019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7518000c00 tx=0x7f751800bde0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 shutdown_connections 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f751c06af80 0x7f751c06d440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75300ff2d0 0x7f753019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 --2- 172.21.15.137:0/3993733253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7530101ad0 0x7f753019ec10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 >> 172.21.15.137:0/3993733253 conn(0x7f7530074df0 msgr2=0x7f75300fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 shutdown_connections 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.693+0000 7f753626d700 1 -- 172.21.15.137:0/3993733253 wait complete. 2024-06-15T01:22:39.681 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 -- 172.21.15.137:0/2348512465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f0105a40 msgr2=0x7f04f0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 --2- 172.21.15.137:0/2348512465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f0105a40 0x7f04f0105e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009910 tx=0x7f04e0009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 -- 172.21.15.137:0/2348512465 shutdown_connections 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 --2- 172.21.15.137:0/2348512465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f00ffbd0 0x7f04f0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 --2- 172.21.15.137:0/2348512465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f0105a40 0x7f04f0105e20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 -- 172.21.15.137:0/2348512465 >> 172.21.15.137:0/2348512465 conn(0x7f04f00755a0 msgr2=0x7f04f00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 -- 172.21.15.137:0/2348512465 shutdown_connections 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.799+0000 7f04f668a700 1 -- 172.21.15.137:0/2348512465 wait complete. 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 Processor -- start 2024-06-15T01:22:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 -- start start 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0105a40 0x7f04f0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f0198020 con 0x7f04f00ffbd0 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04f668a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f0198160 con 0x7f04f0105a40 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04effff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04effff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48614/0 (socket says 172.21.15.137:48614) 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04effff700 1 -- 172.21.15.137:0/1135075526 learned_addr learned my addr 172.21.15.137:0/1135075526 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.800+0000 7f04ef7fe700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0105a40 0x7f04f0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04effff700 1 -- 172.21.15.137:0/1135075526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0105a40 msgr2=0x7f04f0197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04effff700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0105a40 0x7f04f0197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04effff700 1 -- 172.21.15.137:0/1135075526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00095f0 con 0x7f04f00ffbd0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04effff700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04e00092e0 tx=0x7f04e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e001d070 con 0x7f04f00ffbd0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04e0005e00 con 0x7f04f00ffbd0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f01a0530 con 0x7f04f00ffbd0 2024-06-15T01:22:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e000fe60 con 0x7f04f00ffbd0 2024-06-15T01:22:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.801+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f01a0940 con 0x7f04f00ffbd0 2024-06-15T01:22:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.802+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f0108040 con 0x7f04f00ffbd0 2024-06-15T01:22:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.802+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f04e000f6d0 con 0x7f04f00ffbd0 2024-06-15T01:22:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.803+0000 7f04ed7fa700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 0x7f04d806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.803+0000 7f04ef7fe700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 0x7f04d806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.803+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f04e008b560 con 0x7f04f00ffbd0 2024-06-15T01:22:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.803+0000 7f04ef7fe700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 0x7f04d806d440 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005950 tx=0x7f04e40058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:39.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:39.807+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f04e0056560 con 0x7f04f00ffbd0 2024-06-15T01:22:40.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.018+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f00611d0 con 0x7f04d806af80 2024-06-15T01:22:40.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.019+0000 7f04ed7fa700 1 -- 172.21.15.137:0/1135075526 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f04f00611d0 con 0x7f04d806af80 2024-06-15T01:22:40.006 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:22:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:22:40.008 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:22:40.008 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:22:40.008 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:22:40.008 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:22:40.008 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:22:40.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 msgr2=0x7f04d806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 0x7f04d806d440 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005950 tx=0x7f04e40058e0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 msgr2=0x7f04f01974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04e00092e0 tx=0x7f04e000be20 comp rx=0 tx=0).stop 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 shutdown_connections 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04d806af80 0x7f04d806d440 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f00ffbd0 0x7f04f01974c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 --2- 172.21.15.137:0/1135075526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0105a40 0x7f04f0197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.021+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 >> 172.21.15.137:0/1135075526 conn(0x7f04f00755a0 msgr2=0x7f04f00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:40.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.022+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 shutdown_connections 2024-06-15T01:22:40.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.022+0000 7f04f668a700 1 -- 172.21.15.137:0/1135075526 wait complete. 2024-06-15T01:22:40.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1992245075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8101750 msgr2=0x7f4bc81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1992245075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8101750 0x7f4bc81057a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009910 tx=0x7f4bb0009c20 comp rx=0 tx=0).stop 2024-06-15T01:22:40.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1992245075 shutdown_connections 2024-06-15T01:22:40.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1992245075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8101750 0x7f4bc81057a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1992245075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8100da0 0x7f4bc8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1992245075 >> 172.21.15.137:0/1992245075 conn(0x7f4bc80fc650 msgr2=0x7f4bc80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:40.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1992245075 shutdown_connections 2024-06-15T01:22:40.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1992245075 wait complete. 2024-06-15T01:22:40.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 Processor -- start 2024-06-15T01:22:40.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.127+0000 7f4bcd8b4700 1 -- start start 2024-06-15T01:22:40.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:40.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 0x7f4bc80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:40.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8073b10 con 0x7f4bc8100da0 2024-06-15T01:22:40.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8073c50 con 0x7f4bc8101750 2024-06-15T01:22:40.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:40.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48636/0 (socket says 172.21.15.137:48636) 2024-06-15T01:22:40.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 -- 172.21.15.137:0/1020451155 learned_addr learned my addr 172.21.15.137:0/1020451155 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:22:40.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc67fc700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 0x7f4bc80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:40.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 -- 172.21.15.137:0/1020451155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 msgr2=0x7f4bc80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 0x7f4bc80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 -- 172.21.15.137:0/1020451155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00095f0 con 0x7f4bc8100da0 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc67fc700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 0x7f4bc80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bc6ffd700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b4b0 tx=0x7f4bc000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc0011800 con 0x7f4bc8100da0 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bc0011e00 con 0x7f4bc8100da0 2024-06-15T01:22:40.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc81a2960 con 0x7f4bc8100da0 2024-06-15T01:22:40.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc0010e50 con 0x7f4bc8100da0 2024-06-15T01:22:40.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.128+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc81a2d70 con 0x7f4bc8100da0 2024-06-15T01:22:40.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.129+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc81090d0 con 0x7f4bc8100da0 2024-06-15T01:22:40.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.130+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4bc0010380 con 0x7f4bc8100da0 2024-06-15T01:22:40.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.130+0000 7f4bcc8b2700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 0x7f4bb406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:22:40.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.131+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4bc0089f30 con 0x7f4bc8100da0 2024-06-15T01:22:40.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.131+0000 7f4bc67fc700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 0x7f4bb406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:22:40.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.131+0000 7f4bc67fc700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 0x7f4bb406d440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00059d0 tx=0x7f4bb0005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:22:40.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.135+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4bc00bd900 con 0x7f4bc8100da0 2024-06-15T01:22:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:39 smithi142 ceph-mon[82136]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:39 smithi142 ceph-mon[82136]: pgmap v497: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:39 smithi142 ceph-mon[82136]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:39 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3133917219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:22:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:39 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3993733253' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:22:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.406+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4bc8073e10 con 0x7f4bc8100da0 2024-06-15T01:22:40.394 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:39 smithi137 ceph-mon[112295]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:40.394 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:39 smithi137 ceph-mon[112295]: pgmap v497: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:40.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:39 smithi137 ceph-mon[112295]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:40.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:39 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3133917219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:22:40.395 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:39 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3993733253' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:22:40.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.407+0000 7f4bcc8b2700 1 -- 172.21.15.137:0/1020451155 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f4bc0054d60 con 0x7f4bc8100da0 2024-06-15T01:22:40.396 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:22:40.396 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:22:40.396 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:22:40.396 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:22:40.396 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:22:40.397 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:22:40.397 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:22:40.397 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:22:40.397 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:22:40.397 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:22:40.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 msgr2=0x7f4bb406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 0x7f4bb406d440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00059d0 tx=0x7f4bb0005960 comp rx=0 tx=0).stop 2024-06-15T01:22:40.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 msgr2=0x7f4bc8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b4b0 tx=0x7f4bc000b870 comp rx=0 tx=0).stop 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 shutdown_connections 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bb406af80 0x7f4bb406d440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bc8100da0 0x7f4bc8073090 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 --2- 172.21.15.137:0/1020451155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc8101750 0x7f4bc80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:22:40.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 >> 172.21.15.137:0/1020451155 conn(0x7f4bc80fc650 msgr2=0x7f4bc80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:22:40.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 shutdown_connections 2024-06-15T01:22:40.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:22:40.410+0000 7f4bcd8b4700 1 -- 172.21.15.137:0/1020451155 wait complete. 2024-06-15T01:22:41.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:40 smithi142 ceph-mon[82136]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:41.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:40 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1020451155' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:22:41.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:40 smithi137 ceph-mon[112295]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:22:41.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:40 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1020451155' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:22:42.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:42 smithi137 ceph-mon[112295]: pgmap v498: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:42.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:42 smithi142 ceph-mon[82136]: pgmap v498: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:43.762 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:43 smithi137 ceph-mon[112295]: pgmap v499: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:43.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:43 smithi142 ceph-mon[82136]: pgmap v499: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:46.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:45 smithi137 ceph-mon[112295]: pgmap v500: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:46.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:45 smithi142 ceph-mon[82136]: pgmap v500: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:48.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:47 smithi137 ceph-mon[112295]: pgmap v501: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:48.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:47 smithi142 ceph-mon[82136]: pgmap v501: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:48.966 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:22:50.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:49 smithi137 ceph-mon[112295]: pgmap v502: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:50.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:49 smithi142 ceph-mon[82136]: pgmap v502: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:52.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:51 smithi137 ceph-mon[112295]: pgmap v503: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:52.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:51 smithi142 ceph-mon[82136]: pgmap v503: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:53.965 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:53 smithi137 ceph-mon[112295]: pgmap v504: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:54.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:53 smithi142 ceph-mon[82136]: pgmap v504: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:56.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:55 smithi137 ceph-mon[112295]: pgmap v505: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:56.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:55 smithi142 ceph-mon[82136]: pgmap v505: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:58.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:57 smithi137 ceph-mon[112295]: pgmap v506: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:22:58.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:57 smithi142 ceph-mon[82136]: pgmap v506: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:00.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:22:59 smithi137 ceph-mon[112295]: pgmap v507: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:00.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:22:59 smithi142 ceph-mon[82136]: pgmap v507: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:02.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:01 smithi137 ceph-mon[112295]: pgmap v508: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:02.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:01 smithi142 ceph-mon[82136]: pgmap v508: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:04.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:03 smithi137 ceph-mon[112295]: pgmap v509: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:04.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:04.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:03 smithi142 ceph-mon[82136]: pgmap v509: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:04.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:06.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:05 smithi137 ceph-mon[112295]: pgmap v510: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:06.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:05 smithi142 ceph-mon[82136]: pgmap v510: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:08.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:07 smithi137 ceph-mon[112295]: pgmap v511: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:08.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:07 smithi142 ceph-mon[82136]: pgmap v511: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:10.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:09 smithi137 ceph-mon[112295]: pgmap v512: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:10.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:09 smithi142 ceph-mon[82136]: pgmap v512: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:10.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 -- 172.21.15.137:0/4028301250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 msgr2=0x7f7eb8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 --2- 172.21.15.137:0/4028301250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8100050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009910 tx=0x7f7eac009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:10.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 -- 172.21.15.137:0/4028301250 shutdown_connections 2024-06-15T01:23:10.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 --2- 172.21.15.137:0/4028301250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8100050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 --2- 172.21.15.137:0/4028301250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 0x7f7eb8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 -- 172.21.15.137:0/4028301250 >> 172.21.15.137:0/4028301250 conn(0x7f7eb80755a0 msgr2=0x7f7eb80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:10.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 -- 172.21.15.137:0/4028301250 shutdown_connections 2024-06-15T01:23:10.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.528+0000 7f7ebf313700 1 -- 172.21.15.137:0/4028301250 wait complete. 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 Processor -- start 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 -- start start 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 0x7f7eb8101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8102090 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebf313700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb81021d0 con 0x7f7eb8105a40 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebd0af700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebd0af700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57436/0 (socket says 172.21.15.137:57436) 2024-06-15T01:23:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebd0af700 1 -- 172.21.15.137:0/1225837921 learned_addr learned my addr 172.21.15.137:0/1225837921 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.529+0000 7f7ebc8ae700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 0x7f7eb8101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebd0af700 1 -- 172.21.15.137:0/1225837921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 msgr2=0x7f7eb8101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebd0af700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 0x7f7eb8101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebd0af700 1 -- 172.21.15.137:0/1225837921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0095f0 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebd0af700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4b0 tx=0x7f7eb400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb4011800 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7eb4011e00 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb4010e60 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb81a2900 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.530+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb81a2cb0 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.531+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8109070 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.532+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7eb4011960 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.532+0000 7f7eaa7fc700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 0x7f7ea406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.532+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7eb4089e10 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.532+0000 7f7ebc8ae700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 0x7f7ea406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.533+0000 7f7ebc8ae700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 0x7f7ea406d440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8102e00 tx=0x7f7eac01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:10.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.536+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7eb4054cc0 con 0x7f7eb80ffbd0 2024-06-15T01:23:10.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.745+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb802cd80 con 0x7f7ea406af80 2024-06-15T01:23:10.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.746+0000 7f7eaa7fc700 1 -- 172.21.15.137:0/1225837921 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7eb802cd80 con 0x7f7ea406af80 2024-06-15T01:23:10.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 msgr2=0x7f7ea406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 0x7f7ea406d440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8102e00 tx=0x7f7eac01a040 comp rx=0 tx=0).stop 2024-06-15T01:23:10.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 msgr2=0x7f7eb8101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4b0 tx=0x7f7eb400b7c0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 shutdown_connections 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ea406af80 0x7f7ea406d440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7eb80ffbd0 0x7f7eb8101610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 --2- 172.21.15.137:0/1225837921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb8105a40 0x7f7eb8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.748+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 >> 172.21.15.137:0/1225837921 conn(0x7f7eb80755a0 msgr2=0x7f7eb8103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.749+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 shutdown_connections 2024-06-15T01:23:10.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.749+0000 7f7ebf313700 1 -- 172.21.15.137:0/1225837921 wait complete. 2024-06-15T01:23:10.746 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 -- 172.21.15.137:0/1094833393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac8105a40 msgr2=0x7f3ac8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 --2- 172.21.15.137:0/1094833393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac8105a40 0x7f3ac8105e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009910 tx=0x7f3ac4009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 -- 172.21.15.137:0/1094833393 shutdown_connections 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 --2- 172.21.15.137:0/1094833393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 --2- 172.21.15.137:0/1094833393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac8105a40 0x7f3ac8105e20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 -- 172.21.15.137:0/1094833393 >> 172.21.15.137:0/1094833393 conn(0x7f3ac80755a0 msgr2=0x7f3ac80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:10.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 -- 172.21.15.137:0/1094833393 shutdown_connections 2024-06-15T01:23:10.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.853+0000 7f3aceffd700 1 -- 172.21.15.137:0/1094833393 wait complete. 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 Processor -- start 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 -- start start 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac8105a40 0x7f3ac8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8198100 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3aceffd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8198270 con 0x7f3ac8105a40 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3accd99700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3accd99700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57458/0 (socket says 172.21.15.137:57458) 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.854+0000 7f3accd99700 1 -- 172.21.15.137:0/4053438517 learned_addr learned my addr 172.21.15.137:0/4053438517 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3abffff700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac8105a40 0x7f3ac8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3accd99700 1 -- 172.21.15.137:0/4053438517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac8105a40 msgr2=0x7f3ac8197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:10.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3accd99700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac8105a40 0x7f3ac8197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3accd99700 1 -- 172.21.15.137:0/4053438517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40095f0 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3accd99700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4006010 tx=0x7f3ac400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ac401d070 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ac400f460 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ac4021640 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.855+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac81a0610 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.856+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81a09d0 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.856+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8108040 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.857+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3ac400f9e0 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.858+0000 7f3abdffb700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 0x7f3ab006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.858+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3ac408b630 con 0x7f3ac80ffbd0 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.858+0000 7f3abffff700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 0x7f3ab006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.858+0000 7f3abffff700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 0x7f3ab006d440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8068ce0 tx=0x7f3ab8009190 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:10.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:10.862+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ac4056460 con 0x7f3ac80ffbd0 2024-06-15T01:23:11.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.073+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ac802ccc0 con 0x7f3ab006af80 2024-06-15T01:23:11.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.074+0000 7f3abdffb700 1 -- 172.21.15.137:0/4053438517 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ac802ccc0 con 0x7f3ab006af80 2024-06-15T01:23:11.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 msgr2=0x7f3ab006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 0x7f3ab006d440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8068ce0 tx=0x7f3ab8009190 comp rx=0 tx=0).stop 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 msgr2=0x7f3ac81975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4006010 tx=0x7f3ac400bd70 comp rx=0 tx=0).stop 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 shutdown_connections 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3ab006af80 0x7f3ab006d440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ac80ffbd0 0x7f3ac81975a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 --2- 172.21.15.137:0/4053438517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ac8105a40 0x7f3ac8197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 >> 172.21.15.137:0/4053438517 conn(0x7f3ac80755a0 msgr2=0x7f3ac80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.076+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 shutdown_connections 2024-06-15T01:23:11.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.077+0000 7f3aceffd700 1 -- 172.21.15.137:0/4053438517 wait complete. 2024-06-15T01:23:11.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 -- 172.21.15.137:0/384831367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc0ffbd0 msgr2=0x7f8fcc100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 --2- 172.21.15.137:0/384831367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc100050 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009910 tx=0x7f8fc0009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 -- 172.21.15.137:0/384831367 shutdown_connections 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 --2- 172.21.15.137:0/384831367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc100050 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 --2- 172.21.15.137:0/384831367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc105a40 0x7f8fcc105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.180+0000 7f8fd213b700 1 -- 172.21.15.137:0/384831367 >> 172.21.15.137:0/384831367 conn(0x7f8fcc0755a0 msgr2=0x7f8fcc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.181+0000 7f8fd213b700 1 -- 172.21.15.137:0/384831367 shutdown_connections 2024-06-15T01:23:11.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.181+0000 7f8fd213b700 1 -- 172.21.15.137:0/384831367 wait complete. 2024-06-15T01:23:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.181+0000 7f8fd213b700 1 Processor -- start 2024-06-15T01:23:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.181+0000 7f8fd213b700 1 -- start start 2024-06-15T01:23:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fd213b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fd213b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fd213b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc102090 con 0x7f8fcc105a40 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fd213b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1021d0 con 0x7f8fcc0ffbd0 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57482/0 (socket says 172.21.15.137:57482) 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 -- 172.21.15.137:0/493751350 learned_addr learned my addr 172.21.15.137:0/493751350 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcb7fe700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 -- 172.21.15.137:0/493751350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc0ffbd0 msgr2=0x7f8fcc101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 -- 172.21.15.137:0/493751350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00095f0 con 0x7f8fcc105a40 2024-06-15T01:23:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fcaffd700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00092e0 tx=0x7f8fc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc001d070 con 0x7f8fcc105a40 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fc0005e00 con 0x7f8fcc105a40 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.182+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc102450 con 0x7f8fcc105a40 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.183+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc000fe90 con 0x7f8fcc105a40 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.183+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc1a2b70 con 0x7f8fcc105a40 2024-06-15T01:23:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.183+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc04ea90 con 0x7f8fcc105a40 2024-06-15T01:23:11.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.184+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8fc000f6d0 con 0x7f8fcc105a40 2024-06-15T01:23:11.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.185+0000 7f8fc8ff9700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 0x7f8fb406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.185+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8fc008b8f0 con 0x7f8fcc105a40 2024-06-15T01:23:11.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.185+0000 7f8fcb7fe700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 0x7f8fb406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.185+0000 7f8fcb7fe700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 0x7f8fb406d440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005950 tx=0x7f8fbc00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.189+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8fc00567c0 con 0x7f8fcc105a40 2024-06-15T01:23:11.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.387+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fcc0fda40 con 0x7f8fb406af80 2024-06-15T01:23:11.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.397+0000 7f8fc8ff9700 1 -- 172.21.15.137:0/493751350 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8fcc0fda40 con 0x7f8fb406af80 2024-06-15T01:23:11.383 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (13m) 8m ago 24m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (24m) 8m ago 24m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (23m) 7m ago 23m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (11m) 8m ago 24m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (11m) 7m ago 23m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (13m) 8m ago 24m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (8m) 8m ago 18m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (8m) 8m ago 18m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (8m) 7m ago 18m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (7m) 7m ago 18m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (15m) 8m ago 25m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:23:11.384 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (14m) 7m ago 23m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (12m) 8m ago 25m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (12m) 7m ago 23m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (14m) 8m ago 24m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (14m) 7m ago 23m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (11m) 8m ago 22m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (10m) 8m ago 21m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (10m) 8m ago 21m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (9m) 7m ago 21m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (9m) 7m ago 20m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (8m) 7m ago 20m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:23:11.385 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (13m) 8m ago 23m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:23:11.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 msgr2=0x7f8fb406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 0x7f8fb406d440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005950 tx=0x7f8fbc00b410 comp rx=0 tx=0).stop 2024-06-15T01:23:11.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 msgr2=0x7f8fcc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00092e0 tx=0x7f8fc000bd70 comp rx=0 tx=0).stop 2024-06-15T01:23:11.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 shutdown_connections 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8fb406af80 0x7f8fb406d440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fcc0ffbd0 0x7f8fcc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 --2- 172.21.15.137:0/493751350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fcc105a40 0x7f8fcc101b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 >> 172.21.15.137:0/493751350 conn(0x7f8fcc0755a0 msgr2=0x7f8fcc1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 shutdown_connections 2024-06-15T01:23:11.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.399+0000 7f8fd213b700 1 -- 172.21.15.137:0/493751350 wait complete. 2024-06-15T01:23:11.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 -- 172.21.15.137:0/3321776987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 msgr2=0x7f46ec101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 --2- 172.21.15.137:0/3321776987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec101180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009910 tx=0x7f46e0009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 -- 172.21.15.137:0/3321776987 shutdown_connections 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 --2- 172.21.15.137:0/3321776987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 0x7f46ec1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 --2- 172.21.15.137:0/3321776987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec101180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 -- 172.21.15.137:0/3321776987 >> 172.21.15.137:0/3321776987 conn(0x7f46ec0fc630 msgr2=0x7f46ec0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 -- 172.21.15.137:0/3321776987 shutdown_connections 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.504+0000 7f46f2dac700 1 -- 172.21.15.137:0/3321776987 wait complete. 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 Processor -- start 2024-06-15T01:23:11.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 -- start start 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 0x7f46ec196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec197260 con 0x7f46ec100da0 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f2dac700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec199f90 con 0x7f46ec101750 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57494/0 (socket says 172.21.15.137:57494) 2024-06-15T01:23:11.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 -- 172.21.15.137:0/2074745114 learned_addr learned my addr 172.21.15.137:0/2074745114 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 -- 172.21.15.137:0/2074745114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 msgr2=0x7f46ec196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46ebfff700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 0x7f46ec196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 0x7f46ec196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 -- 172.21.15.137:0/2074745114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00095f0 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.505+0000 7f46f0b48700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b5c0 tx=0x7f46e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.506+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e001d070 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.506+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46e000f460 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.506+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e0021640 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.506+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec19a210 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.506+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec19a700 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.507+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec109140 con 0x7f46ec100da0 2024-06-15T01:23:11.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.507+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f46e000f9b0 con 0x7f46ec100da0 2024-06-15T01:23:11.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.508+0000 7f46e9ffb700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 0x7f46dc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.508+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f46e008b6b0 con 0x7f46ec100da0 2024-06-15T01:23:11.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.508+0000 7f46ebfff700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 0x7f46dc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.508+0000 7f46ebfff700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 0x7f46dc06d440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46ec197cb0 tx=0x7f46d8005cf0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.512+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f46e00564e0 con 0x7f46ec100da0 2024-06-15T01:23:11.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.781+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f46ec04ea90 con 0x7f46ec100da0 2024-06-15T01:23:11.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.782+0000 7f46e9ffb700 1 -- 172.21.15.137:0/2074745114 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f46e0059b00 con 0x7f46ec100da0 2024-06-15T01:23:11.768 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:23:11.768 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:23:11.768 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:23:11.769 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 msgr2=0x7f46dc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 0x7f46dc06d440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46ec197cb0 tx=0x7f46d8005cf0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 msgr2=0x7f46ec196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b5c0 tx=0x7f46e000be20 comp rx=0 tx=0).stop 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 shutdown_connections 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f46dc06af80 0x7f46dc06d440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46ec100da0 0x7f46ec196690 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 --2- 172.21.15.137:0/2074745114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46ec101750 0x7f46ec196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 >> 172.21.15.137:0/2074745114 conn(0x7f46ec0fc630 msgr2=0x7f46ec0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 shutdown_connections 2024-06-15T01:23:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.785+0000 7f46f2dac700 1 -- 172.21.15.137:0/2074745114 wait complete. 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 -- 172.21.15.137:0/997199102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8105a40 msgr2=0x7ff5d8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/997199102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8105a40 0x7ff5d8105e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009910 tx=0x7ff5c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 -- 172.21.15.137:0/997199102 shutdown_connections 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/997199102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/997199102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8105a40 0x7ff5d8105e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 -- 172.21.15.137:0/997199102 >> 172.21.15.137:0/997199102 conn(0x7ff5d80755a0 msgr2=0x7ff5d80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 -- 172.21.15.137:0/997199102 shutdown_connections 2024-06-15T01:23:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.890+0000 7ff5dfeae700 1 -- 172.21.15.137:0/997199102 wait complete. 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 Processor -- start 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 -- start start 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 0x7ff5d8195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8196270 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dfeae700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d819a000 con 0x7ff5d8105a40 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57510/0 (socket says 172.21.15.137:57510) 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 -- 172.21.15.137:0/2107285328 learned_addr learned my addr 172.21.15.137:0/2107285328 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dd449700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 0x7ff5d8195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 -- 172.21.15.137:0/2107285328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 msgr2=0x7ff5d8195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 0x7ff5d8195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5ddc4a700 1 -- 172.21.15.137:0/2107285328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80095f0 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.891+0000 7ff5dd449700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 0x7ff5d8195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5ddc4a700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c801d070 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d819a280 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5c800f460 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c8021640 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.892+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d819a770 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.893+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d8108040 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.893+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff5c800fa60 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.894+0000 7ff5ceffd700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 0x7ff5c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.894+0000 7ff5dd449700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 0x7ff5c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.894+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff5c808b6c0 con 0x7ff5d80ffbd0 2024-06-15T01:23:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.894+0000 7ff5dd449700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 0x7ff5c406d440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8196c70 tx=0x7ff5d4008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:11.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:11.898+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff5c80564f0 con 0x7ff5d80ffbd0 2024-06-15T01:23:12.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.134+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff5d804ea90 con 0x7ff5d80ffbd0 2024-06-15T01:23:12.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:11 smithi137 ceph-mon[112295]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:12.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:11 smithi137 ceph-mon[112295]: pgmap v513: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:12.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:11 smithi137 ceph-mon[112295]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:12.122 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:11 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2074745114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.135+0000 7ff5ceffd700 1 -- 172.21.15.137:0/2107285328 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7ff5c8059b10 con 0x7ff5d80ffbd0 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:23:12.122 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:23:12.123 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:23:12.124 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:12.125 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.137+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 msgr2=0x7ff5c406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 0x7ff5c406d440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8196c70 tx=0x7ff5d4008040 comp rx=0 tx=0).stop 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 msgr2=0x7ff5d8195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c800be20 comp rx=0 tx=0).stop 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 shutdown_connections 2024-06-15T01:23:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff5c406af80 0x7ff5c406d440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80ffbd0 0x7ff5d8195650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 --2- 172.21.15.137:0/2107285328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d8105a40 0x7ff5d8195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 >> 172.21.15.137:0/2107285328 conn(0x7ff5d80755a0 msgr2=0x7ff5d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 shutdown_connections 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.138+0000 7ff5dfeae700 1 -- 172.21.15.137:0/2107285328 wait complete. 2024-06-15T01:23:12.127 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:23:12.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 -- 172.21.15.137:0/2082749139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 msgr2=0x7f696c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 --2- 172.21.15.137:0/2082749139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c0ffec0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6954009910 tx=0x7f6954009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 -- 172.21.15.137:0/2082749139 shutdown_connections 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 --2- 172.21.15.137:0/2082749139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c0ffec0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 --2- 172.21.15.137:0/2082749139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 0x7f696c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.242+0000 7f697211f700 1 -- 172.21.15.137:0/2082749139 >> 172.21.15.137:0/2082749139 conn(0x7f696c0fb550 msgr2=0x7f696c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 -- 172.21.15.137:0/2082749139 shutdown_connections 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 -- 172.21.15.137:0/2082749139 wait complete. 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 Processor -- start 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 -- start start 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 0x7f696c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c197260 con 0x7f696c0ffa40 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f697211f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c19aff0 con 0x7f696c105a40 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.243+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57534/0 (socket says 172.21.15.137:57534) 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f696b7fe700 1 -- 172.21.15.137:0/1763550492 learned_addr learned my addr 172.21.15.137:0/1763550492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f6963fff700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 0x7f696c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f696b7fe700 1 -- 172.21.15.137:0/1763550492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 msgr2=0x7f696c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f696b7fe700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 0x7f696c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f696b7fe700 1 -- 172.21.15.137:0/1763550492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69540095f0 con 0x7f696c0ffa40 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f696b7fe700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c8a0 tx=0x7f695c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c00cea0 con 0x7f696c0ffa40 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f695c004830 con 0x7f696c0ffa40 2024-06-15T01:23:12.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c19b2d0 con 0x7f696c0ffa40 2024-06-15T01:23:12.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c005500 con 0x7f696c0ffa40 2024-06-15T01:23:12.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.244+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c19b820 con 0x7f696c0ffa40 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.245+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c108fe0 con 0x7f696c0ffa40 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.246+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f695c0049a0 con 0x7f696c0ffa40 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.246+0000 7f69697fa700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 0x7f695806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.246+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f695c08a080 con 0x7f696c0ffa40 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.246+0000 7f6963fff700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 0x7f695806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.247+0000 7f6963fff700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 0x7f695806d440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f696c197c60 tx=0x7f695401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:12.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.250+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f695c054eb0 con 0x7f696c0ffa40 2024-06-15T01:23:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:11 smithi142 ceph-mon[82136]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:11 smithi142 ceph-mon[82136]: pgmap v513: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:11 smithi142 ceph-mon[82136]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:12.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:11 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2074745114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:23:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.462+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f696c19bbb0 con 0x7f695806af80 2024-06-15T01:23:12.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.463+0000 7f69697fa700 1 -- 172.21.15.137:0/1763550492 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f696c19bbb0 con 0x7f695806af80 2024-06-15T01:23:12.450 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:23:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:23:12.452 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:23:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 msgr2=0x7f695806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 0x7f695806d440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f696c197c60 tx=0x7f695401a040 comp rx=0 tx=0).stop 2024-06-15T01:23:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 msgr2=0x7f696c196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c8a0 tx=0x7f695c00cc60 comp rx=0 tx=0).stop 2024-06-15T01:23:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 shutdown_connections 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f695806af80 0x7f695806d440 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f696c0ffa40 0x7f696c196640 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 --2- 172.21.15.137:0/1763550492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f696c105a40 0x7f696c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 >> 172.21.15.137:0/1763550492 conn(0x7f696c0fb550 msgr2=0x7f696c0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 shutdown_connections 2024-06-15T01:23:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.466+0000 7f697211f700 1 -- 172.21.15.137:0/1763550492 wait complete. 2024-06-15T01:23:12.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 -- 172.21.15.137:0/844728750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1028105a00 msgr2=0x7f1028105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 --2- 172.21.15.137:0/844728750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1028105a00 0x7f1028105de0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f101c009910 tx=0x7f101c009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:12.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 -- 172.21.15.137:0/844728750 shutdown_connections 2024-06-15T01:23:12.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 --2- 172.21.15.137:0/844728750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10280ffa00 0x7f10280ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 --2- 172.21.15.137:0/844728750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1028105a00 0x7f1028105de0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 -- 172.21.15.137:0/844728750 >> 172.21.15.137:0/844728750 conn(0x7f10280fb530 msgr2=0x7f10280fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 -- 172.21.15.137:0/844728750 shutdown_connections 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.570+0000 7f102ec7a700 1 -- 172.21.15.137:0/844728750 wait complete. 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 Processor -- start 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 -- start start 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 0x7f10281979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028197fd0 con 0x7f10280ffa00 2024-06-15T01:23:12.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ec7a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028198110 con 0x7f1028105a00 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.571+0000 7f102ca16700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57548/0 (socket says 172.21.15.137:57548) 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 -- 172.21.15.137:0/3865356822 learned_addr learned my addr 172.21.15.137:0/3865356822 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f1027fff700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 0x7f10281979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 -- 172.21.15.137:0/3865356822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 msgr2=0x7f10281979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 0x7f10281979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 -- 172.21.15.137:0/3865356822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0095f0 con 0x7f10280ffa00 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f1027fff700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 0x7f10281979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ca16700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f101c006010 tx=0x7f101c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c01d070 con 0x7f10280ffa00 2024-06-15T01:23:12.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f101c00f460 con 0x7f10280ffa00 2024-06-15T01:23:12.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c021620 con 0x7f10280ffa00 2024-06-15T01:23:12.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.572+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10281a04e0 con 0x7f10280ffa00 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.573+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a08f0 con 0x7f10280ffa00 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.574+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028108000 con 0x7f10280ffa00 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.574+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f101c00fa40 con 0x7f10280ffa00 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.575+0000 7f1025ffb700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 0x7f101806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.575+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f101c08b4f0 con 0x7f10280ffa00 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.575+0000 7f1027fff700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 0x7f101806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.575+0000 7f1027fff700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 0x7f101806d440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1014005950 tx=0x7f10140058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.579+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f101c056340 con 0x7f10280ffa00 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.852+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f10281982d0 con 0x7f10280ffa00 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.853+0000 7f1025ffb700 1 -- 172.21.15.137:0/3865356822 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f101c059960 con 0x7f10280ffa00 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:23:12.842 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:23:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 msgr2=0x7f101806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 0x7f101806d440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1014005950 tx=0x7f10140058e0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 msgr2=0x7f1028197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f101c006010 tx=0x7f101c00be20 comp rx=0 tx=0).stop 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 shutdown_connections 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f101806af80 0x7f101806d440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10280ffa00 0x7f1028197470 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 --2- 172.21.15.137:0/3865356822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028105a00 0x7f10281979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 >> 172.21.15.137:0/3865356822 conn(0x7f10280fb530 msgr2=0x7f10280fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 shutdown_connections 2024-06-15T01:23:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:12.855+0000 7f102ec7a700 1 -- 172.21.15.137:0/3865356822 wait complete. 2024-06-15T01:23:13.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:12 smithi137 ceph-mon[112295]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:13.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:12 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2107285328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:23:13.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:12 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3865356822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:23:13.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:12 smithi142 ceph-mon[82136]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:13.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:12 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2107285328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:23:13.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:12 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3865356822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:23:14.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:13 smithi137 ceph-mon[112295]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:14.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:13 smithi137 ceph-mon[112295]: pgmap v514: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:14.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:13 smithi142 ceph-mon[82136]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:14.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:13 smithi142 ceph-mon[82136]: pgmap v514: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:16.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:15 smithi137 ceph-mon[112295]: pgmap v515: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:16.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:15 smithi142 ceph-mon[82136]: pgmap v515: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:18.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:17 smithi137 ceph-mon[112295]: pgmap v516: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:17 smithi142 ceph-mon[82136]: pgmap v516: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:19.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:19.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:20.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:19 smithi137 ceph-mon[112295]: pgmap v517: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:20.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:19 smithi142 ceph-mon[82136]: pgmap v517: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:22.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:21 smithi137 ceph-mon[112295]: pgmap v518: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:22.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:21 smithi142 ceph-mon[82136]: pgmap v518: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:24.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:23 smithi137 ceph-mon[112295]: pgmap v519: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:24.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:23 smithi142 ceph-mon[82136]: pgmap v519: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:26.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:25 smithi142 ceph-mon[82136]: pgmap v520: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:26.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:25 smithi137 ceph-mon[112295]: pgmap v520: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:28.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:27 smithi137 ceph-mon[112295]: pgmap v521: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:28.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:27 smithi142 ceph-mon[82136]: pgmap v521: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:30.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:29 smithi137 ceph-mon[112295]: pgmap v522: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:30.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:29 smithi142 ceph-mon[82136]: pgmap v522: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:32.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:31 smithi137 ceph-mon[112295]: pgmap v523: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:31 smithi142 ceph-mon[82136]: pgmap v523: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:34.075 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:33 smithi137 ceph-mon[112295]: pgmap v524: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:34.076 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:34.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:33 smithi142 ceph-mon[82136]: pgmap v524: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:34.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:36.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:35 smithi137 ceph-mon[112295]: pgmap v525: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:36.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:35 smithi142 ceph-mon[82136]: pgmap v525: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:38.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:37 smithi137 ceph-mon[112295]: pgmap v526: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:38.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:37 smithi142 ceph-mon[82136]: pgmap v526: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:40.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:39 smithi137 ceph-mon[112295]: pgmap v527: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:39 smithi142 ceph-mon[82136]: pgmap v527: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:42.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:41 smithi137 ceph-mon[112295]: pgmap v528: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:42.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:41 smithi142 ceph-mon[82136]: pgmap v528: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/1620768673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58069020 msgr2=0x7f2d58105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/1620768673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58069020 0x7f2d58105790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009910 tx=0x7f2d48009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/1620768673 shutdown_connections 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/1620768673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58069020 0x7f2d58105790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/1620768673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58068670 0x7f2d58068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.972+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/1620768673 >> 172.21.15.137:0/1620768673 conn(0x7f2d580755a0 msgr2=0x7f2d580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/1620768673 shutdown_connections 2024-06-15T01:23:42.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/1620768673 wait complete. 2024-06-15T01:23:42.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 Processor -- start 2024-06-15T01:23:42.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 -- start start 2024-06-15T01:23:42.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:42.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58069020 0x7f2d581001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58100720 con 0x7f2d58068670 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.973+0000 7f2d5fdc8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58100860 con 0x7f2d58069020 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51432/0 (socket says 172.21.15.137:51432) 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 -- 172.21.15.137:0/2263711711 learned_addr learned my addr 172.21.15.137:0/2263711711 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5d363700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58069020 0x7f2d581001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 -- 172.21.15.137:0/2263711711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58069020 msgr2=0x7f2d581001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58069020 0x7f2d581001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 -- 172.21.15.137:0/2263711711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480095f0 con 0x7f2d58068670 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5db64700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400b4b0 tx=0x7f2d5400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:42.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d54011800 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d54011e00 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d54010e60 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d581a2950 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.974+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d581a2d60 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.975+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d581090c0 con 0x7f2d58068670 2024-06-15T01:23:42.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.976+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2d54011960 con 0x7f2d58068670 2024-06-15T01:23:42.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.976+0000 7f2d4effd700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 0x7f2d4406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:42.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.976+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2d54089e10 con 0x7f2d58068670 2024-06-15T01:23:42.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.977+0000 7f2d5d363700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 0x7f2d4406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:42.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.977+0000 7f2d5d363700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 0x7f2d4406d440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d58101490 tx=0x7f2d4801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:42.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:42.982+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2d54054cc0 con 0x7f2d58068670 2024-06-15T01:23:43.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.200+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d5802cdd0 con 0x7f2d4406af80 2024-06-15T01:23:43.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.202+0000 7f2d4effd700 1 -- 172.21.15.137:0/2263711711 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d5802cdd0 con 0x7f2d4406af80 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 msgr2=0x7f2d4406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 0x7f2d4406d440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d58101490 tx=0x7f2d4801a040 comp rx=0 tx=0).stop 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 msgr2=0x7f2d580ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400b4b0 tx=0x7f2d5400b7c0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 shutdown_connections 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2d4406af80 0x7f2d4406d440 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d58068670 0x7f2d580ffca0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 --2- 172.21.15.137:0/2263711711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d58069020 0x7f2d581001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 >> 172.21.15.137:0/2263711711 conn(0x7f2d580755a0 msgr2=0x7f2d580fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 shutdown_connections 2024-06-15T01:23:43.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.204+0000 7f2d5fdc8700 1 -- 172.21.15.137:0/2263711711 wait complete. 2024-06-15T01:23:43.201 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:23:43.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 -- 172.21.15.137:0/267169128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c100dc0 msgr2=0x7fbe1c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 --2- 172.21.15.137:0/267169128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c1011a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009910 tx=0x7fbe18009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:43.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 -- 172.21.15.137:0/267169128 shutdown_connections 2024-06-15T01:23:43.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 --2- 172.21.15.137:0/267169128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c101770 0x7fbe1c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 --2- 172.21.15.137:0/267169128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c1011a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.308+0000 7fbe237f3700 1 -- 172.21.15.137:0/267169128 >> 172.21.15.137:0/267169128 conn(0x7fbe1c0fc650 msgr2=0x7fbe1c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 -- 172.21.15.137:0/267169128 shutdown_connections 2024-06-15T01:23:43.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 -- 172.21.15.137:0/267169128 wait complete. 2024-06-15T01:23:43.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 Processor -- start 2024-06-15T01:23:43.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 -- start start 2024-06-15T01:23:43.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c197290 con 0x7fbe1c101770 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.309+0000 7fbe237f3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c19b020 con 0x7fbe1c100dc0 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51442/0 (socket says 172.21.15.137:51442) 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 -- 172.21.15.137:0/472740374 learned_addr learned my addr 172.21.15.137:0/472740374 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe2158f700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 -- 172.21.15.137:0/472740374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c100dc0 msgr2=0x7fbe1c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 -- 172.21.15.137:0/472740374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180095f0 con 0x7fbe1c101770 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe20d8e700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00d900 tx=0x7fbe0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c009900 con 0x7fbe1c101770 2024-06-15T01:23:43.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe0c010460 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c19b300 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c00f5f0 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.310+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c19b850 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.311+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c109160 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.312+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbe0c009a60 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.312+0000 7fbe127fc700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 0x7fbe0806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.312+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbe0c089ed0 con 0x7fbe1c101770 2024-06-15T01:23:43.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.312+0000 7fbe2158f700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 0x7fbe0806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.312+0000 7fbe2158f700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 0x7fbe0806d440 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe18006010 tx=0x7fbe1801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.316+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe0c054d00 con 0x7fbe1c101770 2024-06-15T01:23:43.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.530+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe1c19bb60 con 0x7fbe0806af80 2024-06-15T01:23:43.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.531+0000 7fbe127fc700 1 -- 172.21.15.137:0/472740374 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe1c19bb60 con 0x7fbe0806af80 2024-06-15T01:23:43.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 msgr2=0x7fbe0806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 0x7fbe0806d440 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe18006010 tx=0x7fbe1801a040 comp rx=0 tx=0).stop 2024-06-15T01:23:43.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 msgr2=0x7fbe1c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00d900 tx=0x7fbe0c00dc10 comp rx=0 tx=0).stop 2024-06-15T01:23:43.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 shutdown_connections 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe0806af80 0x7fbe0806d440 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c100dc0 0x7fbe1c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 --2- 172.21.15.137:0/472740374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe1c101770 0x7fbe1c196bb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.533+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 >> 172.21.15.137:0/472740374 conn(0x7fbe1c0fc650 msgr2=0x7fbe1c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.534+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 shutdown_connections 2024-06-15T01:23:43.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.534+0000 7fbe237f3700 1 -- 172.21.15.137:0/472740374 wait complete. 2024-06-15T01:23:43.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 -- 172.21.15.137:0/4176335338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118100de0 msgr2=0x7f41181011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 --2- 172.21.15.137:0/4176335338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118100de0 0x7f41181011c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4114009910 tx=0x7f4114009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 -- 172.21.15.137:0/4176335338 shutdown_connections 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 --2- 172.21.15.137:0/4176335338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118101790 0x7f41181057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 --2- 172.21.15.137:0/4176335338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118100de0 0x7f41181011c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 -- 172.21.15.137:0/4176335338 >> 172.21.15.137:0/4176335338 conn(0x7f41180fc670 msgr2=0x7f41180fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 -- 172.21.15.137:0/4176335338 shutdown_connections 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.638+0000 7f411f658700 1 -- 172.21.15.137:0/4176335338 wait complete. 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 Processor -- start 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 -- start start 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118100de0 0x7f4118196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118197200 con 0x7f4118101790 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411f658700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118197340 con 0x7f4118100de0 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411cbf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411cbf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51464/0 (socket says 172.21.15.137:51464) 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411cbf3700 1 -- 172.21.15.137:0/18902565 learned_addr learned my addr 172.21.15.137:0/18902565 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.639+0000 7f411d3f4700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118100de0 0x7f4118196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411cbf3700 1 -- 172.21.15.137:0/18902565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118100de0 msgr2=0x7f4118196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411cbf3700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118100de0 0x7f4118196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411cbf3700 1 -- 172.21.15.137:0/18902565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140095f0 con 0x7f4118101790 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411cbf3700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f410800c960 tx=0x7f410800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4108007a70 con 0x7f4118101790 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4108007bd0 con 0x7f4118101790 2024-06-15T01:23:43.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411819b190 con 0x7f4118101790 2024-06-15T01:23:43.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4108018630 con 0x7f4118101790 2024-06-15T01:23:43.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.640+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411819b6e0 con 0x7f4118101790 2024-06-15T01:23:43.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.641+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4118109180 con 0x7f4118101790 2024-06-15T01:23:43.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.641+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f410800f450 con 0x7f4118101790 2024-06-15T01:23:43.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.642+0000 7f410e7fc700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 0x7f410406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.642+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f410808a340 con 0x7f4118101790 2024-06-15T01:23:43.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.642+0000 7f411d3f4700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 0x7f410406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.642+0000 7f411d3f4700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 0x7f410406d440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.647+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4108055170 con 0x7f4118101790 2024-06-15T01:23:43.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.851+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f411802cce0 con 0x7f410406af80 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.860+0000 7f410e7fc700 1 -- 172.21.15.137:0/18902565 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f411802cce0 con 0x7f410406af80 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (14m) 8m ago 24m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (25m) 8m ago 25m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (24m) 8m ago 24m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (12m) 8m ago 25m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (12m) 8m ago 23m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (14m) 8m ago 24m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (8m) 8m ago 19m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:23:43.847 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (9m) 8m ago 19m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (8m) 8m ago 19m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (8m) 8m ago 19m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (16m) 8m ago 26m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (15m) 8m ago 23m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (12m) 8m ago 26m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (12m) 8m ago 23m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (14m) 8m ago 25m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (14m) 8m ago 23m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (12m) 8m ago 23m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (11m) 8m ago 22m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:23:43.848 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (10m) 8m ago 22m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (10m) 8m ago 21m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (9m) 8m ago 21m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (9m) 8m ago 20m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (14m) 8m ago 24m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 msgr2=0x7f410406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 0x7f410406d440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411401a040 comp rx=0 tx=0).stop 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 msgr2=0x7f4118196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f410800c960 tx=0x7f410800cc70 comp rx=0 tx=0).stop 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 shutdown_connections 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f410406af80 0x7f410406d440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4118100de0 0x7f4118196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 --2- 172.21.15.137:0/18902565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4118101790 0x7f4118196bb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 >> 172.21.15.137:0/18902565 conn(0x7f41180fc670 msgr2=0x7f41180fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.862+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 shutdown_connections 2024-06-15T01:23:43.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.863+0000 7f411f658700 1 -- 172.21.15.137:0/18902565 wait complete. 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3977109099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 msgr2=0x7f48ac0ff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3977109099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac0ff730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009910 tx=0x7f48a0009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3977109099 shutdown_connections 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3977109099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac0ff730 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3977109099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 0x7f48ac105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3977109099 >> 172.21.15.137:0/3977109099 conn(0x7f48ac074d20 msgr2=0x7f48ac075130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:43.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3977109099 shutdown_connections 2024-06-15T01:23:43.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.965+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3977109099 wait complete. 2024-06-15T01:23:43.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 Processor -- start 2024-06-15T01:23:43.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 -- start start 2024-06-15T01:23:43.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 0x7f48ac196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac197290 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b3bc8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac19b020 con 0x7f48ac105280 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b1964700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b1964700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51482/0 (socket says 172.21.15.137:51482) 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.966+0000 7f48b1964700 1 -- 172.21.15.137:0/3384904078 learned_addr learned my addr 172.21.15.137:0/3384904078 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b1163700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 0x7f48ac196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b1964700 1 -- 172.21.15.137:0/3384904078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 msgr2=0x7f48ac196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b1964700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 0x7f48ac196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b1964700 1 -- 172.21.15.137:0/3384904078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00095f0 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b1964700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fa0 tx=0x7f489c0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c003bd0 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f489c010040 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac19b300 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c0146f0 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.967+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac19b850 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.968+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac109790 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.968+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f489c004030 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.969+0000 7f48aaffd700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 0x7f489406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.969+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f489c08a0f0 con 0x7f48ac0ff2d0 2024-06-15T01:23:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.969+0000 7f48b1163700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 0x7f489406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.969+0000 7f48b1163700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 0x7f489406d440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48ac197c90 tx=0x7f48a0005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:43.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:43.973+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f489c054e80 con 0x7f48ac0ff2d0 2024-06-15T01:23:44.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:43 smithi137 ceph-mon[112295]: pgmap v529: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:44.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:43 smithi137 ceph-mon[112295]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:44.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.260+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48ac04ea90 con 0x7f48ac0ff2d0 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.260+0000 7f48aaffd700 1 -- 172.21.15.137:0/3384904078 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f489c0584a0 con 0x7f48ac0ff2d0 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:44.247 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:23:44.248 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:23:44.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 msgr2=0x7f489406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 0x7f489406d440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48ac197c90 tx=0x7f48a0005a90 comp rx=0 tx=0).stop 2024-06-15T01:23:44.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 msgr2=0x7f48ac196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fa0 tx=0x7f489c0076c0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 shutdown_connections 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f489406af80 0x7f489406d440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48ac0ff2d0 0x7f48ac196670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 --2- 172.21.15.137:0/3384904078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ac105280 0x7f48ac196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 >> 172.21.15.137:0/3384904078 conn(0x7f48ac074d20 msgr2=0x7f48ac1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 shutdown_connections 2024-06-15T01:23:44.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.262+0000 7f48b3bc8700 1 -- 172.21.15.137:0/3384904078 wait complete. 2024-06-15T01:23:44.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:43 smithi142 ceph-mon[82136]: pgmap v529: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:44.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:43 smithi142 ceph-mon[82136]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:44.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 -- 172.21.15.137:0/1882313108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54101c70 msgr2=0x7fbb54102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/1882313108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54101c70 0x7fbb54102050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009910 tx=0x7fbb4c009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 -- 172.21.15.137:0/1882313108 shutdown_connections 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/1882313108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54102620 0x7fbb5410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/1882313108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54101c70 0x7fbb54102050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 -- 172.21.15.137:0/1882313108 >> 172.21.15.137:0/1882313108 conn(0x7fbb540fb550 msgr2=0x7fbb540fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 -- 172.21.15.137:0/1882313108 shutdown_connections 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 -- 172.21.15.137:0/1882313108 wait complete. 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.368+0000 7fbb5aa80700 1 Processor -- start 2024-06-15T01:23:44.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5aa80700 1 -- start start 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5aa80700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 0x7fbb5419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5aa80700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5aa80700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5419d120 con 0x7fbb54102620 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5aa80700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541965d0 con 0x7fbb54101c70 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51494/0 (socket says 172.21.15.137:51494) 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 -- 172.21.15.137:0/232346015 learned_addr learned my addr 172.21.15.137:0/232346015 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5881c700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 0x7fbb5419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 -- 172.21.15.137:0/232346015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 msgr2=0x7fbb5419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 0x7fbb5419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 -- 172.21.15.137:0/232346015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0095f0 con 0x7fbb54102620 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb5881c700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 0x7fbb5419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb53fff700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b9c0 tx=0x7fbb4400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.369+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb4400c980 con 0x7fbb54102620 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.370+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb44014440 con 0x7fbb54102620 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.370+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb44012570 con 0x7fbb54102620 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.370+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541968b0 con 0x7fbb54102620 2024-06-15T01:23:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.370+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb54196e00 con 0x7fbb54102620 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.371+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54108210 con 0x7fbb54102620 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.371+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbb440145b0 con 0x7fbb54102620 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.372+0000 7fbb51ffb700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 0x7fbb4006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.372+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbb44006e40 con 0x7fbb54102620 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.372+0000 7fbb5881c700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 0x7fbb4006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.372+0000 7fbb5881c700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 0x7fbb4006d440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b5c0 tx=0x7fbb4c005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:44.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.376+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb44058640 con 0x7fbb54102620 2024-06-15T01:23:44.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.612+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb5404ea90 con 0x7fbb54102620 2024-06-15T01:23:44.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.612+0000 7fbb51ffb700 1 -- 172.21.15.137:0/232346015 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fbb440581d0 con 0x7fbb54102620 2024-06-15T01:23:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:23:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:23:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:23:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:23:44.600 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:23:44.601 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:44.603 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:23:44.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.614+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 msgr2=0x7fbb4006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.614+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 0x7fbb4006d440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b5c0 tx=0x7fbb4c005960 comp rx=0 tx=0).stop 2024-06-15T01:23:44.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.614+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 msgr2=0x7fbb5419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.614+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b9c0 tx=0x7fbb4400bd80 comp rx=0 tx=0).stop 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 shutdown_connections 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbb4006af80 0x7fbb4006d440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54101c70 0x7fbb5419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 --2- 172.21.15.137:0/232346015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb54102620 0x7fbb5419ca90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 >> 172.21.15.137:0/232346015 conn(0x7fbb540fb550 msgr2=0x7fbb54105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 shutdown_connections 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.615+0000 7fbb5aa80700 1 -- 172.21.15.137:0/232346015 wait complete. 2024-06-15T01:23:44.604 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:23:44.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1650278845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a740ffa40 msgr2=0x7f1a740ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1650278845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a740ffa40 0x7f1a740ffec0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009910 tx=0x7f1a64009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:44.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1650278845 shutdown_connections 2024-06-15T01:23:44.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1650278845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a740ffa40 0x7f1a740ffec0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1650278845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a74105a40 0x7f1a74105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1650278845 >> 172.21.15.137:0/1650278845 conn(0x7f1a740fb550 msgr2=0x7f1a740fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:44.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.719+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1650278845 shutdown_connections 2024-06-15T01:23:44.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1650278845 wait complete. 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 Processor -- start 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 -- start start 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a740ffa40 0x7f1a7419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419f290 con 0x7f1a74105a40 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a7bbcd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74198740 con 0x7f1a740ffa40 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a79168700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a79168700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51518/0 (socket says 172.21.15.137:51518) 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a79168700 1 -- 172.21.15.137:0/1895633292 learned_addr learned my addr 172.21.15.137:0/1895633292 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.720+0000 7f1a79969700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a740ffa40 0x7f1a7419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a79168700 1 -- 172.21.15.137:0/1895633292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a740ffa40 msgr2=0x7f1a7419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a79168700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a740ffa40 0x7f1a7419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a79168700 1 -- 172.21.15.137:0/1895633292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a640095f0 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a79168700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006010 tx=0x7f1a6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a6401d070 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a64005710 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a741989c0 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a6400fe20 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.721+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a74198eb0 con 0x7f1a74105a40 2024-06-15T01:23:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.722+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7404ea90 con 0x7f1a74105a40 2024-06-15T01:23:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.722+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f1a64003a60 con 0x7f1a74105a40 2024-06-15T01:23:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.723+0000 7f1a6affd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 0x7f1a6006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.723+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1a6408b4a0 con 0x7f1a74105a40 2024-06-15T01:23:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.723+0000 7f1a79969700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 0x7f1a6006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.723+0000 7f1a79969700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 0x7f1a6006d440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005950 tx=0x7f1a7000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:44.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.727+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1a64059d60 con 0x7f1a74105a40 2024-06-15T01:23:44.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.936+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a74199200 con 0x7f1a6006af80 2024-06-15T01:23:44.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.937+0000 7f1a6affd700 1 -- 172.21.15.137:0/1895633292 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1a74199200 con 0x7f1a6006af80 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:23:44.924 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:23:44.925 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:23:44.925 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:23:44.925 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:23:44.925 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 msgr2=0x7f1a6006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 0x7f1a6006d440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005950 tx=0x7f1a7000b410 comp rx=0 tx=0).stop 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 msgr2=0x7f1a7419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006010 tx=0x7f1a6400bd70 comp rx=0 tx=0).stop 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 shutdown_connections 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1a6006af80 0x7f1a6006d440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a740ffa40 0x7f1a7419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 --2- 172.21.15.137:0/1895633292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a74105a40 0x7f1a7419ec00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:44.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 >> 172.21.15.137:0/1895633292 conn(0x7f1a740fb550 msgr2=0x7f1a740fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:44.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 shutdown_connections 2024-06-15T01:23:44.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:44.939+0000 7f1a7bbcd700 1 -- 172.21.15.137:0/1895633292 wait complete. 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 -- 172.21.15.137:0/4131968391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 msgr2=0x7f2dd4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/4131968391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4100050 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009910 tx=0x7f2dc4009c20 comp rx=0 tx=0).stop 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 -- 172.21.15.137:0/4131968391 shutdown_connections 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/4131968391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4100050 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/4131968391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 0x7f2dd4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 -- 172.21.15.137:0/4131968391 >> 172.21.15.137:0/4131968391 conn(0x7f2dd40755a0 msgr2=0x7f2dd40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:45.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 -- 172.21.15.137:0/4131968391 shutdown_connections 2024-06-15T01:23:45.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.045+0000 7f2dd8d70700 1 -- 172.21.15.137:0/4131968391 wait complete. 2024-06-15T01:23:45.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 Processor -- start 2024-06-15T01:23:45.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 -- start start 2024-06-15T01:23:45.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:45.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 0x7f2dd4195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4196160 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.046+0000 7f2dd8d70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4199ef0 con 0x7f2dd4105a40 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51534/0 (socket says 172.21.15.137:51534) 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 -- 172.21.15.137:0/2248776597 learned_addr learned my addr 172.21.15.137:0/2248776597 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd1d9b700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 0x7f2dd4195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 -- 172.21.15.137:0/2248776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 msgr2=0x7f2dd4195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 0x7f2dd4195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 -- 172.21.15.137:0/2248776597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40095f0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd259c700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800d900 tx=0x7f2dc800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc80041f0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2dc8004cd0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd419a1d0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc8005660 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.047+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd419a720 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.048+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd4107fd0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.049+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2dc8004e40 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.049+0000 7f2dc37fe700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 0x7f2dbc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:23:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.049+0000 7f2dd1d9b700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 0x7f2dbc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:23:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.049+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2dc808a170 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.050+0000 7f2dd1d9b700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 0x7f2dbc06d440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4196b60 tx=0x7f2dc401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:23:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.054+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2dc8054fc0 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:44 smithi137 ceph-mon[112295]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:45.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:44 smithi137 ceph-mon[112295]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:45.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:44 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3384904078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:23:45.216 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:44 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/232346015' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:23:45.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:44 smithi142 ceph-mon[82136]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:45.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:44 smithi142 ceph-mon[82136]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:45.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:44 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3384904078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:23:45.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:44 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/232346015' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:23:45.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.325+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2dd419aa50 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.325+0000 7f2dc37fe700 1 -- 172.21.15.137:0/2248776597 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f2dc8016330 con 0x7f2dd40ffbd0 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:23:45.313 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:23:45.314 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:23:45.314 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:23:45.314 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:23:45.314 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:23:45.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 msgr2=0x7f2dbc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 0x7f2dbc06d440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4196b60 tx=0x7f2dc401a040 comp rx=0 tx=0).stop 2024-06-15T01:23:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 msgr2=0x7f2dd4195540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:23:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800d900 tx=0x7f2dc800dc10 comp rx=0 tx=0).stop 2024-06-15T01:23:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 shutdown_connections 2024-06-15T01:23:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2dbc06af80 0x7f2dbc06d440 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd40ffbd0 0x7f2dd4195540 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 --2- 172.21.15.137:0/2248776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4105a40 0x7f2dd4195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:23:45.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 >> 172.21.15.137:0/2248776597 conn(0x7f2dd40755a0 msgr2=0x7f2dd40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:23:45.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 shutdown_connections 2024-06-15T01:23:45.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:23:45.328+0000 7f2dd8d70700 1 -- 172.21.15.137:0/2248776597 wait complete. 2024-06-15T01:23:46.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:45 smithi142 ceph-mon[82136]: pgmap v530: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:46.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:45 smithi142 ceph-mon[82136]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:46.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:45 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2248776597' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:23:46.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:45 smithi137 ceph-mon[112295]: pgmap v530: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:46.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:45 smithi137 ceph-mon[112295]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:23:46.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:45 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2248776597' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:23:48.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:47 smithi142 ceph-mon[82136]: pgmap v531: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:48.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:47 smithi137 ceph-mon[112295]: pgmap v531: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:23:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:49 smithi142 ceph-mon[82136]: pgmap v532: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:49 smithi137 ceph-mon[112295]: pgmap v532: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:52.267 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:51 smithi137 ceph-mon[112295]: pgmap v533: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:52.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:51 smithi142 ceph-mon[82136]: pgmap v533: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:54.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:54 smithi137 ceph-mon[112295]: pgmap v534: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:54.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:54 smithi142 ceph-mon[82136]: pgmap v534: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:56.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:56 smithi142 ceph-mon[82136]: pgmap v535: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:56.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:56 smithi137 ceph-mon[112295]: pgmap v535: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:58.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:23:58 smithi142 ceph-mon[82136]: pgmap v536: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:23:58.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:23:58 smithi137 ceph-mon[112295]: pgmap v536: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:00.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:00 smithi142 ceph-mon[82136]: pgmap v537: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:00.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:00 smithi137 ceph-mon[112295]: pgmap v537: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:02.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:02 smithi137 ceph-mon[112295]: pgmap v538: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:02.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:02 smithi142 ceph-mon[82136]: pgmap v538: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:03.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:03.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:04.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:04 smithi137 ceph-mon[112295]: pgmap v539: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:04.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:04 smithi142 ceph-mon[82136]: pgmap v539: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:06.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:06 smithi142 ceph-mon[82136]: pgmap v540: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:06 smithi137 ceph-mon[112295]: pgmap v540: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:08.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:08 smithi142 ceph-mon[82136]: pgmap v541: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:08.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:08 smithi137 ceph-mon[112295]: pgmap v541: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:10.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:10 smithi142 ceph-mon[82136]: pgmap v542: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:10.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:10 smithi137 ceph-mon[112295]: pgmap v542: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:12.300 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:12 smithi137 ceph-mon[112295]: pgmap v543: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:12.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:12 smithi142 ceph-mon[82136]: pgmap v543: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:14.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:14 smithi142 ceph-mon[82136]: pgmap v544: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:14.439 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:14 smithi137 ceph-mon[112295]: pgmap v544: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:15.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 -- 172.21.15.137:0/563656934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 msgr2=0x7fd4201011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 --2- 172.21.15.137:0/563656934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201011c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd410009910 tx=0x7fd410009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:15.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 -- 172.21.15.137:0/563656934 shutdown_connections 2024-06-15T01:24:15.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 --2- 172.21.15.137:0/563656934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 0x7fd4201057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 --2- 172.21.15.137:0/563656934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201011c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 -- 172.21.15.137:0/563656934 >> 172.21.15.137:0/563656934 conn(0x7fd4200fc6b0 msgr2=0x7fd4200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:15.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 -- 172.21.15.137:0/563656934 shutdown_connections 2024-06-15T01:24:15.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.445+0000 7fd425e4c700 1 -- 172.21.15.137:0/563656934 wait complete. 2024-06-15T01:24:15.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 Processor -- start 2024-06-15T01:24:15.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 -- start start 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 0x7fd420196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420197280 con 0x7fd420100de0 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd425e4c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420199fb0 con 0x7fd420101790 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59970/0 (socket says 172.21.15.137:59970) 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd41f7fe700 1 -- 172.21.15.137:0/4199873754 learned_addr learned my addr 172.21.15.137:0/4199873754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.446+0000 7fd416ffd700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 0x7fd420196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41f7fe700 1 -- 172.21.15.137:0/4199873754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 msgr2=0x7fd420196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41f7fe700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 0x7fd420196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41f7fe700 1 -- 172.21.15.137:0/4199873754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100095f0 con 0x7fd420100de0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41f7fe700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd410006010 tx=0x7fd41000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41001d070 con 0x7fd420100de0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd41000f460 con 0x7fd420100de0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd410021640 con 0x7fd420100de0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42019a230 con 0x7fd420100de0 2024-06-15T01:24:15.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.447+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42019a6d0 con 0x7fd420100de0 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.448+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd420109180 con 0x7fd420100de0 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.449+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd41000f9e0 con 0x7fd420100de0 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.449+0000 7fd41d7fa700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 0x7fd40c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.449+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd41008b620 con 0x7fd420100de0 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.449+0000 7fd416ffd700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 0x7fd40c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.450+0000 7fd416ffd700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 0x7fd40c06d440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd420197cd0 tx=0x7fd40800a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:15.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.454+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd410056450 con 0x7fd420100de0 2024-06-15T01:24:15.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.665+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd42019a9b0 con 0x7fd40c06af80 2024-06-15T01:24:15.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.667+0000 7fd41d7fa700 1 -- 172.21.15.137:0/4199873754 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd42019a9b0 con 0x7fd40c06af80 2024-06-15T01:24:15.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 msgr2=0x7fd40c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 0x7fd40c06d440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd420197cd0 tx=0x7fd40800a300 comp rx=0 tx=0).stop 2024-06-15T01:24:15.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 msgr2=0x7fd4201966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd410006010 tx=0x7fd41000bd70 comp rx=0 tx=0).stop 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 shutdown_connections 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd40c06af80 0x7fd40c06d440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420100de0 0x7fd4201966b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 --2- 172.21.15.137:0/4199873754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd420101790 0x7fd420196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 >> 172.21.15.137:0/4199873754 conn(0x7fd4200fc6b0 msgr2=0x7fd4200fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 shutdown_connections 2024-06-15T01:24:15.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.669+0000 7fd425e4c700 1 -- 172.21.15.137:0/4199873754 wait complete. 2024-06-15T01:24:15.666 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 -- 172.21.15.137:0/1447289012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790105a40 msgr2=0x7f1790105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 --2- 172.21.15.137:0/1447289012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790105a40 0x7f1790105e20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1784009910 tx=0x7f1784009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 -- 172.21.15.137:0/1447289012 shutdown_connections 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 --2- 172.21.15.137:0/1447289012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17900ffbd0 0x7f1790100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 --2- 172.21.15.137:0/1447289012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790105a40 0x7f1790105e20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 -- 172.21.15.137:0/1447289012 >> 172.21.15.137:0/1447289012 conn(0x7f17900755a0 msgr2=0x7f17900759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:15.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 -- 172.21.15.137:0/1447289012 shutdown_connections 2024-06-15T01:24:15.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.774+0000 7f1796c16700 1 -- 172.21.15.137:0/1447289012 wait complete. 2024-06-15T01:24:15.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 Processor -- start 2024-06-15T01:24:15.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 -- start start 2024-06-15T01:24:15.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790105a40 0x7f17901979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790198000 con 0x7f17900ffbd0 2024-06-15T01:24:15.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f1796c16700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790198170 con 0x7f1790105a40 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f17949b2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f17949b2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59990/0 (socket says 172.21.15.137:59990) 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f17949b2700 1 -- 172.21.15.137:0/1525530871 learned_addr learned my addr 172.21.15.137:0/1525530871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.775+0000 7f178ffff700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790105a40 0x7f17901979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f17949b2700 1 -- 172.21.15.137:0/1525530871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790105a40 msgr2=0x7f17901979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f17949b2700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790105a40 0x7f17901979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f17949b2700 1 -- 172.21.15.137:0/1525530871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840095f0 con 0x7f17900ffbd0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f17949b2700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1784006010 tx=0x7f178400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178401d070 con 0x7f17900ffbd0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1784005710 con 0x7f17900ffbd0 2024-06-15T01:24:15.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a0510 con 0x7f17900ffbd0 2024-06-15T01:24:15.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178400fe20 con 0x7f17900ffbd0 2024-06-15T01:24:15.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.776+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a0980 con 0x7f17900ffbd0 2024-06-15T01:24:15.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.777+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790108040 con 0x7f17900ffbd0 2024-06-15T01:24:15.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.777+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f1784003a60 con 0x7f17900ffbd0 2024-06-15T01:24:15.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.778+0000 7f178dffb700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 0x7f178006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:15.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.778+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f178408b560 con 0x7f17900ffbd0 2024-06-15T01:24:15.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.778+0000 7f178ffff700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 0x7f178006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:15.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.778+0000 7f178ffff700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 0x7f178006d440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1790068ce0 tx=0x7f177c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:15.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.782+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1784056390 con 0x7f17900ffbd0 2024-06-15T01:24:15.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.992+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17900611d0 con 0x7f178006af80 2024-06-15T01:24:15.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.994+0000 7f178dffb700 1 -- 172.21.15.137:0/1525530871 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f17900611d0 con 0x7f178006af80 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 msgr2=0x7f178006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 0x7f178006d440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1790068ce0 tx=0x7f177c00b410 comp rx=0 tx=0).stop 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 msgr2=0x7f17901974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1784006010 tx=0x7f178400bd70 comp rx=0 tx=0).stop 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 shutdown_connections 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f178006af80 0x7f178006d440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17900ffbd0 0x7f17901974a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 --2- 172.21.15.137:0/1525530871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790105a40 0x7f17901979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 >> 172.21.15.137:0/1525530871 conn(0x7f17900755a0 msgr2=0x7f17900fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 shutdown_connections 2024-06-15T01:24:15.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:15.996+0000 7f1796c16700 1 -- 172.21.15.137:0/1525530871 wait complete. 2024-06-15T01:24:16.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 -- 172.21.15.137:0/3803332966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 msgr2=0x7ff108105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/3803332966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff108105670 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800b0a0 tx=0x7ff0f800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 -- 172.21.15.137:0/3803332966 shutdown_connections 2024-06-15T01:24:16.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/3803332966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 0x7ff1080ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/3803332966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff108105670 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 -- 172.21.15.137:0/3803332966 >> 172.21.15.137:0/3803332966 conn(0x7ff108074bf0 msgr2=0x7ff108075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:16.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 -- 172.21.15.137:0/3803332966 shutdown_connections 2024-06-15T01:24:16.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 -- 172.21.15.137:0/3803332966 wait complete. 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.104+0000 7ff10e0b6700 1 Processor -- start 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff10e0b6700 1 -- start start 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff10e0b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 0x7ff1081944b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff10e0b6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff10e0b6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1081950d0 con 0x7ff108105290 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff10e0b6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108198e60 con 0x7ff1080ff310 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34824/0 (socket says 172.21.15.137:34824) 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 -- 172.21.15.137:0/4032464943 learned_addr learned my addr 172.21.15.137:0/4032464943 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff1077fe700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 0x7ff1081944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 -- 172.21.15.137:0/4032464943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 msgr2=0x7ff1081944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 0x7ff1081944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.105+0000 7ff106ffd700 1 -- 172.21.15.137:0/4032464943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8009d00 con 0x7ff108105290 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff106ffd700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00ed50 tx=0x7ff0fc00a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0fc0165c0 con 0x7ff108105290 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0fc016bc0 con 0x7ff108105290 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff108199140 con 0x7ff108105290 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0fc00bc60 con 0x7ff108105290 2024-06-15T01:24:16.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.106+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff108199690 con 0x7ff108105290 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.107+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff108107790 con 0x7ff108105290 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.107+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff0fc016720 con 0x7ff108105290 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.108+0000 7ff104ff9700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 0x7ff0f006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.108+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff0fc08b400 con 0x7ff108105290 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.108+0000 7ff1077fe700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 0x7ff0f006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.108+0000 7ff1077fe700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 0x7ff0f006d440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006070 tx=0x7ff0f8006000 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.112+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff0fc0562c0 con 0x7ff108105290 2024-06-15T01:24:16.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.312+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff108199a20 con 0x7ff0f006af80 2024-06-15T01:24:16.299 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:16 smithi137 ceph-mon[112295]: pgmap v545: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:16.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:16 smithi142 ceph-mon[82136]: pgmap v545: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:16.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.323+0000 7ff104ff9700 1 -- 172.21.15.137:0/4032464943 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff108199a20 con 0x7ff0f006af80 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (14m) 9m ago 25m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (25m) 9m ago 25m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (24m) 8m ago 24m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (13m) 9m ago 25m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (12m) 8m ago 24m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (14m) 9m ago 25m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (9m) 9m ago 19m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (9m) 9m ago 19m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:24:16.310 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (9m) 8m ago 19m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (8m) 8m ago 19m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (16m) 9m ago 26m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (15m) 8m ago 24m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (13m) 9m ago 27m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (13m) 8m ago 24m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (15m) 9m ago 25m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (15m) 8m ago 24m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (12m) 9m ago 23m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (12m) 9m ago 23m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (11m) 9m ago 22m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (10m) 8m ago 22m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (10m) 8m ago 21m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:24:16.311 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (9m) 8m ago 21m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:24:16.312 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (15m) 9m ago 24m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:24:16.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 msgr2=0x7ff0f006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 0x7ff0f006d440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006070 tx=0x7ff0f8006000 comp rx=0 tx=0).stop 2024-06-15T01:24:16.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 msgr2=0x7ff1081949f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00ed50 tx=0x7ff0fc00a630 comp rx=0 tx=0).stop 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 shutdown_connections 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff0f006af80 0x7ff0f006d440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1080ff310 0x7ff1081944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 --2- 172.21.15.137:0/4032464943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff108105290 0x7ff1081949f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 >> 172.21.15.137:0/4032464943 conn(0x7ff108074bf0 msgr2=0x7ff1080fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 shutdown_connections 2024-06-15T01:24:16.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.326+0000 7ff10e0b6700 1 -- 172.21.15.137:0/4032464943 wait complete. 2024-06-15T01:24:16.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 -- 172.21.15.137:0/2430634000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 msgr2=0x7f44c81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 --2- 172.21.15.137:0/2430634000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c81057c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009910 tx=0x7f44bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:16.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 -- 172.21.15.137:0/2430634000 shutdown_connections 2024-06-15T01:24:16.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 --2- 172.21.15.137:0/2430634000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c81057c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 --2- 172.21.15.137:0/2430634000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 0x7f44c81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.428+0000 7f44cd379700 1 -- 172.21.15.137:0/2430634000 >> 172.21.15.137:0/2430634000 conn(0x7f44c80fc650 msgr2=0x7f44c80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:16.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.429+0000 7f44cd379700 1 -- 172.21.15.137:0/2430634000 shutdown_connections 2024-06-15T01:24:16.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.429+0000 7f44cd379700 1 -- 172.21.15.137:0/2430634000 wait complete. 2024-06-15T01:24:16.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.429+0000 7f44cd379700 1 Processor -- start 2024-06-15T01:24:16.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.429+0000 7f44cd379700 1 -- start start 2024-06-15T01:24:16.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 0x7f44c819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c819f1a0 con 0x7f44c8101770 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c81986e0 con 0x7f44c8100dc0 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34848/0 (socket says 172.21.15.137:34848) 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 -- 172.21.15.137:0/803422094 learned_addr learned my addr 172.21.15.137:0/803422094 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c6ffd700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 0x7f44c819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 -- 172.21.15.137:0/803422094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 msgr2=0x7f44c819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 0x7f44c819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 -- 172.21.15.137:0/803422094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0095f0 con 0x7f44c8101770 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44c67fc700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00bda0 tx=0x7f44bc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44bc01d070 con 0x7f44c8101770 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44bc00f460 con 0x7f44c8101770 2024-06-15T01:24:16.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c8198960 con 0x7f44c8101770 2024-06-15T01:24:16.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44bc021620 con 0x7f44c8101770 2024-06-15T01:24:16.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.430+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c8198e50 con 0x7f44c8101770 2024-06-15T01:24:16.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.431+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c8104f40 con 0x7f44c8101770 2024-06-15T01:24:16.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.432+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f44bc00fa40 con 0x7f44c8101770 2024-06-15T01:24:16.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.432+0000 7f44affff700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 0x7f44b006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.432+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f44bc08b650 con 0x7f44c8101770 2024-06-15T01:24:16.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.432+0000 7f44c6ffd700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 0x7f44b006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.433+0000 7f44c6ffd700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 0x7f44b006d440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44b800ac50 tx=0x7f44b800a380 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.437+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f44bc059e70 con 0x7f44c8101770 2024-06-15T01:24:16.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.707+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f44c804ea90 con 0x7f44c8101770 2024-06-15T01:24:16.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.708+0000 7f44affff700 1 -- 172.21.15.137:0/803422094 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f44bc059a00 con 0x7f44c8101770 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:24:16.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:16.696 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:24:16.696 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:24:16.696 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:24:16.696 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 msgr2=0x7f44b006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 0x7f44b006d440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44b800ac50 tx=0x7f44b800a380 comp rx=0 tx=0).stop 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 msgr2=0x7f44c819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00bda0 tx=0x7f44bc00bdd0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 shutdown_connections 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f44b006af80 0x7f44b006d440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c8100dc0 0x7f44c819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 --2- 172.21.15.137:0/803422094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c8101770 0x7f44c819eba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 >> 172.21.15.137:0/803422094 conn(0x7f44c80fc650 msgr2=0x7f44c80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 shutdown_connections 2024-06-15T01:24:16.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.710+0000 7f44cd379700 1 -- 172.21.15.137:0/803422094 wait complete. 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 -- 172.21.15.137:0/2877917379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 msgr2=0x7f621c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 --2- 172.21.15.137:0/2877917379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c105e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f620c009910 tx=0x7f620c009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 -- 172.21.15.137:0/2877917379 shutdown_connections 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 --2- 172.21.15.137:0/2877917379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 --2- 172.21.15.137:0/2877917379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c105e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 -- 172.21.15.137:0/2877917379 >> 172.21.15.137:0/2877917379 conn(0x7f621c0755a0 msgr2=0x7f621c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 -- 172.21.15.137:0/2877917379 shutdown_connections 2024-06-15T01:24:16.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.814+0000 7f6221996700 1 -- 172.21.15.137:0/2877917379 wait complete. 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 Processor -- start 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 -- start start 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c1971d0 con 0x7f621c105a40 2024-06-15T01:24:16.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f6221996700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c197310 con 0x7f621c0ffbd0 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f621affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f621affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34864/0 (socket says 172.21.15.137:34864) 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.815+0000 7f621affd700 1 -- 172.21.15.137:0/1930093689 learned_addr learned my addr 172.21.15.137:0/1930093689 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6212ffd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f621affd700 1 -- 172.21.15.137:0/1930093689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 msgr2=0x7f621c196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f621affd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f621affd700 1 -- 172.21.15.137:0/1930093689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0095f0 con 0x7f621c105a40 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6212ffd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f621affd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f620400c930 tx=0x7f620400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6204007ad0 con 0x7f621c105a40 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6204007c30 con 0x7f621c105a40 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6204018630 con 0x7f621c105a40 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c19b160 con 0x7f621c105a40 2024-06-15T01:24:16.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.816+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c19b6b0 con 0x7f621c105a40 2024-06-15T01:24:16.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.817+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c109050 con 0x7f621c105a40 2024-06-15T01:24:16.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.818+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f620400f450 con 0x7f621c105a40 2024-06-15T01:24:16.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.818+0000 7f6220994700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 0x7f620806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:16.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.818+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f620408a560 con 0x7f621c105a40 2024-06-15T01:24:16.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.818+0000 7f6212ffd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 0x7f620806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:16.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.819+0000 7f6212ffd700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 0x7f620806d440 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f620c006010 tx=0x7f620c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:16.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:16.823+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f62040554c0 con 0x7f621c105a40 2024-06-15T01:24:17.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.064+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f621c04ea90 con 0x7f621c105a40 2024-06-15T01:24:17.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.065+0000 7f6220994700 1 -- 172.21.15.137:0/1930093689 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f6204058ae0 con 0x7f621c105a40 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:24:17.052 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:24:17.053 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:17.054 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 msgr2=0x7f620806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 0x7f620806d440 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f620c006010 tx=0x7f620c01a040 comp rx=0 tx=0).stop 2024-06-15T01:24:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 msgr2=0x7f621c196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f620400c930 tx=0x7f620400ccf0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 shutdown_connections 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f620806af80 0x7f620806d440 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f621c0ffbd0 0x7f621c196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 --2- 172.21.15.137:0/1930093689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f621c105a40 0x7f621c196b80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 >> 172.21.15.137:0/1930093689 conn(0x7f621c0755a0 msgr2=0x7f621c068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 shutdown_connections 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.067+0000 7f6221996700 1 -- 172.21.15.137:0/1930093689 wait complete. 2024-06-15T01:24:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 -- 172.21.15.137:0/1260492807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc80ffbd0 msgr2=0x7f2cc8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 --2- 172.21.15.137:0/1260492807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc8100050 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009910 tx=0x7f2cbc009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 -- 172.21.15.137:0/1260492807 shutdown_connections 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 --2- 172.21.15.137:0/1260492807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc8100050 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 --2- 172.21.15.137:0/1260492807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8105a40 0x7f2cc8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 -- 172.21.15.137:0/1260492807 >> 172.21.15.137:0/1260492807 conn(0x7f2cc80755a0 msgr2=0x7f2cc80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:17.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 -- 172.21.15.137:0/1260492807 shutdown_connections 2024-06-15T01:24:17.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.172+0000 7f2ccde33700 1 -- 172.21.15.137:0/1260492807 wait complete. 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.173+0000 7f2ccde33700 1 Processor -- start 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.173+0000 7f2ccde33700 1 -- start start 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819d0e0 con 0x7f2cc8105a40 2024-06-15T01:24:17.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8196590 con 0x7f2cc80ffbd0 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34888/0 (socket says 172.21.15.137:34888) 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 -- 172.21.15.137:0/2772274412 learned_addr learned my addr 172.21.15.137:0/2772274412 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc77fe700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 -- 172.21.15.137:0/2772274412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc80ffbd0 msgr2=0x7f2cc819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 -- 172.21.15.137:0/2772274412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0095f0 con 0x7f2cc8105a40 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc6ffd700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006010 tx=0x7f2cbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc01d070 con 0x7f2cc8105a40 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cbc00f460 con 0x7f2cc8105a40 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc021620 con 0x7f2cc8105a40 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc8196810 con 0x7f2cc8105a40 2024-06-15T01:24:17.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.174+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc8196d00 con 0x7f2cc8105a40 2024-06-15T01:24:17.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.175+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8108fe0 con 0x7f2cc8105a40 2024-06-15T01:24:17.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.176+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2cbc00fa40 con 0x7f2cc8105a40 2024-06-15T01:24:17.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.176+0000 7f2cc4ff9700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 0x7f2cb006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.176+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2cbc08b5b0 con 0x7f2cc8105a40 2024-06-15T01:24:17.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.176+0000 7f2cc77fe700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 0x7f2cb006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.177+0000 7f2cc77fe700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 0x7f2cb006d440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005950 tx=0x7f2cb80058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:17.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.181+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cbc059e70 con 0x7f2cc8105a40 2024-06-15T01:24:17.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:17 smithi142 ceph-mon[82136]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:17 smithi142 ceph-mon[82136]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:17 smithi142 ceph-mon[82136]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:17 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/803422094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:24:17.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.392+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc8196f20 con 0x7f2cb006af80 2024-06-15T01:24:17.379 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:17 smithi137 ceph-mon[112295]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.379 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:17 smithi137 ceph-mon[112295]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.379 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:17 smithi137 ceph-mon[112295]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:17.379 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:17 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/803422094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:24:17.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.393+0000 7f2cc4ff9700 1 -- 172.21.15.137:0/2772274412 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2cc8196f20 con 0x7f2cb006af80 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:24:17.380 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:24:17.381 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:24:17.381 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 msgr2=0x7f2cb006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 0x7f2cb006d440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005950 tx=0x7f2cb80058e0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 msgr2=0x7f2cc819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006010 tx=0x7f2cbc00be20 comp rx=0 tx=0).stop 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 shutdown_connections 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cb006af80 0x7f2cb006d440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc80ffbd0 0x7f2cc819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 --2- 172.21.15.137:0/2772274412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cc8105a40 0x7f2cc819ca50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 >> 172.21.15.137:0/2772274412 conn(0x7f2cc80755a0 msgr2=0x7f2cc8103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 shutdown_connections 2024-06-15T01:24:17.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.396+0000 7f2ccde33700 1 -- 172.21.15.137:0/2772274412 wait complete. 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 -- 172.21.15.137:0/2450743232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2174100e10 msgr2=0x7f21741011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 --2- 172.21.15.137:0/2450743232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2174100e10 0x7f21741011f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2164009910 tx=0x7f2164009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 -- 172.21.15.137:0/2450743232 shutdown_connections 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 --2- 172.21.15.137:0/2450743232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21741017c0 0x7f2174105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 --2- 172.21.15.137:0/2450743232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2174100e10 0x7f21741011f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 -- 172.21.15.137:0/2450743232 >> 172.21.15.137:0/2450743232 conn(0x7f21740fc6c0 msgr2=0x7f21740feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 -- 172.21.15.137:0/2450743232 shutdown_connections 2024-06-15T01:24:17.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.502+0000 7f217978a700 1 -- 172.21.15.137:0/2450743232 wait complete. 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 Processor -- start 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 -- start start 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174100e10 0x7f2174073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174073b10 con 0x7f21741017c0 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f217978a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174073c50 con 0x7f2174100e10 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f21727fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f21727fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34898/0 (socket says 172.21.15.137:34898) 2024-06-15T01:24:17.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f21727fc700 1 -- 172.21.15.137:0/3414155941 learned_addr learned my addr 172.21.15.137:0/3414155941 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.503+0000 7f2172ffd700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174100e10 0x7f2174073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f21727fc700 1 -- 172.21.15.137:0/3414155941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174100e10 msgr2=0x7f2174073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f21727fc700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174100e10 0x7f2174073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f21727fc700 1 -- 172.21.15.137:0/3414155941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640095f0 con 0x7f21741017c0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f21727fc700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f216800e8f0 tx=0x7f216800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2168009790 con 0x7f21741017c0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f216800c810 con 0x7f21741017c0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21741a2900 con 0x7f21741017c0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2168005520 con 0x7f21741017c0 2024-06-15T01:24:17.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.504+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21741a2d10 con 0x7f21741017c0 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.505+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21741091b0 con 0x7f21741017c0 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.506+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f216800c980 con 0x7f21741017c0 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.506+0000 7f215bfff700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 0x7f215c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.506+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2168014070 con 0x7f21741017c0 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.506+0000 7f2172ffd700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 0x7f215c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:17.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.507+0000 7f2172ffd700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 0x7f215c06d440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f216400b6e0 tx=0x7f2164005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:17.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.511+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f21680554b0 con 0x7f21741017c0 2024-06-15T01:24:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.782+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f217404ea90 con 0x7f21741017c0 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.782+0000 7f215bfff700 1 -- 172.21.15.137:0/3414155941 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f216800cc30 con 0x7f21741017c0 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:24:17.769 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:24:17.770 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:24:17.770 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:24:17.770 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:24:17.770 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:24:17.770 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:24:17.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 msgr2=0x7f215c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 0x7f215c06d440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f216400b6e0 tx=0x7f2164005a90 comp rx=0 tx=0).stop 2024-06-15T01:24:17.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 msgr2=0x7f21740735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:17.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f216800e8f0 tx=0x7f216800ec00 comp rx=0 tx=0).stop 2024-06-15T01:24:17.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 shutdown_connections 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f215c06af80 0x7f215c06d440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174100e10 0x7f2174073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 --2- 172.21.15.137:0/3414155941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21741017c0 0x7f21740735d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 >> 172.21.15.137:0/3414155941 conn(0x7f21740fc6c0 msgr2=0x7f21740fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 shutdown_connections 2024-06-15T01:24:17.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:17.785+0000 7f217978a700 1 -- 172.21.15.137:0/3414155941 wait complete. 2024-06-15T01:24:18.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:18 smithi142 ceph-mon[82136]: pgmap v546: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:18.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:18 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1930093689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:24:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:18 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3414155941' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:24:18.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:18 smithi137 ceph-mon[112295]: pgmap v546: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:18 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1930093689' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:24:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:18 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3414155941' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:24:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:19.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:19 smithi142 ceph-mon[82136]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:19.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:19 smithi137 ceph-mon[112295]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:20.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:20 smithi142 ceph-mon[82136]: pgmap v547: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:20.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:20 smithi137 ceph-mon[112295]: pgmap v547: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:22.299 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:22 smithi137 ceph-mon[112295]: pgmap v548: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:22.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:22 smithi142 ceph-mon[82136]: pgmap v548: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:24.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:24 smithi142 ceph-mon[82136]: pgmap v549: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:24.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:24 smithi137 ceph-mon[112295]: pgmap v549: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:26.299 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:26 smithi142 ceph-mon[82136]: pgmap v550: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:26.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:26 smithi137 ceph-mon[112295]: pgmap v550: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:28.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:28 smithi142 ceph-mon[82136]: pgmap v551: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:28.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:28 smithi137 ceph-mon[112295]: pgmap v551: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:30.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:30 smithi142 ceph-mon[82136]: pgmap v552: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:30.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:30 smithi137 ceph-mon[112295]: pgmap v552: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:32 smithi142 ceph-mon[82136]: pgmap v553: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:32.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:32 smithi137 ceph-mon[112295]: pgmap v553: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:33.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:33.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:34.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:34 smithi137 ceph-mon[112295]: pgmap v554: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:34 smithi142 ceph-mon[82136]: pgmap v554: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:36.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:36 smithi142 ceph-mon[82136]: pgmap v555: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:36.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:36 smithi137 ceph-mon[112295]: pgmap v555: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:38.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:38 smithi137 ceph-mon[112295]: pgmap v556: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:38.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:38 smithi142 ceph-mon[82136]: pgmap v556: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:40.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:40 smithi137 ceph-mon[112295]: pgmap v557: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:40.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:40 smithi142 ceph-mon[82136]: pgmap v557: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:42.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:42 smithi137 ceph-mon[112295]: pgmap v558: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:42.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:42 smithi142 ceph-mon[82136]: pgmap v558: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:44.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:44 smithi137 ceph-mon[112295]: pgmap v559: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:44.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:44 smithi142 ceph-mon[82136]: pgmap v559: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:46.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:46 smithi137 ceph-mon[112295]: pgmap v560: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:46.480 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:46 smithi142 ceph-mon[82136]: pgmap v560: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/2234814484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 msgr2=0x7f6d080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/2234814484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d080ffed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009910 tx=0x7f6cfc009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/2234814484 shutdown_connections 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/2234814484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d080ffed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/2234814484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 0x7f6d08105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/2234814484 >> 172.21.15.137:0/2234814484 conn(0x7f6d080fb540 msgr2=0x7f6d080fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/2234814484 shutdown_connections 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/2234814484 wait complete. 2024-06-15T01:24:47.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.902+0000 7f6d0dfd2700 1 Processor -- start 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d0dfd2700 1 -- start start 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d0dfd2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d0dfd2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 0x7f6d081979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d0dfd2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08198090 con 0x7f6d080ffa50 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d0dfd2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d081981d0 con 0x7f6d08105a50 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51012/0 (socket says 172.21.15.137:51012) 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 -- 172.21.15.137:0/1541326940 learned_addr learned my addr 172.21.15.137:0/1541326940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d06ffd700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 0x7f6d081979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 -- 172.21.15.137:0/1541326940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 msgr2=0x7f6d081979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 0x7f6d081979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.903+0000 7f6d077fe700 1 -- 172.21.15.137:0/1541326940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc0095f0 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d077fe700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800ada0 tx=0x7f6cf800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf800f070 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d081a04e0 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cf8007bf0 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf8005630 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.904+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d081a0a00 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.905+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08107fe0 con 0x7f6d080ffa50 2024-06-15T01:24:47.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.905+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6cf80058a0 con 0x7f6d080ffa50 2024-06-15T01:24:47.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.906+0000 7f6d04ff9700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 0x7f6cf006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:47.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.906+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6cf8089da0 con 0x7f6d080ffa50 2024-06-15T01:24:47.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.906+0000 7f6d06ffd700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 0x7f6cf006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:47.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.906+0000 7f6d06ffd700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 0x7f6cf006d440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d08069ed0 tx=0x7f6cfc01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:47.910+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6cf8054c80 con 0x7f6d080ffa50 2024-06-15T01:24:48.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.121+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d0802cdd0 con 0x7f6cf006af80 2024-06-15T01:24:48.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.122+0000 7f6d04ff9700 1 -- 172.21.15.137:0/1541326940 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6d0802cdd0 con 0x7f6cf006af80 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.124+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 msgr2=0x7f6cf006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.124+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 0x7f6cf006d440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d08069ed0 tx=0x7f6cfc01a040 comp rx=0 tx=0).stop 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.124+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 msgr2=0x7f6d08197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.124+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800ada0 tx=0x7f6cf800c330 comp rx=0 tx=0).stop 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 shutdown_connections 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6cf006af80 0x7f6cf006d440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d080ffa50 0x7f6d08197470 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 --2- 172.21.15.137:0/1541326940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d08105a50 0x7f6d081979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 >> 172.21.15.137:0/1541326940 conn(0x7f6d080fb540 msgr2=0x7f6d080fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 shutdown_connections 2024-06-15T01:24:48.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.125+0000 7f6d0dfd2700 1 -- 172.21.15.137:0/1541326940 wait complete. 2024-06-15T01:24:48.121 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:24:48.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:48 smithi137 ceph-mon[112295]: pgmap v561: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:48.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:48.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- 172.21.15.137:0/767511782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec100dd0 msgr2=0x7f53ec1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 --2- 172.21.15.137:0/767511782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec100dd0 0x7f53ec1011b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009910 tx=0x7f53e0009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- 172.21.15.137:0/767511782 shutdown_connections 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 --2- 172.21.15.137:0/767511782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec101780 0x7f53ec1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 --2- 172.21.15.137:0/767511782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec100dd0 0x7f53ec1011b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- 172.21.15.137:0/767511782 >> 172.21.15.137:0/767511782 conn(0x7f53ec0fc640 msgr2=0x7f53ec0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- 172.21.15.137:0/767511782 shutdown_connections 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- 172.21.15.137:0/767511782 wait complete. 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 Processor -- start 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.229+0000 7f53f2d19700 1 -- start start 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53f2d19700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec100dd0 0x7f53ec19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53f2d19700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53f2d19700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec19d0c0 con 0x7f53ec101780 2024-06-15T01:24:48.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53f2d19700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec196570 con 0x7f53ec100dd0 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51034/0 (socket says 172.21.15.137:51034) 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 -- 172.21.15.137:0/2131408173 learned_addr learned my addr 172.21.15.137:0/2131408173 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53f0ab5700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec100dd0 0x7f53ec19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 -- 172.21.15.137:0/2131408173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec100dd0 msgr2=0x7f53ec19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec100dd0 0x7f53ec19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 -- 172.21.15.137:0/2131408173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00095f0 con 0x7f53ec101780 2024-06-15T01:24:48.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.230+0000 7f53ebfff700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c8f0 tx=0x7f53d800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.231+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d80078a0 con 0x7f53ec101780 2024-06-15T01:24:48.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.231+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec196850 con 0x7f53ec101780 2024-06-15T01:24:48.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.231+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f53d800f450 con 0x7f53ec101780 2024-06-15T01:24:48.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.231+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec196da0 con 0x7f53ec101780 2024-06-15T01:24:48.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.231+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d8007ea0 con 0x7f53ec101780 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.232+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec109170 con 0x7f53ec101780 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.232+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f53d800f5c0 con 0x7f53ec101780 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.233+0000 7f53e9ffb700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 0x7f53dc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.233+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f53d808a210 con 0x7f53ec101780 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.233+0000 7f53f0ab5700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 0x7f53dc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.233+0000 7f53f0ab5700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 0x7f53dc06d440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006010 tx=0x7f53e001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.238+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f53d8058b50 con 0x7f53ec101780 2024-06-15T01:24:48.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.450+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53ec197070 con 0x7f53dc06af80 2024-06-15T01:24:48.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.451+0000 7f53e9ffb700 1 -- 172.21.15.137:0/2131408173 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f53ec197070 con 0x7f53dc06af80 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 msgr2=0x7f53dc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 0x7f53dc06d440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006010 tx=0x7f53e001a040 comp rx=0 tx=0).stop 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 msgr2=0x7f53ec19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c8f0 tx=0x7f53d800cc00 comp rx=0 tx=0).stop 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 shutdown_connections 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f53dc06af80 0x7f53dc06d440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53ec100dd0 0x7f53ec19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 --2- 172.21.15.137:0/2131408173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ec101780 0x7f53ec19ca30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.453+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 >> 172.21.15.137:0/2131408173 conn(0x7f53ec0fc640 msgr2=0x7f53ec0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.454+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 shutdown_connections 2024-06-15T01:24:48.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.454+0000 7f53f2d19700 1 -- 172.21.15.137:0/2131408173 wait complete. 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 -- 172.21.15.137:0/2150946810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118100de0 msgr2=0x7f21181011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 --2- 172.21.15.137:0/2150946810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118100de0 0x7f21181011c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f210c009910 tx=0x7f210c009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 -- 172.21.15.137:0/2150946810 shutdown_connections 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 --2- 172.21.15.137:0/2150946810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118101790 0x7f21181057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 --2- 172.21.15.137:0/2150946810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118100de0 0x7f21181011c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 -- 172.21.15.137:0/2150946810 >> 172.21.15.137:0/2150946810 conn(0x7f21180fc670 msgr2=0x7f21180fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 -- 172.21.15.137:0/2150946810 shutdown_connections 2024-06-15T01:24:48.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.557+0000 7f211e409700 1 -- 172.21.15.137:0/2150946810 wait complete. 2024-06-15T01:24:48.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 Processor -- start 2024-06-15T01:24:48.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 -- start start 2024-06-15T01:24:48.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118100de0 0x7f2118196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118197290 con 0x7f2118101790 2024-06-15T01:24:48.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f211e409700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211819b020 con 0x7f2118100de0 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f21177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f21177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51064/0 (socket says 172.21.15.137:51064) 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f21177fe700 1 -- 172.21.15.137:0/1848547462 learned_addr learned my addr 172.21.15.137:0/1848547462 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.558+0000 7f2117fff700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118100de0 0x7f2118196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21177fe700 1 -- 172.21.15.137:0/1848547462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118100de0 msgr2=0x7f2118196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21177fe700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118100de0 0x7f2118196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21177fe700 1 -- 172.21.15.137:0/1848547462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0095f0 con 0x7f2118101790 2024-06-15T01:24:48.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21177fe700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f210800c8f0 tx=0x7f210800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21080078a0 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f210800f450 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211819b300 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2108007ea0 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.559+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211819b850 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.560+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2118109180 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.561+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f210800f5c0 con 0x7f2118101790 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.561+0000 7f21157fa700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 0x7f210006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.561+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f210808a330 con 0x7f2118101790 2024-06-15T01:24:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.561+0000 7f2117fff700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 0x7f210006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.562+0000 7f2117fff700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 0x7f210006d440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.566+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f21080550c0 con 0x7f2118101790 2024-06-15T01:24:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:48 smithi142 ceph-mon[82136]: pgmap v561: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:24:48.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.766+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f211819bbc0 con 0x7f210006af80 2024-06-15T01:24:48.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.775+0000 7f21157fa700 1 -- 172.21.15.137:0/1848547462 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f211819bbc0 con 0x7f210006af80 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (15m) 9m ago 25m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (26m) 9m ago 26m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (25m) 9m ago 25m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (13m) 9m ago 26m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (13m) 9m ago 25m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (15m) 9m ago 25m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (9m) 9m ago 20m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (10m) 9m ago 20m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:24:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (9m) 9m ago 20m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (9m) 9m ago 20m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (17m) 9m ago 27m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (16m) 9m ago 24m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (14m) 9m ago 27m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (13m) 9m ago 24m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (15m) 9m ago 26m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (15m) 9m ago 24m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (13m) 9m ago 24m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (12m) 9m ago 23m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (12m) 9m ago 23m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (11m) 9m ago 22m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:24:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (10m) 9m ago 22m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:24:48.764 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (10m) 9m ago 21m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:24:48.764 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (15m) 9m ago 25m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:24:48.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.777+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 msgr2=0x7f210006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.777+0000 7f211e409700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 0x7f210006d440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c01a040 comp rx=0 tx=0).stop 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.777+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 msgr2=0x7f2118196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.777+0000 7f211e409700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f210800c8f0 tx=0x7f210800cc00 comp rx=0 tx=0).stop 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 shutdown_connections 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f210006af80 0x7f210006d440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2118100de0 0x7f2118196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 --2- 172.21.15.137:0/1848547462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2118101790 0x7f2118196bb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 >> 172.21.15.137:0/1848547462 conn(0x7f21180fc670 msgr2=0x7f21180fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 shutdown_connections 2024-06-15T01:24:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.778+0000 7f211e409700 1 -- 172.21.15.137:0/1848547462 wait complete. 2024-06-15T01:24:48.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 -- 172.21.15.137:0/3032621874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500101770 msgr2=0x7fc5001057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 --2- 172.21.15.137:0/3032621874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500101770 0x7fc5001057c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009910 tx=0x7fc4e8009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 -- 172.21.15.137:0/3032621874 shutdown_connections 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 --2- 172.21.15.137:0/3032621874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500101770 0x7fc5001057c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 --2- 172.21.15.137:0/3032621874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500100dc0 0x7fc5001011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 -- 172.21.15.137:0/3032621874 >> 172.21.15.137:0/3032621874 conn(0x7fc5000fc650 msgr2=0x7fc5000fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 -- 172.21.15.137:0/3032621874 shutdown_connections 2024-06-15T01:24:48.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.881+0000 7fc504a90700 1 -- 172.21.15.137:0/3032621874 wait complete. 2024-06-15T01:24:48.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 Processor -- start 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 -- start start 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500101770 0x7fc5000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500073b10 con 0x7fc500100dc0 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc504a90700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500073c50 con 0x7fc500101770 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51082/0 (socket says 172.21.15.137:51082) 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 -- 172.21.15.137:0/2571235589 learned_addr learned my addr 172.21.15.137:0/2571235589 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fdd9b700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500101770 0x7fc5000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 -- 172.21.15.137:0/2571235589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500101770 msgr2=0x7fc5000735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500101770 0x7fc5000735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.882+0000 7fc4fe59c700 1 -- 172.21.15.137:0/2571235589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80095f0 con 0x7fc500100dc0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc4fe59c700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800d8d0 tx=0x7fc4f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f8009940 con 0x7fc500100dc0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5001a28a0 con 0x7fc500100dc0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4f8010460 con 0x7fc500100dc0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f800f5f0 con 0x7fc500100dc0 2024-06-15T01:24:48.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.883+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5001a2cb0 con 0x7fc500100dc0 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.884+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5001091c0 con 0x7fc500100dc0 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.885+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc4f8009aa0 con 0x7fc500100dc0 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.885+0000 7fc4f77fe700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 0x7fc4ec06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.885+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc4f808a390 con 0x7fc500100dc0 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.885+0000 7fc4fdd9b700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 0x7fc4ec06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.886+0000 7fc4fdd9b700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 0x7fc4ec06d440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc500074880 tx=0x7fc4e801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:48.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:48.890+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc4f80551c0 con 0x7fc500100dc0 2024-06-15T01:24:49.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.162+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc50004ea90 con 0x7fc500100dc0 2024-06-15T01:24:49.149 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:49 smithi137 ceph-mon[112295]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:49.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.162+0000 7fc4f77fe700 1 -- 172.21.15.137:0/2571235589 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc4f80587e0 con 0x7fc500100dc0 2024-06-15T01:24:49.149 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:24:49.149 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:24:49.149 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:24:49.150 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 msgr2=0x7fc4ec06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 0x7fc4ec06d440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc500074880 tx=0x7fc4e801a040 comp rx=0 tx=0).stop 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 msgr2=0x7fc500073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800d8d0 tx=0x7fc4f800dc90 comp rx=0 tx=0).stop 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 shutdown_connections 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc4ec06af80 0x7fc4ec06d440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc500100dc0 0x7fc500073090 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 --2- 172.21.15.137:0/2571235589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc500101770 0x7fc5000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 >> 172.21.15.137:0/2571235589 conn(0x7fc5000fc650 msgr2=0x7fc5000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 shutdown_connections 2024-06-15T01:24:49.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.165+0000 7fc504a90700 1 -- 172.21.15.137:0/2571235589 wait complete. 2024-06-15T01:24:49.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3362425862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c80ffa40 msgr2=0x7fb3c80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3362425862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c80ffa40 0x7fb3c80ffec0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009910 tx=0x7fb3b8009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3362425862 shutdown_connections 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3362425862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c80ffa40 0x7fb3c80ffec0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3362425862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c8105a40 0x7fb3c8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3362425862 >> 172.21.15.137:0/3362425862 conn(0x7fb3c80fb550 msgr2=0x7fb3c80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3362425862 shutdown_connections 2024-06-15T01:24:49.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.269+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3362425862 wait complete. 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 Processor -- start 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 -- start start 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 0x7fb3c8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8198260 con 0x7fb3c8105a40 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3cd2f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81983a0 con 0x7fb3c80ffa40 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3c67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3c67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51088/0 (socket says 172.21.15.137:51088) 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.270+0000 7fb3c67fc700 1 -- 172.21.15.137:0/3273162529 learned_addr learned my addr 172.21.15.137:0/3273162529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c6ffd700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 0x7fb3c8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c67fc700 1 -- 172.21.15.137:0/3273162529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 msgr2=0x7fb3c8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c67fc700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 0x7fb3c8197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c67fc700 1 -- 172.21.15.137:0/3273162529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80095f0 con 0x7fb3c8105a40 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c6ffd700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 0x7fb3c8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3c67fc700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8006010 tx=0x7fb3b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b801d070 con 0x7fb3c8105a40 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c81a06b0 con 0x7fb3c8105a40 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3b800f460 con 0x7fb3c8105a40 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b8021640 con 0x7fb3c8105a40 2024-06-15T01:24:49.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.271+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c81a0b20 con 0x7fb3c8105a40 2024-06-15T01:24:49.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.272+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c8108060 con 0x7fb3c8105a40 2024-06-15T01:24:49.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.272+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb3b800fa60 con 0x7fb3c8105a40 2024-06-15T01:24:49.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.273+0000 7fb3affff700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 0x7fb3b006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.273+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb3b808b5f0 con 0x7fb3c8105a40 2024-06-15T01:24:49.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.273+0000 7fb3c6ffd700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 0x7fb3b006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.273+0000 7fb3c6ffd700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 0x7fb3b006d440 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0099d0 tx=0x7fb3bc008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.277+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb3b80564d0 con 0x7fb3c8105a40 2024-06-15T01:24:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.513+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3c802cc70 con 0x7fb3c8105a40 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.513+0000 7fb3affff700 1 -- 172.21.15.137:0/3273162529 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fb3b8059af0 con 0x7fb3c8105a40 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:24:49.500 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:24:49.501 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:24:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:49.503 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 msgr2=0x7fb3b006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 0x7fb3b006d440 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0099d0 tx=0x7fb3bc008040 comp rx=0 tx=0).stop 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 msgr2=0x7fb3c8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8006010 tx=0x7fb3b800be20 comp rx=0 tx=0).stop 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 shutdown_connections 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb3b006af80 0x7fb3b006d440 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c80ffa40 0x7fb3c8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 --2- 172.21.15.137:0/3273162529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c8105a40 0x7fb3c8197b80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 >> 172.21.15.137:0/3273162529 conn(0x7fb3c80fb550 msgr2=0x7fb3c80fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 shutdown_connections 2024-06-15T01:24:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.516+0000 7fb3cd2f5700 1 -- 172.21.15.137:0/3273162529 wait complete. 2024-06-15T01:24:49.505 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:24:49.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:49 smithi142 ceph-mon[82136]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:49.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 -- 172.21.15.137:0/3694810205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a00ffa70 msgr2=0x7f35a00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 --2- 172.21.15.137:0/3694810205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a00ffa70 0x7f35a00ffef0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3590009910 tx=0x7f3590009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 -- 172.21.15.137:0/3694810205 shutdown_connections 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 --2- 172.21.15.137:0/3694810205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a00ffa70 0x7f35a00ffef0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 --2- 172.21.15.137:0/3694810205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a0105a70 0x7f35a0105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.620+0000 7f35a816d700 1 -- 172.21.15.137:0/3694810205 >> 172.21.15.137:0/3694810205 conn(0x7f35a00fb560 msgr2=0x7f35a00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 -- 172.21.15.137:0/3694810205 shutdown_connections 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 -- 172.21.15.137:0/3694810205 wait complete. 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 Processor -- start 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 -- start start 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a00ffa70 0x7f35a019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a019d110 con 0x7f35a0105a70 2024-06-15T01:24:49.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.621+0000 7f35a816d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a01965c0 con 0x7f35a00ffa70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51094/0 (socket says 172.21.15.137:51094) 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 -- 172.21.15.137:0/2382170362 learned_addr learned my addr 172.21.15.137:0/2382170362 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5f09700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a00ffa70 0x7f35a019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 -- 172.21.15.137:0/2382170362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a00ffa70 msgr2=0x7f35a019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a00ffa70 0x7f35a019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 -- 172.21.15.137:0/2382170362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900095f0 con 0x7f35a0105a70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a5708700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3590006010 tx=0x7f359000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f359001d070 con 0x7f35a0105a70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f359000f460 con 0x7f35a0105a70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3590021640 con 0x7f35a0105a70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a0196840 con 0x7f35a0105a70 2024-06-15T01:24:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.622+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a0196d30 con 0x7f35a0105a70 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.623+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a01090a0 con 0x7f35a0105a70 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.624+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f359000fa20 con 0x7f35a0105a70 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.624+0000 7f3596ffd700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 0x7f358c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.624+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f359008b670 con 0x7f35a0105a70 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.624+0000 7f35a5f09700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 0x7f358c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.625+0000 7f35a5f09700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 0x7f358c06d440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.629+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3590059f30 con 0x7f35a0105a70 2024-06-15T01:24:49.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.845+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35a002cca0 con 0x7f358c06af80 2024-06-15T01:24:49.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.846+0000 7f3596ffd700 1 -- 172.21.15.137:0/2382170362 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35a002cca0 con 0x7f358c06af80 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:24:49.833 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:24:49.834 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:24:49.834 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 msgr2=0x7f358c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 0x7f358c06d440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c008040 comp rx=0 tx=0).stop 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 msgr2=0x7f35a019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3590006010 tx=0x7f359000be20 comp rx=0 tx=0).stop 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 shutdown_connections 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f358c06af80 0x7f358c06d440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35a00ffa70 0x7f35a019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 --2- 172.21.15.137:0/2382170362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35a0105a70 0x7f35a019ca80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.848+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 >> 172.21.15.137:0/2382170362 conn(0x7f35a00fb560 msgr2=0x7f35a00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.849+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 shutdown_connections 2024-06-15T01:24:49.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.849+0000 7f35a816d700 1 -- 172.21.15.137:0/2382170362 wait complete. 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- 172.21.15.137:0/2229732458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ffee0 msgr2=0x7fe454105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 --2- 172.21.15.137:0/2229732458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ffee0 0x7fe454105870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe448009910 tx=0x7fe448009c20 comp rx=0 tx=0).stop 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- 172.21.15.137:0/2229732458 shutdown_connections 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 --2- 172.21.15.137:0/2229732458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ffee0 0x7fe454105870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 --2- 172.21.15.137:0/2229732458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4540ff530 0x7fe4540ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- 172.21.15.137:0/2229732458 >> 172.21.15.137:0/2229732458 conn(0x7fe454074df0 msgr2=0x7fe454075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:49.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- 172.21.15.137:0/2229732458 shutdown_connections 2024-06-15T01:24:49.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- 172.21.15.137:0/2229732458 wait complete. 2024-06-15T01:24:49.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 Processor -- start 2024-06-15T01:24:49.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.952+0000 7fe45a753700 1 -- start start 2024-06-15T01:24:49.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe45a753700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe45a753700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4540ffee0 0x7fe45419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe45a753700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45419d0c0 con 0x7fe4540ff530 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe45a753700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454196570 con 0x7fe4540ffee0 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51106/0 (socket says 172.21.15.137:51106) 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 -- 172.21.15.137:0/2151206453 learned_addr learned my addr 172.21.15.137:0/2151206453 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 -- 172.21.15.137:0/2151206453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4540ffee0 msgr2=0x7fe45419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4540ffee0 0x7fe45419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 -- 172.21.15.137:0/2151206453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480095f0 con 0x7fe4540ff530 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe453fff700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d8d0 tx=0x7fe44400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.953+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe444009940 con 0x7fe4540ff530 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.954+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe454196850 con 0x7fe4540ff530 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.954+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe444010460 con 0x7fe4540ff530 2024-06-15T01:24:49.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.954+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe44400f5f0 con 0x7fe4540ff530 2024-06-15T01:24:49.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.954+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe454196da0 con 0x7fe4540ff530 2024-06-15T01:24:49.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.955+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe454109150 con 0x7fe4540ff530 2024-06-15T01:24:49.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.955+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe4440105d0 con 0x7fe4540ff530 2024-06-15T01:24:49.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.955+0000 7fe4517fa700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 0x7fe43c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:24:49.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.956+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe44408a3e0 con 0x7fe4540ff530 2024-06-15T01:24:49.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.956+0000 7fe4537fe700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 0x7fe43c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:24:49.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.956+0000 7fe4537fe700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 0x7fe43c06d440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b5c0 tx=0x7fe448005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:24:49.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:49.960+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe444058ca0 con 0x7fe4540ff530 2024-06-15T01:24:50.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.233+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe454197080 con 0x7fe4540ff530 2024-06-15T01:24:50.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.233+0000 7fe4517fa700 1 -- 172.21.15.137:0/2151206453 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fe444010880 con 0x7fe4540ff530 2024-06-15T01:24:50.219 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:24:50.219 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:24:50.220 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:24:50.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 msgr2=0x7fe43c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:50.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 0x7fe43c06d440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b5c0 tx=0x7fe448005960 comp rx=0 tx=0).stop 2024-06-15T01:24:50.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 msgr2=0x7fe45419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:24:50.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d8d0 tx=0x7fe44400dc90 comp rx=0 tx=0).stop 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 shutdown_connections 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe43c06af80 0x7fe43c06d440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4540ff530 0x7fe45419c4f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 --2- 172.21.15.137:0/2151206453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4540ffee0 0x7fe45419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 >> 172.21.15.137:0/2151206453 conn(0x7fe454074df0 msgr2=0x7fe4541050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 shutdown_connections 2024-06-15T01:24:50.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:24:50.235+0000 7fe45a753700 1 -- 172.21.15.137:0/2151206453 wait complete. 2024-06-15T01:24:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:50 smithi137 ceph-mon[112295]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:50 smithi137 ceph-mon[112295]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:50 smithi137 ceph-mon[112295]: pgmap v562: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:50 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2571235589' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:24:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:50 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3273162529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:24:50.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:50 smithi142 ceph-mon[82136]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:50.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:50 smithi142 ceph-mon[82136]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:50.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:50 smithi142 ceph-mon[82136]: pgmap v562: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:50.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:50 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2571235589' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:24:50.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:50 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3273162529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:24:51.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:51 smithi137 ceph-mon[112295]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:51.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2151206453' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:24:51.525 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:51 smithi142 ceph-mon[82136]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:24:51.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2151206453' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:24:52.366 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:52 smithi137 ceph-mon[112295]: pgmap v563: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:52.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:52 smithi142 ceph-mon[82136]: pgmap v563: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:54.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:54 smithi137 ceph-mon[112295]: pgmap v564: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:54.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:54 smithi142 ceph-mon[82136]: pgmap v564: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:56.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:56 smithi137 ceph-mon[112295]: pgmap v565: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:56.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:56 smithi142 ceph-mon[82136]: pgmap v565: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:58.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:24:58 smithi137 ceph-mon[112295]: pgmap v566: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:24:58.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:24:58 smithi142 ceph-mon[82136]: pgmap v566: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:00.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:00 smithi137 ceph-mon[112295]: pgmap v567: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:00.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:00 smithi142 ceph-mon[82136]: pgmap v567: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:02.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:02 smithi137 ceph-mon[112295]: pgmap v568: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:02.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:02 smithi142 ceph-mon[82136]: pgmap v568: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:03.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:03.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:04.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:04 smithi137 ceph-mon[112295]: pgmap v569: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:04.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:04 smithi142 ceph-mon[82136]: pgmap v569: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:06.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:06 smithi137 ceph-mon[112295]: pgmap v570: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:06.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:06 smithi142 ceph-mon[82136]: pgmap v570: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:08.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:08 smithi137 ceph-mon[112295]: pgmap v571: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:08.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:08 smithi142 ceph-mon[82136]: pgmap v571: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:10.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:10 smithi137 ceph-mon[112295]: pgmap v572: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:10.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:10 smithi142 ceph-mon[82136]: pgmap v572: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:12.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:12 smithi137 ceph-mon[112295]: pgmap v573: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:12.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:12 smithi142 ceph-mon[82136]: pgmap v573: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:14.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:14 smithi137 ceph-mon[112295]: pgmap v574: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:14.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:14 smithi142 ceph-mon[82136]: pgmap v574: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:16.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:16 smithi137 ceph-mon[112295]: pgmap v575: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:16.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:16 smithi142 ceph-mon[82136]: pgmap v575: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:18 smithi137 ceph-mon[112295]: pgmap v576: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:18.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:18.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:18 smithi142 ceph-mon[82136]: pgmap v576: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:18.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:20.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 -- 172.21.15.137:0/1004630759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830101750 msgr2=0x7f28301057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 --2- 172.21.15.137:0/1004630759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830101750 0x7f28301057a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2818009910 tx=0x7f2818009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:20.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 -- 172.21.15.137:0/1004630759 shutdown_connections 2024-06-15T01:25:20.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 --2- 172.21.15.137:0/1004630759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830101750 0x7f28301057a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 --2- 172.21.15.137:0/1004630759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830100da0 0x7f2830101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 -- 172.21.15.137:0/1004630759 >> 172.21.15.137:0/1004630759 conn(0x7f28300fc630 msgr2=0x7f28300fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:20.340 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:20 smithi137 ceph-mon[112295]: pgmap v577: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:20.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 -- 172.21.15.137:0/1004630759 shutdown_connections 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.353+0000 7f2834da4700 1 -- 172.21.15.137:0/1004630759 wait complete. 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 Processor -- start 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 -- start start 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830101750 0x7f28300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830073b10 con 0x7f2830100da0 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f2834da4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830073c50 con 0x7f2830101750 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f282e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f282e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54256/0 (socket says 172.21.15.137:54256) 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.354+0000 7f282e59c700 1 -- 172.21.15.137:0/3881799710 learned_addr learned my addr 172.21.15.137:0/3881799710 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2825d9b700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830101750 0x7f28300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f282e59c700 1 -- 172.21.15.137:0/3881799710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830101750 msgr2=0x7f28300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f282e59c700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830101750 0x7f28300735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f282e59c700 1 -- 172.21.15.137:0/3881799710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28180095f0 con 0x7f2830100da0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f282e59c700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f282000c930 tx=0x7f282000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2820007ad0 con 0x7f2830100da0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28301a2820 con 0x7f2830100da0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2820007c30 con 0x7f2830100da0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2820018630 con 0x7f2830100da0 2024-06-15T01:25:20.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.355+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28301a2b80 con 0x7f2830100da0 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.356+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28301091a0 con 0x7f2830100da0 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.357+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f282000f450 con 0x7f2830100da0 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.357+0000 7f2827fff700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 0x7f281c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.357+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f282008a5b0 con 0x7f2830100da0 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.357+0000 7f2825d9b700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 0x7f281c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.358+0000 7f2825d9b700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 0x7f281c06d440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2830074880 tx=0x7f281801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.362+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f28200553e0 con 0x7f2830100da0 2024-06-15T01:25:20.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.568+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f283002cd60 con 0x7f281c06af80 2024-06-15T01:25:20.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.569+0000 7f2827fff700 1 -- 172.21.15.137:0/3881799710 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f283002cd60 con 0x7f281c06af80 2024-06-15T01:25:20.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:20 smithi142 ceph-mon[82136]: pgmap v577: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.571+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 msgr2=0x7f281c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.571+0000 7f2834da4700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 0x7f281c06d440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2830074880 tx=0x7f281801a040 comp rx=0 tx=0).stop 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.571+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 msgr2=0x7f2830073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.571+0000 7f2834da4700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f282000c930 tx=0x7f282000ccf0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 shutdown_connections 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f281c06af80 0x7f281c06d440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2830100da0 0x7f2830073090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 --2- 172.21.15.137:0/3881799710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2830101750 0x7f28300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 >> 172.21.15.137:0/3881799710 conn(0x7f28300fc630 msgr2=0x7f28300fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:20.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 shutdown_connections 2024-06-15T01:25:20.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.572+0000 7f2834da4700 1 -- 172.21.15.137:0/3881799710 wait complete. 2024-06-15T01:25:20.569 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:25:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 -- 172.21.15.137:0/3182147402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 msgr2=0x7efd74102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 --2- 172.21.15.137:0/3182147402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd74102060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b0a0 tx=0x7efd6400b3b0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 -- 172.21.15.137:0/3182147402 shutdown_connections 2024-06-15T01:25:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 --2- 172.21.15.137:0/3182147402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7410ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 --2- 172.21.15.137:0/3182147402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd74102060 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 -- 172.21.15.137:0/3182147402 >> 172.21.15.137:0/3182147402 conn(0x7efd740fb540 msgr2=0x7efd740fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 -- 172.21.15.137:0/3182147402 shutdown_connections 2024-06-15T01:25:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 -- 172.21.15.137:0/3182147402 wait complete. 2024-06-15T01:25:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.677+0000 7efd79cd9700 1 Processor -- start 2024-06-15T01:25:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd79cd9700 1 -- start start 2024-06-15T01:25:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd79cd9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd79cd9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd79cd9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7419d0e0 con 0x7efd74101c80 2024-06-15T01:25:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd79cd9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd74196590 con 0x7efd74102630 2024-06-15T01:25:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54260/0 (socket says 172.21.15.137:54260) 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 -- 172.21.15.137:0/4177144822 learned_addr learned my addr 172.21.15.137:0/4177144822 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd72ffd700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 -- 172.21.15.137:0/4177144822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 msgr2=0x7efd7419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 -- 172.21.15.137:0/4177144822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64009d00 con 0x7efd74101c80 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd72ffd700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.678+0000 7efd737fe700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efd64015040 tx=0x7efd64007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.679+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6400e040 con 0x7efd74101c80 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.679+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd640044e0 con 0x7efd74101c80 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.679+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd74196810 con 0x7efd74101c80 2024-06-15T01:25:20.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.679+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6401d9e0 con 0x7efd74101c80 2024-06-15T01:25:20.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.679+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd74196d00 con 0x7efd74101c80 2024-06-15T01:25:20.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.680+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd74108220 con 0x7efd74101c80 2024-06-15T01:25:20.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.680+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efd64004650 con 0x7efd74101c80 2024-06-15T01:25:20.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.681+0000 7efd70ff9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 0x7efd5c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.681+0000 7efd72ffd700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 0x7efd5c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.681+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efd6408bde0 con 0x7efd74101c80 2024-06-15T01:25:20.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.681+0000 7efd72ffd700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 0x7efd5c06d440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd740fdf40 tx=0x7efd68008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.685+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efd6405a6a0 con 0x7efd74101c80 2024-06-15T01:25:20.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.893+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd7402ccc0 con 0x7efd5c06af80 2024-06-15T01:25:20.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.894+0000 7efd70ff9700 1 -- 172.21.15.137:0/4177144822 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efd7402ccc0 con 0x7efd5c06af80 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 msgr2=0x7efd5c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 0x7efd5c06d440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd740fdf40 tx=0x7efd68008040 comp rx=0 tx=0).stop 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 msgr2=0x7efd7419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efd64015040 tx=0x7efd64007be0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 shutdown_connections 2024-06-15T01:25:20.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efd5c06af80 0x7efd5c06d440 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd74101c80 0x7efd7419c510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 --2- 172.21.15.137:0/4177144822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd74102630 0x7efd7419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.896+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 >> 172.21.15.137:0/4177144822 conn(0x7efd740fb540 msgr2=0x7efd74105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:20.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.897+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 shutdown_connections 2024-06-15T01:25:20.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:20.897+0000 7efd79cd9700 1 -- 172.21.15.137:0/4177144822 wait complete. 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 -- 172.21.15.137:0/2725379813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ff510 msgr2=0x7fe8100ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 --2- 172.21.15.137:0/2725379813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ff510 0x7fe8100ff8f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe80000aa20 tx=0x7fe80000ad30 comp rx=0 tx=0).stop 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 -- 172.21.15.137:0/2725379813 shutdown_connections 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 --2- 172.21.15.137:0/2725379813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ffec0 0x7fe810103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 --2- 172.21.15.137:0/2725379813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ff510 0x7fe8100ff8f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 -- 172.21.15.137:0/2725379813 >> 172.21.15.137:0/2725379813 conn(0x7fe810074d80 msgr2=0x7fe810075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 -- 172.21.15.137:0/2725379813 shutdown_connections 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.000+0000 7fe818418700 1 -- 172.21.15.137:0/2725379813 wait complete. 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 Processor -- start 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 -- start start 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 0x7fe81019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019d100 con 0x7fe8100ffec0 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe818418700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8101965b0 con 0x7fe8100ff510 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe8159b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe8159b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54282/0 (socket says 172.21.15.137:54282) 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe8159b3700 1 -- 172.21.15.137:0/273224134 learned_addr learned my addr 172.21.15.137:0/273224134 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.001+0000 7fe8161b4700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 0x7fe81019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8159b3700 1 -- 172.21.15.137:0/273224134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 msgr2=0x7fe81019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8159b3700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 0x7fe81019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8159b3700 1 -- 172.21.15.137:0/273224134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80000a700 con 0x7fe8100ffec0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8161b4700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 0x7fe81019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8159b3700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b480 tx=0x7fe80c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80c0118c0 con 0x7fe8100ffec0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe80c010380 con 0x7fe8100ffec0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810196890 con 0x7fe8100ffec0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80c00f510 con 0x7fe8100ffec0 2024-06-15T01:25:20.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.002+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe810196de0 con 0x7fe8100ffec0 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.003+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8101097e0 con 0x7fe8100ffec0 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.004+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe80c0104f0 con 0x7fe8100ffec0 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.004+0000 7fe8077fe700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 0x7fe7fc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.004+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe80c089fc0 con 0x7fe8100ffec0 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.004+0000 7fe8161b4700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 0x7fe7fc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:20.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.004+0000 7fe8161b4700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 0x7fe7fc06d440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c6d0 tx=0x7fe800005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:20.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.008+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe80c058880 con 0x7fe8100ffec0 2024-06-15T01:25:21.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.210+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe810197100 con 0x7fe7fc06af80 2024-06-15T01:25:21.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.219+0000 7fe8077fe700 1 -- 172.21.15.137:0/273224134 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe810197100 con 0x7fe7fc06af80 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (15m) 10m ago 26m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (26m) 10m ago 26m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (25m) 9m ago 25m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (14m) 10m ago 26m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (14m) 9m ago 25m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (15m) 10m ago 26m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (10m) 10m ago 20m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (10m) 10m ago 20m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (10m) 9m ago 20m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (9m) 9m ago 20m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (17m) 10m ago 28m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:25:21.206 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (16m) 9m ago 25m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (14m) 10m ago 28m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (14m) 9m ago 25m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (16m) 10m ago 26m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (16m) 9m ago 25m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (13m) 10m ago 24m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (13m) 10m ago 24m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (12m) 10m ago 23m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (12m) 9m ago 23m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (11m) 9m ago 22m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (10m) 9m ago 22m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:25:21.207 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (16m) 10m ago 25m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 msgr2=0x7fe7fc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 0x7fe7fc06d440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c6d0 tx=0x7fe800005960 comp rx=0 tx=0).stop 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 msgr2=0x7fe81019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b480 tx=0x7fe80c00b840 comp rx=0 tx=0).stop 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 shutdown_connections 2024-06-15T01:25:21.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe7fc06af80 0x7fe7fc06d440 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8100ff510 0x7fe81019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 --2- 172.21.15.137:0/273224134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8100ffec0 0x7fe81019ca70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.221+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 >> 172.21.15.137:0/273224134 conn(0x7fe810074d80 msgr2=0x7fe8100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:21.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.222+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 shutdown_connections 2024-06-15T01:25:21.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.222+0000 7fe818418700 1 -- 172.21.15.137:0/273224134 wait complete. 2024-06-15T01:25:21.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 -- 172.21.15.137:0/2081185360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 msgr2=0x7f41300735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 --2- 172.21.15.137:0/2081185360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f41300735a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f412c009910 tx=0x7f412c009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 -- 172.21.15.137:0/2081185360 shutdown_connections 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 --2- 172.21.15.137:0/2081185360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 0x7f413010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 --2- 172.21.15.137:0/2081185360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f41300735a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 -- 172.21.15.137:0/2081185360 >> 172.21.15.137:0/2081185360 conn(0x7f41300fc650 msgr2=0x7f41300fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 -- 172.21.15.137:0/2081185360 shutdown_connections 2024-06-15T01:25:21.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.324+0000 7f413805e700 1 -- 172.21.15.137:0/2081185360 wait complete. 2024-06-15T01:25:21.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 Processor -- start 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 -- start start 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 0x7f413019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019f290 con 0x7f41300731c0 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f413805e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130198740 con 0x7f4130073ae0 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f4135dfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f4135dfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54286/0 (socket says 172.21.15.137:54286) 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.325+0000 7f4135dfa700 1 -- 172.21.15.137:0/3677264255 learned_addr learned my addr 172.21.15.137:0/3677264255 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f41355f9700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 0x7f413019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4135dfa700 1 -- 172.21.15.137:0/3677264255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 msgr2=0x7f413019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4135dfa700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 0x7f413019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4135dfa700 1 -- 172.21.15.137:0/3677264255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0095f0 con 0x7f41300731c0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4135dfa700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f412c006010 tx=0x7f412c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c01d070 con 0x7f41300731c0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f412c00f460 con 0x7f41300731c0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c021620 con 0x7f41300731c0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41301989c0 con 0x7f41300731c0 2024-06-15T01:25:21.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.326+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4130198e60 con 0x7f41300731c0 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.327+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413010a560 con 0x7f41300731c0 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.328+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f412c00f5d0 con 0x7f41300731c0 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.328+0000 7f4126ffd700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 0x7f411c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.328+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f412c08b540 con 0x7f41300731c0 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.328+0000 7f41355f9700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 0x7f411c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.329+0000 7f41355f9700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 0x7f411c06d440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f41200058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:21.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.333+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f412c059fa0 con 0x7f41300731c0 2024-06-15T01:25:21.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.601+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4130199d50 con 0x7f41300731c0 2024-06-15T01:25:21.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.602+0000 7f4126ffd700 1 -- 172.21.15.137:0/3677264255 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f4130199d50 con 0x7f41300731c0 2024-06-15T01:25:21.588 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:25:21.588 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:25:21.588 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:25:21.589 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.604+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 msgr2=0x7f411c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.604+0000 7f413805e700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 0x7f411c06d440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f41200058e0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.604+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 msgr2=0x7f413019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.604+0000 7f413805e700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f412c006010 tx=0x7f412c00be20 comp rx=0 tx=0).stop 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 shutdown_connections 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f411c06af80 0x7f411c06d440 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41300731c0 0x7f413019e6c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 --2- 172.21.15.137:0/3677264255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4130073ae0 0x7f413019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 >> 172.21.15.137:0/3677264255 conn(0x7f41300fc650 msgr2=0x7f41301076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 shutdown_connections 2024-06-15T01:25:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.605+0000 7f413805e700 1 -- 172.21.15.137:0/3677264255 wait complete. 2024-06-15T01:25:21.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 -- 172.21.15.137:0/1361450103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60100ffa40 msgr2=0x7f60100ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 --2- 172.21.15.137:0/1361450103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60100ffa40 0x7f60100ffec0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6004009910 tx=0x7f6004009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 -- 172.21.15.137:0/1361450103 shutdown_connections 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 --2- 172.21.15.137:0/1361450103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60100ffa40 0x7f60100ffec0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 --2- 172.21.15.137:0/1361450103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6010105a40 0x7f6010105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 -- 172.21.15.137:0/1361450103 >> 172.21.15.137:0/1361450103 conn(0x7f60100fb550 msgr2=0x7f60100fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 -- 172.21.15.137:0/1361450103 shutdown_connections 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 -- 172.21.15.137:0/1361450103 wait complete. 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.708+0000 7f60149f3700 1 Processor -- start 2024-06-15T01:25:21.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f60149f3700 1 -- start start 2024-06-15T01:25:21.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f60149f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60100ffa40 0x7f6010101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f60149f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f60149f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010102090 con 0x7f6010105a40 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f60149f3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60101021d0 con 0x7f60100ffa40 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54310/0 (socket says 172.21.15.137:54310) 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 -- 172.21.15.137:0/897900784 learned_addr learned my addr 172.21.15.137:0/897900784 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600e59c700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60100ffa40 0x7f6010101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 -- 172.21.15.137:0/897900784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60100ffa40 msgr2=0x7f6010101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60100ffa40 0x7f6010101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 -- 172.21.15.137:0/897900784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040095f0 con 0x7f6010105a40 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.709+0000 7f600dd9b700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6004000c00 tx=0x7f600400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.710+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f600401d070 con 0x7f6010105a40 2024-06-15T01:25:21.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.710+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6004005dc0 con 0x7f6010105a40 2024-06-15T01:25:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.710+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6010102450 con 0x7f6010105a40 2024-06-15T01:25:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.710+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6004003940 con 0x7f6010105a40 2024-06-15T01:25:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.710+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60101a2bf0 con 0x7f6010105a40 2024-06-15T01:25:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.711+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6010107fd0 con 0x7f6010105a40 2024-06-15T01:25:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.711+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6004004ae0 con 0x7f6010105a40 2024-06-15T01:25:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.711+0000 7f5fff7fe700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 0x7f5ff806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.712+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f600408b940 con 0x7f6010105a40 2024-06-15T01:25:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.712+0000 7f600e59c700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 0x7f5ff806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.712+0000 7f600e59c700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 0x7f5ff806d440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6000009c60 tx=0x7f6000009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.716+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6004056770 con 0x7f6010105a40 2024-06-15T01:25:21.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.949+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f60101079b0 con 0x7f6010105a40 2024-06-15T01:25:21.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.950+0000 7f5fff7fe700 1 -- 172.21.15.137:0/897900784 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f6004059d90 con 0x7f6010105a40 2024-06-15T01:25:21.936 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:25:21.936 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:25:21.936 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:25:21.937 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:25:21.938 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:21.939 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:21.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 msgr2=0x7f5ff806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 0x7f5ff806d440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6000009c60 tx=0x7f6000009260 comp rx=0 tx=0).stop 2024-06-15T01:25:21.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 msgr2=0x7f6010101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6004000c00 tx=0x7f600400bde0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 shutdown_connections 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5ff806af80 0x7f5ff806d440 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60100ffa40 0x7f6010101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 --2- 172.21.15.137:0/897900784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6010105a40 0x7f6010101b50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 >> 172.21.15.137:0/897900784 conn(0x7f60100fb550 msgr2=0x7f6010109b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 shutdown_connections 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:21.953+0000 7f60149f3700 1 -- 172.21.15.137:0/897900784 wait complete. 2024-06-15T01:25:21.941 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:25:22.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 -- 172.21.15.137:0/3540306207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620100dc0 msgr2=0x7f86201011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 --2- 172.21.15.137:0/3540306207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620100dc0 0x7f86201011a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f861800b0a0 tx=0x7f861800b3b0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 -- 172.21.15.137:0/3540306207 shutdown_connections 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 --2- 172.21.15.137:0/3540306207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620101770 0x7f86201057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 --2- 172.21.15.137:0/3540306207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620100dc0 0x7f86201011a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 -- 172.21.15.137:0/3540306207 >> 172.21.15.137:0/3540306207 conn(0x7f86200fc670 msgr2=0x7f86200fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 -- 172.21.15.137:0/3540306207 shutdown_connections 2024-06-15T01:25:22.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.058+0000 7f86281bb700 1 -- 172.21.15.137:0/3540306207 wait complete. 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 Processor -- start 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 -- start start 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 0x7f8620196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8620197250 con 0x7f8620101770 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f86281bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86201973c0 con 0x7f8620100dc0 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f8625756700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f8625756700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54326/0 (socket says 172.21.15.137:54326) 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f8625756700 1 -- 172.21.15.137:0/1928963441 learned_addr learned my addr 172.21.15.137:0/1928963441 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f8625f57700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 0x7f8620196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.059+0000 7f8625756700 1 -- 172.21.15.137:0/1928963441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 msgr2=0x7f8620196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f8625756700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 0x7f8620196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f8625756700 1 -- 172.21.15.137:0/1928963441 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009d00 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f8625f57700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 0x7f8620196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f8625756700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f861400c960 tx=0x7f861400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8614007a70 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8614007bd0 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862019a160 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8614018630 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.060+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862019a6b0 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.061+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8620109160 con 0x7f8620101770 2024-06-15T01:25:22.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.061+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f861400f9f0 con 0x7f8620101770 2024-06-15T01:25:22.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.062+0000 7f861effd700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 0x7f860806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.062+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f861408a2e0 con 0x7f8620101770 2024-06-15T01:25:22.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.062+0000 7f8625f57700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 0x7f860806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.062+0000 7f8625f57700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 0x7f860806d440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8618009480 tx=0x7f8618006000 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:22.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.066+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8614058c20 con 0x7f8620101770 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: pgmap v578: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3677264255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:25:22.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:22 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/897900784' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:25:22.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.277+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f862002cce0 con 0x7f860806af80 2024-06-15T01:25:22.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.278+0000 7f861effd700 1 -- 172.21.15.137:0/1928963441 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f862002cce0 con 0x7f860806af80 2024-06-15T01:25:22.264 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:25:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:25:22.265 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:25:22.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.280+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 msgr2=0x7f860806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.280+0000 7f86281bb700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 0x7f860806d440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8618009480 tx=0x7f8618006000 comp rx=0 tx=0).stop 2024-06-15T01:25:22.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 msgr2=0x7f8620196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f861400c960 tx=0x7f861400cd20 comp rx=0 tx=0).stop 2024-06-15T01:25:22.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 shutdown_connections 2024-06-15T01:25:22.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f860806af80 0x7f860806d440 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8620100dc0 0x7f8620196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 --2- 172.21.15.137:0/1928963441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8620101770 0x7f8620196c50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 >> 172.21.15.137:0/1928963441 conn(0x7f86200fc670 msgr2=0x7f86200fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:22.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 shutdown_connections 2024-06-15T01:25:22.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.281+0000 7f86281bb700 1 -- 172.21.15.137:0/1928963441 wait complete. 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1456583011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 msgr2=0x7fd4c01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1456583011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c01011a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009910 tx=0x7fd4bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1456583011 shutdown_connections 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1456583011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 0x7fd4c01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1456583011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c01011a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.385+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1456583011 >> 172.21.15.137:0/1456583011 conn(0x7fd4c00fc650 msgr2=0x7fd4c00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1456583011 shutdown_connections 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1456583011 wait complete. 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 Processor -- start 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 -- start start 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 0x7fd4c00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0073b10 con 0x7fd4c0100dc0 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c7c19700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0073c50 con 0x7fd4c0101770 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.386+0000 7fd4c59b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54344/0 (socket says 172.21.15.137:54344) 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 -- 172.21.15.137:0/1614890784 learned_addr learned my addr 172.21.15.137:0/1614890784 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c51b4700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 0x7fd4c00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 -- 172.21.15.137:0/1614890784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 msgr2=0x7fd4c00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 0x7fd4c00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 -- 172.21.15.137:0/1614890784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0095f0 con 0x7fd4c0100dc0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c59b5700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0092e0 tx=0x7fd4bc00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc01d070 con 0x7fd4c0100dc0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4bc0044c0 con 0x7fd4c0100dc0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c0073ed0 con 0x7fd4c0100dc0 2024-06-15T01:25:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4bc003a50 con 0x7fd4c0100dc0 2024-06-15T01:25:22.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.387+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c01a2b10 con 0x7fd4c0100dc0 2024-06-15T01:25:22.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.388+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c0109160 con 0x7fd4c0100dc0 2024-06-15T01:25:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.389+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd4bc0048e0 con 0x7fd4c0100dc0 2024-06-15T01:25:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.389+0000 7fd4b6ffd700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 0x7fd4ac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.389+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd4bc08b950 con 0x7fd4c0100dc0 2024-06-15T01:25:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.389+0000 7fd4c51b4700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 0x7fd4ac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.390+0000 7fd4c51b4700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 0x7fd4ac06d440 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0074880 tx=0x7fd4b0009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:22.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.393+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd4bc056800 con 0x7fd4c0100dc0 2024-06-15T01:25:22.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: pgmap v578: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:22.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:22.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3677264255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:25:22.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:22 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/897900784' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:25:22.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.662+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4c0108b40 con 0x7fd4c0100dc0 2024-06-15T01:25:22.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.662+0000 7fd4b6ffd700 1 -- 172.21.15.137:0/1614890784 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fd4bc026330 con 0x7fd4c0100dc0 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:25:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 msgr2=0x7fd4ac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 0x7fd4ac06d440 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0074880 tx=0x7fd4b0009260 comp rx=0 tx=0).stop 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 msgr2=0x7fd4c0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0092e0 tx=0x7fd4bc00be90 comp rx=0 tx=0).stop 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 shutdown_connections 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd4ac06af80 0x7fd4ac06d440 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0100dc0 0x7fd4c0073090 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 --2- 172.21.15.137:0/1614890784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4c0101770 0x7fd4c00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 >> 172.21.15.137:0/1614890784 conn(0x7fd4c00fc650 msgr2=0x7fd4c00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 shutdown_connections 2024-06-15T01:25:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:22.665+0000 7fd4c7c19700 1 -- 172.21.15.137:0/1614890784 wait complete. 2024-06-15T01:25:23.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:23 smithi137 ceph-mon[112295]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:23.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:23 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1614890784' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:25:23.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:23 smithi142 ceph-mon[82136]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:23.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:23 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1614890784' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:25:24.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:24 smithi137 ceph-mon[112295]: pgmap v579: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:24.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:24 smithi142 ceph-mon[82136]: pgmap v579: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:26.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:26 smithi137 ceph-mon[112295]: pgmap v580: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:26.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:26 smithi142 ceph-mon[82136]: pgmap v580: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:28.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:28 smithi137 ceph-mon[112295]: pgmap v581: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:28.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:28 smithi142 ceph-mon[82136]: pgmap v581: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:30.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:30 smithi137 ceph-mon[112295]: pgmap v582: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:30.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:30 smithi142 ceph-mon[82136]: pgmap v582: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:32.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:32 smithi137 ceph-mon[112295]: pgmap v583: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:32.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:32 smithi142 ceph-mon[82136]: pgmap v583: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:33.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:33.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:34.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:34 smithi137 ceph-mon[112295]: pgmap v584: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:34 smithi142 ceph-mon[82136]: pgmap v584: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:36.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:36 smithi137 ceph-mon[112295]: pgmap v585: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:36.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:36 smithi142 ceph-mon[82136]: pgmap v585: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:38.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:38 smithi137 ceph-mon[112295]: pgmap v586: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:38.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:38 smithi142 ceph-mon[82136]: pgmap v586: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:40.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:40 smithi137 ceph-mon[112295]: pgmap v587: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:40.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:40 smithi142 ceph-mon[82136]: pgmap v587: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:42.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:42 smithi142 ceph-mon[82136]: pgmap v588: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:42.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:42 smithi137 ceph-mon[112295]: pgmap v588: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:44.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:44 smithi137 ceph-mon[112295]: pgmap v589: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:44.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:44 smithi142 ceph-mon[82136]: pgmap v589: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:46.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:46 smithi137 ceph-mon[112295]: pgmap v590: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:46.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:46 smithi142 ceph-mon[82136]: pgmap v590: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:48.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:48 smithi137 ceph-mon[112295]: pgmap v591: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:48.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:48 smithi142 ceph-mon[82136]: pgmap v591: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:25:50.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:50 smithi137 ceph-mon[112295]: pgmap v592: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:50.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:50 smithi142 ceph-mon[82136]: pgmap v592: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:52.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:52 smithi137 ceph-mon[112295]: pgmap v593: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:52.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:52 smithi142 ceph-mon[82136]: pgmap v593: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:52.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 -- 172.21.15.137:0/4061827197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c101750 msgr2=0x7f796c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:52.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 --2- 172.21.15.137:0/4061827197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c101750 0x7f796c1057a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7954009910 tx=0x7f7954009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:52.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 -- 172.21.15.137:0/4061827197 shutdown_connections 2024-06-15T01:25:52.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 --2- 172.21.15.137:0/4061827197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c101750 0x7f796c1057a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 --2- 172.21.15.137:0/4061827197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c100da0 0x7f796c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 -- 172.21.15.137:0/4061827197 >> 172.21.15.137:0/4061827197 conn(0x7f796c0fc630 msgr2=0x7f796c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:52.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 -- 172.21.15.137:0/4061827197 shutdown_connections 2024-06-15T01:25:52.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.782+0000 7f7971148700 1 -- 172.21.15.137:0/4061827197 wait complete. 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 Processor -- start 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 -- start start 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c101750 0x7f796c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c197280 con 0x7f796c100da0 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f7971148700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c199fb0 con 0x7f796c101750 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f796ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f796ad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59624/0 (socket says 172.21.15.137:59624) 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.783+0000 7f796ad9d700 1 -- 172.21.15.137:0/4010022883 learned_addr learned my addr 172.21.15.137:0/4010022883 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f796a59c700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c101750 0x7f796c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f796ad9d700 1 -- 172.21.15.137:0/4010022883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c101750 msgr2=0x7f796c196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f796ad9d700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c101750 0x7f796c196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f796ad9d700 1 -- 172.21.15.137:0/4010022883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540095f0 con 0x7f796c100da0 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f796ad9d700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c8f0 tx=0x7f795c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795c004a00 con 0x7f796c100da0 2024-06-15T01:25:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c19a290 con 0x7f796c100da0 2024-06-15T01:25:52.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f795c005020 con 0x7f796c100da0 2024-06-15T01:25:52.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795c007c40 con 0x7f796c100da0 2024-06-15T01:25:52.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.784+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c19a7b0 con 0x7f796c100da0 2024-06-15T01:25:52.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.785+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c1091a0 con 0x7f796c100da0 2024-06-15T01:25:52.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.786+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f795c005190 con 0x7f796c100da0 2024-06-15T01:25:52.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.786+0000 7f7963fff700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 0x7f795806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:52.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.786+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f795c013070 con 0x7f796c100da0 2024-06-15T01:25:52.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.786+0000 7f796a59c700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 0x7f795806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:52.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.787+0000 7f796a59c700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 0x7f795806d440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7954006010 tx=0x7f795401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:52.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.791+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f795c055010 con 0x7f796c100da0 2024-06-15T01:25:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:52.998+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f796c19aa70 con 0x7f795806af80 2024-06-15T01:25:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.000+0000 7f7963fff700 1 -- 172.21.15.137:0/4010022883 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f796c19aa70 con 0x7f795806af80 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 msgr2=0x7f795806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 0x7f795806d440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7954006010 tx=0x7f795401a040 comp rx=0 tx=0).stop 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 msgr2=0x7f796c1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c8f0 tx=0x7f795c00cc00 comp rx=0 tx=0).stop 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 shutdown_connections 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f795806af80 0x7f795806d440 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f796c100da0 0x7f796c1966b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 --2- 172.21.15.137:0/4010022883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f796c101750 0x7f796c196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 >> 172.21.15.137:0/4010022883 conn(0x7f796c0fc630 msgr2=0x7f796c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 shutdown_connections 2024-06-15T01:25:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.002+0000 7f7971148700 1 -- 172.21.15.137:0/4010022883 wait complete. 2024-06-15T01:25:53.001 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:25:53.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 -- 172.21.15.137:0/3371328352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 msgr2=0x7f13d4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 --2- 172.21.15.137:0/3371328352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d4101180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009910 tx=0x7f13c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 -- 172.21.15.137:0/3371328352 shutdown_connections 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 --2- 172.21.15.137:0/3371328352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 0x7f13d41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 --2- 172.21.15.137:0/3371328352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d4101180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 -- 172.21.15.137:0/3371328352 >> 172.21.15.137:0/3371328352 conn(0x7f13d40fc650 msgr2=0x7f13d40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.107+0000 7f13dad96700 1 -- 172.21.15.137:0/3371328352 shutdown_connections 2024-06-15T01:25:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 -- 172.21.15.137:0/3371328352 wait complete. 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 Processor -- start 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 -- start start 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 0x7f13d419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d419d0d0 con 0x7f13d4100da0 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13dad96700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4196580 con 0x7f13d4101750 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13d8b32700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13d8b32700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59648/0 (socket says 172.21.15.137:59648) 2024-06-15T01:25:53.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.108+0000 7f13d8b32700 1 -- 172.21.15.137:0/2800599056 learned_addr learned my addr 172.21.15.137:0/2800599056 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d3fff700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 0x7f13d419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d8b32700 1 -- 172.21.15.137:0/2800599056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 msgr2=0x7f13d419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d8b32700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 0x7f13d419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d8b32700 1 -- 172.21.15.137:0/2800599056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80095f0 con 0x7f13d4100da0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d8b32700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13c8006010 tx=0x7f13c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c801d070 con 0x7f13d4100da0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13c8005710 con 0x7f13d4100da0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d4196800 con 0x7f13d4100da0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c800fe20 con 0x7f13d4100da0 2024-06-15T01:25:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.109+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d4196cf0 con 0x7f13d4100da0 2024-06-15T01:25:53.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.111+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f13c8003a60 con 0x7f13d4100da0 2024-06-15T01:25:53.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.111+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d4109140 con 0x7f13d4100da0 2024-06-15T01:25:53.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.111+0000 7f13d1ffb700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.111+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f13c808b520 con 0x7f13d4100da0 2024-06-15T01:25:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.111+0000 7f13d3fff700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.112+0000 7f13d3fff700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13d40fce70 tx=0x7f13c000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.115+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f13c8059de0 con 0x7f13d4100da0 2024-06-15T01:25:53.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.325+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13d4196f10 con 0x7f13c406af80 2024-06-15T01:25:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.326+0000 7f13d1ffb700 1 -- 172.21.15.137:0/2800599056 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f13d4196f10 con 0x7f13c406af80 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 msgr2=0x7f13c406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13d40fce70 tx=0x7f13c000b410 comp rx=0 tx=0).stop 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 msgr2=0x7f13d419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f13c8006010 tx=0x7f13c800bd70 comp rx=0 tx=0).stop 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 shutdown_connections 2024-06-15T01:25:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d4100da0 0x7f13d419c500 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 --2- 172.21.15.137:0/2800599056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d4101750 0x7f13d419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 >> 172.21.15.137:0/2800599056 conn(0x7f13d40fc650 msgr2=0x7f13d40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:53.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 shutdown_connections 2024-06-15T01:25:53.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.328+0000 7f13dad96700 1 -- 172.21.15.137:0/2800599056 wait complete. 2024-06-15T01:25:53.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 -- 172.21.15.137:0/1359909812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba64105a40 msgr2=0x7fba64105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/1359909812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba64105a40 0x7fba64105e20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009910 tx=0x7fba5c009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:53.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 -- 172.21.15.137:0/1359909812 shutdown_connections 2024-06-15T01:25:53.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/1359909812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640ffbd0 0x7fba64100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/1359909812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba64105a40 0x7fba64105e20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 -- 172.21.15.137:0/1359909812 >> 172.21.15.137:0/1359909812 conn(0x7fba640755a0 msgr2=0x7fba640759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:53.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 -- 172.21.15.137:0/1359909812 shutdown_connections 2024-06-15T01:25:53.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.433+0000 7fba6aa9e700 1 -- 172.21.15.137:0/1359909812 wait complete. 2024-06-15T01:25:53.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 Processor -- start 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 -- start start 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba64105a40 0x7fba64195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba64196170 con 0x7fba640ffbd0 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6aa9e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba64199f00 con 0x7fba64105a40 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6883a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6883a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59660/0 (socket says 172.21.15.137:59660) 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba6883a700 1 -- 172.21.15.137:0/731837777 learned_addr learned my addr 172.21.15.137:0/731837777 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.434+0000 7fba63fff700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba64105a40 0x7fba64195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6883a700 1 -- 172.21.15.137:0/731837777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba64105a40 msgr2=0x7fba64195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6883a700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba64105a40 0x7fba64195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6883a700 1 -- 172.21.15.137:0/731837777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0095f0 con 0x7fba640ffbd0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6883a700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba5c006010 tx=0x7fba5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5c01f070 con 0x7fba640ffbd0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba5c005710 con 0x7fba640ffbd0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5c00f460 con 0x7fba640ffbd0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6419a180 con 0x7fba640ffbd0 2024-06-15T01:25:53.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.435+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6419a620 con 0x7fba640ffbd0 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.436+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6404ea90 con 0x7fba640ffbd0 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.437+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fba5c00f5e0 con 0x7fba640ffbd0 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.437+0000 7fba61ffb700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 0x7fba5006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.437+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fba5c08d6c0 con 0x7fba640ffbd0 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.437+0000 7fba63fff700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 0x7fba5006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.438+0000 7fba63fff700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 0x7fba5006d440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba54009550 tx=0x7fba540091b0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.442+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fba5c0584f0 con 0x7fba640ffbd0 2024-06-15T01:25:53.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.645+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fba6419aad0 con 0x7fba5006af80 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.655+0000 7fba61ffb700 1 -- 172.21.15.137:0/731837777 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fba6419aad0 con 0x7fba5006af80 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (16m) 10m ago 26m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (27m) 10m ago 27m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (26m) 10m ago 26m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (14m) 10m ago 27m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (14m) 10m ago 26m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (16m) 10m ago 26m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:25:53.641 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (10m) 10m ago 21m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (11m) 10m ago 21m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (10m) 10m ago 21m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (10m) 10m ago 21m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (18m) 10m ago 28m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (17m) 10m ago 25m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (15m) 10m ago 28m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (14m) 10m ago 25m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (17m) 10m ago 27m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (16m) 10m ago 26m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (14m) 10m ago 25m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (13m) 10m ago 24m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (13m) 10m ago 24m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (12m) 10m ago 23m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:25:53.642 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (12m) 10m ago 23m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:25:53.643 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (11m) 10m ago 22m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:25:53.643 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (16m) 10m ago 26m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.657+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 msgr2=0x7fba5006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.657+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 0x7fba5006d440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba54009550 tx=0x7fba540091b0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.657+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 msgr2=0x7fba64195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.657+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba5c006010 tx=0x7fba5c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 shutdown_connections 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fba5006af80 0x7fba5006d440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba640ffbd0 0x7fba64195550 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 --2- 172.21.15.137:0/731837777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba64105a40 0x7fba64195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 >> 172.21.15.137:0/731837777 conn(0x7fba640755a0 msgr2=0x7fba640fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 shutdown_connections 2024-06-15T01:25:53.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.658+0000 7fba6aa9e700 1 -- 172.21.15.137:0/731837777 wait complete. 2024-06-15T01:25:53.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 -- 172.21.15.137:0/3629128614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78101d60 msgr2=0x7f9d78102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 --2- 172.21.15.137:0/3629128614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78101d60 0x7f9d78102140 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800b0d0 tx=0x7f9d6800b3e0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 -- 172.21.15.137:0/3629128614 shutdown_connections 2024-06-15T01:25:53.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 --2- 172.21.15.137:0/3629128614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78102680 0x7f9d7810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 --2- 172.21.15.137:0/3629128614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78101d60 0x7f9d78102140 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 -- 172.21.15.137:0/3629128614 >> 172.21.15.137:0/3629128614 conn(0x7f9d780fb550 msgr2=0x7f9d780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:53.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 -- 172.21.15.137:0/3629128614 shutdown_connections 2024-06-15T01:25:53.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.761+0000 7f9d7d493700 1 -- 172.21.15.137:0/3629128614 wait complete. 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 Processor -- start 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 -- start start 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78101d60 0x7f9d7819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7819d0f0 con 0x7f9d78102680 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d7d493700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781965a0 con 0x7f9d78101d60 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d767fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d767fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59688/0 (socket says 172.21.15.137:59688) 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d767fc700 1 -- 172.21.15.137:0/2962340138 learned_addr learned my addr 172.21.15.137:0/2962340138 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.762+0000 7f9d76ffd700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78101d60 0x7f9d7819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d767fc700 1 -- 172.21.15.137:0/2962340138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78101d60 msgr2=0x7f9d7819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d767fc700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78101d60 0x7f9d7819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d767fc700 1 -- 172.21.15.137:0/2962340138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68009d00 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d767fc700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8c0 tx=0x7f9d6000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d600041f0 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d6000cc80 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d78196880 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d60003cd0 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.763+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d78196dd0 con 0x7f9d78102680 2024-06-15T01:25:53.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.764+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d78108270 con 0x7f9d78102680 2024-06-15T01:25:53.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.765+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f9d6000c7e0 con 0x7f9d78102680 2024-06-15T01:25:53.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.765+0000 7f9d6ffff700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 0x7f9d6406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:53.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.765+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9d60014070 con 0x7f9d78102680 2024-06-15T01:25:53.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.765+0000 7f9d76ffd700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 0x7f9d6406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:53.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.765+0000 7f9d76ffd700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 0x7f9d6406d440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009480 tx=0x7f9d6800b040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:53.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:53.769+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9d60058f70 con 0x7f9d78102680 2024-06-15T01:25:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.039+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d78197050 con 0x7f9d78102680 2024-06-15T01:25:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.039+0000 7f9d6ffff700 1 -- 172.21.15.137:0/2962340138 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9d60058b00 con 0x7f9d78102680 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:25:54.026 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:25:54.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:25:54.027 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:25:54.027 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:25:54.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:25:54.027 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 msgr2=0x7f9d6406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 0x7f9d6406d440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009480 tx=0x7f9d6800b040 comp rx=0 tx=0).stop 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 msgr2=0x7f9d7819ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8c0 tx=0x7f9d6000ebd0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 shutdown_connections 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9d6406af80 0x7f9d6406d440 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d78101d60 0x7f9d7819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 --2- 172.21.15.137:0/2962340138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d78102680 0x7f9d7819ca60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 >> 172.21.15.137:0/2962340138 conn(0x7f9d780fb550 msgr2=0x7f9d781053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 shutdown_connections 2024-06-15T01:25:54.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.042+0000 7f9d7d493700 1 -- 172.21.15.137:0/2962340138 wait complete. 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.146+0000 7ff763bf1700 1 -- 172.21.15.137:0/1141113713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c101770 msgr2=0x7ff75c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.146+0000 7ff763bf1700 1 --2- 172.21.15.137:0/1141113713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c101770 0x7ff75c1057c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff758009910 tx=0x7ff758009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 -- 172.21.15.137:0/1141113713 shutdown_connections 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 --2- 172.21.15.137:0/1141113713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c101770 0x7ff75c1057c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 --2- 172.21.15.137:0/1141113713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c100dc0 0x7ff75c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 -- 172.21.15.137:0/1141113713 >> 172.21.15.137:0/1141113713 conn(0x7ff75c0fc650 msgr2=0x7ff75c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 -- 172.21.15.137:0/1141113713 shutdown_connections 2024-06-15T01:25:54.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 -- 172.21.15.137:0/1141113713 wait complete. 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 Processor -- start 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.147+0000 7ff763bf1700 1 -- start start 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c101770 0x7ff75c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c197210 con 0x7ff75c100dc0 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c197380 con 0x7ff75c101770 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59696/0 (socket says 172.21.15.137:59696) 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 -- 172.21.15.137:0/2708674890 learned_addr learned my addr 172.21.15.137:0/2708674890 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76118c700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c101770 0x7ff75c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 -- 172.21.15.137:0/2708674890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c101770 msgr2=0x7ff75c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c101770 0x7ff75c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 -- 172.21.15.137:0/2708674890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580095f0 con 0x7ff75c100dc0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff76198d700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c930 tx=0x7ff74c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c007ad0 con 0x7ff75c100dc0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff74c00ce80 con 0x7ff75c100dc0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c19a120 con 0x7ff75c100dc0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c018630 con 0x7ff75c100dc0 2024-06-15T01:25:54.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.148+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c19a640 con 0x7ff75c100dc0 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.149+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c1091c0 con 0x7ff75c100dc0 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.150+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff74c007c30 con 0x7ff75c100dc0 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.150+0000 7ff752ffd700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 0x7ff74806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.150+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff74c08a380 con 0x7ff75c100dc0 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.150+0000 7ff76118c700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 0x7ff74806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.151+0000 7ff76118c700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 0x7ff74806d440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff75c197df0 tx=0x7ff75801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.155+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff74c0551b0 con 0x7ff75c100dc0 2024-06-15T01:25:54.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.390+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff75c04ea90 con 0x7ff75c100dc0 2024-06-15T01:25:54.376 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:54 smithi137 ceph-mon[112295]: pgmap v594: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:54.376 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:54 smithi137 ceph-mon[112295]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:54.377 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:54 smithi137 ceph-mon[112295]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:54.377 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:54 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2962340138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:25:54.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.390+0000 7ff752ffd700 1 -- 172.21.15.137:0/2708674890 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7ff74c0587d0 con 0x7ff75c100dc0 2024-06-15T01:25:54.377 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:25:54.377 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:25:54.377 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:25:54.377 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:25:54.378 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:25:54.379 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:54.380 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:25:54.381 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:25:54.381 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:54.381 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:54.381 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:54.381 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 msgr2=0x7ff74806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 0x7ff74806d440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff75c197df0 tx=0x7ff75801a040 comp rx=0 tx=0).stop 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 msgr2=0x7ff75c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c930 tx=0x7ff74c00cc40 comp rx=0 tx=0).stop 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 shutdown_connections 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff74806af80 0x7ff74806d440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff75c100dc0 0x7ff75c1966d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 --2- 172.21.15.137:0/2708674890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c101770 0x7ff75c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 >> 172.21.15.137:0/2708674890 conn(0x7ff75c0fc650 msgr2=0x7ff75c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 shutdown_connections 2024-06-15T01:25:54.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.393+0000 7ff763bf1700 1 -- 172.21.15.137:0/2708674890 wait complete. 2024-06-15T01:25:54.383 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:25:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 -- 172.21.15.137:0/872005717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 msgr2=0x7fb244105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 --2- 172.21.15.137:0/872005717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244105e20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb240009910 tx=0x7fb240009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 -- 172.21.15.137:0/872005717 shutdown_connections 2024-06-15T01:25:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 --2- 172.21.15.137:0/872005717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2440ffbd0 0x7fb244100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 --2- 172.21.15.137:0/872005717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244105e20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 -- 172.21.15.137:0/872005717 >> 172.21.15.137:0/872005717 conn(0x7fb2440755a0 msgr2=0x7fb2440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 -- 172.21.15.137:0/872005717 shutdown_connections 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.498+0000 7fb24c50f700 1 -- 172.21.15.137:0/872005717 wait complete. 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 Processor -- start 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 -- start start 2024-06-15T01:25:54.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2440ffbd0 0x7fb244195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2441960e0 con 0x7fb244105a40 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb24c50f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb244196220 con 0x7fb2440ffbd0 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59706/0 (socket says 172.21.15.137:59706) 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 -- 172.21.15.137:0/3088161856 learned_addr learned my addr 172.21.15.137:0/3088161856 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 -- 172.21.15.137:0/3088161856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2440ffbd0 msgr2=0x7fb244195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2440ffbd0 0x7fb244195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 -- 172.21.15.137:0/3088161856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400095f0 con 0x7fb244105a40 2024-06-15T01:25:54.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.499+0000 7fb249aaa700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c930 tx=0x7fb23400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.500+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb234007ad0 con 0x7fb244105a40 2024-06-15T01:25:54.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.500+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb234007c30 con 0x7fb244105a40 2024-06-15T01:25:54.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.500+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24419a070 con 0x7fb244105a40 2024-06-15T01:25:54.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.500+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb234018630 con 0x7fb244105a40 2024-06-15T01:25:54.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.500+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24419a5c0 con 0x7fb244105a40 2024-06-15T01:25:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.501+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb244108040 con 0x7fb244105a40 2024-06-15T01:25:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.501+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb23400f450 con 0x7fb244105a40 2024-06-15T01:25:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.501+0000 7fb23b7fe700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 0x7fb23006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.502+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb23408a5b0 con 0x7fb244105a40 2024-06-15T01:25:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.502+0000 7fb24a2ab700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 0x7fb23006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.502+0000 7fb24a2ab700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 0x7fb23006d440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b5c0 tx=0x7fb240005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.506+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb2340553e0 con 0x7fb244105a40 2024-06-15T01:25:54.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:54 smithi142 ceph-mon[82136]: pgmap v594: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:54.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:54 smithi142 ceph-mon[82136]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:54.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:54 smithi142 ceph-mon[82136]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:54.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:54 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2962340138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:25:54.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.711+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24402cce0 con 0x7fb23006af80 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.713+0000 7fb23b7fe700 1 -- 172.21.15.137:0/3088161856 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb24402cce0 con 0x7fb23006af80 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:25:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:25:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:25:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:25:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:25:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:25:54.700 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 msgr2=0x7fb23006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 0x7fb23006d440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b5c0 tx=0x7fb240005fd0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 msgr2=0x7fb244195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c930 tx=0x7fb23400ccf0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 shutdown_connections 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb23006af80 0x7fb23006d440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2440ffbd0 0x7fb244195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 --2- 172.21.15.137:0/3088161856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb244105a40 0x7fb244195a90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 >> 172.21.15.137:0/3088161856 conn(0x7fb2440755a0 msgr2=0x7fb2440fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 shutdown_connections 2024-06-15T01:25:54.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.715+0000 7fb24c50f700 1 -- 172.21.15.137:0/3088161856 wait complete. 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 -- 172.21.15.137:0/3153851658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc100dd0 msgr2=0x7f75fc1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 --2- 172.21.15.137:0/3153851658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc100dd0 0x7f75fc1011b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009910 tx=0x7f75ec009c20 comp rx=0 tx=0).stop 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 -- 172.21.15.137:0/3153851658 shutdown_connections 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 --2- 172.21.15.137:0/3153851658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc101780 0x7f75fc1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 --2- 172.21.15.137:0/3153851658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc100dd0 0x7f75fc1011b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.819+0000 7f76023d2700 1 -- 172.21.15.137:0/3153851658 >> 172.21.15.137:0/3153851658 conn(0x7f75fc0fc640 msgr2=0x7f75fc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.820+0000 7f76023d2700 1 -- 172.21.15.137:0/3153851658 shutdown_connections 2024-06-15T01:25:54.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.820+0000 7f76023d2700 1 -- 172.21.15.137:0/3153851658 wait complete. 2024-06-15T01:25:54.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.820+0000 7f76023d2700 1 Processor -- start 2024-06-15T01:25:54.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.820+0000 7f76023d2700 1 -- start start 2024-06-15T01:25:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.820+0000 7f76023d2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 0x7f75fc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f76023d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f76023d2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc19f1c0 con 0x7f75fc101780 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f76023d2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc198700 con 0x7f75fc100dd0 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59718/0 (socket says 172.21.15.137:59718) 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 -- 172.21.15.137:0/4051810302 learned_addr learned my addr 172.21.15.137:0/4051810302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fbfff700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 0x7f75fc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 -- 172.21.15.137:0/4051810302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 msgr2=0x7f75fc19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 0x7f75fc19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 -- 172.21.15.137:0/4051810302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0095f0 con 0x7f75fc101780 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fbfff700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 0x7f75fc19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75fb7fe700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c960 tx=0x7f75f000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75f0007a70 con 0x7f75fc101780 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc1989e0 con 0x7f75fc101780 2024-06-15T01:25:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f75f0007bd0 con 0x7f75fc101780 2024-06-15T01:25:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75f0018630 con 0x7f75fc101780 2024-06-15T01:25:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.821+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc198f30 con 0x7f75fc101780 2024-06-15T01:25:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.822+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc104f50 con 0x7f75fc101780 2024-06-15T01:25:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.823+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f75f000f9f0 con 0x7f75fc101780 2024-06-15T01:25:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.824+0000 7f75f97fa700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 0x7f75e406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:25:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.824+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f75f008a3d0 con 0x7f75fc101780 2024-06-15T01:25:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.824+0000 7f75fbfff700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 0x7f75e406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:25:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.824+0000 7f75fbfff700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 0x7f75e406d440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b5c0 tx=0x7f75ec005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:25:54.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:54.828+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f75f0058a00 con 0x7f75fc101780 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.100+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f75fc04ea90 con 0x7f75fc101780 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.101+0000 7f75f97fa700 1 -- 172.21.15.137:0/4051810302 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f75f0058820 con 0x7f75fc101780 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:25:55.087 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:25:55.088 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:25:55.088 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:25:55.088 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:25:55.088 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:25:55.088 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:25:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 msgr2=0x7f75e406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 0x7f75e406d440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b5c0 tx=0x7f75ec005960 comp rx=0 tx=0).stop 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 msgr2=0x7f75fc19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c960 tx=0x7f75f000cd20 comp rx=0 tx=0).stop 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 shutdown_connections 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f75e406af80 0x7f75e406d440 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75fc100dd0 0x7f75fc19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 --2- 172.21.15.137:0/4051810302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75fc101780 0x7f75fc19ebc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 >> 172.21.15.137:0/4051810302 conn(0x7f75fc0fc640 msgr2=0x7f75fc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.103+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 shutdown_connections 2024-06-15T01:25:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:25:55.104+0000 7f76023d2700 1 -- 172.21.15.137:0/4051810302 wait complete. 2024-06-15T01:25:55.342 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:55 smithi137 ceph-mon[112295]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:55.342 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2708674890' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:25:55.342 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4051810302' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:25:55.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:55 smithi142 ceph-mon[82136]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:55.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2708674890' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:25:55.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4051810302' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:25:56.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:56 smithi137 ceph-mon[112295]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:56.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:56 smithi137 ceph-mon[112295]: pgmap v595: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:56.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:56 smithi142 ceph-mon[82136]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:25:56.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:56 smithi142 ceph-mon[82136]: pgmap v595: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:58.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:25:58 smithi137 ceph-mon[112295]: pgmap v596: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:25:58.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:25:58 smithi142 ceph-mon[82136]: pgmap v596: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:00.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:00 smithi142 ceph-mon[82136]: pgmap v597: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:00.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:00 smithi137 ceph-mon[112295]: pgmap v597: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:02.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:02 smithi142 ceph-mon[82136]: pgmap v598: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:02.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:02 smithi137 ceph-mon[112295]: pgmap v598: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:03.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:03.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:04.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:04 smithi142 ceph-mon[82136]: pgmap v599: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:04.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:04 smithi137 ceph-mon[112295]: pgmap v599: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:06.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:06 smithi142 ceph-mon[82136]: pgmap v600: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:06.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:06 smithi137 ceph-mon[112295]: pgmap v600: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:08.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:08 smithi142 ceph-mon[82136]: pgmap v601: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:08.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:08 smithi137 ceph-mon[112295]: pgmap v601: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:10.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:10 smithi142 ceph-mon[82136]: pgmap v602: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:10.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:10 smithi137 ceph-mon[112295]: pgmap v602: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:12.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:12 smithi142 ceph-mon[82136]: pgmap v603: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:12.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:12 smithi137 ceph-mon[112295]: pgmap v603: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:14.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:14 smithi142 ceph-mon[82136]: pgmap v604: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:14.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:14 smithi137 ceph-mon[112295]: pgmap v604: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:16.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:16 smithi142 ceph-mon[82136]: pgmap v605: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:16.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:16 smithi137 ceph-mon[112295]: pgmap v605: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:18.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:18 smithi142 ceph-mon[82136]: pgmap v606: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:18.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:18.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:18 smithi137 ceph-mon[112295]: pgmap v606: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:18.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:20.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:20 smithi142 ceph-mon[82136]: pgmap v607: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:20.565 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:20 smithi137 ceph-mon[112295]: pgmap v607: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:22.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:22 smithi142 ceph-mon[82136]: pgmap v608: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:22.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:22 smithi137 ceph-mon[112295]: pgmap v608: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:24.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:24 smithi142 ceph-mon[82136]: pgmap v609: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:24.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:24 smithi137 ceph-mon[112295]: pgmap v609: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 -- 172.21.15.137:0/637945805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f40ffa60 msgr2=0x7f50f40ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 --2- 172.21.15.137:0/637945805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f40ffa60 0x7f50f40ffee0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009910 tx=0x7f50e4009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 -- 172.21.15.137:0/637945805 shutdown_connections 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 --2- 172.21.15.137:0/637945805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f40ffa60 0x7f50f40ffee0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 --2- 172.21.15.137:0/637945805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f4105a60 0x7f50f4105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 -- 172.21.15.137:0/637945805 >> 172.21.15.137:0/637945805 conn(0x7f50f40fb550 msgr2=0x7f50f40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 -- 172.21.15.137:0/637945805 shutdown_connections 2024-06-15T01:26:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.216+0000 7f50fc60f700 1 -- 172.21.15.137:0/637945805 wait complete. 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 Processor -- start 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 -- start start 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f40ffa60 0x7f50f4196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4197240 con 0x7f50f4105a60 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fc60f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f419afd0 con 0x7f50f40ffa60 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48402/0 (socket says 172.21.15.137:48402) 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 -- 172.21.15.137:0/900143002 learned_addr learned my addr 172.21.15.137:0/900143002 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50fa3ab700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f40ffa60 0x7f50f4196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 -- 172.21.15.137:0/900143002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f40ffa60 msgr2=0x7f50f4196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f40ffa60 0x7f50f4196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.217+0000 7f50f9baa700 1 -- 172.21.15.137:0/900143002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40095f0 con 0x7f50f4105a60 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50f9baa700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f50e40092e0 tx=0x7f50e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50e401d070 con 0x7f50f4105a60 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50e4005e00 con 0x7f50f4105a60 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f419b250 con 0x7f50f4105a60 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50e400fe60 con 0x7f50f4105a60 2024-06-15T01:26:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.218+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f419b740 con 0x7f50f4105a60 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.219+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4109000 con 0x7f50f4105a60 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.219+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f50e400f6d0 con 0x7f50f4105a60 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.220+0000 7f50eb7fe700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 0x7f50e006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.220+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f50e408b630 con 0x7f50f4105a60 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.220+0000 7f50fa3ab700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 0x7f50e006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.220+0000 7f50fa3ab700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 0x7f50e006d440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005950 tx=0x7f50f00058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.225+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f50e4056580 con 0x7f50f4105a60 2024-06-15T01:26:25.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.435+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50f419bae0 con 0x7f50e006af80 2024-06-15T01:26:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.437+0000 7f50eb7fe700 1 -- 172.21.15.137:0/900143002 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f50f419bae0 con 0x7f50e006af80 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 msgr2=0x7f50e006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 0x7f50e006d440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005950 tx=0x7f50f00058e0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 msgr2=0x7f50f4196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f50e40092e0 tx=0x7f50e400be20 comp rx=0 tx=0).stop 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 shutdown_connections 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f50e006af80 0x7f50e006d440 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50f40ffa60 0x7f50f4196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 --2- 172.21.15.137:0/900143002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50f4105a60 0x7f50f4196b60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 >> 172.21.15.137:0/900143002 conn(0x7f50f40fb550 msgr2=0x7f50f4069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 shutdown_connections 2024-06-15T01:26:25.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.439+0000 7f50fc60f700 1 -- 172.21.15.137:0/900143002 wait complete. 2024-06-15T01:26:25.435 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:26:25.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 -- 172.21.15.137:0/3446855477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c069020 msgr2=0x7f2d0c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 --2- 172.21.15.137:0/3446855477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c069020 0x7f2d0c105790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009910 tx=0x7f2d00009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 -- 172.21.15.137:0/3446855477 shutdown_connections 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 --2- 172.21.15.137:0/3446855477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c069020 0x7f2d0c105790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 --2- 172.21.15.137:0/3446855477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c068670 0x7f2d0c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.543+0000 7f2d146c5700 1 -- 172.21.15.137:0/3446855477 >> 172.21.15.137:0/3446855477 conn(0x7f2d0c0755a0 msgr2=0x7f2d0c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.544+0000 7f2d146c5700 1 -- 172.21.15.137:0/3446855477 shutdown_connections 2024-06-15T01:26:25.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.544+0000 7f2d146c5700 1 -- 172.21.15.137:0/3446855477 wait complete. 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.544+0000 7f2d146c5700 1 Processor -- start 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.544+0000 7f2d146c5700 1 -- start start 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.544+0000 7f2d146c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d146c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c069020 0x7f2d0c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d146c5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c197260 con 0x7f2d0c068670 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d146c5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19aff0 con 0x7f2d0c069020 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48410/0 (socket says 172.21.15.137:48410) 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 -- 172.21.15.137:0/2557730505 learned_addr learned my addr 172.21.15.137:0/2557730505 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d11c60700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c069020 0x7f2d0c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 -- 172.21.15.137:0/2557730505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c069020 msgr2=0x7f2d0c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c069020 0x7f2d0c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 -- 172.21.15.137:0/2557730505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000095f0 con 0x7f2d0c068670 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d12461700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0060b0 tx=0x7f2cfc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc00dda0 con 0x7f2d0c068670 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cfc00f040 con 0x7f2d0c068670 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c19b2d0 con 0x7f2d0c068670 2024-06-15T01:26:25.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc0145d0 con 0x7f2d0c068670 2024-06-15T01:26:25.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.545+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c19b820 con 0x7f2d0c068670 2024-06-15T01:26:25.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.546+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c1090a0 con 0x7f2d0c068670 2024-06-15T01:26:25.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.547+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2cfc015400 con 0x7f2d0c068670 2024-06-15T01:26:25.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.547+0000 7f2d0b7fe700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 0x7f2cf406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.547+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2cfc089dd0 con 0x7f2d0c068670 2024-06-15T01:26:25.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.547+0000 7f2d11c60700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 0x7f2cf406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.548+0000 7f2d11c60700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 0x7f2cf406d440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c197c60 tx=0x7f2d0001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.551+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cfc054cb0 con 0x7f2d0c068670 2024-06-15T01:26:25.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.765+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d0c19bb90 con 0x7f2cf406af80 2024-06-15T01:26:25.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.766+0000 7f2d0b7fe700 1 -- 172.21.15.137:0/2557730505 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d0c19bb90 con 0x7f2cf406af80 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 msgr2=0x7f2cf406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 0x7f2cf406d440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c197c60 tx=0x7f2d0001a040 comp rx=0 tx=0).stop 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 msgr2=0x7f2d0c196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0060b0 tx=0x7f2cfc00d3b0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 shutdown_connections 2024-06-15T01:26:25.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cf406af80 0x7f2cf406d440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d0c068670 0x7f2d0c196640 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 --2- 172.21.15.137:0/2557730505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d0c069020 0x7f2d0c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 >> 172.21.15.137:0/2557730505 conn(0x7f2d0c0755a0 msgr2=0x7f2d0c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:25.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 shutdown_connections 2024-06-15T01:26:25.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.769+0000 7f2d146c5700 1 -- 172.21.15.137:0/2557730505 wait complete. 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/183913895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 msgr2=0x7f5f180ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/183913895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f180ffec0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009910 tx=0x7f5f0c009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/183913895 shutdown_connections 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/183913895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f180ffec0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/183913895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/183913895 >> 172.21.15.137:0/183913895 conn(0x7f5f180fb570 msgr2=0x7f5f180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/183913895 shutdown_connections 2024-06-15T01:26:25.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/183913895 wait complete. 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.872+0000 7f5f1d4ea700 1 Processor -- start 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f1d4ea700 1 -- start start 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f1d4ea700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f1d4ea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f1d4ea700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18197280 con 0x7f5f180ffa40 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f1d4ea700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1819b010 con 0x7f5f18105a40 2024-06-15T01:26:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48430/0 (socket says 172.21.15.137:48430) 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 -- 172.21.15.137:0/283496168 learned_addr learned my addr 172.21.15.137:0/283496168 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f167fc700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 -- 172.21.15.137:0/283496168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 msgr2=0x7f5f18196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f16ffd700 1 -- 172.21.15.137:0/283496168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0095f0 con 0x7f5f180ffa40 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.873+0000 7f5f167fc700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5f16ffd700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c8a0 tx=0x7f5f0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f0800cea0 con 0x7f5f180ffa40 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f08004830 con 0x7f5f180ffa40 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1819b2f0 con 0x7f5f180ffa40 2024-06-15T01:26:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f08005500 con 0x7f5f180ffa40 2024-06-15T01:26:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.874+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1819b840 con 0x7f5f180ffa40 2024-06-15T01:26:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.875+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f18108fe0 con 0x7f5f180ffa40 2024-06-15T01:26:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.875+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5f08007ad0 con 0x7f5f180ffa40 2024-06-15T01:26:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.876+0000 7f5efffff700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 0x7f5f0006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.876+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5f0808a140 con 0x7f5f180ffa40 2024-06-15T01:26:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.876+0000 7f5f167fc700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 0x7f5f0006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.877+0000 7f5f167fc700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 0x7f5f0006d440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00b5c0 tx=0x7f5f0c005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:25.880+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f08054ed0 con 0x7f5f180ffa40 2024-06-15T01:26:26.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.078+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f1819bbd0 con 0x7f5f0006af80 2024-06-15T01:26:26.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.089+0000 7f5efffff700 1 -- 172.21.15.137:0/283496168 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5f1819bbd0 con 0x7f5f0006af80 2024-06-15T01:26:26.074 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:26:26.074 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (16m) 11m ago 27m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:26:26.074 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (27m) 11m ago 27m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (26m) 10m ago 26m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (15m) 11m ago 27m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (15m) 10m ago 26m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (16m) 11m ago 27m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (11m) 11m ago 21m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (11m) 11m ago 22m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (11m) 10m ago 22m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (11m) 10m ago 21m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (18m) 11m ago 29m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (17m) 10m ago 26m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (15m) 11m ago 29m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (15m) 10m ago 26m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (17m) 11m ago 27m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (17m) 10m ago 26m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:26:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (14m) 11m ago 25m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (14m) 11m ago 25m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (13m) 11m ago 24m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (13m) 10m ago 24m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (12m) 10m ago 23m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (12m) 10m ago 23m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:26:26.076 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (17m) 11m ago 27m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 msgr2=0x7f5f0006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 0x7f5f0006d440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00b5c0 tx=0x7f5f0c005960 comp rx=0 tx=0).stop 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 msgr2=0x7f5f18196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c8a0 tx=0x7f5f0800cc60 comp rx=0 tx=0).stop 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 shutdown_connections 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5f0006af80 0x7f5f0006d440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f180ffa40 0x7f5f18196660 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 --2- 172.21.15.137:0/283496168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f18105a40 0x7f5f18196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 >> 172.21.15.137:0/283496168 conn(0x7f5f180fb570 msgr2=0x7f5f18069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 shutdown_connections 2024-06-15T01:26:26.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.091+0000 7f5f1d4ea700 1 -- 172.21.15.137:0/283496168 wait complete. 2024-06-15T01:26:26.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 -- 172.21.15.137:0/8309685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394101770 msgr2=0x7f33941057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 --2- 172.21.15.137:0/8309685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394101770 0x7f33941057c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3380009910 tx=0x7f3380009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:26.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 -- 172.21.15.137:0/8309685 shutdown_connections 2024-06-15T01:26:26.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 --2- 172.21.15.137:0/8309685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394101770 0x7f33941057c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 --2- 172.21.15.137:0/8309685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394100dc0 0x7f33941011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 -- 172.21.15.137:0/8309685 >> 172.21.15.137:0/8309685 conn(0x7f33940fc650 msgr2=0x7f33940fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 -- 172.21.15.137:0/8309685 shutdown_connections 2024-06-15T01:26:26.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 -- 172.21.15.137:0/8309685 wait complete. 2024-06-15T01:26:26.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.193+0000 7f339aacf700 1 Processor -- start 2024-06-15T01:26:26.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339aacf700 1 -- start start 2024-06-15T01:26:26.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339aacf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339aacf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 0x7f33940735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339aacf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394073b10 con 0x7f3394100dc0 2024-06-15T01:26:26.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339aacf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394073c50 con 0x7f3394101770 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60294/0 (socket says 172.21.15.137:60294) 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 -- 172.21.15.137:0/3555612327 learned_addr learned my addr 172.21.15.137:0/3555612327 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f3393fff700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 0x7f33940735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 -- 172.21.15.137:0/3555612327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 msgr2=0x7f33940735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 0x7f33940735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 -- 172.21.15.137:0/3555612327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800095f0 con 0x7f3394100dc0 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f3393fff700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 0x7f33940735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.194+0000 7f339886b700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f33880060b0 tx=0x7f338800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.195+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338800dda0 con 0x7f3394100dc0 2024-06-15T01:26:26.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.195+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f338800f040 con 0x7f3394100dc0 2024-06-15T01:26:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.195+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33880146f0 con 0x7f3394100dc0 2024-06-15T01:26:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.195+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33941a28a0 con 0x7f3394100dc0 2024-06-15T01:26:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.195+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33941a2cb0 con 0x7f3394100dc0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.196+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33941091c0 con 0x7f3394100dc0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.196+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3388015400 con 0x7f3394100dc0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.197+0000 7f3391ffb700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 0x7f338406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.197+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3388089d30 con 0x7f3394100dc0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.197+0000 7f3393fff700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 0x7f338406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.197+0000 7f3393fff700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 0x7f338406d440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3394074880 tx=0x7f338001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.201+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3388054c90 con 0x7f3394100dc0 2024-06-15T01:26:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.472+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f339404ea90 con 0x7f3394100dc0 2024-06-15T01:26:26.458 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:26 smithi137 ceph-mon[112295]: pgmap v610: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.472+0000 7f3391ffb700 1 -- 172.21.15.137:0/3555612327 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f33880582b0 con 0x7f3394100dc0 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:26:26.459 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:26:26.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:26.460 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:26:26.460 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:26:26.460 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:26:26.460 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 msgr2=0x7f338406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 0x7f338406d440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3394074880 tx=0x7f338001a040 comp rx=0 tx=0).stop 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 msgr2=0x7f3394073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f33880060b0 tx=0x7f338800d3b0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 shutdown_connections 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f338406af80 0x7f338406d440 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3394100dc0 0x7f3394073090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 --2- 172.21.15.137:0/3555612327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3394101770 0x7f33940735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 >> 172.21.15.137:0/3555612327 conn(0x7f33940fc650 msgr2=0x7f33940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 shutdown_connections 2024-06-15T01:26:26.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.475+0000 7f339aacf700 1 -- 172.21.15.137:0/3555612327 wait complete. 2024-06-15T01:26:26.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:26 smithi142 ceph-mon[82136]: pgmap v610: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:26.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 -- 172.21.15.137:0/2615985370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ff400 msgr2=0x7f4b940ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 --2- 172.21.15.137:0/2615985370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ff400 0x7f4b940ff7e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009910 tx=0x7f4b7c009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:26.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 -- 172.21.15.137:0/2615985370 shutdown_connections 2024-06-15T01:26:26.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 --2- 172.21.15.137:0/2615985370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ffdb0 0x7f4b94103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 --2- 172.21.15.137:0/2615985370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ff400 0x7f4b940ff7e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 -- 172.21.15.137:0/2615985370 >> 172.21.15.137:0/2615985370 conn(0x7f4b94075e30 msgr2=0x7f4b940fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 -- 172.21.15.137:0/2615985370 shutdown_connections 2024-06-15T01:26:26.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.579+0000 7f4b9a592700 1 -- 172.21.15.137:0/2615985370 wait complete. 2024-06-15T01:26:26.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 Processor -- start 2024-06-15T01:26:26.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 -- start start 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ff400 0x7f4b94196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b94197270 con 0x7f4b940ffdb0 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b9a592700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9419b000 con 0x7f4b940ff400 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b937fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b937fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60316/0 (socket says 172.21.15.137:60316) 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b937fe700 1 -- 172.21.15.137:0/47810241 learned_addr learned my addr 172.21.15.137:0/47810241 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b93fff700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ff400 0x7f4b94196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b937fe700 1 -- 172.21.15.137:0/47810241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ff400 msgr2=0x7f4b94196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.580+0000 7f4b937fe700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ff400 0x7f4b94196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b937fe700 1 -- 172.21.15.137:0/47810241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0095f0 con 0x7f4b940ffdb0 2024-06-15T01:26:26.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b937fe700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400dd70 tx=0x7f4b8400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b8400ad10 con 0x7f4b940ffdb0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9419b2e0 con 0x7f4b940ffdb0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b8400ae70 con 0x7f4b940ffdb0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b84016810 con 0x7f4b940ffdb0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.581+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9419b830 con 0x7f4b940ffdb0 2024-06-15T01:26:26.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.582+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b941097b0 con 0x7f4b940ffdb0 2024-06-15T01:26:26.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.582+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4b8400e450 con 0x7f4b940ffdb0 2024-06-15T01:26:26.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.583+0000 7f4b917fa700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 0x7f4b8006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.583+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4b8408a1e0 con 0x7f4b940ffdb0 2024-06-15T01:26:26.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.583+0000 7f4b93fff700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 0x7f4b8006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.583+0000 7f4b93fff700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 0x7f4b8006d440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006010 tx=0x7f4b7c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.587+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4b84055140 con 0x7f4b940ffdb0 2024-06-15T01:26:26.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.819+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b9404ea90 con 0x7f4b940ffdb0 2024-06-15T01:26:26.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.819+0000 7f4b917fa700 1 -- 172.21.15.137:0/47810241 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f4b84058760 con 0x7f4b940ffdb0 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:26:26.805 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:26:26.806 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:26:26.807 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:26.808 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:26.808 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:26.808 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:26.808 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:26.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 msgr2=0x7f4b8006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 0x7f4b8006d440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006010 tx=0x7f4b7c01a040 comp rx=0 tx=0).stop 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 msgr2=0x7f4b94196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400dd70 tx=0x7f4b8400a530 comp rx=0 tx=0).stop 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 shutdown_connections 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4b8006af80 0x7f4b8006d440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b940ff400 0x7f4b94196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 --2- 172.21.15.137:0/47810241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b940ffdb0 0x7f4b94196b90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 >> 172.21.15.137:0/47810241 conn(0x7f4b94075e30 msgr2=0x7f4b94103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 shutdown_connections 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.822+0000 7f4b9a592700 1 -- 172.21.15.137:0/47810241 wait complete. 2024-06-15T01:26:26.809 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:26:26.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3535620489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 msgr2=0x7fbf580ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3535620489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580ffed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009910 tx=0x7fbf40009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3535620489 shutdown_connections 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3535620489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580ffed0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3535620489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf58105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3535620489 >> 172.21.15.137:0/3535620489 conn(0x7fbf580fb540 msgr2=0x7fbf580fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3535620489 shutdown_connections 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.927+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3535620489 wait complete. 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 Processor -- start 2024-06-15T01:26:26.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 -- start start 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf580fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580fdc10 con 0x7fbf580ffa50 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf5dde1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580fdd80 con 0x7fbf58105a50 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60328/0 (socket says 172.21.15.137:60328) 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 -- 172.21.15.137:0/3440168360 learned_addr learned my addr 172.21.15.137:0/3440168360 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:26.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 -- 172.21.15.137:0/3440168360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 msgr2=0x7fbf580fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf56ffd700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf580fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf580fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf577fe700 1 -- 172.21.15.137:0/3440168360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf400095f0 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.928+0000 7fbf56ffd700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf580fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf577fe700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbf480060b0 tx=0x7fbf4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf4800dda0 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf580fc9a0 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf4800f040 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf581a7250 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.929+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf480146f0 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.930+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58108ff0 con 0x7fbf580ffa50 2024-06-15T01:26:26.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.930+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbf48015400 con 0x7fbf580ffa50 2024-06-15T01:26:26.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.931+0000 7fbf54ff9700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 0x7fbf4406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:26.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.931+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbf48089e70 con 0x7fbf580ffa50 2024-06-15T01:26:26.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.931+0000 7fbf56ffd700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 0x7fbf4406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:26.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.931+0000 7fbf56ffd700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 0x7fbf4406d440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf400059d0 tx=0x7fbf40005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:26.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:26.935+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbf48054d50 con 0x7fbf580ffa50 2024-06-15T01:26:27.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.145+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf580611d0 con 0x7fbf4406af80 2024-06-15T01:26:27.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.147+0000 7fbf54ff9700 1 -- 172.21.15.137:0/3440168360 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf580611d0 con 0x7fbf4406af80 2024-06-15T01:26:27.131 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:26:27.132 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 msgr2=0x7fbf4406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 0x7fbf4406d440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf400059d0 tx=0x7fbf40005960 comp rx=0 tx=0).stop 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 msgr2=0x7fbf580fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbf480060b0 tx=0x7fbf4800d3b0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 shutdown_connections 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbf4406af80 0x7fbf4406d440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf580ffa50 0x7fbf580fbe60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 --2- 172.21.15.137:0/3440168360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58105a50 0x7fbf580fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 >> 172.21.15.137:0/3440168360 conn(0x7fbf580fb540 msgr2=0x7fbf58069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 shutdown_connections 2024-06-15T01:26:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.149+0000 7fbf5dde1700 1 -- 172.21.15.137:0/3440168360 wait complete. 2024-06-15T01:26:27.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 -- 172.21.15.137:0/1164377717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 msgr2=0x7f7850100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 --2- 172.21.15.137:0/1164377717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f7850100050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7838009910 tx=0x7f7838009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 -- 172.21.15.137:0/1164377717 shutdown_connections 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 --2- 172.21.15.137:0/1164377717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f7850100050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 --2- 172.21.15.137:0/1164377717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 0x7f7850105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 -- 172.21.15.137:0/1164377717 >> 172.21.15.137:0/1164377717 conn(0x7f78500755a0 msgr2=0x7f78500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 -- 172.21.15.137:0/1164377717 shutdown_connections 2024-06-15T01:26:27.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.253+0000 7f785603c700 1 -- 172.21.15.137:0/1164377717 wait complete. 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 Processor -- start 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 -- start start 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 0x7f785019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019d0e0 con 0x7f78500ffbd0 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f785603c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850196590 con 0x7f7850105a40 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f784f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f784f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60340/0 (socket says 172.21.15.137:60340) 2024-06-15T01:26:27.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f784f7fe700 1 -- 172.21.15.137:0/3123405444 learned_addr learned my addr 172.21.15.137:0/3123405444 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.254+0000 7f7847fff700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 0x7f785019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784f7fe700 1 -- 172.21.15.137:0/3123405444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 msgr2=0x7f785019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784f7fe700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 0x7f785019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784f7fe700 1 -- 172.21.15.137:0/3123405444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380095f0 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784f7fe700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f784000d8d0 tx=0x7f784000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78400098a0 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7840010460 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7840009ea0 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7850196870 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.255+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7850196d90 con 0x7f78500ffbd0 2024-06-15T01:26:27.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.256+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850108fe0 con 0x7f78500ffbd0 2024-06-15T01:26:27.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.256+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f78400105d0 con 0x7f78500ffbd0 2024-06-15T01:26:27.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.257+0000 7f784d7fa700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 0x7f783c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:27.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.257+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f784008a180 con 0x7f78500ffbd0 2024-06-15T01:26:27.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.257+0000 7f7847fff700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 0x7f783c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:27.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.257+0000 7f7847fff700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 0x7f783c06d440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f783800b5c0 tx=0x7f783801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:27.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.261+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7840058ac0 con 0x7f78500ffbd0 2024-06-15T01:26:27.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:27 smithi137 ceph-mon[112295]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:27 smithi137 ceph-mon[112295]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:27 smithi137 ceph-mon[112295]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:27 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3555612327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:26:27.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:27 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/47810241' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:26:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.531+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f785004ea90 con 0x7f78500ffbd0 2024-06-15T01:26:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.531+0000 7f784d7fa700 1 -- 172.21.15.137:0/3123405444 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f7840016330 con 0x7f78500ffbd0 2024-06-15T01:26:27.516 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:26:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:26:27.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 msgr2=0x7f783c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 0x7f783c06d440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f783800b5c0 tx=0x7f783801a040 comp rx=0 tx=0).stop 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 msgr2=0x7f785019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f784000d8d0 tx=0x7f784000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 shutdown_connections 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f783c06af80 0x7f783c06d440 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78500ffbd0 0x7f785019c510 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.533+0000 7f785603c700 1 --2- 172.21.15.137:0/3123405444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7850105a40 0x7f785019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.534+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 >> 172.21.15.137:0/3123405444 conn(0x7f78500755a0 msgr2=0x7f7850103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.534+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 shutdown_connections 2024-06-15T01:26:27.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:27.534+0000 7f785603c700 1 -- 172.21.15.137:0/3123405444 wait complete. 2024-06-15T01:26:27.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:27 smithi142 ceph-mon[82136]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:27 smithi142 ceph-mon[82136]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:27 smithi142 ceph-mon[82136]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:27.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:27 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3555612327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:26:27.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:27 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/47810241' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:26:28.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:28 smithi142 ceph-mon[82136]: pgmap v611: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:28.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:28 smithi142 ceph-mon[82136]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:28.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:28 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3123405444' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:26:28.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:28 smithi137 ceph-mon[112295]: pgmap v611: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:28.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:28 smithi137 ceph-mon[112295]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:28.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:28 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3123405444' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:26:30.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:30 smithi142 ceph-mon[82136]: pgmap v612: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:30.652 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:30 smithi137 ceph-mon[112295]: pgmap v612: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:32.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:32 smithi142 ceph-mon[82136]: pgmap v613: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:32.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:32 smithi137 ceph-mon[112295]: pgmap v613: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:33.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:33.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:34 smithi142 ceph-mon[82136]: pgmap v614: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:34.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:34 smithi137 ceph-mon[112295]: pgmap v614: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:36.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:36 smithi142 ceph-mon[82136]: pgmap v615: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:36 smithi137 ceph-mon[112295]: pgmap v615: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:38.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:38 smithi142 ceph-mon[82136]: pgmap v616: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:38.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:38 smithi137 ceph-mon[112295]: pgmap v616: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:40.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:40 smithi142 ceph-mon[82136]: pgmap v617: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:40.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:40 smithi137 ceph-mon[112295]: pgmap v617: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:42.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:42 smithi142 ceph-mon[82136]: pgmap v618: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:42.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:42 smithi137 ceph-mon[112295]: pgmap v618: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:44.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:44 smithi142 ceph-mon[82136]: pgmap v619: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:44.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:44 smithi137 ceph-mon[112295]: pgmap v619: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:46.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:46 smithi142 ceph-mon[82136]: pgmap v620: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:46.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:46 smithi137 ceph-mon[112295]: pgmap v620: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:48 smithi142 ceph-mon[82136]: pgmap v621: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:48.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:48.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:48 smithi137 ceph-mon[112295]: pgmap v621: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:48.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:26:50.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:50 smithi142 ceph-mon[82136]: pgmap v622: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:50.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:50 smithi137 ceph-mon[112295]: pgmap v622: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:52.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:52 smithi142 ceph-mon[82136]: pgmap v623: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:52.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:52 smithi137 ceph-mon[112295]: pgmap v623: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:54.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:54 smithi137 ceph-mon[112295]: pgmap v624: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:54.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:54 smithi142 ceph-mon[82136]: pgmap v624: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:56.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:56 smithi137 ceph-mon[112295]: pgmap v625: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:56.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:56 smithi142 ceph-mon[82136]: pgmap v625: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- 172.21.15.137:0/2652902530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c100400 msgr2=0x7efc0c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 --2- 172.21.15.137:0/2652902530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c100400 0x7efc0c1048d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009910 tx=0x7efbfc009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- 172.21.15.137:0/2652902530 shutdown_connections 2024-06-15T01:26:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 --2- 172.21.15.137:0/2652902530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c100400 0x7efc0c1048d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 --2- 172.21.15.137:0/2652902530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c0ffa50 0x7efc0c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- 172.21.15.137:0/2652902530 >> 172.21.15.137:0/2652902530 conn(0x7efc0c0fb520 msgr2=0x7efc0c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- 172.21.15.137:0/2652902530 shutdown_connections 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- 172.21.15.137:0/2652902530 wait complete. 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 Processor -- start 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.650+0000 7efc137a8700 1 -- start start 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc137a8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc137a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c100400 0x7efc0c1a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc137a8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c1a3620 con 0x7efc0c0ffa50 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc137a8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c19cad0 con 0x7efc0c100400 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36816/0 (socket says 172.21.15.137:36816) 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 -- 172.21.15.137:0/808775805 learned_addr learned my addr 172.21.15.137:0/808775805 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc10d43700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c100400 0x7efc0c1a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 -- 172.21.15.137:0/808775805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c100400 msgr2=0x7efc0c1a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c100400 0x7efc0c1a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 -- 172.21.15.137:0/808775805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0095f0 con 0x7efc0c0ffa50 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc11544700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c8f0 tx=0x7efc0800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.651+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc08004a00 con 0x7efc0c0ffa50 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.652+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c19cdb0 con 0x7efc0c0ffa50 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.652+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc08005020 con 0x7efc0c0ffa50 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.652+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc08007c40 con 0x7efc0c0ffa50 2024-06-15T01:26:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.652+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c19d300 con 0x7efc0c0ffa50 2024-06-15T01:26:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.653+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c102030 con 0x7efc0c0ffa50 2024-06-15T01:26:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.653+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efc08005190 con 0x7efc0c0ffa50 2024-06-15T01:26:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.653+0000 7efc027fc700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 0x7efbf806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.654+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efc08013070 con 0x7efc0c0ffa50 2024-06-15T01:26:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.654+0000 7efc10d43700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 0x7efbf806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.654+0000 7efc10d43700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 0x7efbf806d440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0fccb0 tx=0x7efbfc01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.658+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efc08058aa0 con 0x7efc0c0ffa50 2024-06-15T01:26:57.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.866+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc0c19d5d0 con 0x7efbf806af80 2024-06-15T01:26:57.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.868+0000 7efc027fc700 1 -- 172.21.15.137:0/808775805 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efc0c19d5d0 con 0x7efbf806af80 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 msgr2=0x7efbf806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 0x7efbf806d440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0fccb0 tx=0x7efbfc01a040 comp rx=0 tx=0).stop 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 msgr2=0x7efc0c1a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c8f0 tx=0x7efc0800cc00 comp rx=0 tx=0).stop 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 shutdown_connections 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efbf806af80 0x7efbf806d440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c0ffa50 0x7efc0c1a2a50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 --2- 172.21.15.137:0/808775805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc0c100400 0x7efc0c1a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 >> 172.21.15.137:0/808775805 conn(0x7efc0c0fb520 msgr2=0x7efc0c110f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 shutdown_connections 2024-06-15T01:26:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.870+0000 7efc137a8700 1 -- 172.21.15.137:0/808775805 wait complete. 2024-06-15T01:26:57.866 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:26:57.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 -- 172.21.15.137:0/1966526932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c105a40 msgr2=0x7f738c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 --2- 172.21.15.137:0/1966526932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c105a40 0x7f738c105e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7380009910 tx=0x7f7380009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 -- 172.21.15.137:0/1966526932 shutdown_connections 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 --2- 172.21.15.137:0/1966526932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c0ffa40 0x7f738c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 --2- 172.21.15.137:0/1966526932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c105a40 0x7f738c105e20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 -- 172.21.15.137:0/1966526932 >> 172.21.15.137:0/1966526932 conn(0x7f738c0fb550 msgr2=0x7f738c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 -- 172.21.15.137:0/1966526932 shutdown_connections 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.976+0000 7f739186e700 1 -- 172.21.15.137:0/1966526932 wait complete. 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 Processor -- start 2024-06-15T01:26:57.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 -- start start 2024-06-15T01:26:57.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c105a40 0x7f738c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c198060 con 0x7f738c0ffa40 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f739186e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c1981a0 con 0x7f738c105a40 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f738affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f738affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36834/0 (socket says 172.21.15.137:36834) 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f738affd700 1 -- 172.21.15.137:0/3275853736 learned_addr learned my addr 172.21.15.137:0/3275853736 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.977+0000 7f738a7fc700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c105a40 0x7f738c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f738affd700 1 -- 172.21.15.137:0/3275853736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c105a40 msgr2=0x7f738c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f738affd700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c105a40 0x7f738c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f738affd700 1 -- 172.21.15.137:0/3275853736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800095f0 con 0x7f738c0ffa40 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f738affd700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7380006010 tx=0x7f738000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738001d070 con 0x7f738c0ffa40 2024-06-15T01:26:57.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f738000f460 con 0x7f738c0ffa40 2024-06-15T01:26:57.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c1a0570 con 0x7f738c0ffa40 2024-06-15T01:26:57.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7380021620 con 0x7f738c0ffa40 2024-06-15T01:26:57.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.978+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1a0980 con 0x7f738c0ffa40 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.979+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c108040 con 0x7f738c0ffa40 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.980+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f738000f5d0 con 0x7f738c0ffa40 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.980+0000 7f739086c700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 0x7f737406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.980+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f738008b5f0 con 0x7f738c0ffa40 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.980+0000 7f738a7fc700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 0x7f737406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:57.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.981+0000 7f738a7fc700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 0x7f737406d440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f738c069660 tx=0x7f737c00a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:57.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:57.984+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7380056540 con 0x7f738c0ffa40 2024-06-15T01:26:58.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.194+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f738c02cd30 con 0x7f737406af80 2024-06-15T01:26:58.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.195+0000 7f739086c700 1 -- 172.21.15.137:0/3275853736 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f738c02cd30 con 0x7f737406af80 2024-06-15T01:26:58.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 msgr2=0x7f737406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 0x7f737406d440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f738c069660 tx=0x7f737c00a300 comp rx=0 tx=0).stop 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 msgr2=0x7f738c197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7380006010 tx=0x7f738000bd70 comp rx=0 tx=0).stop 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 shutdown_connections 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f737406af80 0x7f737406d440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f738c0ffa40 0x7f738c197500 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 --2- 172.21.15.137:0/3275853736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738c105a40 0x7f738c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 >> 172.21.15.137:0/3275853736 conn(0x7f738c0fb550 msgr2=0x7f738c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 shutdown_connections 2024-06-15T01:26:58.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.197+0000 7f739186e700 1 -- 172.21.15.137:0/3275853736 wait complete. 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 -- 172.21.15.137:0/2187653722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370101780 msgr2=0x7fa3701057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 --2- 172.21.15.137:0/2187653722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370101780 0x7fa3701057d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009910 tx=0x7fa36c009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 -- 172.21.15.137:0/2187653722 shutdown_connections 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 --2- 172.21.15.137:0/2187653722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370101780 0x7fa3701057d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 --2- 172.21.15.137:0/2187653722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370100dd0 0x7fa3701011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 -- 172.21.15.137:0/2187653722 >> 172.21.15.137:0/2187653722 conn(0x7fa3700fc640 msgr2=0x7fa3700fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 -- 172.21.15.137:0/2187653722 shutdown_connections 2024-06-15T01:26:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.301+0000 7fa37729b700 1 -- 172.21.15.137:0/2187653722 wait complete. 2024-06-15T01:26:58.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 Processor -- start 2024-06-15T01:26:58.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 -- start start 2024-06-15T01:26:58.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 0x7fa370196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701972d0 con 0x7fa370100dd0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa37729b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37019a000 con 0x7fa370101780 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36860/0 (socket says 172.21.15.137:36860) 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 -- 172.21.15.137:0/4046931098 learned_addr learned my addr 172.21.15.137:0/4046931098 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa374836700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 0x7fa370196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 -- 172.21.15.137:0/4046931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 msgr2=0x7fa370196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 0x7fa370196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 -- 172.21.15.137:0/4046931098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0095f0 con 0x7fa370100dd0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa374836700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 0x7fa370196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.302+0000 7fa375037700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d8d0 tx=0x7fa36000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.303+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa360009940 con 0x7fa370100dd0 2024-06-15T01:26:58.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.303+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa360010460 con 0x7fa370100dd0 2024-06-15T01:26:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.303+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37019a2e0 con 0x7fa370100dd0 2024-06-15T01:26:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.303+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa36000f5f0 con 0x7fa370100dd0 2024-06-15T01:26:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.303+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37019a800 con 0x7fa370100dd0 2024-06-15T01:26:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.304+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3701091d0 con 0x7fa370100dd0 2024-06-15T01:26:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.304+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa3600105d0 con 0x7fa370100dd0 2024-06-15T01:26:58.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.305+0000 7fa3667fc700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 0x7fa35c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.305+0000 7fa374836700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 0x7fa35c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.305+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa36008a3e0 con 0x7fa370100dd0 2024-06-15T01:26:58.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.305+0000 7fa374836700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 0x7fa35c06d440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa370197e20 tx=0x7fa36c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:58.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.309+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa360055210 con 0x7fa370100dd0 2024-06-15T01:26:58.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.510+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa37019aa20 con 0x7fa35c06af80 2024-06-15T01:26:58.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.519+0000 7fa3667fc700 1 -- 172.21.15.137:0/4046931098 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa37019aa20 con 0x7fa35c06af80 2024-06-15T01:26:58.505 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:26:58.505 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (17m) 11m ago 28m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (28m) 11m ago 28m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (27m) 11m ago 27m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (15m) 11m ago 28m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (15m) 11m ago 27m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (17m) 11m ago 27m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (12m) 11m ago 22m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (12m) 11m ago 22m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (11m) 11m ago 22m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (11m) 11m ago 22m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (19m) 11m ago 29m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (18m) 11m ago 27m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (16m) 11m ago 29m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:26:58.506 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (15m) 11m ago 26m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (18m) 11m ago 28m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (18m) 11m ago 27m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (15m) 11m ago 26m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (14m) 11m ago 25m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (14m) 11m ago 25m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (13m) 11m ago 24m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (13m) 11m ago 24m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (12m) 11m ago 24m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:26:58.507 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (17m) 11m ago 27m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.521+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 msgr2=0x7fa35c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.521+0000 7fa37729b700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 0x7fa35c06d440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa370197e20 tx=0x7fa36c01a040 comp rx=0 tx=0).stop 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.521+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 msgr2=0x7fa370196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.521+0000 7fa37729b700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d8d0 tx=0x7fa36000dc90 comp rx=0 tx=0).stop 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.521+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 shutdown_connections 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa35c06af80 0x7fa35c06d440 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa370100dd0 0x7fa370196700 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 --2- 172.21.15.137:0/4046931098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa370101780 0x7fa370196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 >> 172.21.15.137:0/4046931098 conn(0x7fa3700fc640 msgr2=0x7fa3700fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 shutdown_connections 2024-06-15T01:26:58.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.522+0000 7fa37729b700 1 -- 172.21.15.137:0/4046931098 wait complete. 2024-06-15T01:26:58.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 -- 172.21.15.137:0/1982768373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 msgr2=0x7f42381057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 --2- 172.21.15.137:0/1982768373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42381057a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f422c009910 tx=0x7f422c009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:58.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 -- 172.21.15.137:0/1982768373 shutdown_connections 2024-06-15T01:26:58.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 --2- 172.21.15.137:0/1982768373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42381057a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 --2- 172.21.15.137:0/1982768373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 0x7f4238101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.624+0000 7f423fed0700 1 -- 172.21.15.137:0/1982768373 >> 172.21.15.137:0/1982768373 conn(0x7f42380fc650 msgr2=0x7f42380fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.611 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:58 smithi137 ceph-mon[112295]: pgmap v626: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:58.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 -- 172.21.15.137:0/1982768373 shutdown_connections 2024-06-15T01:26:58.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 -- 172.21.15.137:0/1982768373 wait complete. 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 Processor -- start 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 -- start start 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 0x7f4238073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238073b10 con 0x7f4238101750 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.625+0000 7f423fed0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238073c50 con 0x7f4238100da0 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36892/0 (socket says 172.21.15.137:36892) 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 -- 172.21.15.137:0/4108414932 learned_addr learned my addr 172.21.15.137:0/4108414932 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423dc6c700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 0x7f4238073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 -- 172.21.15.137:0/4108414932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 msgr2=0x7f4238073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 0x7f4238073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 -- 172.21.15.137:0/4108414932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0095f0 con 0x7f4238101750 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423d46b700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f422c0092e0 tx=0x7f422c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f422c01d070 con 0x7f4238101750 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f422c005dc0 con 0x7f4238101750 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4238073ed0 con 0x7f4238101750 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f422c00fea0 con 0x7f4238101750 2024-06-15T01:26:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.626+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42381a2b70 con 0x7f4238101750 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.627+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42381091a0 con 0x7f4238101750 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.628+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f422c003a60 con 0x7f4238101750 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.628+0000 7f422affd700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 0x7f422406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.629+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f422c08b4f0 con 0x7f4238101750 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.629+0000 7f423dc6c700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 0x7f422406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.629+0000 7f423dc6c700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 0x7f422406d440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4234005950 tx=0x7f423400b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:58.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.633+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f422c056320 con 0x7f4238101750 2024-06-15T01:26:58.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:58 smithi142 ceph-mon[82136]: pgmap v626: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:26:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.904+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f423804ea90 con 0x7f4238101750 2024-06-15T01:26:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.905+0000 7f422affd700 1 -- 172.21.15.137:0/4108414932 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f422c059940 con 0x7f4238101750 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:58.891 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:26:58.892 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 msgr2=0x7f422406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 0x7f422406d440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4234005950 tx=0x7f423400b410 comp rx=0 tx=0).stop 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 msgr2=0x7f42380735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f422c0092e0 tx=0x7f422c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 shutdown_connections 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f422406af80 0x7f422406d440 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4238100da0 0x7f4238073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 --2- 172.21.15.137:0/4108414932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4238101750 0x7f42380735d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 >> 172.21.15.137:0/4108414932 conn(0x7f42380fc650 msgr2=0x7f42380fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 shutdown_connections 2024-06-15T01:26:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:58.907+0000 7f423fed0700 1 -- 172.21.15.137:0/4108414932 wait complete. 2024-06-15T01:26:58.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 -- 172.21.15.137:0/2886485865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c100da0 msgr2=0x7efe7c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:58.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 --2- 172.21.15.137:0/2886485865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c100da0 0x7efe7c101180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe78009910 tx=0x7efe78009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:58.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 -- 172.21.15.137:0/2886485865 shutdown_connections 2024-06-15T01:26:58.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 --2- 172.21.15.137:0/2886485865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c101750 0x7efe7c1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 --2- 172.21.15.137:0/2886485865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c100da0 0x7efe7c101180 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 -- 172.21.15.137:0/2886485865 >> 172.21.15.137:0/2886485865 conn(0x7efe7c0fc650 msgr2=0x7efe7c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 -- 172.21.15.137:0/2886485865 shutdown_connections 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.011+0000 7efe84069700 1 -- 172.21.15.137:0/2886485865 wait complete. 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 Processor -- start 2024-06-15T01:26:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 -- start start 2024-06-15T01:26:58.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c100da0 0x7efe7c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:58.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c19d0d0 con 0x7efe7c101750 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe84069700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c196580 con 0x7efe7c100da0 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36914/0 (socket says 172.21.15.137:36914) 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 -- 172.21.15.137:0/2125776051 learned_addr learned my addr 172.21.15.137:0/2125776051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81e05700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c100da0 0x7efe7c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 -- 172.21.15.137:0/2125776051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c100da0 msgr2=0x7efe7c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c100da0 0x7efe7c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 -- 172.21.15.137:0/2125776051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780095f0 con 0x7efe7c101750 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.012+0000 7efe81604700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c8f0 tx=0x7efe6c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.013+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6c0078a0 con 0x7efe7c101750 2024-06-15T01:26:59.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.013+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe6c00f450 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.013+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c196860 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.013+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6c007ea0 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.013+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c196db0 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.014+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c109140 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.014+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7efe6c00f5c0 con 0x7efe7c101750 2024-06-15T01:26:59.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.015+0000 7efe72ffd700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 0x7efe6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.015+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efe6c08a290 con 0x7efe7c101750 2024-06-15T01:26:59.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.015+0000 7efe81e05700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 0x7efe6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.015+0000 7efe81e05700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 0x7efe6806d440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efe78005f50 tx=0x7efe7801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.019+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efe6c058b50 con 0x7efe7c101750 2024-06-15T01:26:59.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.250+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe7c197030 con 0x7efe7c101750 2024-06-15T01:26:59.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.251+0000 7efe72ffd700 1 -- 172.21.15.137:0/2125776051 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7efe6c0586e0 con 0x7efe7c101750 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:26:59.237 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:26:59.238 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:59.239 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:59.240 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:59.240 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:26:59.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 msgr2=0x7efe6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 0x7efe6806d440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efe78005f50 tx=0x7efe7801a040 comp rx=0 tx=0).stop 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 msgr2=0x7efe7c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c8f0 tx=0x7efe6c00cc00 comp rx=0 tx=0).stop 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 shutdown_connections 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7efe6806af80 0x7efe6806d440 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c100da0 0x7efe7c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 --2- 172.21.15.137:0/2125776051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe7c101750 0x7efe7c19ca40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 >> 172.21.15.137:0/2125776051 conn(0x7efe7c0fc650 msgr2=0x7efe7c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 shutdown_connections 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.253+0000 7efe84069700 1 -- 172.21.15.137:0/2125776051 wait complete. 2024-06-15T01:26:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:26:59.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- 172.21.15.137:0/748408852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 msgr2=0x7fe94c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 --2- 172.21.15.137:0/748408852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c1057c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe934009910 tx=0x7fe934009c20 comp rx=0 tx=0).stop 2024-06-15T01:26:59.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- 172.21.15.137:0/748408852 shutdown_connections 2024-06-15T01:26:59.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 --2- 172.21.15.137:0/748408852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c101770 0x7fe94c1057c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 --2- 172.21.15.137:0/748408852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c100dc0 0x7fe94c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- 172.21.15.137:0/748408852 >> 172.21.15.137:0/748408852 conn(0x7fe94c0fc650 msgr2=0x7fe94c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- 172.21.15.137:0/748408852 shutdown_connections 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- 172.21.15.137:0/748408852 wait complete. 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 Processor -- start 2024-06-15T01:26:59.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.359+0000 7fe951f52700 1 -- start start 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe951f52700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe951f52700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe951f52700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c1972a0 con 0x7fe94c100dc0 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe951f52700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c199fd0 con 0x7fe94c101770 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36944/0 (socket says 172.21.15.137:36944) 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 -- 172.21.15.137:0/3314306996 learned_addr learned my addr 172.21.15.137:0/3314306996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe942ffd700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 -- 172.21.15.137:0/3314306996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 msgr2=0x7fe94c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 -- 172.21.15.137:0/3314306996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340095f0 con 0x7fe94c100dc0 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe942ffd700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.360+0000 7fe94b7fe700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00d8d0 tx=0x7fe93c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.361+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93c009940 con 0x7fe94c100dc0 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.361+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe93c010460 con 0x7fe94c100dc0 2024-06-15T01:26:59.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.361+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c19a2b0 con 0x7fe94c100dc0 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.361+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93c00f5f0 con 0x7fe94c100dc0 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.361+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c19a7d0 con 0x7fe94c100dc0 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.362+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c1091c0 con 0x7fe94c100dc0 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.362+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe93c0105d0 con 0x7fe94c100dc0 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.362+0000 7fe9497fa700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.363+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe93c08a3e0 con 0x7fe94c100dc0 2024-06-15T01:26:59.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.363+0000 7fe942ffd700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.363+0000 7fe942ffd700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe94c197df0 tx=0x7fe93401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.367+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe93c055210 con 0x7fe94c100dc0 2024-06-15T01:26:59.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.574+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe94c19a9f0 con 0x7fe93806af80 2024-06-15T01:26:59.560 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:59 smithi137 ceph-mon[112295]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:59.560 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:59 smithi137 ceph-mon[112295]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:59.560 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:59 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4108414932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:26:59.560 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:26:59 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2125776051' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.575+0000 7fe9497fa700 1 -- 172.21.15.137:0/3314306996 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe94c19a9f0 con 0x7fe93806af80 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:26:59.561 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:26:59.562 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 msgr2=0x7fe93806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe94c197df0 tx=0x7fe93401a040 comp rx=0 tx=0).stop 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 msgr2=0x7fe94c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00d8d0 tx=0x7fe93c00dc90 comp rx=0 tx=0).stop 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 shutdown_connections 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe93806af80 0x7fe93806d440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe94c100dc0 0x7fe94c1966d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 --2- 172.21.15.137:0/3314306996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe94c101770 0x7fe94c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 >> 172.21.15.137:0/3314306996 conn(0x7fe94c0fc650 msgr2=0x7fe94c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 shutdown_connections 2024-06-15T01:26:59.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.578+0000 7fe951f52700 1 -- 172.21.15.137:0/3314306996 wait complete. 2024-06-15T01:26:59.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 -- 172.21.15.137:0/3719281468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 msgr2=0x7f4d140ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3719281468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d140ff8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400aa20 tx=0x7f4d0400ad30 comp rx=0 tx=0).stop 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 -- 172.21.15.137:0/3719281468 shutdown_connections 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3719281468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3719281468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d140ff8f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.681+0000 7f4d19aef700 1 -- 172.21.15.137:0/3719281468 >> 172.21.15.137:0/3719281468 conn(0x7f4d14074d80 msgr2=0x7f4d14075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 -- 172.21.15.137:0/3719281468 shutdown_connections 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 -- 172.21.15.137:0/3719281468 wait complete. 2024-06-15T01:26:59.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 Processor -- start 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 -- start start 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14101980 con 0x7f4d140ff510 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.682+0000 7f4d19aef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14101ac0 con 0x7f4d140ffec0 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36960/0 (socket says 172.21.15.137:36960) 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 -- 172.21.15.137:0/3640715353 learned_addr learned my addr 172.21.15.137:0/3640715353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d12ffd700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 -- 172.21.15.137:0/3640715353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 msgr2=0x7f4d14101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 -- 172.21.15.137:0/3640715353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0400a700 con 0x7f4d140ff510 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d12ffd700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d137fe700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006010 tx=0x7f4d0400fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0400ccc0 con 0x7f4d140ff510 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d04005070 con 0x7f4d140ff510 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d14101d40 con 0x7f4d140ff510 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d040166f0 con 0x7f4d140ff510 2024-06-15T01:26:59.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.683+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d141a2b00 con 0x7f4d140ff510 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.684+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d141097e0 con 0x7f4d140ff510 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.685+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4d04016960 con 0x7f4d140ff510 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.685+0000 7f4d10ff9700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.685+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4d04012070 con 0x7f4d140ff510 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.685+0000 7f4d12ffd700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:26:59.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.686+0000 7f4d12ffd700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005950 tx=0x7f4d080058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:26:59.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.689+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4d040bf900 con 0x7f4d140ff510 2024-06-15T01:26:59.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:59 smithi142 ceph-mon[82136]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:59.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:59 smithi142 ceph-mon[82136]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:26:59.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:59 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4108414932' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:26:59.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:26:59 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2125776051' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:26:59.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.958+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d1402cd30 con 0x7f4d140ff510 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.959+0000 7f4d10ff9700 1 -- 172.21.15.137:0/3640715353 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f4d04056f10 con 0x7f4d140ff510 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:26:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:26:59.946 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:26:59.946 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:26:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 msgr2=0x7f4cfc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005950 tx=0x7f4d080058e0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 msgr2=0x7f4d14100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:26:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006010 tx=0x7f4d0400fe40 comp rx=0 tx=0).stop 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 shutdown_connections 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4cfc06af80 0x7f4cfc06d440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d140ff510 0x7f4d14100f00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 --2- 172.21.15.137:0/3640715353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d140ffec0 0x7f4d14101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.961+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 >> 172.21.15.137:0/3640715353 conn(0x7f4d14074d80 msgr2=0x7f4d140fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.962+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 shutdown_connections 2024-06-15T01:26:59.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:26:59.962+0000 7f4d19aef700 1 -- 172.21.15.137:0/3640715353 wait complete. 2024-06-15T01:27:00.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:00 smithi137 ceph-mon[112295]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:00.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:00 smithi137 ceph-mon[112295]: pgmap v627: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:00.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:00 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3640715353' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:27:00.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:00 smithi142 ceph-mon[82136]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:00.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:00 smithi142 ceph-mon[82136]: pgmap v627: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:00.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:00 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3640715353' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:27:01.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:01 smithi137 ceph-mon[112295]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:01.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:01 smithi142 ceph-mon[82136]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:02.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:02 smithi142 ceph-mon[82136]: pgmap v628: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:02.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:02 smithi137 ceph-mon[112295]: pgmap v628: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:03.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:03.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:04.717 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:04 smithi137 ceph-mon[112295]: pgmap v629: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:04.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:04 smithi142 ceph-mon[82136]: pgmap v629: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:06.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:06 smithi137 ceph-mon[112295]: pgmap v630: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:06.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:06 smithi142 ceph-mon[82136]: pgmap v630: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:08.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:08 smithi137 ceph-mon[112295]: pgmap v631: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:08.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:08 smithi142 ceph-mon[82136]: pgmap v631: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:10.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:10 smithi137 ceph-mon[112295]: pgmap v632: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:10.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:10 smithi142 ceph-mon[82136]: pgmap v632: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:12.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:12 smithi137 ceph-mon[112295]: pgmap v633: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:12.794 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:12 smithi142 ceph-mon[82136]: pgmap v633: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:14.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:14 smithi137 ceph-mon[112295]: pgmap v634: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:14.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:14 smithi142 ceph-mon[82136]: pgmap v634: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:16.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:16 smithi137 ceph-mon[112295]: pgmap v635: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:16.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:16 smithi142 ceph-mon[82136]: pgmap v635: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:18.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:18 smithi137 ceph-mon[112295]: pgmap v636: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:18.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:18.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:18 smithi142 ceph-mon[82136]: pgmap v636: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:18.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:20.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:20 smithi137 ceph-mon[112295]: pgmap v637: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:20.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:20 smithi142 ceph-mon[82136]: pgmap v637: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:22.715 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:22 smithi137 ceph-mon[112295]: pgmap v638: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:22.807 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:22 smithi142 ceph-mon[82136]: pgmap v638: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:24.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:24 smithi137 ceph-mon[112295]: pgmap v639: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:24.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:24 smithi142 ceph-mon[82136]: pgmap v639: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:25.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:25 smithi137 ceph-mon[112295]: pgmap v640: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:25.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:25 smithi142 ceph-mon[82136]: pgmap v640: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:28.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:27 smithi137 ceph-mon[112295]: pgmap v641: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:27 smithi142 ceph-mon[82136]: pgmap v641: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:30.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 -- 172.21.15.137:0/805481276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 msgr2=0x7f6cb81048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 --2- 172.21.15.137:0/805481276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81048e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009910 tx=0x7f6cac009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 -- 172.21.15.137:0/805481276 shutdown_connections 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 --2- 172.21.15.137:0/805481276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81048e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 --2- 172.21.15.137:0/805481276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 0x7f6cb80ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 -- 172.21.15.137:0/805481276 >> 172.21.15.137:0/805481276 conn(0x7f6cb80fb530 msgr2=0x7f6cb80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 -- 172.21.15.137:0/805481276 shutdown_connections 2024-06-15T01:27:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.080+0000 7f6cbd240700 1 -- 172.21.15.137:0/805481276 wait complete. 2024-06-15T01:27:30.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 Processor -- start 2024-06-15T01:27:30.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 -- start start 2024-06-15T01:27:30.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 0x7f6cb81a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb81a3620 con 0x7f6cb8100410 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cbd240700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb819cad0 con 0x7f6cb80ffa60 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cb659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cb659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45276/0 (socket says 172.21.15.137:45276) 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cb659c700 1 -- 172.21.15.137:0/3356164420 learned_addr learned my addr 172.21.15.137:0/3356164420 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.081+0000 7f6cb6d9d700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 0x7f6cb81a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cb659c700 1 -- 172.21.15.137:0/3356164420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 msgr2=0x7f6cb81a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cb659c700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 0x7f6cb81a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cb659c700 1 -- 172.21.15.137:0/3356164420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0095f0 con 0x7f6cb8100410 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cb659c700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0092e0 tx=0x7f6cac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac01d070 con 0x7f6cb8100410 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cac005dc0 con 0x7f6cb8100410 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac00fea0 con 0x7f6cb8100410 2024-06-15T01:27:30.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb819cd50 con 0x7f6cb8100410 2024-06-15T01:27:30.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.082+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb819d1f0 con 0x7f6cb8100410 2024-06-15T01:27:30.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.083+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb8102040 con 0x7f6cb8100410 2024-06-15T01:27:30.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.084+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6cac003a60 con 0x7f6cb8100410 2024-06-15T01:27:30.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.084+0000 7f6c9ffff700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 0x7f6ca006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.084+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6cac08b470 con 0x7f6cb8100410 2024-06-15T01:27:30.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.084+0000 7f6cb6d9d700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 0x7f6ca006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.085+0000 7f6cb6d9d700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 0x7f6ca006d440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005950 tx=0x7f6ca800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.089+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6cac059db0 con 0x7f6cb8100410 2024-06-15T01:27:30.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:29 smithi137 ceph-mon[112295]: pgmap v642: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.300+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cb802cce0 con 0x7f6ca006af80 2024-06-15T01:27:30.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.301+0000 7f6c9ffff700 1 -- 172.21.15.137:0/3356164420 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6cb802cce0 con 0x7f6ca006af80 2024-06-15T01:27:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 msgr2=0x7f6ca006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 0x7f6ca006d440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005950 tx=0x7f6ca800b410 comp rx=0 tx=0).stop 2024-06-15T01:27:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 msgr2=0x7f6cb81a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0092e0 tx=0x7f6cac00bd70 comp rx=0 tx=0).stop 2024-06-15T01:27:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 shutdown_connections 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6ca006af80 0x7f6ca006d440 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb80ffa60 0x7f6cb81a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 --2- 172.21.15.137:0/3356164420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cb8100410 0x7f6cb81a2f90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 >> 172.21.15.137:0/3356164420 conn(0x7f6cb80fb530 msgr2=0x7f6cb8110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 shutdown_connections 2024-06-15T01:27:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.303+0000 7f6cbd240700 1 -- 172.21.15.137:0/3356164420 wait complete. 2024-06-15T01:27:30.298 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:27:30.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:29 smithi142 ceph-mon[82136]: pgmap v642: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:30.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/840134363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c74101d80 msgr2=0x7f2c74102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/840134363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c74101d80 0x7f2c74102160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009910 tx=0x7f2c70009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/840134363 shutdown_connections 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/840134363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c741026a0 0x7f2c7410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/840134363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c74101d80 0x7f2c74102160 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/840134363 >> 172.21.15.137:0/840134363 conn(0x7f2c740fb550 msgr2=0x7f2c740fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/840134363 shutdown_connections 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.409+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/840134363 wait complete. 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 Processor -- start 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 -- start start 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 0x7f2c74196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74197280 con 0x7f2c741026a0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c7b8ab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7419b010 con 0x7f2c74101d80 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45290/0 (socket says 172.21.15.137:45290) 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 -- 172.21.15.137:0/209970908 learned_addr learned my addr 172.21.15.137:0/209970908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c79647700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 0x7f2c74196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 -- 172.21.15.137:0/209970908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 msgr2=0x7f2c74196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 0x7f2c74196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 -- 172.21.15.137:0/209970908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700095f0 con 0x7f2c741026a0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c79647700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 0x7f2c74196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.410+0000 7f2c78e46700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e8f0 tx=0x7f2c6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.411+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c64009790 con 0x7f2c741026a0 2024-06-15T01:27:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.411+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c64004cd0 con 0x7f2c741026a0 2024-06-15T01:27:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.411+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7419b2f0 con 0x7f2c741026a0 2024-06-15T01:27:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.411+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c64005580 con 0x7f2c741026a0 2024-06-15T01:27:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.411+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7419b840 con 0x7f2c741026a0 2024-06-15T01:27:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.412+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74108290 con 0x7f2c741026a0 2024-06-15T01:27:30.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.412+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2c64004750 con 0x7f2c741026a0 2024-06-15T01:27:30.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.413+0000 7f2c6a7fc700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 0x7f2c6006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.413+0000 7f2c79647700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 0x7f2c6006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.413+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2c64014070 con 0x7f2c741026a0 2024-06-15T01:27:30.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.413+0000 7f2c79647700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 0x7f2c6006d440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c700059d0 tx=0x7f2c70005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.417+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2c64055540 con 0x7f2c741026a0 2024-06-15T01:27:30.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.625+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c7419bbe0 con 0x7f2c6006af80 2024-06-15T01:27:30.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.626+0000 7f2c6a7fc700 1 -- 172.21.15.137:0/209970908 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2c7419bbe0 con 0x7f2c6006af80 2024-06-15T01:27:30.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 msgr2=0x7f2c6006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 0x7f2c6006d440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c700059d0 tx=0x7f2c70005940 comp rx=0 tx=0).stop 2024-06-15T01:27:30.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 msgr2=0x7f2c74196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e8f0 tx=0x7f2c6400ecb0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 shutdown_connections 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2c6006af80 0x7f2c6006d440 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c74101d80 0x7f2c74196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 --2- 172.21.15.137:0/209970908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c741026a0 0x7f2c74196ba0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.629+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 >> 172.21.15.137:0/209970908 conn(0x7f2c740fb550 msgr2=0x7f2c741053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.630+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 shutdown_connections 2024-06-15T01:27:30.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.630+0000 7f2c7b8ab700 1 -- 172.21.15.137:0/209970908 wait complete. 2024-06-15T01:27:30.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 -- 172.21.15.137:0/343432411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4102640 msgr2=0x7f04f410ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/343432411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4102640 0x7f04f410ab30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009910 tx=0x7f04e8009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 -- 172.21.15.137:0/343432411 shutdown_connections 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/343432411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4102640 0x7f04f410ab30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/343432411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4101c90 0x7f04f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 -- 172.21.15.137:0/343432411 >> 172.21.15.137:0/343432411 conn(0x7f04f40fb550 msgr2=0x7f04f40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 -- 172.21.15.137:0/343432411 shutdown_connections 2024-06-15T01:27:30.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.733+0000 7f04f9d6b700 1 -- 172.21.15.137:0/343432411 wait complete. 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 Processor -- start 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 -- start start 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 0x7f04f4196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4197280 con 0x7f04f4101c90 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f9d6b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f419b010 con 0x7f04f4102640 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45322/0 (socket says 172.21.15.137:45322) 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 -- 172.21.15.137:0/3743070200 learned_addr learned my addr 172.21.15.137:0/3743070200 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f2ffd700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 0x7f04f4196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 -- 172.21.15.137:0/3743070200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 msgr2=0x7f04f4196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 0x7f04f4196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f37fe700 1 -- 172.21.15.137:0/3743070200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e80095f0 con 0x7f04f4101c90 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.734+0000 7f04f2ffd700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 0x7f04f4196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f37fe700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f04e40060b0 tx=0x7f04e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e400dda0 con 0x7f04f4101c90 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04e400f040 con 0x7f04f4101c90 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f419b2f0 con 0x7f04f4101c90 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e40146f0 con 0x7f04f4101c90 2024-06-15T01:27:30.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.735+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f419b840 con 0x7f04f4101c90 2024-06-15T01:27:30.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.736+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4108230 con 0x7f04f4101c90 2024-06-15T01:27:30.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.736+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f04e4015400 con 0x7f04f4101c90 2024-06-15T01:27:30.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.737+0000 7f04f0ff9700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 0x7f04dc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:30.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.737+0000 7f04f2ffd700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 0x7f04dc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:30.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.737+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f04e4089d40 con 0x7f04f4101c90 2024-06-15T01:27:30.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.737+0000 7f04f2ffd700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 0x7f04dc06d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f04e80059d0 tx=0x7f04e8005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:30.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.741+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f04e4054ca0 con 0x7f04f4101c90 2024-06-15T01:27:30.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.943+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f419bbe0 con 0x7f04dc06af80 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.953+0000 7f04f0ff9700 1 -- 172.21.15.137:0/3743070200 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f04f419bbe0 con 0x7f04dc06af80 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (18m) 12m ago 28m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (28m) 12m ago 28m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (27m) 12m ago 27m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (16m) 12m ago 28m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (16m) 12m ago 27m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (17m) 12m ago 28m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:27:30.939 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (12m) 12m ago 23m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (12m) 12m ago 23m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (12m) 12m ago 23m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (12m) 12m ago 22m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (19m) 12m ago 30m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (18m) 12m ago 27m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (16m) 12m ago 30m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (16m) 12m ago 27m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (18m) 12m ago 28m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (18m) 12m ago 27m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (15m) 12m ago 26m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (15m) 12m ago 26m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (14m) 12m ago 25m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (14m) 12m ago 25m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:27:30.940 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (13m) 12m ago 24m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (13m) 12m ago 24m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (18m) 12m ago 28m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 msgr2=0x7f04dc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 0x7f04dc06d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f04e80059d0 tx=0x7f04e8005960 comp rx=0 tx=0).stop 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 msgr2=0x7f04f4196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:30.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f04e40060b0 tx=0x7f04e400d3b0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 shutdown_connections 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f04dc06af80 0x7f04dc06d440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04f4101c90 0x7f04f4196660 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 --2- 172.21.15.137:0/3743070200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f4102640 0x7f04f4196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.955+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 >> 172.21.15.137:0/3743070200 conn(0x7f04f40fb550 msgr2=0x7f04f4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.956+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 shutdown_connections 2024-06-15T01:27:30.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:30.956+0000 7f04f9d6b700 1 -- 172.21.15.137:0/3743070200 wait complete. 2024-06-15T01:27:31.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 -- 172.21.15.137:0/2361450765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 msgr2=0x7fe6341011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 --2- 172.21.15.137:0/2361450765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341011c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009910 tx=0x7fe62c009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 -- 172.21.15.137:0/2361450765 shutdown_connections 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 --2- 172.21.15.137:0/2361450765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 0x7fe6341057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 --2- 172.21.15.137:0/2361450765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341011c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 -- 172.21.15.137:0/2361450765 >> 172.21.15.137:0/2361450765 conn(0x7fe6340fc6b0 msgr2=0x7fe6340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 -- 172.21.15.137:0/2361450765 shutdown_connections 2024-06-15T01:27:31.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.061+0000 7fe63ad72700 1 -- 172.21.15.137:0/2361450765 wait complete. 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 Processor -- start 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 -- start start 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 0x7fe634196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634197280 con 0x7fe634100de0 2024-06-15T01:27:31.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe63ad72700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634199fb0 con 0x7fe634101790 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45346/0 (socket says 172.21.15.137:45346) 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 -- 172.21.15.137:0/1721106927 learned_addr learned my addr 172.21.15.137:0/1721106927 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe633fff700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 0x7fe634196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 -- 172.21.15.137:0/1721106927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 msgr2=0x7fe634196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 0x7fe634196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 -- 172.21.15.137:0/1721106927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0095f0 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe638b0e700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b5c0 tx=0x7fe62c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.062+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c01d070 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.063+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63419a230 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.063+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe62c005e00 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.063+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c00fe90 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.063+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63419a720 con 0x7fe634100de0 2024-06-15T01:27:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.064+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634109180 con 0x7fe634100de0 2024-06-15T01:27:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.064+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe62c00f6d0 con 0x7fe634100de0 2024-06-15T01:27:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.065+0000 7fe631ffb700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 0x7fe62006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.065+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe62c08b5c0 con 0x7fe634100de0 2024-06-15T01:27:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.065+0000 7fe633fff700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 0x7fe62006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.065+0000 7fe633fff700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 0x7fe62006d440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe624009550 tx=0x7fe6240091b0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.069+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe62c056510 con 0x7fe634100de0 2024-06-15T01:27:31.197 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:30 smithi137 ceph-mon[112295]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:31.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:30 smithi142 ceph-mon[82136]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:31.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.349+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe63404ea90 con 0x7fe634100de0 2024-06-15T01:27:31.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.350+0000 7fe631ffb700 1 -- 172.21.15.137:0/1721106927 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe62c059b30 con 0x7fe634100de0 2024-06-15T01:27:31.335 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:27:31.336 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:27:31.337 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:27:31.337 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 msgr2=0x7fe62006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 0x7fe62006d440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe624009550 tx=0x7fe6240091b0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 msgr2=0x7fe6341966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b5c0 tx=0x7fe62c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 shutdown_connections 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe62006af80 0x7fe62006d440 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe634100de0 0x7fe6341966b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 --2- 172.21.15.137:0/1721106927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101790 0x7fe634196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 >> 172.21.15.137:0/1721106927 conn(0x7fe6340fc6b0 msgr2=0x7fe6340fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 shutdown_connections 2024-06-15T01:27:31.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.352+0000 7fe63ad72700 1 -- 172.21.15.137:0/1721106927 wait complete. 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 -- 172.21.15.137:0/3523714655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 msgr2=0x7fd38c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 --2- 172.21.15.137:0/3523714655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c1057a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd374009910 tx=0x7fd374009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 -- 172.21.15.137:0/3523714655 shutdown_connections 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 --2- 172.21.15.137:0/3523714655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c1057a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 --2- 172.21.15.137:0/3523714655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 0x7fd38c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 -- 172.21.15.137:0/3523714655 >> 172.21.15.137:0/3523714655 conn(0x7fd38c0fc630 msgr2=0x7fd38c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:31.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 -- 172.21.15.137:0/3523714655 shutdown_connections 2024-06-15T01:27:31.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.456+0000 7fd391681700 1 -- 172.21.15.137:0/3523714655 wait complete. 2024-06-15T01:27:31.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 Processor -- start 2024-06-15T01:27:31.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 -- start start 2024-06-15T01:27:31.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 0x7fd38c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c073b10 con 0x7fd38c101750 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd391681700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c073c50 con 0x7fd38c100da0 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45368/0 (socket says 172.21.15.137:45368) 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 -- 172.21.15.137:0/4244547675 learned_addr learned my addr 172.21.15.137:0/4244547675 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd38affd700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 0x7fd38c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 -- 172.21.15.137:0/4244547675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 msgr2=0x7fd38c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 0x7fd38c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 -- 172.21.15.137:0/4244547675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740095f0 con 0x7fd38c101750 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.457+0000 7fd383fff700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd374000c00 tx=0x7fd37400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.458+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37401d070 con 0x7fd38c101750 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.458+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd37400bed0 con 0x7fd38c101750 2024-06-15T01:27:31.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.458+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c073ed0 con 0x7fd38c101750 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.458+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37400fd90 con 0x7fd38c101750 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.458+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1a2af0 con 0x7fd38c101750 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.459+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c1091a0 con 0x7fd38c101750 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.459+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd374021410 con 0x7fd38c101750 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.459+0000 7fd388ff9700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 0x7fd37806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.460+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd37408b530 con 0x7fd38c101750 2024-06-15T01:27:31.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.460+0000 7fd38affd700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 0x7fd37806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.460+0000 7fd38affd700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 0x7fd37806d440 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009c60 tx=0x7fd37c009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.464+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd374056360 con 0x7fd38c101750 2024-06-15T01:27:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.696+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd38c04ea90 con 0x7fd38c101750 2024-06-15T01:27:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.696+0000 7fd388ff9700 1 -- 172.21.15.137:0/4244547675 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fd374059980 con 0x7fd38c101750 2024-06-15T01:27:31.682 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:27:31.682 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:27:31.682 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:27:31.683 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:27:31.684 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:27:31.685 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 msgr2=0x7fd37806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 0x7fd37806d440 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009c60 tx=0x7fd37c009260 comp rx=0 tx=0).stop 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 msgr2=0x7fd38c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd374000c00 tx=0x7fd37400ba70 comp rx=0 tx=0).stop 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 shutdown_connections 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd37806af80 0x7fd37806d440 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd38c100da0 0x7fd38c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 --2- 172.21.15.137:0/4244547675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd38c101750 0x7fd38c0735d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 >> 172.21.15.137:0/4244547675 conn(0x7fd38c0fc630 msgr2=0x7fd38c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:31.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.699+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 shutdown_connections 2024-06-15T01:27:31.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.700+0000 7fd391681700 1 -- 172.21.15.137:0/4244547675 wait complete. 2024-06-15T01:27:31.687 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:27:31.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 -- 172.21.15.137:0/1907321619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024105a40 msgr2=0x7fd024105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/1907321619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024105a40 0x7fd024105e20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd018009910 tx=0x7fd018009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 -- 172.21.15.137:0/1907321619 shutdown_connections 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/1907321619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0240ffbd0 0x7fd024100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/1907321619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024105a40 0x7fd024105e20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 -- 172.21.15.137:0/1907321619 >> 172.21.15.137:0/1907321619 conn(0x7fd0240755a0 msgr2=0x7fd0240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 -- 172.21.15.137:0/1907321619 shutdown_connections 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.805+0000 7fd02b1ac700 1 -- 172.21.15.137:0/1907321619 wait complete. 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 Processor -- start 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 -- start start 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd024105a40 0x7fd024101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd024102120 con 0x7fd0240ffbd0 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd02b1ac700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd024102290 con 0x7fd024105a40 2024-06-15T01:27:31.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45382/0 (socket says 172.21.15.137:45382) 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 -- 172.21.15.137:0/257628408 learned_addr learned my addr 172.21.15.137:0/257628408 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd023fff700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd024105a40 0x7fd024101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 -- 172.21.15.137:0/257628408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd024105a40 msgr2=0x7fd024101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd024105a40 0x7fd024101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.806+0000 7fd028f48700 1 -- 172.21.15.137:0/257628408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0180095f0 con 0x7fd0240ffbd0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd028f48700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd018000c00 tx=0x7fd01800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01801d070 con 0x7fd0240ffbd0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0180044c0 con 0x7fd0240ffbd0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0241a2810 con 0x7fd0240ffbd0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0180039d0 con 0x7fd0240ffbd0 2024-06-15T01:27:31.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.807+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0241a2b70 con 0x7fd0240ffbd0 2024-06-15T01:27:31.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.808+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd024109050 con 0x7fd0240ffbd0 2024-06-15T01:27:31.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.808+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd0180048e0 con 0x7fd0240ffbd0 2024-06-15T01:27:31.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.809+0000 7fd021ffb700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 0x7fd01406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:31.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.809+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd01808ba30 con 0x7fd0240ffbd0 2024-06-15T01:27:31.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.809+0000 7fd023fff700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 0x7fd01406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:31.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.809+0000 7fd023fff700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 0x7fd01406d440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd024102e00 tx=0x7fd010009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:31.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:31.813+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd018056860 con 0x7fd0240ffbd0 2024-06-15T01:27:32.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.024+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd02402cce0 con 0x7fd01406af80 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.025+0000 7fd021ffb700 1 -- 172.21.15.137:0/257628408 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd02402cce0 con 0x7fd01406af80 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:27:32.011 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:27:32.012 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:27:32.012 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:27:32.012 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:27:32.012 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 msgr2=0x7fd01406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 0x7fd01406d440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd024102e00 tx=0x7fd010009260 comp rx=0 tx=0).stop 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 msgr2=0x7fd024101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd018000c00 tx=0x7fd01800be90 comp rx=0 tx=0).stop 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 shutdown_connections 2024-06-15T01:27:32.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd01406af80 0x7fd01406d440 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0240ffbd0 0x7fd024101610 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 --2- 172.21.15.137:0/257628408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd024105a40 0x7fd024101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.027+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 >> 172.21.15.137:0/257628408 conn(0x7fd0240755a0 msgr2=0x7fd024068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:32.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.028+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 shutdown_connections 2024-06-15T01:27:32.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.028+0000 7fd02b1ac700 1 -- 172.21.15.137:0/257628408 wait complete. 2024-06-15T01:27:32.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.131+0000 7f1674f7b700 1 -- 172.21.15.137:0/434939873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 msgr2=0x7f1670102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.131+0000 7f1674f7b700 1 --2- 172.21.15.137:0/434939873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f1670102160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1660009910 tx=0x7f1660009c20 comp rx=0 tx=0).stop 2024-06-15T01:27:32.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 -- 172.21.15.137:0/434939873 shutdown_connections 2024-06-15T01:27:32.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 --2- 172.21.15.137:0/434939873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 --2- 172.21.15.137:0/434939873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f1670102160 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 -- 172.21.15.137:0/434939873 >> 172.21.15.137:0/434939873 conn(0x7f16700fb550 msgr2=0x7f16700fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:32.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:31 smithi137 ceph-mon[112295]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:32.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:31 smithi137 ceph-mon[112295]: pgmap v643: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:32.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:31 smithi137 ceph-mon[112295]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:32.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:31 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1721106927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:27:32.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:31 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4244547675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:27:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 -- 172.21.15.137:0/434939873 shutdown_connections 2024-06-15T01:27:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 -- 172.21.15.137:0/434939873 wait complete. 2024-06-15T01:27:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.132+0000 7f1674f7b700 1 Processor -- start 2024-06-15T01:27:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f1674f7b700 1 -- start start 2024-06-15T01:27:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f1674f7b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f1674f7b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f1674f7b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167019d1d0 con 0x7f1670101d80 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f1674f7b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1670196680 con 0x7f16701026a0 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45398/0 (socket says 172.21.15.137:45398) 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 -- 172.21.15.137:0/3306162751 learned_addr learned my addr 172.21.15.137:0/3306162751 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166dd9b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 -- 172.21.15.137:0/3306162751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 msgr2=0x7f167019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 -- 172.21.15.137:0/3306162751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600095f0 con 0x7f1670101d80 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166dd9b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.133+0000 7f166e59c700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1660006010 tx=0x7f166000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.134+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166001d070 con 0x7f1670101d80 2024-06-15T01:27:32.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.134+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f166000f460 con 0x7f1670101d80 2024-06-15T01:27:32.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.134+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1670196900 con 0x7f1670101d80 2024-06-15T01:27:32.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.134+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1660021640 con 0x7f1670101d80 2024-06-15T01:27:32.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.134+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1670196df0 con 0x7f1670101d80 2024-06-15T01:27:32.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.135+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1670108290 con 0x7f1670101d80 2024-06-15T01:27:32.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.135+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f166000f9e0 con 0x7f1670101d80 2024-06-15T01:27:32.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.136+0000 7f165f7fe700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 0x7f165806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:27:32.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.136+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f166008b620 con 0x7f1670101d80 2024-06-15T01:27:32.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.136+0000 7f166dd9b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 0x7f165806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:27:32.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.136+0000 7f166dd9b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 0x7f165806d440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16700fcd80 tx=0x7f1664008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:27:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.140+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1660059ee0 con 0x7f1670101d80 2024-06-15T01:27:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:31 smithi142 ceph-mon[82136]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:31 smithi142 ceph-mon[82136]: pgmap v643: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:31 smithi142 ceph-mon[82136]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:31 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1721106927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:27:32.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:31 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4244547675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:27:32.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.408+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f167004ea90 con 0x7f1670101d80 2024-06-15T01:27:32.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.409+0000 7f165f7fe700 1 -- 172.21.15.137:0/3306162751 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f166005ee60 con 0x7f1670101d80 2024-06-15T01:27:32.394 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:27:32.395 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:27:32.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 msgr2=0x7f165806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 0x7f165806d440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16700fcd80 tx=0x7f1664008040 comp rx=0 tx=0).stop 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 msgr2=0x7f167019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1660006010 tx=0x7f166000be20 comp rx=0 tx=0).stop 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 shutdown_connections 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f165806af80 0x7f165806d440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670101d80 0x7f167019c600 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 --2- 172.21.15.137:0/3306162751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16701026a0 0x7f167019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 >> 172.21.15.137:0/3306162751 conn(0x7f16700fb550 msgr2=0x7f16701053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 shutdown_connections 2024-06-15T01:27:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:27:32.411+0000 7f1674f7b700 1 -- 172.21.15.137:0/3306162751 wait complete. 2024-06-15T01:27:33.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:32 smithi137 ceph-mon[112295]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:33.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:32 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3306162751' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:27:33.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:32 smithi142 ceph-mon[82136]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:27:33.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:32 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3306162751' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:27:34.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:33 smithi137 ceph-mon[112295]: pgmap v644: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:34.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:34.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:33 smithi142 ceph-mon[82136]: pgmap v644: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:34.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:36.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:35 smithi137 ceph-mon[112295]: pgmap v645: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:36.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:35 smithi142 ceph-mon[82136]: pgmap v645: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:38.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:37 smithi142 ceph-mon[82136]: pgmap v646: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:38.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:37 smithi137 ceph-mon[112295]: pgmap v646: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:40.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:39 smithi142 ceph-mon[82136]: pgmap v647: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:40.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:39 smithi137 ceph-mon[112295]: pgmap v647: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:42.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:41 smithi137 ceph-mon[112295]: pgmap v648: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:41 smithi142 ceph-mon[82136]: pgmap v648: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:44.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:43 smithi137 ceph-mon[112295]: pgmap v649: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:44.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:43 smithi142 ceph-mon[82136]: pgmap v649: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:46.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:45 smithi142 ceph-mon[82136]: pgmap v650: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:46.334 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:45 smithi137 ceph-mon[112295]: pgmap v650: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:48.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:47 smithi142 ceph-mon[82136]: pgmap v651: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:48.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:47 smithi137 ceph-mon[112295]: pgmap v651: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:49.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:49.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:27:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:50 smithi142 ceph-mon[82136]: pgmap v652: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:50 smithi137 ceph-mon[112295]: pgmap v652: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:52.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:52 smithi137 ceph-mon[112295]: pgmap v653: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:52.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:52 smithi142 ceph-mon[82136]: pgmap v653: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:54.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:54 smithi137 ceph-mon[112295]: pgmap v654: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:54.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:54 smithi142 ceph-mon[82136]: pgmap v654: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:56.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:56 smithi142 ceph-mon[82136]: pgmap v655: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:56.425 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:56 smithi137 ceph-mon[112295]: pgmap v655: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:58.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:27:58 smithi142 ceph-mon[82136]: pgmap v656: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:27:58.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:27:58 smithi137 ceph-mon[112295]: pgmap v656: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:00.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:00 smithi142 ceph-mon[82136]: pgmap v657: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:00.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:00 smithi137 ceph-mon[112295]: pgmap v657: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:02.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:02 smithi137 ceph-mon[112295]: pgmap v658: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:02.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:02 smithi142 ceph-mon[82136]: pgmap v658: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 -- 172.21.15.137:0/1263289095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c102650 msgr2=0x7fdd2c10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 --2- 172.21.15.137:0/1263289095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c102650 0x7fdd2c10ab40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009910 tx=0x7fdd20009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 -- 172.21.15.137:0/1263289095 shutdown_connections 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 --2- 172.21.15.137:0/1263289095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c102650 0x7fdd2c10ab40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 --2- 172.21.15.137:0/1263289095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c101d30 0x7fdd2c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 -- 172.21.15.137:0/1263289095 >> 172.21.15.137:0/1263289095 conn(0x7fdd2c0fb560 msgr2=0x7fdd2c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 -- 172.21.15.137:0/1263289095 shutdown_connections 2024-06-15T01:28:02.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 -- 172.21.15.137:0/1263289095 wait complete. 2024-06-15T01:28:02.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.528+0000 7fdd31958700 1 Processor -- start 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd31958700 1 -- start start 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd31958700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd31958700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c102650 0x7fdd2c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd31958700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1972b0 con 0x7fdd2c101d30 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd31958700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c199fe0 con 0x7fdd2c102650 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:02.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41374/0 (socket says 172.21.15.137:41374) 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 -- 172.21.15.137:0/159928327 learned_addr learned my addr 172.21.15.137:0/159928327 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2a7fc700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c102650 0x7fdd2c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 -- 172.21.15.137:0/159928327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c102650 msgr2=0x7fdd2c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c102650 0x7fdd2c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.529+0000 7fdd2affd700 1 -- 172.21.15.137:0/159928327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200095f0 con 0x7fdd2c101d30 2024-06-15T01:28:02.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd2affd700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c007e90 tx=0x7fdd1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:02.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1c017070 con 0x7fdd2c101d30 2024-06-15T01:28:02.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c19a2c0 con 0x7fdd2c101d30 2024-06-15T01:28:02.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd1c00f040 con 0x7fdd2c101d30 2024-06-15T01:28:02.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1c005860 con 0x7fdd2c101d30 2024-06-15T01:28:02.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.530+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c19a7e0 con 0x7fdd2c101d30 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.531+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c108240 con 0x7fdd2c101d30 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.531+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fdd1c004c40 con 0x7fdd2c101d30 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.532+0000 7fdd30956700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 0x7fdd1406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.532+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fdd1c089f00 con 0x7fdd2c101d30 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.532+0000 7fdd2a7fc700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 0x7fdd1406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:02.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.533+0000 7fdd2a7fc700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 0x7fdd1406d440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd2001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:02.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.537+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd1c054d30 con 0x7fdd2c101d30 2024-06-15T01:28:02.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.745+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd2c19a9d0 con 0x7fdd1406af80 2024-06-15T01:28:02.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.747+0000 7fdd30956700 1 -- 172.21.15.137:0/159928327 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd2c19a9d0 con 0x7fdd1406af80 2024-06-15T01:28:02.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 msgr2=0x7fdd1406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 0x7fdd1406d440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd2001a040 comp rx=0 tx=0).stop 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 msgr2=0x7fdd2c1966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c007e90 tx=0x7fdd1c00d3b0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 shutdown_connections 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd1406af80 0x7fdd1406d440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd2c101d30 0x7fdd2c1966e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 --2- 172.21.15.137:0/159928327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2c102650 0x7fdd2c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 >> 172.21.15.137:0/159928327 conn(0x7fdd2c0fb560 msgr2=0x7fdd2c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 shutdown_connections 2024-06-15T01:28:02.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.749+0000 7fdd31958700 1 -- 172.21.15.137:0/159928327 wait complete. 2024-06-15T01:28:02.745 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 -- 172.21.15.137:0/918103396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950101750 msgr2=0x7f99501057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 --2- 172.21.15.137:0/918103396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950101750 0x7f99501057a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f994c009910 tx=0x7f994c009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 -- 172.21.15.137:0/918103396 shutdown_connections 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 --2- 172.21.15.137:0/918103396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950101750 0x7f99501057a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 --2- 172.21.15.137:0/918103396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9950100da0 0x7f9950101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 -- 172.21.15.137:0/918103396 >> 172.21.15.137:0/918103396 conn(0x7f99500fc630 msgr2=0x7f99500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 -- 172.21.15.137:0/918103396 shutdown_connections 2024-06-15T01:28:02.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.853+0000 7f99574f4700 1 -- 172.21.15.137:0/918103396 wait complete. 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 Processor -- start 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 -- start start 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9950101750 0x7f9950196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950197270 con 0x7f9950100da0 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f99574f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019b000 con 0x7f9950101750 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41386/0 (socket says 172.21.15.137:41386) 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 -- 172.21.15.137:0/1689813991 learned_addr learned my addr 172.21.15.137:0/1689813991 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 -- 172.21.15.137:0/1689813991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9950101750 msgr2=0x7f9950196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:02.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9950101750 0x7f9950196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.854+0000 7f9955290700 1 -- 172.21.15.137:0/1689813991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0095f0 con 0x7f9950100da0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f9955290700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f994000c8a0 tx=0x7f994000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994000cea0 con 0x7f9950100da0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9940004830 con 0x7f9950100da0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995019b2e0 con 0x7f9950100da0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940005500 con 0x7f9950100da0 2024-06-15T01:28:02.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.855+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995019b830 con 0x7f9950100da0 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.856+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99501091a0 con 0x7f9950100da0 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.856+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f99400049a0 con 0x7f9950100da0 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.857+0000 7f99467fc700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 0x7f993c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.857+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f994008a050 con 0x7f9950100da0 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.857+0000 7f9954a8f700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 0x7f993c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:02.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.857+0000 7f9954a8f700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 0x7f993c06d440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994c00b5c0 tx=0x7f994c005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:02.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:02.861+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9940054e80 con 0x7f9950100da0 2024-06-15T01:28:03.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.074+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f995019bb90 con 0x7f993c06af80 2024-06-15T01:28:03.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.075+0000 7f99467fc700 1 -- 172.21.15.137:0/1689813991 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f995019bb90 con 0x7f993c06af80 2024-06-15T01:28:03.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 msgr2=0x7f993c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 0x7f993c06d440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994c00b5c0 tx=0x7f994c005960 comp rx=0 tx=0).stop 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 msgr2=0x7f9950196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f994000c8a0 tx=0x7f994000cc60 comp rx=0 tx=0).stop 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 shutdown_connections 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993c06af80 0x7f993c06d440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9950100da0 0x7f9950196650 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 --2- 172.21.15.137:0/1689813991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9950101750 0x7f9950196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 >> 172.21.15.137:0/1689813991 conn(0x7f99500fc630 msgr2=0x7f99500fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 shutdown_connections 2024-06-15T01:28:03.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.078+0000 7f99574f4700 1 -- 172.21.15.137:0/1689813991 wait complete. 2024-06-15T01:28:03.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- 172.21.15.137:0/2472910669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 msgr2=0x7fc424105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/2472910669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424105e30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc414009910 tx=0x7fc414009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- 172.21.15.137:0/2472910669 shutdown_connections 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/2472910669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 0x7fc4240ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/2472910669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424105e30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- 172.21.15.137:0/2472910669 >> 172.21.15.137:0/2472910669 conn(0x7fc4240fb540 msgr2=0x7fc4240fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- 172.21.15.137:0/2472910669 shutdown_connections 2024-06-15T01:28:03.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- 172.21.15.137:0/2472910669 wait complete. 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 Processor -- start 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.183+0000 7fc42b3b7700 1 -- start start 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 0x7fc424196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424197260 con 0x7fc424105a50 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424199f90 con 0x7fc4240ffa50 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41408/0 (socket says 172.21.15.137:41408) 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 -- 172.21.15.137:0/3781372006 learned_addr learned my addr 172.21.15.137:0/3781372006 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc429153700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 0x7fc424196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 -- 172.21.15.137:0/3781372006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 msgr2=0x7fc424196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 0x7fc424196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 -- 172.21.15.137:0/3781372006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140095f0 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc428952700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc42000d8d0 tx=0x7fc42000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4200098a0 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc420010460 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc420009ea0 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42419a270 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.184+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42419a7c0 con 0x7fc424105a50 2024-06-15T01:28:03.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.185+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42404ea90 con 0x7fc424105a50 2024-06-15T01:28:03.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.186+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc4200105d0 con 0x7fc424105a50 2024-06-15T01:28:03.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.186+0000 7fc41a7fc700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 0x7fc41006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.186+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc42008a220 con 0x7fc424105a50 2024-06-15T01:28:03.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.186+0000 7fc429153700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 0x7fc41006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.187+0000 7fc429153700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 0x7fc41006d440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc414006010 tx=0x7fc41401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.190+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc420055030 con 0x7fc424105a50 2024-06-15T01:28:03.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:03.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.392+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc42419aba0 con 0x7fc41006af80 2024-06-15T01:28:03.378 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:03.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.401+0000 7fc41a7fc700 1 -- 172.21.15.137:0/3781372006 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc42419aba0 con 0x7fc41006af80 2024-06-15T01:28:03.387 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:28:03.387 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (18m) 13m ago 29m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (29m) 13m ago 29m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (28m) 12m ago 28m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (16m) 13m ago 29m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (16m) 12m ago 28m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (18m) 13m ago 28m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (13m) 13m ago 23m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (13m) 13m ago 23m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (12m) 12m ago 23m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (12m) 12m ago 23m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (20m) 13m ago 30m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (19m) 12m ago 28m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (17m) 13m ago 30m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (17m) 12m ago 28m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:28:03.388 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (19m) 13m ago 29m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (19m) 12m ago 28m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (16m) 13m ago 27m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (15m) 13m ago 26m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (15m) 13m ago 26m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (14m) 12m ago 25m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (14m) 12m ago 25m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (13m) 12m ago 25m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:28:03.389 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (18m) 13m ago 28m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:28:03.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 msgr2=0x7fc41006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 0x7fc41006d440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc414006010 tx=0x7fc41401a040 comp rx=0 tx=0).stop 2024-06-15T01:28:03.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 msgr2=0x7fc424196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc42000d8d0 tx=0x7fc42000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 shutdown_connections 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc41006af80 0x7fc41006d440 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4240ffa50 0x7fc424196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 --2- 172.21.15.137:0/3781372006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc424105a50 0x7fc424196bd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 >> 172.21.15.137:0/3781372006 conn(0x7fc4240fb540 msgr2=0x7fc4240fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 shutdown_connections 2024-06-15T01:28:03.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.404+0000 7fc42b3b7700 1 -- 172.21.15.137:0/3781372006 wait complete. 2024-06-15T01:28:03.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 -- 172.21.15.137:0/722264121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 msgr2=0x7f9560105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 --2- 172.21.15.137:0/722264121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f9560105f30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f955c009910 tx=0x7f955c009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 -- 172.21.15.137:0/722264121 shutdown_connections 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 --2- 172.21.15.137:0/722264121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f95600ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 --2- 172.21.15.137:0/722264121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f9560105f30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 -- 172.21.15.137:0/722264121 >> 172.21.15.137:0/722264121 conn(0x7f9560075e30 msgr2=0x7f95600fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 -- 172.21.15.137:0/722264121 shutdown_connections 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.507+0000 7f9567c9c700 1 -- 172.21.15.137:0/722264121 wait complete. 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 Processor -- start 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 -- start start 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f956019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956019d040 con 0x7f9560105b50 2024-06-15T01:28:03.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9567c9c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560196570 con 0x7f95600fef70 2024-06-15T01:28:03.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9565237700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9565237700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41420/0 (socket says 172.21.15.137:41420) 2024-06-15T01:28:03.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9565237700 1 -- 172.21.15.137:0/776453307 learned_addr learned my addr 172.21.15.137:0/776453307 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:03.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.508+0000 7f9565a38700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f956019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9565237700 1 -- 172.21.15.137:0/776453307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 msgr2=0x7f956019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9565237700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f956019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9565237700 1 -- 172.21.15.137:0/776453307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0095f0 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9565a38700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f956019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9565237700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f955000c8a0 tx=0x7f955000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955000cea0 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9560196850 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9550004830 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9550005500 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.509+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9560196da0 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.510+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9560109140 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.510+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f95500049a0 con 0x7f9560105b50 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.511+0000 7f9556ffd700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 0x7f954c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.511+0000 7f9565a38700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 0x7f954c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.511+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f955008a050 con 0x7f9560105b50 2024-06-15T01:28:03.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.512+0000 7f9565a38700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 0x7f954c06d440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c00b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.515+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9550058910 con 0x7f9560105b50 2024-06-15T01:28:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.793+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f956004ea90 con 0x7f9560105b50 2024-06-15T01:28:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.793+0000 7f9556ffd700 1 -- 172.21.15.137:0/776453307 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f95500584a0 con 0x7f9560105b50 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:03.779 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:28:03.780 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 msgr2=0x7f954c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 0x7f954c06d440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c00b540 comp rx=0 tx=0).stop 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 msgr2=0x7f956019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f955000c8a0 tx=0x7f955000cc60 comp rx=0 tx=0).stop 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 shutdown_connections 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f954c06af80 0x7f954c06d440 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95600fef70 0x7f956019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 --2- 172.21.15.137:0/776453307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9560105b50 0x7f956019c960 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 >> 172.21.15.137:0/776453307 conn(0x7f9560075e30 msgr2=0x7f9560104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 shutdown_connections 2024-06-15T01:28:03.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.795+0000 7f9567c9c700 1 -- 172.21.15.137:0/776453307 wait complete. 2024-06-15T01:28:03.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 -- 172.21.15.137:0/2054926648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264102620 msgr2=0x7fd26410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 --2- 172.21.15.137:0/2054926648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264102620 0x7fd26410ab10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd258009910 tx=0x7fd258009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:03.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 -- 172.21.15.137:0/2054926648 shutdown_connections 2024-06-15T01:28:03.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 --2- 172.21.15.137:0/2054926648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264102620 0x7fd26410ab10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 --2- 172.21.15.137:0/2054926648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264101d00 0x7fd2641020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 -- 172.21.15.137:0/2054926648 >> 172.21.15.137:0/2054926648 conn(0x7fd2640fb570 msgr2=0x7fd2640fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:03.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.899+0000 7fd268c9c700 1 -- 172.21.15.137:0/2054926648 shutdown_connections 2024-06-15T01:28:03.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.900+0000 7fd268c9c700 1 -- 172.21.15.137:0/2054926648 wait complete. 2024-06-15T01:28:03.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.900+0000 7fd268c9c700 1 Processor -- start 2024-06-15T01:28:03.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.900+0000 7fd268c9c700 1 -- start start 2024-06-15T01:28:03.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.900+0000 7fd268c9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd268c9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264102620 0x7fd2640ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd268c9c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641003e0 con 0x7fd264101d00 2024-06-15T01:28:03.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd268c9c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264100520 con 0x7fd264102620 2024-06-15T01:28:03.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41432/0 (socket says 172.21.15.137:41432) 2024-06-15T01:28:03.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 -- 172.21.15.137:0/1369109073 learned_addr learned my addr 172.21.15.137:0/1369109073 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:03.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd261d9b700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264102620 0x7fd2640ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 -- 172.21.15.137:0/1369109073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264102620 msgr2=0x7fd2640ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:03.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264102620 0x7fd2640ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 -- 172.21.15.137:0/1369109073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580095f0 con 0x7fd264101d00 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd26259c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d8d0 tx=0x7fd25400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd254009940 con 0x7fd264101d00 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2641a28a0 con 0x7fd264101d00 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd254010460 con 0x7fd264101d00 2024-06-15T01:28:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25400f5f0 con 0x7fd264101d00 2024-06-15T01:28:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.901+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2641a2cb0 con 0x7fd264101d00 2024-06-15T01:28:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.902+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd264108210 con 0x7fd264101d00 2024-06-15T01:28:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.903+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd2540105d0 con 0x7fd264101d00 2024-06-15T01:28:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.903+0000 7fd2537fe700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 0x7fd24c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.903+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd25408a3e0 con 0x7fd264101d00 2024-06-15T01:28:03.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.903+0000 7fd261d9b700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 0x7fd24c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:03.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.904+0000 7fd261d9b700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 0x7fd24c06d440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd264101150 tx=0x7fd25801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:03.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:03.907+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd254055210 con 0x7fd264101d00 2024-06-15T01:28:04.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.144+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd2641006e0 con 0x7fd264101d00 2024-06-15T01:28:04.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:04 smithi137 ceph-mon[112295]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:04.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:04 smithi137 ceph-mon[112295]: pgmap v659: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:04.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:04 smithi137 ceph-mon[112295]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:04.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:04 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/776453307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.145+0000 7fd2537fe700 1 -- 172.21.15.137:0/1369109073 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fd254010880 con 0x7fd264101d00 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:28:04.131 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:28:04.132 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:04.133 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:28:04.134 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:04.134 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:04.134 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:04.134 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:04.134 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 msgr2=0x7fd24c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 0x7fd24c06d440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd264101150 tx=0x7fd25801a040 comp rx=0 tx=0).stop 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 msgr2=0x7fd2640ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d8d0 tx=0x7fd25400dc90 comp rx=0 tx=0).stop 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 shutdown_connections 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd24c06af80 0x7fd24c06d440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd264101d00 0x7fd2640ff960 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 --2- 172.21.15.137:0/1369109073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd264102620 0x7fd2640ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 >> 172.21.15.137:0/1369109073 conn(0x7fd2640fb570 msgr2=0x7fd264105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:04.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 shutdown_connections 2024-06-15T01:28:04.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.147+0000 7fd268c9c700 1 -- 172.21.15.137:0/1369109073 wait complete. 2024-06-15T01:28:04.136 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:28:04.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2730947407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 msgr2=0x7fe3f41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2730947407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f41057e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009910 tx=0x7fe3dc009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2730947407 shutdown_connections 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2730947407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f41057e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2730947407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2730947407 >> 172.21.15.137:0/2730947407 conn(0x7fe3f40fc670 msgr2=0x7fe3f40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2730947407 shutdown_connections 2024-06-15T01:28:04.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.253+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2730947407 wait complete. 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 Processor -- start 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 -- start start 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41972a0 con 0x7fe3f4101790 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f9f51700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4199fd0 con 0x7fe3f4100de0 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41456/0 (socket says 172.21.15.137:41456) 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 -- 172.21.15.137:0/2259726168 learned_addr learned my addr 172.21.15.137:0/2259726168 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f37fe700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 -- 172.21.15.137:0/2259726168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 msgr2=0x7fe3f41966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 -- 172.21.15.137:0/2259726168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0095f0 con 0x7fe3f4101790 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3f37fe700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.254+0000 7fe3eaffd700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0092e0 tx=0x7fe3dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.255+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc01d070 con 0x7fe3f4101790 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.255+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3dc00f460 con 0x7fe3f4101790 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.255+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f419a250 con 0x7fe3f4101790 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.255+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc021600 con 0x7fe3f4101790 2024-06-15T01:28:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.255+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f419a740 con 0x7fe3f4101790 2024-06-15T01:28:04.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.256+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f41091e0 con 0x7fe3f4101790 2024-06-15T01:28:04.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.256+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe3dc00f5d0 con 0x7fe3f4101790 2024-06-15T01:28:04.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.257+0000 7fe3f17fa700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 0x7fe3e006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.257+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe3dc08b630 con 0x7fe3f4101790 2024-06-15T01:28:04.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.257+0000 7fe3f37fe700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 0x7fe3e006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.257+0000 7fe3f37fe700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 0x7fe3e006d440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4005fd0 tx=0x7fe3e4005dc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.261+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3dc056460 con 0x7fe3f4101790 2024-06-15T01:28:04.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:04 smithi142 ceph-mon[82136]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:04.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:04 smithi142 ceph-mon[82136]: pgmap v659: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:04.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:04 smithi142 ceph-mon[82136]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:04.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:04 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/776453307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:28:04.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.471+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3f402cce0 con 0x7fe3e006af80 2024-06-15T01:28:04.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.473+0000 7fe3f17fa700 1 -- 172.21.15.137:0/2259726168 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe3f402cce0 con 0x7fe3e006af80 2024-06-15T01:28:04.458 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:28:04.458 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:28:04.459 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:28:04.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 msgr2=0x7fe3e006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 0x7fe3e006d440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4005fd0 tx=0x7fe3e4005dc0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 msgr2=0x7fe3f4196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0092e0 tx=0x7fe3dc00be20 comp rx=0 tx=0).stop 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 shutdown_connections 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe3e006af80 0x7fe3e006d440 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f4100de0 0x7fe3f41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 --2- 172.21.15.137:0/2259726168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3f4101790 0x7fe3f4196c10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 >> 172.21.15.137:0/2259726168 conn(0x7fe3f40fc670 msgr2=0x7fe3f40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 shutdown_connections 2024-06-15T01:28:04.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.475+0000 7fe3f9f51700 1 -- 172.21.15.137:0/2259726168 wait complete. 2024-06-15T01:28:04.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.578+0000 7ff1ed46c700 1 -- 172.21.15.137:0/2575661213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8102680 msgr2=0x7ff1e810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.578+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/2575661213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8102680 0x7ff1e810ab70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009940 tx=0x7ff1dc009c50 comp rx=0 tx=0).stop 2024-06-15T01:28:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 -- 172.21.15.137:0/2575661213 shutdown_connections 2024-06-15T01:28:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/2575661213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8102680 0x7ff1e810ab70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/2575661213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8101d60 0x7ff1e8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 -- 172.21.15.137:0/2575661213 >> 172.21.15.137:0/2575661213 conn(0x7ff1e80fb550 msgr2=0x7ff1e80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 -- 172.21.15.137:0/2575661213 shutdown_connections 2024-06-15T01:28:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 -- 172.21.15.137:0/2575661213 wait complete. 2024-06-15T01:28:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 Processor -- start 2024-06-15T01:28:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.579+0000 7ff1ed46c700 1 -- start start 2024-06-15T01:28:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 0x7ff1e8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e81972a0 con 0x7ff1e8101d60 2024-06-15T01:28:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8199fd0 con 0x7ff1e8102680 2024-06-15T01:28:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41482/0 (socket says 172.21.15.137:41482) 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 -- 172.21.15.137:0/912418239 learned_addr learned my addr 172.21.15.137:0/912418239 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e67fc700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 0x7ff1e8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 -- 172.21.15.137:0/912418239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 msgr2=0x7ff1e8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 0x7ff1e8196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 -- 172.21.15.137:0/912418239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0095f0 con 0x7ff1e8101d60 2024-06-15T01:28:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e67fc700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 0x7ff1e8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:28:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1e6ffd700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d900 tx=0x7ff1d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d80041f0 con 0x7ff1e8101d60 2024-06-15T01:28:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1d8004cd0 con 0x7ff1e8101d60 2024-06-15T01:28:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e819a2b0 con 0x7ff1e8101d60 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d80056d0 con 0x7ff1e8101d60 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.580+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e819a7d0 con 0x7ff1e8101d60 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.582+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e8108270 con 0x7ff1e8101d60 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.582+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff1d8004830 con 0x7ff1e8101d60 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.582+0000 7ff1cffff700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 0x7ff1d006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.582+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff1d808a530 con 0x7ff1e8101d60 2024-06-15T01:28:04.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.582+0000 7ff1e67fc700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 0x7ff1d006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:04.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.583+0000 7ff1e67fc700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 0x7ff1d006d3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009910 tx=0x7ff1dc005ab0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:04.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.587+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff1d8055360 con 0x7ff1e8101d60 2024-06-15T01:28:04.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.858+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff1e804ea90 con 0x7ff1e8101d60 2024-06-15T01:28:04.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.859+0000 7ff1cffff700 1 -- 172.21.15.137:0/912418239 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7ff1d8058980 con 0x7ff1e8101d60 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:28:04.845 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:28:04.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 msgr2=0x7ff1d006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 0x7ff1d006d3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009910 tx=0x7ff1dc005ab0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 msgr2=0x7ff1e81966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d900 tx=0x7ff1d800dcc0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 shutdown_connections 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff1d006af20 0x7ff1d006d3e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e8101d60 0x7ff1e81966d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 --2- 172.21.15.137:0/912418239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1e8102680 0x7ff1e8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 >> 172.21.15.137:0/912418239 conn(0x7ff1e80fb550 msgr2=0x7ff1e81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 shutdown_connections 2024-06-15T01:28:04.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:04.861+0000 7ff1ed46c700 1 -- 172.21.15.137:0/912418239 wait complete. 2024-06-15T01:28:05.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:05 smithi137 ceph-mon[112295]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:05.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:05 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1369109073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:28:05.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:05 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/912418239' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:28:05.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:05 smithi142 ceph-mon[82136]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:05.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:05 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1369109073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:28:05.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:05 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/912418239' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:28:06.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:06 smithi142 ceph-mon[82136]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:06.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:06 smithi142 ceph-mon[82136]: pgmap v660: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:06.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:06 smithi137 ceph-mon[112295]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:06.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:06 smithi137 ceph-mon[112295]: pgmap v660: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:08.288 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:08 smithi142 ceph-mon[82136]: pgmap v661: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:08.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:08 smithi137 ceph-mon[112295]: pgmap v661: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:10.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:10 smithi142 ceph-mon[82136]: pgmap v662: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:10.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:10 smithi137 ceph-mon[112295]: pgmap v662: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:12.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:12 smithi137 ceph-mon[112295]: pgmap v663: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:12.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:12 smithi142 ceph-mon[82136]: pgmap v663: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:14.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:14 smithi142 ceph-mon[82136]: pgmap v664: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:14.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:14 smithi137 ceph-mon[112295]: pgmap v664: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:16.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:16 smithi142 ceph-mon[82136]: pgmap v665: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:16.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:16 smithi137 ceph-mon[112295]: pgmap v665: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:18.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:18 smithi142 ceph-mon[82136]: pgmap v666: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:18.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:18.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:18 smithi137 ceph-mon[112295]: pgmap v666: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:18.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:20.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:20 smithi142 ceph-mon[82136]: pgmap v667: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:20.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:20 smithi137 ceph-mon[112295]: pgmap v667: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:22.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:22 smithi137 ceph-mon[112295]: pgmap v668: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:22.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:22 smithi142 ceph-mon[82136]: pgmap v668: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:24.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:24 smithi142 ceph-mon[82136]: pgmap v669: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:24.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:24 smithi137 ceph-mon[112295]: pgmap v669: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:26.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:26 smithi142 ceph-mon[82136]: pgmap v670: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:26.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:26 smithi137 ceph-mon[112295]: pgmap v670: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:28 smithi142 ceph-mon[82136]: pgmap v671: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:28.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:28 smithi137 ceph-mon[112295]: pgmap v671: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:30.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:30 smithi142 ceph-mon[82136]: pgmap v672: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:30.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:30 smithi137 ceph-mon[112295]: pgmap v672: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:32.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:32 smithi137 ceph-mon[112295]: pgmap v673: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:32.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:32 smithi142 ceph-mon[82136]: pgmap v673: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:33.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:33.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:34.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:34 smithi137 ceph-mon[112295]: pgmap v674: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:34 smithi142 ceph-mon[82136]: pgmap v674: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 -- 172.21.15.137:0/3232663581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 msgr2=0x7fc0d41057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 --2- 172.21.15.137:0/3232663581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d41057b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009910 tx=0x7fc0bc009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 -- 172.21.15.137:0/3232663581 shutdown_connections 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 --2- 172.21.15.137:0/3232663581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d41057b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 --2- 172.21.15.137:0/3232663581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 0x7fc0d40ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 -- 172.21.15.137:0/3232663581 >> 172.21.15.137:0/3232663581 conn(0x7fc0d4074e00 msgr2=0x7fc0d4075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 -- 172.21.15.137:0/3232663581 shutdown_connections 2024-06-15T01:28:34.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.978+0000 7fc0d8971700 1 -- 172.21.15.137:0/3232663581 wait complete. 2024-06-15T01:28:34.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.979+0000 7fc0d8971700 1 Processor -- start 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.979+0000 7fc0d8971700 1 -- start start 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d8971700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 0x7fc0d41966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d8971700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d8971700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4197290 con 0x7fc0d40ffec0 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d8971700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4199fc0 con 0x7fc0d40ff510 2024-06-15T01:28:34.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51580/0 (socket says 172.21.15.137:51580) 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 -- 172.21.15.137:0/340256438 learned_addr learned my addr 172.21.15.137:0/340256438 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d259c700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 0x7fc0d41966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 -- 172.21.15.137:0/340256438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 msgr2=0x7fc0d41966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 0x7fc0d41966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 -- 172.21.15.137:0/340256438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0095f0 con 0x7fc0d40ffec0 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0c9d9b700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0092e0 tx=0x7fc0bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:34.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc01d070 con 0x7fc0d40ffec0 2024-06-15T01:28:34.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d419a240 con 0x7fc0d40ffec0 2024-06-15T01:28:34.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0bc005dc0 con 0x7fc0d40ffec0 2024-06-15T01:28:34.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.980+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc00fea0 con 0x7fc0d40ffec0 2024-06-15T01:28:34.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.981+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d419a730 con 0x7fc0d40ffec0 2024-06-15T01:28:34.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.981+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d4109170 con 0x7fc0d40ffec0 2024-06-15T01:28:34.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.982+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc0bc003a60 con 0x7fc0d40ffec0 2024-06-15T01:28:34.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.982+0000 7fc0cbfff700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:34.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.983+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc0bc08b4c0 con 0x7fc0d40ffec0 2024-06-15T01:28:34.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.983+0000 7fc0d259c700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:34.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.983+0000 7fc0d259c700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005950 tx=0x7fc0c400b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:34.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:34.987+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc0bc0562f0 con 0x7fc0d40ffec0 2024-06-15T01:28:35.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.201+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0d419aa10 con 0x7fc0c006af80 2024-06-15T01:28:35.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.202+0000 7fc0cbfff700 1 -- 172.21.15.137:0/340256438 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc0d419aa10 con 0x7fc0c006af80 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.204+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 msgr2=0x7fc0c006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.204+0000 7fc0d8971700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005950 tx=0x7fc0c400b410 comp rx=0 tx=0).stop 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 msgr2=0x7fc0d4196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0092e0 tx=0x7fc0bc00bd70 comp rx=0 tx=0).stop 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 shutdown_connections 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc0c006af80 0x7fc0c006d440 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d40ff510 0x7fc0d41966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 --2- 172.21.15.137:0/340256438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0d40ffec0 0x7fc0d4196c00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 >> 172.21.15.137:0/340256438 conn(0x7fc0d4074e00 msgr2=0x7fc0d40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 shutdown_connections 2024-06-15T01:28:35.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.205+0000 7fc0d8971700 1 -- 172.21.15.137:0/340256438 wait complete. 2024-06-15T01:28:35.200 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:28:35.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 -- 172.21.15.137:0/4230289905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 msgr2=0x7ff2d41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/4230289905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d41011c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009910 tx=0x7ff2c4009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:35.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 -- 172.21.15.137:0/4230289905 shutdown_connections 2024-06-15T01:28:35.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/4230289905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/4230289905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d41011c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 -- 172.21.15.137:0/4230289905 >> 172.21.15.137:0/4230289905 conn(0x7ff2d40fc670 msgr2=0x7ff2d40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 -- 172.21.15.137:0/4230289905 shutdown_connections 2024-06-15T01:28:35.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.309+0000 7ff2d99fb700 1 -- 172.21.15.137:0/4230289905 wait complete. 2024-06-15T01:28:35.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 Processor -- start 2024-06-15T01:28:35.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 -- start start 2024-06-15T01:28:35.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d4196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4197290 con 0x7ff2d4100de0 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d99fb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419b020 con 0x7ff2d4101790 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51598/0 (socket says 172.21.15.137:51598) 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2d2ffd700 1 -- 172.21.15.137:0/717824239 learned_addr learned my addr 172.21.15.137:0/717824239 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.310+0000 7ff2ca7fc700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d4196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d2ffd700 1 -- 172.21.15.137:0/717824239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 msgr2=0x7ff2d4196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d2ffd700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d4196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d2ffd700 1 -- 172.21.15.137:0/717824239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40095f0 con 0x7ff2d4100de0 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2ca7fc700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d4196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d2ffd700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400b5c0 tx=0x7ff2c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c401d070 con 0x7ff2d4100de0 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2c400f460 con 0x7ff2d4100de0 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d419b2a0 con 0x7ff2d4100de0 2024-06-15T01:28:35.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c4021640 con 0x7ff2d4100de0 2024-06-15T01:28:35.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.311+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d419b790 con 0x7ff2d4100de0 2024-06-15T01:28:35.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.312+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4109180 con 0x7ff2d4100de0 2024-06-15T01:28:35.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.312+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff2c400fa20 con 0x7ff2d4100de0 2024-06-15T01:28:35.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.313+0000 7ff2d89f9700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 0x7ff2c006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.313+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff2c408b490 con 0x7ff2d4100de0 2024-06-15T01:28:35.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.313+0000 7ff2ca7fc700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 0x7ff2c006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.313+0000 7ff2ca7fc700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 0x7ff2c006d440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc005950 tx=0x7ff2bc0058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.317+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2c4056340 con 0x7ff2d4100de0 2024-06-15T01:28:35.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.524+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2d419bac0 con 0x7ff2c006af80 2024-06-15T01:28:35.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.525+0000 7ff2d89f9700 1 -- 172.21.15.137:0/717824239 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff2d419bac0 con 0x7ff2c006af80 2024-06-15T01:28:35.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 msgr2=0x7ff2c006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 0x7ff2c006d440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc005950 tx=0x7ff2bc0058e0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 msgr2=0x7ff2d4196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400b5c0 tx=0x7ff2c400be20 comp rx=0 tx=0).stop 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 shutdown_connections 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff2c006af80 0x7ff2c006d440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d4100de0 0x7ff2d4196670 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 --2- 172.21.15.137:0/717824239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2d4101790 0x7ff2d4196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 >> 172.21.15.137:0/717824239 conn(0x7ff2d40fc670 msgr2=0x7ff2d40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 shutdown_connections 2024-06-15T01:28:35.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.528+0000 7ff2d99fb700 1 -- 172.21.15.137:0/717824239 wait complete. 2024-06-15T01:28:35.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 -- 172.21.15.137:0/4251178729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 msgr2=0x7f0090102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 --2- 172.21.15.137:0/4251178729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090102110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f008000b0a0 tx=0x7f008000b3b0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 -- 172.21.15.137:0/4251178729 shutdown_connections 2024-06-15T01:28:35.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 --2- 172.21.15.137:0/4251178729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f009010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 --2- 172.21.15.137:0/4251178729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090102110 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 -- 172.21.15.137:0/4251178729 >> 172.21.15.137:0/4251178729 conn(0x7f00900fb560 msgr2=0x7f00900fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 -- 172.21.15.137:0/4251178729 shutdown_connections 2024-06-15T01:28:35.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.632+0000 7f009604b700 1 -- 172.21.15.137:0/4251178729 wait complete. 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 Processor -- start 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 -- start start 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f0090196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090197260 con 0x7f0090101d30 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f009604b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009019aff0 con 0x7f0090102650 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51610/0 (socket says 172.21.15.137:51610) 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008f7fe700 1 -- 172.21.15.137:0/407751889 learned_addr learned my addr 172.21.15.137:0/407751889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:35.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008effd700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f0090196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008f7fe700 1 -- 172.21.15.137:0/407751889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 msgr2=0x7f0090196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.633+0000 7f008f7fe700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f0090196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008f7fe700 1 -- 172.21.15.137:0/407751889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009d00 con 0x7f0090101d30 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008effd700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f0090196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008f7fe700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f008000d010 tx=0x7f0080007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008000e040 con 0x7f0090101d30 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0080003e50 con 0x7f0090101d30 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008001da10 con 0x7f0090101d30 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009019b270 con 0x7f0090101d30 2024-06-15T01:28:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.634+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009019b760 con 0x7f0090101d30 2024-06-15T01:28:35.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.635+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090108240 con 0x7f0090101d30 2024-06-15T01:28:35.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.636+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0080004210 con 0x7f0090101d30 2024-06-15T01:28:35.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.636+0000 7f008cff9700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 0x7f007806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.637+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f008008bd60 con 0x7f0090101d30 2024-06-15T01:28:35.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.637+0000 7f008effd700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 0x7f007806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.637+0000 7f008effd700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 0x7f007806d440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0090197c60 tx=0x7f0084005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.641+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0080056c10 con 0x7f0090101d30 2024-06-15T01:28:35.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.842+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f009002cd60 con 0x7f007806af80 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.851+0000 7f008cff9700 1 -- 172.21.15.137:0/407751889 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f009002cd60 con 0x7f007806af80 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (19m) 13m ago 29m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (30m) 13m ago 30m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (28m) 13m ago 28m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (17m) 13m ago 29m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:28:35.837 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (17m) 13m ago 28m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (18m) 13m ago 29m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (13m) 13m ago 24m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (13m) 13m ago 24m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (13m) 13m ago 24m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (13m) 13m ago 24m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (20m) 13m ago 31m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (19m) 13m ago 28m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (17m) 13m ago 31m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (17m) 13m ago 28m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (19m) 13m ago 29m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (19m) 13m ago 28m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:28:35.838 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (16m) 13m ago 27m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (16m) 13m ago 27m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (15m) 13m ago 26m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (15m) 13m ago 26m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (14m) 13m ago 26m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (14m) 13m ago 25m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:28:35.839 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (19m) 13m ago 29m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 msgr2=0x7f007806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 0x7f007806d440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0090197c60 tx=0x7f0084005f90 comp rx=0 tx=0).stop 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 msgr2=0x7f0090196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f008000d010 tx=0x7f0080007be0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 shutdown_connections 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f007806af80 0x7f007806d440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0090101d30 0x7f0090196640 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 --2- 172.21.15.137:0/407751889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090102650 0x7f0090196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 >> 172.21.15.137:0/407751889 conn(0x7f00900fb560 msgr2=0x7f0090105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 shutdown_connections 2024-06-15T01:28:35.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.854+0000 7f009604b700 1 -- 172.21.15.137:0/407751889 wait complete. 2024-06-15T01:28:35.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- 172.21.15.137:0/3131691540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ffeb0 msgr2=0x7f89ac103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 --2- 172.21.15.137:0/3131691540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac103e80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009910 tx=0x7f89a8009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:35.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- 172.21.15.137:0/3131691540 shutdown_connections 2024-06-15T01:28:35.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 --2- 172.21.15.137:0/3131691540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac103e80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 --2- 172.21.15.137:0/3131691540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ff500 0x7f89ac0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- 172.21.15.137:0/3131691540 >> 172.21.15.137:0/3131691540 conn(0x7f89ac074d80 msgr2=0x7f89ac075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:35.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- 172.21.15.137:0/3131691540 shutdown_connections 2024-06-15T01:28:35.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- 172.21.15.137:0/3131691540 wait complete. 2024-06-15T01:28:35.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 Processor -- start 2024-06-15T01:28:35.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.957+0000 7f89b3724700 1 -- start start 2024-06-15T01:28:35.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b3724700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b3724700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b3724700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac19d0c0 con 0x7f89ac0ff500 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b3724700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac196570 con 0x7f89ac0ffeb0 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51634/0 (socket says 172.21.15.137:51634) 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 -- 172.21.15.137:0/3882484081 learned_addr learned my addr 172.21.15.137:0/3882484081 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b0cbf700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 -- 172.21.15.137:0/3882484081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ffeb0 msgr2=0x7f89ac19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 -- 172.21.15.137:0/3882484081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80095f0 con 0x7f89ac0ff500 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.958+0000 7f89b14c0700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f899c00d8d0 tx=0x7f899c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.959+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899c0098a0 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.959+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f899c010460 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.959+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac196850 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.959+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899c009ea0 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.959+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac196da0 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.960+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac04ea90 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.960+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f899c0105d0 con 0x7f89ac0ff500 2024-06-15T01:28:35.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.961+0000 7f89a27fc700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 0x7f899806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:35.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.961+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f899c08a180 con 0x7f89ac0ff500 2024-06-15T01:28:35.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.961+0000 7f89b0cbf700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 0x7f899806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:35.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.961+0000 7f89b0cbf700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 0x7f899806d440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89ac105e30 tx=0x7f89a801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:35.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:35.965+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f899c058ac0 con 0x7f89ac0ff500 2024-06-15T01:28:36.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:36 smithi137 ceph-mon[112295]: pgmap v675: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:36.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:36 smithi137 ceph-mon[112295]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:36.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.235+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f89ac197180 con 0x7f89ac0ff500 2024-06-15T01:28:36.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.235+0000 7f89a27fc700 1 -- 172.21.15.137:0/3882484081 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f899c010880 con 0x7f89ac0ff500 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:28:36.221 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:28:36.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:28:36.222 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:28:36.222 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:28:36.222 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:28:36.222 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 msgr2=0x7f899806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 0x7f899806d440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89ac105e30 tx=0x7f89a801a040 comp rx=0 tx=0).stop 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 msgr2=0x7f89ac19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f899c00d8d0 tx=0x7f899c00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 shutdown_connections 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899806af80 0x7f899806d440 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac0ff500 0x7f89ac19c4f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 --2- 172.21.15.137:0/3882484081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89ac0ffeb0 0x7f89ac19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.237+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 >> 172.21.15.137:0/3882484081 conn(0x7f89ac074d80 msgr2=0x7f89ac1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.238+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 shutdown_connections 2024-06-15T01:28:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.238+0000 7f89b3724700 1 -- 172.21.15.137:0/3882484081 wait complete. 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 -- 172.21.15.137:0/2739879328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c100de0 msgr2=0x7f630c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 --2- 172.21.15.137:0/2739879328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c100de0 0x7f630c1011c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009910 tx=0x7f62fc009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 -- 172.21.15.137:0/2739879328 shutdown_connections 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 --2- 172.21.15.137:0/2739879328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c101790 0x7f630c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 --2- 172.21.15.137:0/2739879328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c100de0 0x7f630c1011c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 -- 172.21.15.137:0/2739879328 >> 172.21.15.137:0/2739879328 conn(0x7f630c0fc670 msgr2=0x7f630c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 -- 172.21.15.137:0/2739879328 shutdown_connections 2024-06-15T01:28:36.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.341+0000 7f63121bb700 1 -- 172.21.15.137:0/2739879328 wait complete. 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 Processor -- start 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 -- start start 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c100de0 0x7f630c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c197290 con 0x7f630c101790 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f63121bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c19b020 con 0x7f630c100de0 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f6303fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f6303fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40670/0 (socket says 172.21.15.137:40670) 2024-06-15T01:28:36.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f6303fff700 1 -- 172.21.15.137:0/160784882 learned_addr learned my addr 172.21.15.137:0/160784882 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.342+0000 7f630b7fe700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c100de0 0x7f630c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f6303fff700 1 -- 172.21.15.137:0/160784882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c100de0 msgr2=0x7f630c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f6303fff700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c100de0 0x7f630c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f6303fff700 1 -- 172.21.15.137:0/160784882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0095f0 con 0x7f630c101790 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f6303fff700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d900 tx=0x7f62f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f4009900 con 0x7f630c101790 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62f4010460 con 0x7f630c101790 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c19b300 con 0x7f630c101790 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f400f5f0 con 0x7f630c101790 2024-06-15T01:28:36.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.343+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c19b850 con 0x7f630c101790 2024-06-15T01:28:36.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.344+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c109180 con 0x7f630c101790 2024-06-15T01:28:36.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.345+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f62f40105d0 con 0x7f630c101790 2024-06-15T01:28:36.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.345+0000 7f63097fa700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 0x7f62f806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.345+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f62f4089dc0 con 0x7f630c101790 2024-06-15T01:28:36.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.345+0000 7f630b7fe700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 0x7f62f806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.346+0000 7f630b7fe700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 0x7f62f806d440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62fc006010 tx=0x7f62fc01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:36.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.349+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f62f4054c70 con 0x7f630c101790 2024-06-15T01:28:36.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:36 smithi142 ceph-mon[82136]: pgmap v675: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:36.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:36 smithi142 ceph-mon[82136]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:36.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.583+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f630c04ea90 con 0x7f630c101790 2024-06-15T01:28:36.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.584+0000 7f63097fa700 1 -- 172.21.15.137:0/160784882 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f62f4017750 con 0x7f630c101790 2024-06-15T01:28:36.569 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:28:36.570 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:28:36.571 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:28:36.572 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:36.573 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:36.573 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:36.573 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:28:36.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 msgr2=0x7f62f806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 0x7f62f806d440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62fc006010 tx=0x7f62fc01a040 comp rx=0 tx=0).stop 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 msgr2=0x7f630c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d900 tx=0x7f62f400dc10 comp rx=0 tx=0).stop 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 shutdown_connections 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f62f806af80 0x7f62f806d440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f630c100de0 0x7f630c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 --2- 172.21.15.137:0/160784882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f630c101790 0x7f630c196bb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 >> 172.21.15.137:0/160784882 conn(0x7f630c0fc670 msgr2=0x7f630c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 shutdown_connections 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.586+0000 7f63121bb700 1 -- 172.21.15.137:0/160784882 wait complete. 2024-06-15T01:28:36.574 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:28:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 -- 172.21.15.137:0/1670222509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480739d0 msgr2=0x7f994810cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 --2- 172.21.15.137:0/1670222509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480739d0 0x7f994810cee0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9938009910 tx=0x7f9938009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 -- 172.21.15.137:0/1670222509 shutdown_connections 2024-06-15T01:28:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 --2- 172.21.15.137:0/1670222509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480739d0 0x7f994810cee0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 --2- 172.21.15.137:0/1670222509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480730b0 0x7f9948073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.691+0000 7f994d944700 1 -- 172.21.15.137:0/1670222509 >> 172.21.15.137:0/1670222509 conn(0x7f99480fc650 msgr2=0x7f99480fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:36.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 -- 172.21.15.137:0/1670222509 shutdown_connections 2024-06-15T01:28:36.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 -- 172.21.15.137:0/1670222509 wait complete. 2024-06-15T01:28:36.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 Processor -- start 2024-06-15T01:28:36.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 -- start start 2024-06-15T01:28:36.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480739d0 0x7f994819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819f230 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.692+0000 7f994d944700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99481986e0 con 0x7f99480739d0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40688/0 (socket says 172.21.15.137:40688) 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 -- 172.21.15.137:0/3257109606 learned_addr learned my addr 172.21.15.137:0/3257109606 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f99467fc700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480739d0 0x7f994819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 -- 172.21.15.137:0/3257109606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480739d0 msgr2=0x7f994819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480739d0 0x7f994819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 -- 172.21.15.137:0/3257109606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380095f0 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f9946ffd700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d8d0 tx=0x7f993c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c0098a0 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f993c010460 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99481989c0 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c009ea0 con 0x7f99480730b0 2024-06-15T01:28:36.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.693+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9948198f10 con 0x7f99480730b0 2024-06-15T01:28:36.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.694+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994810a5e0 con 0x7f99480730b0 2024-06-15T01:28:36.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.695+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f993c0105d0 con 0x7f99480730b0 2024-06-15T01:28:36.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.695+0000 7f994c942700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 0x7f993006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:36.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.695+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f993c08a250 con 0x7f99480730b0 2024-06-15T01:28:36.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.695+0000 7f99467fc700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 0x7f993006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:36.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.696+0000 7f99467fc700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 0x7f993006d440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9948199b50 tx=0x7f993801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.700+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f993c058b10 con 0x7f99480730b0 2024-06-15T01:28:36.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.933+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9948199130 con 0x7f993006af80 2024-06-15T01:28:36.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.934+0000 7f994c942700 1 -- 172.21.15.137:0/3257109606 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9948199130 con 0x7f993006af80 2024-06-15T01:28:36.919 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:28:36.920 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:28:36.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 msgr2=0x7f993006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 0x7f993006d440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9948199b50 tx=0x7f993801a040 comp rx=0 tx=0).stop 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 msgr2=0x7f994819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d8d0 tx=0x7f993c00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 shutdown_connections 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f993006af80 0x7f993006d440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480730b0 0x7f994819e660 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 --2- 172.21.15.137:0/3257109606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99480739d0 0x7f994819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 >> 172.21.15.137:0/3257109606 conn(0x7f99480fc650 msgr2=0x7f9948107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 shutdown_connections 2024-06-15T01:28:36.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:36.936+0000 7f994d944700 1 -- 172.21.15.137:0/3257109606 wait complete. 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 -- 172.21.15.137:0/4254057947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 msgr2=0x7f9b8c103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 --2- 172.21.15.137:0/4254057947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c103eb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009910 tx=0x7f9b88009c20 comp rx=0 tx=0).stop 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 -- 172.21.15.137:0/4254057947 shutdown_connections 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 --2- 172.21.15.137:0/4254057947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c103eb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 --2- 172.21.15.137:0/4254057947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 0x7f9b8c0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 -- 172.21.15.137:0/4254057947 >> 172.21.15.137:0/4254057947 conn(0x7f9b8c074df0 msgr2=0x7f9b8c075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 -- 172.21.15.137:0/4254057947 shutdown_connections 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 -- 172.21.15.137:0/4254057947 wait complete. 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.040+0000 7f9b938be700 1 Processor -- start 2024-06-15T01:28:37.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 -- start start 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 0x7f9b8c1060c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c106b40 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c106c80 con 0x7f9b8c0ff530 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40720/0 (socket says 172.21.15.137:40720) 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 -- 172.21.15.137:0/3033236089 learned_addr learned my addr 172.21.15.137:0/3033236089 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:28:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b9165a700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 0x7f9b8c1060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 -- 172.21.15.137:0/3033236089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 msgr2=0x7f9b8c1060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 0x7f9b8c1060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 -- 172.21.15.137:0/3033236089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880095f0 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b90e59700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800bda0 tx=0x7f9b8800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8801d070 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b88005710 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8800fe90 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.041+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c106f00 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.042+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c1a2bd0 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.043+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c04ea90 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.043+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f9b8800f6f0 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.044+0000 7f9b827fc700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 0x7f9b7806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:28:37.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.044+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9b8808b4e0 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.044+0000 7f9b9165a700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 0x7f9b7806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:28:37.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.044+0000 7f9b9165a700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 0x7f9b7806d440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005950 tx=0x7f9b7c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:28:37.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.048+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9b88056310 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:37 smithi137 ceph-mon[112295]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:37.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:37 smithi137 ceph-mon[112295]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:37.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:37 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3882484081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:28:37.215 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:37 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/160784882' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:28:37.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.323+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b8c1075f0 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.324+0000 7f9b827fc700 1 -- 172.21.15.137:0/3033236089 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f9b88059930 con 0x7f9b8c0ffee0 2024-06-15T01:28:37.309 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:28:37.309 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:28:37.310 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:28:37.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 msgr2=0x7f9b7806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:37.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 0x7f9b7806d440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005950 tx=0x7f9b7c00b410 comp rx=0 tx=0).stop 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 msgr2=0x7f9b8c106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800bda0 tx=0x7f9b8800bdd0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 shutdown_connections 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f9b7806af80 0x7f9b7806d440 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b8c0ff530 0x7f9b8c1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 --2- 172.21.15.137:0/3033236089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c0ffee0 0x7f9b8c106600 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 >> 172.21.15.137:0/3033236089 conn(0x7f9b8c074df0 msgr2=0x7f9b8c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 shutdown_connections 2024-06-15T01:28:37.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:28:37.326+0000 7f9b938be700 1 -- 172.21.15.137:0/3033236089 wait complete. 2024-06-15T01:28:37.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:37 smithi142 ceph-mon[82136]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:37.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:37 smithi142 ceph-mon[82136]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:37.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:37 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3882484081' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:28:37.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:37 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/160784882' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:28:38.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:38 smithi142 ceph-mon[82136]: pgmap v676: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:38.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:38 smithi142 ceph-mon[82136]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:38.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:38 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3033236089' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:28:38.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:38 smithi137 ceph-mon[112295]: pgmap v676: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:38.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:38 smithi137 ceph-mon[112295]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:28:38.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:38 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3033236089' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:28:40.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:40 smithi137 ceph-mon[112295]: pgmap v677: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:40.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:40 smithi142 ceph-mon[82136]: pgmap v677: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:42.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:42 smithi137 ceph-mon[112295]: pgmap v678: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:42.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:42 smithi142 ceph-mon[82136]: pgmap v678: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:44.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:44 smithi137 ceph-mon[112295]: pgmap v679: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:44.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:44 smithi142 ceph-mon[82136]: pgmap v679: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:46.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:46 smithi137 ceph-mon[112295]: pgmap v680: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:46.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:46 smithi142 ceph-mon[82136]: pgmap v680: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:48.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:48 smithi137 ceph-mon[112295]: pgmap v681: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:48.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:48.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:48 smithi142 ceph-mon[82136]: pgmap v681: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:48.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:28:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:50 smithi137 ceph-mon[112295]: pgmap v682: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:50.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:50 smithi142 ceph-mon[82136]: pgmap v682: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:52.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:52 smithi137 ceph-mon[112295]: pgmap v683: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:52.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:52 smithi142 ceph-mon[82136]: pgmap v683: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:54.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:54 smithi137 ceph-mon[112295]: pgmap v684: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:54.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:54 smithi142 ceph-mon[82136]: pgmap v684: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:56.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:56 smithi137 ceph-mon[112295]: pgmap v685: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:56.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:56 smithi142 ceph-mon[82136]: pgmap v685: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:58.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:28:58 smithi137 ceph-mon[112295]: pgmap v686: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:28:58.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:28:58 smithi142 ceph-mon[82136]: pgmap v686: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:00.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:00 smithi137 ceph-mon[112295]: pgmap v687: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:00.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:00 smithi142 ceph-mon[82136]: pgmap v687: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:02.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:02 smithi137 ceph-mon[112295]: pgmap v688: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:02.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:02 smithi142 ceph-mon[82136]: pgmap v688: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:03.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:03.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:04.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:04 smithi137 ceph-mon[112295]: pgmap v689: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:04.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:04 smithi142 ceph-mon[82136]: pgmap v689: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:06.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:06 smithi137 ceph-mon[112295]: pgmap v690: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:06.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:06 smithi142 ceph-mon[82136]: pgmap v690: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 -- 172.21.15.137:0/1349376418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544101750 msgr2=0x7f05441057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 --2- 172.21.15.137:0/1349376418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544101750 0x7f05441057a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0538009910 tx=0x7f0538009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 -- 172.21.15.137:0/1349376418 shutdown_connections 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 --2- 172.21.15.137:0/1349376418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544101750 0x7f05441057a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 --2- 172.21.15.137:0/1349376418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544100da0 0x7f0544101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 -- 172.21.15.137:0/1349376418 >> 172.21.15.137:0/1349376418 conn(0x7f05440fc630 msgr2=0x7f05440fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 -- 172.21.15.137:0/1349376418 shutdown_connections 2024-06-15T01:29:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.443+0000 7f0549a56700 1 -- 172.21.15.137:0/1349376418 wait complete. 2024-06-15T01:29:07.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 Processor -- start 2024-06-15T01:29:07.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 -- start start 2024-06-15T01:29:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 0x7f0544196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544197270 con 0x7f0544100da0 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0549a56700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054419b000 con 0x7f0544101750 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0542ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0542ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47278/0 (socket says 172.21.15.137:47278) 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0542ffd700 1 -- 172.21.15.137:0/2966649919 learned_addr learned my addr 172.21.15.137:0/2966649919 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f05427fc700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 0x7f0544196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0542ffd700 1 -- 172.21.15.137:0/2966649919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 msgr2=0x7f0544196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.444+0000 7f0542ffd700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 0x7f0544196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0542ffd700 1 -- 172.21.15.137:0/2966649919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380095f0 con 0x7f0544100da0 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f05427fc700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 0x7f0544196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:29:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0542ffd700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f053400ef10 tx=0x7f053400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053400ca50 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f053400f040 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054419b280 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05340146b0 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.445+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054419b7d0 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.446+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05441091a0 con 0x7f0544100da0 2024-06-15T01:29:07.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.446+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f053400cbb0 con 0x7f0544100da0 2024-06-15T01:29:07.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.447+0000 7f0548a54700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 0x7f052c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.447+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f053408a860 con 0x7f0544100da0 2024-06-15T01:29:07.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.447+0000 7f05427fc700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 0x7f052c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.447+0000 7f05427fc700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 0x7f052c06d440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f0538005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:07.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.451+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f05340555f0 con 0x7f0544100da0 2024-06-15T01:29:07.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.660+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f054402cd60 con 0x7f052c06af80 2024-06-15T01:29:07.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.661+0000 7f0548a54700 1 -- 172.21.15.137:0/2966649919 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f054402cd60 con 0x7f052c06af80 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 msgr2=0x7f052c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 0x7f052c06d440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f0538005940 comp rx=0 tx=0).stop 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 msgr2=0x7f0544196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f053400ef10 tx=0x7f053400c3a0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 shutdown_connections 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f052c06af80 0x7f052c06d440 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0544100da0 0x7f0544196650 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.663+0000 7f0549a56700 1 --2- 172.21.15.137:0/2966649919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0544101750 0x7f0544196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.664+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 >> 172.21.15.137:0/2966649919 conn(0x7f05440fc630 msgr2=0x7f05440fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:07.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.664+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 shutdown_connections 2024-06-15T01:29:07.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.664+0000 7f0549a56700 1 -- 172.21.15.137:0/2966649919 wait complete. 2024-06-15T01:29:07.659 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.770+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1076811141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ffee0 msgr2=0x7f7fd8103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.770+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1076811141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8103eb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009910 tx=0x7f7fc8009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1076811141 shutdown_connections 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1076811141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8103eb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1076811141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ff530 0x7f7fd80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1076811141 >> 172.21.15.137:0/1076811141 conn(0x7f7fd8074df0 msgr2=0x7f7fd8075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:07.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1076811141 shutdown_connections 2024-06-15T01:29:07.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1076811141 wait complete. 2024-06-15T01:29:07.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 Processor -- start 2024-06-15T01:29:07.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- start start 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.771+0000 7f7fdfc4a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81019d0 con 0x7f7fd80ff530 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdfc4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8101b10 con 0x7f7fd80ffee0 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47302/0 (socket says 172.21.15.137:47302) 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 -- 172.21.15.137:0/1298957936 learned_addr learned my addr 172.21.15.137:0/1298957936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd1e5700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 -- 172.21.15.137:0/1298957936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ffee0 msgr2=0x7f7fd8101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 -- 172.21.15.137:0/1298957936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80095f0 con 0x7f7fd80ff530 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdd9e6700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400dd40 tx=0x7f7fd400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:07.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd400ad10 con 0x7f7fd80ff530 2024-06-15T01:29:07.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7fd400ae70 con 0x7f7fd80ff530 2024-06-15T01:29:07.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd81a2860 con 0x7f7fd80ff530 2024-06-15T01:29:07.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd81a2c70 con 0x7f7fd80ff530 2024-06-15T01:29:07.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.772+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd40168a0 con 0x7f7fd80ff530 2024-06-15T01:29:07.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.773+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd81097a0 con 0x7f7fd80ff530 2024-06-15T01:29:07.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.774+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7fd400e450 con 0x7f7fd80ff530 2024-06-15T01:29:07.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.774+0000 7f7fceffd700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 0x7f7fc406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:07.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.774+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7fd408a460 con 0x7f7fd80ff530 2024-06-15T01:29:07.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.774+0000 7f7fdd1e5700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 0x7f7fc406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:07.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.775+0000 7f7fdd1e5700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 0x7f7fc406d440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8102740 tx=0x7f7fc801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:07.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.778+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7fd40553c0 con 0x7f7fd80ff530 2024-06-15T01:29:07.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.988+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fd802cdd0 con 0x7f7fc406af80 2024-06-15T01:29:07.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.990+0000 7f7fceffd700 1 -- 172.21.15.137:0/1298957936 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fd802cdd0 con 0x7f7fc406af80 2024-06-15T01:29:07.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 msgr2=0x7f7fc406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 0x7f7fc406d440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8102740 tx=0x7f7fc801a040 comp rx=0 tx=0).stop 2024-06-15T01:29:07.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 msgr2=0x7f7fd8100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400dd40 tx=0x7f7fd400a530 comp rx=0 tx=0).stop 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 shutdown_connections 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7fc406af80 0x7f7fc406d440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80ff530 0x7f7fd8100f50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 --2- 172.21.15.137:0/1298957936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd80ffee0 0x7f7fd8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 >> 172.21.15.137:0/1298957936 conn(0x7f7fd8074df0 msgr2=0x7f7fd80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 shutdown_connections 2024-06-15T01:29:07.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:07.992+0000 7f7fdfc4a700 1 -- 172.21.15.137:0/1298957936 wait complete. 2024-06-15T01:29:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 -- 172.21.15.137:0/1023712598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c680ffbd0 msgr2=0x7f5c68100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 --2- 172.21.15.137:0/1023712598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c680ffbd0 0x7f5c68100050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009910 tx=0x7f5c64009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 -- 172.21.15.137:0/1023712598 shutdown_connections 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 --2- 172.21.15.137:0/1023712598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c680ffbd0 0x7f5c68100050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 --2- 172.21.15.137:0/1023712598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c68105a40 0x7f5c68105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 -- 172.21.15.137:0/1023712598 >> 172.21.15.137:0/1023712598 conn(0x7f5c680755a0 msgr2=0x7f5c680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 -- 172.21.15.137:0/1023712598 shutdown_connections 2024-06-15T01:29:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.095+0000 7f5c7019a700 1 -- 172.21.15.137:0/1023712598 wait complete. 2024-06-15T01:29:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 Processor -- start 2024-06-15T01:29:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 -- start start 2024-06-15T01:29:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 0x7f5c681956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68196280 con 0x7f5c68105a40 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c7019a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68198fb0 con 0x7f5c680ffbd0 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6d735700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6d735700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47322/0 (socket says 172.21.15.137:47322) 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6d735700 1 -- 172.21.15.137:0/605060455 learned_addr learned my addr 172.21.15.137:0/605060455 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6df36700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 0x7f5c681956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6d735700 1 -- 172.21.15.137:0/605060455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 msgr2=0x7f5c681956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.096+0000 7f5c6d735700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 0x7f5c681956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c6d735700 1 -- 172.21.15.137:0/605060455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640095f0 con 0x7f5c68105a40 2024-06-15T01:29:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c6df36700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 0x7f5c681956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c6d735700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c64006010 tx=0x7f5c6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6401f070 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c64019610 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c64005de0 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c68199230 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.097+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68199720 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.098+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6804ea90 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.098+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5c6400f7a0 con 0x7f5c68105a40 2024-06-15T01:29:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.099+0000 7f5c5affd700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 0x7f5c5406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.099+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5c6408d910 con 0x7f5c68105a40 2024-06-15T01:29:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.099+0000 7f5c6df36700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 0x7f5c5406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.099+0000 7f5c6df36700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 0x7f5c5406d440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0099d0 tx=0x7f5c5c008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.103+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c640587b0 con 0x7f5c68105a40 2024-06-15T01:29:08.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.304+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c681999b0 con 0x7f5c5406af80 2024-06-15T01:29:08.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.314+0000 7f5c5affd700 1 -- 172.21.15.137:0/605060455 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5c681999b0 con 0x7f5c5406af80 2024-06-15T01:29:08.299 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:29:08.299 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (19m) 14m ago 30m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:29:08.299 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (30m) 14m ago 30m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:29:08.299 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (29m) 13m ago 29m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (17m) 14m ago 30m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (17m) 13m ago 29m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (19m) 14m ago 29m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (14m) 14m ago 24m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (14m) 14m ago 24m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (13m) 13m ago 24m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (13m) 13m ago 24m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (21m) 14m ago 31m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (20m) 13m ago 29m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (18m) 14m ago 31m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (18m) 13m ago 29m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (20m) 14m ago 30m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (20m) 13m ago 29m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:29:08.300 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (17m) 14m ago 28m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (16m) 14m ago 27m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (16m) 14m ago 27m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (15m) 13m ago 27m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (15m) 13m ago 26m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (14m) 13m ago 26m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:29:08.301 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (19m) 14m ago 29m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 msgr2=0x7f5c5406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 0x7f5c5406d440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0099d0 tx=0x7f5c5c008040 comp rx=0 tx=0).stop 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 msgr2=0x7f5c68195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c64006010 tx=0x7f5c6400be20 comp rx=0 tx=0).stop 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 shutdown_connections 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.316+0000 7f5c7019a700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c5406af80 0x7f5c5406d440 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.317+0000 7f5c7019a700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c680ffbd0 0x7f5c681956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.317+0000 7f5c7019a700 1 --2- 172.21.15.137:0/605060455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c68105a40 0x7f5c68195bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.317+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 >> 172.21.15.137:0/605060455 conn(0x7f5c680755a0 msgr2=0x7f5c680fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:08.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.317+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 shutdown_connections 2024-06-15T01:29:08.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.317+0000 7f5c7019a700 1 -- 172.21.15.137:0/605060455 wait complete. 2024-06-15T01:29:08.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 -- 172.21.15.137:0/2248841274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8105a40 msgr2=0x7fcff8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 --2- 172.21.15.137:0/2248841274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8105a40 0x7fcff8105e20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009910 tx=0x7fcfec009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:08.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 -- 172.21.15.137:0/2248841274 shutdown_connections 2024-06-15T01:29:08.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 --2- 172.21.15.137:0/2248841274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff80ffbd0 0x7fcff8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 --2- 172.21.15.137:0/2248841274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8105a40 0x7fcff8105e20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 -- 172.21.15.137:0/2248841274 >> 172.21.15.137:0/2248841274 conn(0x7fcff80755a0 msgr2=0x7fcff80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:08.406 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:08 smithi137 ceph-mon[112295]: pgmap v691: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:08.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 -- 172.21.15.137:0/2248841274 shutdown_connections 2024-06-15T01:29:08.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.419+0000 7fcffead4700 1 -- 172.21.15.137:0/2248841274 wait complete. 2024-06-15T01:29:08.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 Processor -- start 2024-06-15T01:29:08.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 -- start start 2024-06-15T01:29:08.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8105a40 0x7fcff8195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81960e0 con 0x7fcff80ffbd0 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffead4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8196220 con 0x7fcff8105a40 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffc870700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffc870700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47338/0 (socket says 172.21.15.137:47338) 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.420+0000 7fcffc870700 1 -- 172.21.15.137:0/251302295 learned_addr learned my addr 172.21.15.137:0/251302295 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcff7fff700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8105a40 0x7fcff8195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcffc870700 1 -- 172.21.15.137:0/251302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8105a40 msgr2=0x7fcff8195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcffc870700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8105a40 0x7fcff8195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcffc870700 1 -- 172.21.15.137:0/251302295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0095f0 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcffc870700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfec01f070 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfec005940 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff819a010 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.421+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfec00f690 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.422+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff819a500 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.423+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff804ea90 con 0x7fcff80ffbd0 2024-06-15T01:29:08.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.423+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcfec00f7f0 con 0x7fcff80ffbd0 2024-06-15T01:29:08.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.423+0000 7fcff5ffb700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 0x7fcfe806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.424+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcfec08d7e0 con 0x7fcff80ffbd0 2024-06-15T01:29:08.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.424+0000 7fcff7fff700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 0x7fcfe806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.424+0000 7fcff7fff700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 0x7fcfe806d440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcff8196b70 tx=0x7fcfe400a380 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.428+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfec058610 con 0x7fcff80ffbd0 2024-06-15T01:29:08.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:08 smithi142 ceph-mon[82136]: pgmap v691: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:08.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.696+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcff8107c20 con 0x7fcff80ffbd0 2024-06-15T01:29:08.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.697+0000 7fcff5ffb700 1 -- 172.21.15.137:0/251302295 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fcfec05bc30 con 0x7fcff80ffbd0 2024-06-15T01:29:08.682 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:29:08.682 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:29:08.682 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:29:08.683 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:29:08.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 msgr2=0x7fcfe806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 0x7fcfe806d440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcff8196b70 tx=0x7fcfe400a380 comp rx=0 tx=0).stop 2024-06-15T01:29:08.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 msgr2=0x7fcff8195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec00bd70 comp rx=0 tx=0).stop 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 shutdown_connections 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfe806af80 0x7fcfe806d440 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff80ffbd0 0x7fcff8195550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 --2- 172.21.15.137:0/251302295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8105a40 0x7fcff8195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 >> 172.21.15.137:0/251302295 conn(0x7fcff80755a0 msgr2=0x7fcff80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 shutdown_connections 2024-06-15T01:29:08.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.699+0000 7fcffead4700 1 -- 172.21.15.137:0/251302295 wait complete. 2024-06-15T01:29:08.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 -- 172.21.15.137:0/669716561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20101770 msgr2=0x7f0f201057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 --2- 172.21.15.137:0/669716561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20101770 0x7f0f201057c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009910 tx=0x7f0f10009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:08.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 -- 172.21.15.137:0/669716561 shutdown_connections 2024-06-15T01:29:08.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 --2- 172.21.15.137:0/669716561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20101770 0x7f0f201057c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 --2- 172.21.15.137:0/669716561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f20100dc0 0x7f0f201011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 -- 172.21.15.137:0/669716561 >> 172.21.15.137:0/669716561 conn(0x7f0f200fc650 msgr2=0x7f0f200fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:08.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 -- 172.21.15.137:0/669716561 shutdown_connections 2024-06-15T01:29:08.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.803+0000 7f0f25eea700 1 -- 172.21.15.137:0/669716561 wait complete. 2024-06-15T01:29:08.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 Processor -- start 2024-06-15T01:29:08.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 -- start start 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f20101770 0x7f0f20196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201972a0 con 0x7f0f20100dc0 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f25eea700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20199fd0 con 0x7f0f20101770 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f1f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f1f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47370/0 (socket says 172.21.15.137:47370) 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.804+0000 7f0f1f7fe700 1 -- 172.21.15.137:0/2561322867 learned_addr learned my addr 172.21.15.137:0/2561322867 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1f7fe700 1 -- 172.21.15.137:0/2561322867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f20101770 msgr2=0x7f0f20196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1f7fe700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f20101770 0x7f0f20196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1f7fe700 1 -- 172.21.15.137:0/2561322867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100095f0 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1f7fe700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c8a0 tx=0x7f0f1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1400cea0 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2019a2b0 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0f14004830 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f14005500 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.805+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2019a800 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.806+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f201091c0 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.806+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0f140049a0 con 0x7f0f20100dc0 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.807+0000 7f0f1cff9700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 0x7f0f0806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:08.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.807+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0f1408a0a0 con 0x7f0f20100dc0 2024-06-15T01:29:08.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.807+0000 7f0f1effd700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 0x7f0f0806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:08.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.807+0000 7f0f1effd700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 0x7f0f0806d440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0f100059d0 tx=0x7f0f10005960 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:08.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:08.811+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f14054ed0 con 0x7f0f20100dc0 2024-06-15T01:29:09.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.044+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f2019aad0 con 0x7f0f20100dc0 2024-06-15T01:29:09.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.044+0000 7f0f1cff9700 1 -- 172.21.15.137:0/2561322867 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f0f1401d750 con 0x7f0f20100dc0 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:29:09.030 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:29:09.031 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:29:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:29:09.033 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:09.034 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:09.034 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:09.034 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:09.034 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:09.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 msgr2=0x7f0f0806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 0x7f0f0806d440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0f100059d0 tx=0x7f0f10005960 comp rx=0 tx=0).stop 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 msgr2=0x7f0f201966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c8a0 tx=0x7f0f1400cc60 comp rx=0 tx=0).stop 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 shutdown_connections 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0f0806af80 0x7f0f0806d440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f20100dc0 0x7f0f201966d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 --2- 172.21.15.137:0/2561322867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f20101770 0x7f0f20196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 >> 172.21.15.137:0/2561322867 conn(0x7f0f200fc650 msgr2=0x7f0f200fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 shutdown_connections 2024-06-15T01:29:09.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.046+0000 7f0f25eea700 1 -- 172.21.15.137:0/2561322867 wait complete. 2024-06-15T01:29:09.037 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.151+0000 7f2ad46b5700 1 -- 172.21.15.137:0/734886418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc101770 msgr2=0x7f2acc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.151+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/734886418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc101770 0x7f2acc1057c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009910 tx=0x7f2ac8009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.151+0000 7f2ad46b5700 1 -- 172.21.15.137:0/734886418 shutdown_connections 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.151+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/734886418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc101770 0x7f2acc1057c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.151+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/734886418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc100dc0 0x7f2acc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- 172.21.15.137:0/734886418 >> 172.21.15.137:0/734886418 conn(0x7f2acc0fc650 msgr2=0x7f2acc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- 172.21.15.137:0/734886418 shutdown_connections 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- 172.21.15.137:0/734886418 wait complete. 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 Processor -- start 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- start start 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc101770 0x7f2acc196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc1972a0 con 0x7f2acc100dc0 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.152+0000 7f2ad46b5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc199fd0 con 0x7f2acc101770 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:09.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47398/0 (socket says 172.21.15.137:47398) 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 -- 172.21.15.137:0/2396617385 learned_addr learned my addr 172.21.15.137:0/2396617385 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad1c50700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc101770 0x7f2acc196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 -- 172.21.15.137:0/2396617385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc101770 msgr2=0x7f2acc196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc101770 0x7f2acc196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 -- 172.21.15.137:0/2396617385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80095f0 con 0x7f2acc100dc0 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad2451700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d8d0 tx=0x7f2abc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2abc0098a0 con 0x7f2acc100dc0 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2abc010460 con 0x7f2acc100dc0 2024-06-15T01:29:09.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc19a2b0 con 0x7f2acc100dc0 2024-06-15T01:29:09.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2abc009ea0 con 0x7f2acc100dc0 2024-06-15T01:29:09.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.153+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc19a7d0 con 0x7f2acc100dc0 2024-06-15T01:29:09.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.154+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc1091c0 con 0x7f2acc100dc0 2024-06-15T01:29:09.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.155+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2abc0105d0 con 0x7f2acc100dc0 2024-06-15T01:29:09.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.155+0000 7f2ac37fe700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 0x7f2ab806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.155+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2abc08a220 con 0x7f2acc100dc0 2024-06-15T01:29:09.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.155+0000 7f2ad1c50700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 0x7f2ab806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:09.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.156+0000 7f2ad1c50700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 0x7f2ab806d440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2acc197df0 tx=0x7f2ac801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:09.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.160+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2abc055030 con 0x7f2acc100dc0 2024-06-15T01:29:09.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.370+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2acc19aab0 con 0x7f2ab806af80 2024-06-15T01:29:09.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:09 smithi137 ceph-mon[112295]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:09 smithi137 ceph-mon[112295]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:09 smithi137 ceph-mon[112295]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.356 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:09 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/251302295' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:29:09.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:09 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2561322867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:29:09.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.372+0000 7f2ac37fe700 1 -- 172.21.15.137:0/2396617385 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2acc19aab0 con 0x7f2ab806af80 2024-06-15T01:29:09.357 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:29:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:29:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:29:09.358 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:29:09.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 msgr2=0x7f2ab806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 0x7f2ab806d440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2acc197df0 tx=0x7f2ac801a040 comp rx=0 tx=0).stop 2024-06-15T01:29:09.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 msgr2=0x7f2acc1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d8d0 tx=0x7f2abc00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 shutdown_connections 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2ab806af80 0x7f2ab806d440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2acc100dc0 0x7f2acc1966d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 --2- 172.21.15.137:0/2396617385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2acc101770 0x7f2acc196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 >> 172.21.15.137:0/2396617385 conn(0x7f2acc0fc650 msgr2=0x7f2acc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 shutdown_connections 2024-06-15T01:29:09.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.374+0000 7f2ad46b5700 1 -- 172.21.15.137:0/2396617385 wait complete. 2024-06-15T01:29:09.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 -- 172.21.15.137:0/4163619872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 msgr2=0x7f61d81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 --2- 172.21.15.137:0/4163619872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d81011a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009910 tx=0x7f61c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:09.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 -- 172.21.15.137:0/4163619872 shutdown_connections 2024-06-15T01:29:09.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 --2- 172.21.15.137:0/4163619872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d8101770 0x7f61d81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 --2- 172.21.15.137:0/4163619872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d81011a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 -- 172.21.15.137:0/4163619872 >> 172.21.15.137:0/4163619872 conn(0x7f61d80fc650 msgr2=0x7f61d80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:09.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 -- 172.21.15.137:0/4163619872 shutdown_connections 2024-06-15T01:29:09.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.476+0000 7f61df87f700 1 -- 172.21.15.137:0/4163619872 wait complete. 2024-06-15T01:29:09.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 Processor -- start 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 -- start start 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d8101770 0x7f61d8196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d8197260 con 0x7f61d8100dc0 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61df87f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d8199f90 con 0x7f61d8101770 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47408/0 (socket says 172.21.15.137:47408) 2024-06-15T01:29:09.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 -- 172.21.15.137:0/3989555869 learned_addr learned my addr 172.21.15.137:0/3989555869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 -- 172.21.15.137:0/3989555869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d8101770 msgr2=0x7f61d8196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d8101770 0x7f61d8196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.477+0000 7f61dd61b700 1 -- 172.21.15.137:0/3989555869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80095f0 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61dd61b700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f61c80092e0 tx=0x7f61c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c801d070 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61c8005e00 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d819a210 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c800fe60 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.478+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d819a700 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.479+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d8109160 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.479+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f61c800f6d0 con 0x7f61d8100dc0 2024-06-15T01:29:09.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.480+0000 7f61ce7fc700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 0x7f61c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:09.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.480+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f61c808b590 con 0x7f61d8100dc0 2024-06-15T01:29:09.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.480+0000 7f61dce1a700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 0x7f61c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:09.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.480+0000 7f61dce1a700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 0x7f61c406d440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005950 tx=0x7f61d40058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:09.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.484+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f61c80564e0 con 0x7f61d8100dc0 2024-06-15T01:29:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:09 smithi142 ceph-mon[82136]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:09 smithi142 ceph-mon[82136]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:09 smithi142 ceph-mon[82136]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:09 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/251302295' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:29:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:09 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2561322867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:29:09.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.754+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f61d802cc70 con 0x7f61d8100dc0 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.754+0000 7f61ce7fc700 1 -- 172.21.15.137:0/3989555869 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f61c8059b00 con 0x7f61d8100dc0 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:29:09.740 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:29:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:29:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:29:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:29:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:29:09.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 msgr2=0x7f61c406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 0x7f61c406d440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005950 tx=0x7f61d40058e0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 msgr2=0x7f61d8196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f61c80092e0 tx=0x7f61c800be20 comp rx=0 tx=0).stop 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 shutdown_connections 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f61c406af80 0x7f61c406d440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8100dc0 0x7f61d8196690 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 --2- 172.21.15.137:0/3989555869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d8101770 0x7f61d8196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 >> 172.21.15.137:0/3989555869 conn(0x7f61d80fc650 msgr2=0x7f61d80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:09.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.756+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 shutdown_connections 2024-06-15T01:29:09.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:09.757+0000 7f61df87f700 1 -- 172.21.15.137:0/3989555869 wait complete. 2024-06-15T01:29:10.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:10 smithi137 ceph-mon[112295]: pgmap v692: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:10.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:10 smithi137 ceph-mon[112295]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:10.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:10 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3989555869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:29:10.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:10 smithi142 ceph-mon[82136]: pgmap v692: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:10.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:10 smithi142 ceph-mon[82136]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:10.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:10 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3989555869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:29:12.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:12 smithi137 ceph-mon[112295]: pgmap v693: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:12.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:12 smithi142 ceph-mon[82136]: pgmap v693: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:14.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:14 smithi137 ceph-mon[112295]: pgmap v694: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:14.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:14 smithi142 ceph-mon[82136]: pgmap v694: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:16.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:16 smithi137 ceph-mon[112295]: pgmap v695: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:16.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:16 smithi142 ceph-mon[82136]: pgmap v695: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:18.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:18 smithi137 ceph-mon[112295]: pgmap v696: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:18.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:18.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:18 smithi142 ceph-mon[82136]: pgmap v696: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:18.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:20.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:20 smithi137 ceph-mon[112295]: pgmap v697: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:20.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:20 smithi142 ceph-mon[82136]: pgmap v697: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:22.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:22 smithi137 ceph-mon[112295]: pgmap v698: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:22.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:22 smithi142 ceph-mon[82136]: pgmap v698: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:23.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:23 smithi142 ceph-mon[82136]: pgmap v699: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:23.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:23 smithi137 ceph-mon[112295]: pgmap v699: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:26.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:25 smithi142 ceph-mon[82136]: pgmap v700: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:26.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:25 smithi137 ceph-mon[112295]: pgmap v700: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:27 smithi142 ceph-mon[82136]: pgmap v701: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:28.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:27 smithi137 ceph-mon[112295]: pgmap v701: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:30.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:29 smithi142 ceph-mon[82136]: pgmap v702: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:30.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:29 smithi137 ceph-mon[112295]: pgmap v702: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:32.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:31 smithi137 ceph-mon[112295]: pgmap v703: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:32.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:31 smithi142 ceph-mon[82136]: pgmap v703: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:34.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:34 smithi137 ceph-mon[112295]: pgmap v704: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:34.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:34.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:34 smithi142 ceph-mon[82136]: pgmap v704: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:34.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:36.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:36 smithi142 ceph-mon[82136]: pgmap v705: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:36.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:36 smithi137 ceph-mon[112295]: pgmap v705: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:38.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:38 smithi142 ceph-mon[82136]: pgmap v706: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:38.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:38 smithi137 ceph-mon[112295]: pgmap v706: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:39.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 -- 172.21.15.137:0/2631934458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 msgr2=0x7fb924100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 --2- 172.21.15.137:0/2631934458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb924100050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb918009910 tx=0x7fb918009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 -- 172.21.15.137:0/2631934458 shutdown_connections 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 --2- 172.21.15.137:0/2631934458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb924100050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 --2- 172.21.15.137:0/2631934458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 0x7fb924105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.874+0000 7fb9295d1700 1 -- 172.21.15.137:0/2631934458 >> 172.21.15.137:0/2631934458 conn(0x7fb9240755a0 msgr2=0x7fb9240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.875+0000 7fb9295d1700 1 -- 172.21.15.137:0/2631934458 shutdown_connections 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.875+0000 7fb9295d1700 1 -- 172.21.15.137:0/2631934458 wait complete. 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.875+0000 7fb9295d1700 1 Processor -- start 2024-06-15T01:29:39.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.875+0000 7fb9295d1700 1 -- start start 2024-06-15T01:29:39.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:39.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 0x7fb92419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92419d0e0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb924196590 con 0x7fb924105a40 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37602/0 (socket says 172.21.15.137:37602) 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 -- 172.21.15.137:0/1132767175 learned_addr learned my addr 172.21.15.137:0/1132767175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9227fc700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 0x7fb92419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 -- 172.21.15.137:0/1132767175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 msgr2=0x7fb92419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 0x7fb92419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 -- 172.21.15.137:0/1132767175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180095f0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb922ffd700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d8d0 tx=0x7fb91400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb914009940 con 0x7fb9240ffbd0 2024-06-15T01:29:39.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb914010460 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb924196870 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91400f5f0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.876+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb924196dc0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.877+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb924108fe0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.878+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb9140105d0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.878+0000 7fb90bfff700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 0x7fb90c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.878+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb91408a3e0 con 0x7fb9240ffbd0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.878+0000 7fb9227fc700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 0x7fb90c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:39.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.879+0000 7fb9227fc700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 0x7fb90c06d440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb924104ef0 tx=0x7fb91801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:39.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:39.883+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb914058ca0 con 0x7fb9240ffbd0 2024-06-15T01:29:40.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.092+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb924196fe0 con 0x7fb90c06af80 2024-06-15T01:29:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.093+0000 7fb90bfff700 1 -- 172.21.15.137:0/1132767175 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb924196fe0 con 0x7fb90c06af80 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 msgr2=0x7fb90c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 0x7fb90c06d440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb924104ef0 tx=0x7fb91801a040 comp rx=0 tx=0).stop 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 msgr2=0x7fb92419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d8d0 tx=0x7fb91400dc90 comp rx=0 tx=0).stop 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 shutdown_connections 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb90c06af80 0x7fb90c06d440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9240ffbd0 0x7fb92419c510 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 --2- 172.21.15.137:0/1132767175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb924105a40 0x7fb92419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 >> 172.21.15.137:0/1132767175 conn(0x7fb9240755a0 msgr2=0x7fb924103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 shutdown_connections 2024-06-15T01:29:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.095+0000 7fb9295d1700 1 -- 172.21.15.137:0/1132767175 wait complete. 2024-06-15T01:29:40.091 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:29:40.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.200+0000 7f8f59597700 1 -- 172.21.15.137:0/2870594062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54100dc0 msgr2=0x7f8f541011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.200+0000 7f8f59597700 1 --2- 172.21.15.137:0/2870594062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54100dc0 0x7f8f541011a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009910 tx=0x7f8f44009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:40.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.200+0000 7f8f59597700 1 -- 172.21.15.137:0/2870594062 shutdown_connections 2024-06-15T01:29:40.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.200+0000 7f8f59597700 1 --2- 172.21.15.137:0/2870594062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54101770 0x7f8f541057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.186 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:40 smithi137 ceph-mon[112295]: pgmap v707: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:40.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.200+0000 7f8f59597700 1 --2- 172.21.15.137:0/2870594062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54100dc0 0x7f8f541011a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.201+0000 7f8f59597700 1 -- 172.21.15.137:0/2870594062 >> 172.21.15.137:0/2870594062 conn(0x7f8f540fc650 msgr2=0x7f8f540fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.201+0000 7f8f59597700 1 -- 172.21.15.137:0/2870594062 shutdown_connections 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.201+0000 7f8f59597700 1 -- 172.21.15.137:0/2870594062 wait complete. 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 Processor -- start 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 -- start start 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54100dc0 0x7f8f54196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f54197290 con 0x7f8f54101770 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f59597700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5419b020 con 0x7f8f54100dc0 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37622/0 (socket says 172.21.15.137:37622) 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 -- 172.21.15.137:0/3810880152 learned_addr learned my addr 172.21.15.137:0/3810880152 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f52ffd700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54100dc0 0x7f8f54196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 -- 172.21.15.137:0/3810880152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54100dc0 msgr2=0x7f8f54196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54100dc0 0x7f8f54196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.202+0000 7f8f527fc700 1 -- 172.21.15.137:0/3810880152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440095f0 con 0x7f8f54101770 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f527fc700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800c8f0 tx=0x7f8f4800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f480078a0 con 0x7f8f54101770 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f4800f450 con 0x7f8f54101770 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5419b300 con 0x7f8f54101770 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f48007ea0 con 0x7f8f54101770 2024-06-15T01:29:40.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.203+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5419b850 con 0x7f8f54101770 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.204+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54109160 con 0x7f8f54101770 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.204+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f8f4800f5c0 con 0x7f8f54101770 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.205+0000 7f8f3bfff700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 0x7f8f3c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.205+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8f4808a210 con 0x7f8f54101770 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.205+0000 7f8f52ffd700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 0x7f8f3c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.206+0000 7f8f52ffd700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 0x7f8f3c06d440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006010 tx=0x7f8f4401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.210+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8f480550c0 con 0x7f8f54101770 2024-06-15T01:29:40.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:40 smithi142 ceph-mon[82136]: pgmap v707: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:40.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.423+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f5419bb90 con 0x7f8f3c06af80 2024-06-15T01:29:40.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.424+0000 7f8f3bfff700 1 -- 172.21.15.137:0/3810880152 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8f5419bb90 con 0x7f8f3c06af80 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 msgr2=0x7f8f3c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 0x7f8f3c06d440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006010 tx=0x7f8f4401a040 comp rx=0 tx=0).stop 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 msgr2=0x7f8f54196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800c8f0 tx=0x7f8f4800cc00 comp rx=0 tx=0).stop 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 shutdown_connections 2024-06-15T01:29:40.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f8f3c06af80 0x7f8f3c06d440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f54100dc0 0x7f8f54196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 --2- 172.21.15.137:0/3810880152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f54101770 0x7f8f54196bb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.426+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 >> 172.21.15.137:0/3810880152 conn(0x7f8f540fc650 msgr2=0x7f8f540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.427+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 shutdown_connections 2024-06-15T01:29:40.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.427+0000 7f8f59597700 1 -- 172.21.15.137:0/3810880152 wait complete. 2024-06-15T01:29:40.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.530+0000 7ffa1ee73700 1 -- 172.21.15.137:0/1539275247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa180ffa80 msgr2=0x7ffa180fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.530+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/1539275247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa180ffa80 0x7ffa180fff00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009910 tx=0x7ffa04009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:40.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- 172.21.15.137:0/1539275247 shutdown_connections 2024-06-15T01:29:40.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/1539275247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa180ffa80 0x7ffa180fff00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/1539275247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa18105a80 0x7ffa18105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- 172.21.15.137:0/1539275247 >> 172.21.15.137:0/1539275247 conn(0x7ffa180fb590 msgr2=0x7ffa180fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- 172.21.15.137:0/1539275247 shutdown_connections 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- 172.21.15.137:0/1539275247 wait complete. 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 Processor -- start 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- start start 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa180ffa80 0x7ffa181965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa181971f0 con 0x7ffa18105a80 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.531+0000 7ffa1ee73700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1819af80 con 0x7ffa180ffa80 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37642/0 (socket says 172.21.15.137:37642) 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 -- 172.21.15.137:0/907617692 learned_addr learned my addr 172.21.15.137:0/907617692 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa1cc0f700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa180ffa80 0x7ffa181965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 -- 172.21.15.137:0/907617692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa180ffa80 msgr2=0x7ffa181965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa180ffa80 0x7ffa181965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 -- 172.21.15.137:0/907617692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040095f0 con 0x7ffa18105a80 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa17fff700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b5c0 tx=0x7ffa0400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0401d070 con 0x7ffa18105a80 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa04005e00 con 0x7ffa18105a80 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1819b200 con 0x7ffa18105a80 2024-06-15T01:29:40.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.532+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0400fe90 con 0x7ffa18105a80 2024-06-15T01:29:40.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.533+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1819b6f0 con 0x7ffa18105a80 2024-06-15T01:29:40.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.533+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa18109020 con 0x7ffa18105a80 2024-06-15T01:29:40.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.534+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ffa0400f6d0 con 0x7ffa18105a80 2024-06-15T01:29:40.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.534+0000 7ffa15ffb700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 0x7ffa0806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.534+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ffa0408b660 con 0x7ffa18105a80 2024-06-15T01:29:40.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.534+0000 7ffa1cc0f700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 0x7ffa0806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.535+0000 7ffa1cc0f700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 0x7ffa0806d440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c005950 tx=0x7ffa0c00b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.539+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffa04056510 con 0x7ffa18105a80 2024-06-15T01:29:40.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.737+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa1802cd60 con 0x7ffa0806af80 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.747+0000 7ffa15ffb700 1 -- 172.21.15.137:0/907617692 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ffa1802cd60 con 0x7ffa0806af80 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (20m) 14m ago 30m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (31m) 14m ago 31m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (29m) 14m ago 29m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (18m) 14m ago 31m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (18m) 14m ago 29m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:29:40.732 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (19m) 14m ago 30m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (14m) 14m ago 25m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (15m) 14m ago 25m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (14m) 14m ago 25m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (14m) 14m ago 25m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (22m) 14m ago 32m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (20m) 14m ago 29m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (18m) 14m ago 32m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (18m) 14m ago 29m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (20m) 14m ago 30m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (20m) 14m ago 29m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (18m) 14m ago 28m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (17m) 14m ago 28m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (16m) 14m ago 28m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (16m) 14m ago 27m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (15m) 14m ago 27m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:29:40.733 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (15m) 14m ago 26m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:29:40.734 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (20m) 14m ago 30m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:29:40.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 msgr2=0x7ffa0806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 0x7ffa0806d440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c005950 tx=0x7ffa0c00b410 comp rx=0 tx=0).stop 2024-06-15T01:29:40.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 msgr2=0x7ffa18196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b5c0 tx=0x7ffa0400bd70 comp rx=0 tx=0).stop 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 shutdown_connections 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffa0806af80 0x7ffa0806d440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa180ffa80 0x7ffa181965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 --2- 172.21.15.137:0/907617692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa18105a80 0x7ffa18196b10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 >> 172.21.15.137:0/907617692 conn(0x7ffa180fb590 msgr2=0x7ffa18069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 shutdown_connections 2024-06-15T01:29:40.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.749+0000 7ffa1ee73700 1 -- 172.21.15.137:0/907617692 wait complete. 2024-06-15T01:29:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 -- 172.21.15.137:0/505874981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100620 msgr2=0x7f897c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 --2- 172.21.15.137:0/505874981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100620 0x7f897c100a00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8964009910 tx=0x7f8964009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 -- 172.21.15.137:0/505874981 shutdown_connections 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 --2- 172.21.15.137:0/505874981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100fd0 0x7f897c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 --2- 172.21.15.137:0/505874981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100620 0x7f897c100a00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 -- 172.21.15.137:0/505874981 >> 172.21.15.137:0/505874981 conn(0x7f897c074d80 msgr2=0x7f897c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 -- 172.21.15.137:0/505874981 shutdown_connections 2024-06-15T01:29:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.851+0000 7f898321e700 1 -- 172.21.15.137:0/505874981 wait complete. 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 Processor -- start 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 -- start start 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100620 0x7f897c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c197260 con 0x7f897c100fd0 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f898321e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c19aff0 con 0x7f897c100620 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f897bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f897bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37660/0 (socket says 172.21.15.137:37660) 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.852+0000 7f897bfff700 1 -- 172.21.15.137:0/1020654696 learned_addr learned my addr 172.21.15.137:0/1020654696 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f8980fba700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100620 0x7f897c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f897bfff700 1 -- 172.21.15.137:0/1020654696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100620 msgr2=0x7f897c196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f897bfff700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100620 0x7f897c196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f897bfff700 1 -- 172.21.15.137:0/1020654696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640095f0 con 0x7f897c100fd0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f897bfff700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c8f0 tx=0x7f896c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f896c0078a0 con 0x7f897c100fd0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c19b2d0 con 0x7f897c100fd0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f896c00f450 con 0x7f897c100fd0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f896c007ea0 con 0x7f897c100fd0 2024-06-15T01:29:40.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.853+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c19b820 con 0x7f897c100fd0 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.854+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c109060 con 0x7f897c100fd0 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.855+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f896c007a00 con 0x7f897c100fd0 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.855+0000 7f8979ffb700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 0x7f896806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.855+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f896c08a2d0 con 0x7f897c100fd0 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.855+0000 7f8980fba700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 0x7f896806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.856+0000 7f8980fba700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 0x7f896806d440 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8964005f50 tx=0x7f896401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:40.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:40.860+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f896c055100 con 0x7f897c100fd0 2024-06-15T01:29:41.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.130+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f897c04ea90 con 0x7f897c100fd0 2024-06-15T01:29:41.116 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:41 smithi137 ceph-mon[112295]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.131+0000 7f8979ffb700 1 -- 172.21.15.137:0/1020654696 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f896c00f5c0 con 0x7f897c100fd0 2024-06-15T01:29:41.116 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:29:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:29:41.117 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 msgr2=0x7f896806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 0x7f896806d440 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8964005f50 tx=0x7f896401a040 comp rx=0 tx=0).stop 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 msgr2=0x7f897c196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c8f0 tx=0x7f896c00cc00 comp rx=0 tx=0).stop 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 shutdown_connections 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f896806af80 0x7f896806d440 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c100620 0x7f897c196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 --2- 172.21.15.137:0/1020654696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f897c100fd0 0x7f897c196b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 >> 172.21.15.137:0/1020654696 conn(0x7f897c074d80 msgr2=0x7f897c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 shutdown_connections 2024-06-15T01:29:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.133+0000 7f898321e700 1 -- 172.21.15.137:0/1020654696 wait complete. 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 -- 172.21.15.137:0/2784041295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8350105b20 msgr2=0x7f8350105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 --2- 172.21.15.137:0/2784041295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8350105b20 0x7f8350105f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8340009910 tx=0x7f8340009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 -- 172.21.15.137:0/2784041295 shutdown_connections 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 --2- 172.21.15.137:0/2784041295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83500ff050 0x7f83500ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 --2- 172.21.15.137:0/2784041295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8350105b20 0x7f8350105f00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 -- 172.21.15.137:0/2784041295 >> 172.21.15.137:0/2784041295 conn(0x7f8350075e30 msgr2=0x7f83500fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 -- 172.21.15.137:0/2784041295 shutdown_connections 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.237+0000 7f8356228700 1 -- 172.21.15.137:0/2784041295 wait complete. 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 Processor -- start 2024-06-15T01:29:41.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 -- start start 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8350105b20 0x7f8350196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350197270 con 0x7f83500ff050 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8356228700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350199fa0 con 0x7f8350105b20 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37682/0 (socket says 172.21.15.137:37682) 2024-06-15T01:29:41.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 -- 172.21.15.137:0/538751404 learned_addr learned my addr 172.21.15.137:0/538751404 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f8347fff700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8350105b20 0x7f8350196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 -- 172.21.15.137:0/538751404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8350105b20 msgr2=0x7f8350196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8350105b20 0x7f8350196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.238+0000 7f834f7fe700 1 -- 172.21.15.137:0/538751404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400095f0 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f834f7fe700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f834000be10 tx=0x7f834000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834001d070 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835019a220 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8340005dc0 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8340003a70 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.239+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835019a6b0 con 0x7f83500ff050 2024-06-15T01:29:41.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.240+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8350109060 con 0x7f83500ff050 2024-06-15T01:29:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.240+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f83400048e0 con 0x7f83500ff050 2024-06-15T01:29:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.241+0000 7f834d7fa700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 0x7f833c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.241+0000 7f8347fff700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 0x7f833c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.241+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f834008bb10 con 0x7f83500ff050 2024-06-15T01:29:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.241+0000 7f8347fff700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 0x7f833c06d440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8350197cc0 tx=0x7f8338009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.245+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8340056940 con 0x7f83500ff050 2024-06-15T01:29:41.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:41 smithi142 ceph-mon[82136]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:41.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.476+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8350197aa0 con 0x7f83500ff050 2024-06-15T01:29:41.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.476+0000 7f834d7fa700 1 -- 172.21.15.137:0/538751404 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f8340059f60 con 0x7f83500ff050 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:29:41.462 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:29:41.463 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:29:41.464 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:41.465 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:41.465 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:41.465 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 msgr2=0x7f833c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 0x7f833c06d440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8350197cc0 tx=0x7f8338009260 comp rx=0 tx=0).stop 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 msgr2=0x7f83501966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f834000be10 tx=0x7f834000bef0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 shutdown_connections 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f833c06af80 0x7f833c06d440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83500ff050 0x7f83501966a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 --2- 172.21.15.137:0/538751404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8350105b20 0x7f8350196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.479+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 >> 172.21.15.137:0/538751404 conn(0x7f8350075e30 msgr2=0x7f83500fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.480+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 shutdown_connections 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.480+0000 7f8356228700 1 -- 172.21.15.137:0/538751404 wait complete. 2024-06-15T01:29:41.466 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.584+0000 7fb9596e1700 1 -- 172.21.15.137:0/3170848296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 msgr2=0x7fb9541011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.584+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3170848296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb9541011a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb944009910 tx=0x7fb944009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 -- 172.21.15.137:0/3170848296 shutdown_connections 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3170848296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 0x7fb9541057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3170848296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb9541011a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 -- 172.21.15.137:0/3170848296 >> 172.21.15.137:0/3170848296 conn(0x7fb9540fc650 msgr2=0x7fb9540fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 -- 172.21.15.137:0/3170848296 shutdown_connections 2024-06-15T01:29:41.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 -- 172.21.15.137:0/3170848296 wait complete. 2024-06-15T01:29:41.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 Processor -- start 2024-06-15T01:29:41.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.585+0000 7fb9596e1700 1 -- start start 2024-06-15T01:29:41.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 0x7fb954196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954197290 con 0x7fb954100dc0 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95419b020 con 0x7fb954101770 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37702/0 (socket says 172.21.15.137:37702) 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 -- 172.21.15.137:0/3810922060 learned_addr learned my addr 172.21.15.137:0/3810922060 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9527fc700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 0x7fb954196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 -- 172.21.15.137:0/3810922060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 msgr2=0x7fb954196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 0x7fb954196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 -- 172.21.15.137:0/3810922060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440095f0 con 0x7fb954100dc0 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb952ffd700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb94400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94401d070 con 0x7fb954100dc0 2024-06-15T01:29:41.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb944005710 con 0x7fb954100dc0 2024-06-15T01:29:41.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95419b2a0 con 0x7fb954100dc0 2024-06-15T01:29:41.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94400fe20 con 0x7fb954100dc0 2024-06-15T01:29:41.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.586+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95419b790 con 0x7fb954100dc0 2024-06-15T01:29:41.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.588+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb954109160 con 0x7fb954100dc0 2024-06-15T01:29:41.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.588+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb944003a60 con 0x7fb954100dc0 2024-06-15T01:29:41.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.588+0000 7fb93bfff700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 0x7fb93c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.589+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb94408b590 con 0x7fb954100dc0 2024-06-15T01:29:41.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.589+0000 7fb9527fc700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 0x7fb93c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.589+0000 7fb9527fc700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 0x7fb93c06d440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb954197c90 tx=0x7fb94800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.593+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb9440563c0 con 0x7fb954100dc0 2024-06-15T01:29:41.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.801+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb95419ba70 con 0x7fb93c06af80 2024-06-15T01:29:41.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.803+0000 7fb93bfff700 1 -- 172.21.15.137:0/3810922060 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb95419ba70 con 0x7fb93c06af80 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:29:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:29:41.789 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:29:41.789 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:29:41.789 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:29:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 msgr2=0x7fb93c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 0x7fb93c06d440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb954197c90 tx=0x7fb94800b410 comp rx=0 tx=0).stop 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 msgr2=0x7fb954196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb94400bd70 comp rx=0 tx=0).stop 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 shutdown_connections 2024-06-15T01:29:41.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb93c06af80 0x7fb93c06d440 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb954100dc0 0x7fb954196670 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 --2- 172.21.15.137:0/3810922060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb954101770 0x7fb954196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 >> 172.21.15.137:0/3810922060 conn(0x7fb9540fc650 msgr2=0x7fb9540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 shutdown_connections 2024-06-15T01:29:41.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.805+0000 7fb9596e1700 1 -- 172.21.15.137:0/3810922060 wait complete. 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- 172.21.15.137:0/1065329331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770101780 msgr2=0x7fc7701057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1065329331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770101780 0x7fc7701057d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009910 tx=0x7fc75c009c20 comp rx=0 tx=0).stop 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- 172.21.15.137:0/1065329331 shutdown_connections 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1065329331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770101780 0x7fc7701057d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1065329331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770100dd0 0x7fc7701011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- 172.21.15.137:0/1065329331 >> 172.21.15.137:0/1065329331 conn(0x7fc7700fc640 msgr2=0x7fc7700fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- 172.21.15.137:0/1065329331 shutdown_connections 2024-06-15T01:29:41.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- 172.21.15.137:0/1065329331 wait complete. 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 Processor -- start 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- start start 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 0x7fc770196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701972d0 con 0x7fc770100dd0 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.909+0000 7fc776ac6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77019a000 con 0x7fc770101780 2024-06-15T01:29:41.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37724/0 (socket says 172.21.15.137:37724) 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 -- 172.21.15.137:0/1371101486 learned_addr learned my addr 172.21.15.137:0/1371101486 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc76ffff700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 0x7fc770196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 -- 172.21.15.137:0/1371101486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 msgr2=0x7fc770196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 0x7fc770196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 -- 172.21.15.137:0/1371101486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0095f0 con 0x7fc770100dd0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc76ffff700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 0x7fc770196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc774862700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d8d0 tx=0x7fc76400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc764009940 con 0x7fc770100dd0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc764010460 con 0x7fc770100dd0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77019a2e0 con 0x7fc770100dd0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76400f5f0 con 0x7fc770100dd0 2024-06-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.910+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77019a800 con 0x7fc770100dd0 2024-06-15T01:29:41.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.911+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7701091d0 con 0x7fc770100dd0 2024-06-15T01:29:41.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.912+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc7640105d0 con 0x7fc770100dd0 2024-06-15T01:29:41.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.912+0000 7fc76dffb700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 0x7fc76006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:29:41.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.912+0000 7fc76ffff700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 0x7fc76006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:29:41.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.912+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc76408a480 con 0x7fc770100dd0 2024-06-15T01:29:41.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.913+0000 7fc76ffff700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 0x7fc76006d440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc770197e20 tx=0x7fc75c01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:29:41.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:41.917+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc764055210 con 0x7fc770100dd0 2024-06-15T01:29:42.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.187+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc77004ea90 con 0x7fc770100dd0 2024-06-15T01:29:42.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:42 smithi137 ceph-mon[112295]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:42.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:42 smithi137 ceph-mon[112295]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:42.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:42 smithi137 ceph-mon[112295]: pgmap v708: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:42.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:42 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1020654696' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:29:42.173 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:42 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/538751404' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:29:42.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.188+0000 7fc76dffb700 1 -- 172.21.15.137:0/1371101486 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7fc764010880 con 0x7fc770100dd0 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:29:42.174 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:29:42.175 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:29:42.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 msgr2=0x7fc76006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:42.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 0x7fc76006d440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc770197e20 tx=0x7fc75c01a040 comp rx=0 tx=0).stop 2024-06-15T01:29:42.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 msgr2=0x7fc770196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:29:42.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d8d0 tx=0x7fc76400dc90 comp rx=0 tx=0).stop 2024-06-15T01:29:42.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 shutdown_connections 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc76006af80 0x7fc76006d440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770100dd0 0x7fc770196700 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 --2- 172.21.15.137:0/1371101486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc770101780 0x7fc770196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 >> 172.21.15.137:0/1371101486 conn(0x7fc7700fc640 msgr2=0x7fc7700fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 shutdown_connections 2024-06-15T01:29:42.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:29:42.191+0000 7fc776ac6700 1 -- 172.21.15.137:0/1371101486 wait complete. 2024-06-15T01:29:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:42 smithi142 ceph-mon[82136]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:42 smithi142 ceph-mon[82136]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:42 smithi142 ceph-mon[82136]: pgmap v708: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:42 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1020654696' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:29:42.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:42 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/538751404' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:29:43.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:43 smithi142 ceph-mon[82136]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:43.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:43 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1371101486' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:29:43.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:43 smithi137 ceph-mon[112295]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:29:43.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:43 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1371101486' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:29:44.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:44 smithi137 ceph-mon[112295]: pgmap v709: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:44.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:44 smithi142 ceph-mon[82136]: pgmap v709: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:46.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:46 smithi142 ceph-mon[82136]: pgmap v710: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:46.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:46 smithi137 ceph-mon[112295]: pgmap v710: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:48.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:48 smithi142 ceph-mon[82136]: pgmap v711: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:48.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:48.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:48 smithi137 ceph-mon[112295]: pgmap v711: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:48.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:29:50.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:50 smithi142 ceph-mon[82136]: pgmap v712: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:50 smithi137 ceph-mon[112295]: pgmap v712: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:52.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:52 smithi137 ceph-mon[112295]: pgmap v713: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:52.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:52 smithi142 ceph-mon[82136]: pgmap v713: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:54.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:54 smithi142 ceph-mon[82136]: pgmap v714: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:54.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:54 smithi137 ceph-mon[112295]: pgmap v714: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:56.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:56 smithi142 ceph-mon[82136]: pgmap v715: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:56.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:56 smithi137 ceph-mon[112295]: pgmap v715: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:58.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:29:58 smithi142 ceph-mon[82136]: pgmap v716: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:29:58.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:29:58 smithi137 ceph-mon[112295]: pgmap v716: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:00.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:00 smithi142 ceph-mon[82136]: pgmap v717: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:00.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:00 smithi142 ceph-mon[82136]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:30:00.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:00 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:29:59.999+0000 7f9ac4846640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:30:00.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:00 smithi137 ceph-mon[112295]: pgmap v717: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:00.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:00 smithi137 ceph-mon[112295]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:30:02.298 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:02 smithi137 ceph-mon[112295]: pgmap v718: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:02.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:02 smithi142 ceph-mon[82136]: pgmap v718: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:03.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:03 smithi142 ceph-mon[82136]: pgmap v719: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:03.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:03.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:03 smithi137 ceph-mon[112295]: pgmap v719: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:03.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:06.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:05 smithi142 ceph-mon[82136]: pgmap v720: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:06.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:05 smithi137 ceph-mon[112295]: pgmap v720: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:08.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:08 smithi142 ceph-mon[82136]: pgmap v721: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:08.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:08 smithi137 ceph-mon[112295]: pgmap v721: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:10.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:10 smithi142 ceph-mon[82136]: pgmap v722: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:10.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:10 smithi137 ceph-mon[112295]: pgmap v722: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 -- 172.21.15.137:0/1121759150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100fd0 msgr2=0x7faaa4104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 --2- 172.21.15.137:0/1121759150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100fd0 0x7faaa4104720 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa90009910 tx=0x7faa90009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 -- 172.21.15.137:0/1121759150 shutdown_connections 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 --2- 172.21.15.137:0/1121759150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100fd0 0x7faaa4104720 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 --2- 172.21.15.137:0/1121759150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100620 0x7faaa4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 -- 172.21.15.137:0/1121759150 >> 172.21.15.137:0/1121759150 conn(0x7faaa4074e00 msgr2=0x7faaa4075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:12.296 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:12 smithi137 ceph-mon[112295]: pgmap v723: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:12.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 -- 172.21.15.137:0/1121759150 shutdown_connections 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.309+0000 7faaaad59700 1 -- 172.21.15.137:0/1121759150 wait complete. 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 Processor -- start 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 -- start start 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100fd0 0x7faaa4196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4197270 con 0x7faaa4100620 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaaad59700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa419b000 con 0x7faaa4100fd0 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.310+0000 7faaa8af5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46690/0 (socket says 172.21.15.137:46690) 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 -- 172.21.15.137:0/1994114278 learned_addr learned my addr 172.21.15.137:0/1994114278 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa3fff700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100fd0 0x7faaa4196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 -- 172.21.15.137:0/1994114278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100fd0 msgr2=0x7faaa4196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100fd0 0x7faaa4196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 -- 172.21.15.137:0/1994114278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900095f0 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa8af5700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d8d0 tx=0x7faa9800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa980098a0 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa419b2e0 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa98010460 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa98009ea0 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.311+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa419b800 con 0x7faaa4100620 2024-06-15T01:30:12.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.313+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4109010 con 0x7faaa4100620 2024-06-15T01:30:12.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.313+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7faa980105d0 con 0x7faaa4100620 2024-06-15T01:30:12.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.314+0000 7faaa1ffb700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 0x7faa9406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.314+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7faa9808a2f0 con 0x7faaa4100620 2024-06-15T01:30:12.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.314+0000 7faaa3fff700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 0x7faa9406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.314+0000 7faaa3fff700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 0x7faa9406d440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa90006010 tx=0x7faa9001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.318+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faa98055080 con 0x7faaa4100620 2024-06-15T01:30:12.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:12 smithi142 ceph-mon[82136]: pgmap v723: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:12.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.528+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaa419bb30 con 0x7faa9406af80 2024-06-15T01:30:12.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.529+0000 7faaa1ffb700 1 -- 172.21.15.137:0/1994114278 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faaa419bb30 con 0x7faa9406af80 2024-06-15T01:30:12.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 msgr2=0x7faa9406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 0x7faa9406d440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa90006010 tx=0x7faa9001a040 comp rx=0 tx=0).stop 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 msgr2=0x7faaa4196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d8d0 tx=0x7faa9800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 shutdown_connections 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7faa9406af80 0x7faa9406d440 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaa4100620 0x7faaa4196650 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 --2- 172.21.15.137:0/1994114278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faaa4100fd0 0x7faaa4196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 >> 172.21.15.137:0/1994114278 conn(0x7faaa4074e00 msgr2=0x7faaa4103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 shutdown_connections 2024-06-15T01:30:12.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.531+0000 7faaaad59700 1 -- 172.21.15.137:0/1994114278 wait complete. 2024-06-15T01:30:12.529 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:30:12.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 -- 172.21.15.137:0/3389644141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 msgr2=0x7f5c8c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 --2- 172.21.15.137:0/3389644141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0ffec0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009910 tx=0x7f5c88009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:12.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 -- 172.21.15.137:0/3389644141 shutdown_connections 2024-06-15T01:30:12.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 --2- 172.21.15.137:0/3389644141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0ffec0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 --2- 172.21.15.137:0/3389644141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 0x7f5c8c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 -- 172.21.15.137:0/3389644141 >> 172.21.15.137:0/3389644141 conn(0x7f5c8c0fb570 msgr2=0x7f5c8c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:12.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 -- 172.21.15.137:0/3389644141 shutdown_connections 2024-06-15T01:30:12.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.636+0000 7f5c93612700 1 -- 172.21.15.137:0/3389644141 wait complete. 2024-06-15T01:30:12.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 Processor -- start 2024-06-15T01:30:12.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 -- start start 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 0x7f5c8c0fc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0fc9c0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c93612700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0fcb00 con 0x7f5c8c105a40 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c913ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c913ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46708/0 (socket says 172.21.15.137:46708) 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c913ae700 1 -- 172.21.15.137:0/756288077 learned_addr learned my addr 172.21.15.137:0/756288077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c90bad700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 0x7f5c8c0fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.637+0000 7f5c913ae700 1 -- 172.21.15.137:0/756288077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 msgr2=0x7f5c8c0fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c913ae700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 0x7f5c8c0fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c913ae700 1 -- 172.21.15.137:0/756288077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880095f0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c913ae700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00d8d0 tx=0x7f5c7c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7c0098a0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c7c010460 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c1a27e0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7c009ea0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.638+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c1a2bf0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.639+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c108fe0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.639+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5c7c0105d0 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.640+0000 7f5c827fc700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 0x7f5c7806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.640+0000 7f5c90bad700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 0x7f5c7806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.640+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5c7c08a250 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.640+0000 7f5c90bad700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 0x7f5c7806d440 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0fd730 tx=0x7f5c8801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.644+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c7c055080 con 0x7f5c8c0ffa40 2024-06-15T01:30:12.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.872+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c8c0fccc0 con 0x7f5c7806af80 2024-06-15T01:30:12.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.874+0000 7f5c827fc700 1 -- 172.21.15.137:0/756288077 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c8c0fccc0 con 0x7f5c7806af80 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 msgr2=0x7f5c7806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 0x7f5c7806d440 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0fd730 tx=0x7f5c8801a040 comp rx=0 tx=0).stop 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 msgr2=0x7f5c8c0fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00d8d0 tx=0x7f5c7c00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 shutdown_connections 2024-06-15T01:30:12.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5c7806af80 0x7f5c7806d440 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c8c0ffa40 0x7f5c8c0fbf40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 --2- 172.21.15.137:0/756288077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c8c105a40 0x7f5c8c0fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 >> 172.21.15.137:0/756288077 conn(0x7f5c8c0fb570 msgr2=0x7f5c8c068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:12.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 shutdown_connections 2024-06-15T01:30:12.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.876+0000 7f5c93612700 1 -- 172.21.15.137:0/756288077 wait complete. 2024-06-15T01:30:12.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 -- 172.21.15.137:0/1376709046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 msgr2=0x7fa3c8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/1376709046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8100050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009910 tx=0x7fa3c4009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:12.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 -- 172.21.15.137:0/1376709046 shutdown_connections 2024-06-15T01:30:12.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/1376709046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8100050 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/1376709046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 0x7fa3c8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 -- 172.21.15.137:0/1376709046 >> 172.21.15.137:0/1376709046 conn(0x7fa3c80755a0 msgr2=0x7fa3c80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:12.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 -- 172.21.15.137:0/1376709046 shutdown_connections 2024-06-15T01:30:12.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.979+0000 7fa3d00ae700 1 -- 172.21.15.137:0/1376709046 wait complete. 2024-06-15T01:30:12.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 Processor -- start 2024-06-15T01:30:12.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 -- start start 2024-06-15T01:30:12.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 0x7fa3c8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8197290 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3d00ae700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c819b020 con 0x7fa3c8105a40 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3cde4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3cde4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46728/0 (socket says 172.21.15.137:46728) 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3cde4a700 1 -- 172.21.15.137:0/3255674331 learned_addr learned my addr 172.21.15.137:0/3255674331 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.980+0000 7fa3cd649700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 0x7fa3c8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3cde4a700 1 -- 172.21.15.137:0/3255674331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 msgr2=0x7fa3c8196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3cde4a700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 0x7fa3c8196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3cde4a700 1 -- 172.21.15.137:0/3255674331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40095f0 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3cde4a700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800d8d0 tx=0x7fa3b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b80098a0 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c819b300 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c819b850 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3b8010460 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.981+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b8009ea0 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.982+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac005320 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.982+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa3b8010a40 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.983+0000 7fa3beffd700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 0x7fa3b406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:12.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.983+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa3b808a1d0 con 0x7fa3c80ffbd0 2024-06-15T01:30:12.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.983+0000 7fa3cd649700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 0x7fa3b406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:12.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.983+0000 7fa3cd649700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 0x7fa3b406d440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8197c90 tx=0x7fa3c401a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:12.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:12.987+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa3b8055000 con 0x7fa3c80ffbd0 2024-06-15T01:30:13.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.189+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3ac000bf0 con 0x7fa3b406af80 2024-06-15T01:30:13.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.199+0000 7fa3beffd700 1 -- 172.21.15.137:0/3255674331 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa3ac000bf0 con 0x7fa3b406af80 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (20m) 15m ago 31m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (31m) 15m ago 31m 8409k - 18.2.1 5be31c24972a d95288182022 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (30m) 14m ago 30m 8086k - 18.2.1 5be31c24972a 379fa66a02cd 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (18m) 15m ago 31m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (18m) 14m ago 30m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (20m) 15m ago 31m 87.3M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (15m) 15m ago 25m 11.7M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (15m) 15m ago 25m 15.2M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:30:13.184 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (15m) 14m ago 25m 15.1M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (14m) 14m ago 25m 12.9M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (22m) 15m ago 32m 499M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (21m) 14m ago 30m 440M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (19m) 15m ago 33m 50.0M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (19m) 14m ago 30m 42.8M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (21m) 15m ago 31m 15.6M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (21m) 14m ago 30m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (18m) 15m ago 29m 188M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (18m) 15m ago 29m 121M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (17m) 15m ago 28m 136M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (16m) 14m ago 28m 115M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (16m) 14m ago 27m 109M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (15m) 14m ago 27m 101M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:30:13.185 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (21m) 15m ago 30m 74.1M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:30:13.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 msgr2=0x7fa3b406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 0x7fa3b406d440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8197c90 tx=0x7fa3c401a040 comp rx=0 tx=0).stop 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 msgr2=0x7fa3c8196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800d8d0 tx=0x7fa3b800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 shutdown_connections 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa3b406af80 0x7fa3b406d440 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3c80ffbd0 0x7fa3c8196670 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 --2- 172.21.15.137:0/3255674331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3c8105a40 0x7fa3c8196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 >> 172.21.15.137:0/3255674331 conn(0x7fa3c80755a0 msgr2=0x7fa3c81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 shutdown_connections 2024-06-15T01:30:13.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.201+0000 7fa3d00ae700 1 -- 172.21.15.137:0/3255674331 wait complete. 2024-06-15T01:30:13.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 -- 172.21.15.137:0/1405271820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170102680 msgr2=0x7f717010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 --2- 172.21.15.137:0/1405271820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170102680 0x7f717010ab70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7164009940 tx=0x7f7164009c50 comp rx=0 tx=0).stop 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 -- 172.21.15.137:0/1405271820 shutdown_connections 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 --2- 172.21.15.137:0/1405271820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170102680 0x7f717010ab70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 --2- 172.21.15.137:0/1405271820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170101d60 0x7f7170102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.304+0000 7f7177a07700 1 -- 172.21.15.137:0/1405271820 >> 172.21.15.137:0/1405271820 conn(0x7f71700fb550 msgr2=0x7f71700fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 -- 172.21.15.137:0/1405271820 shutdown_connections 2024-06-15T01:30:13.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 -- 172.21.15.137:0/1405271820 wait complete. 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 Processor -- start 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 -- start start 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170102680 0x7f71700fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7170100440 con 0x7f7170101d60 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.305+0000 7f7177a07700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7170100580 con 0x7f7170102680 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46738/0 (socket says 172.21.15.137:46738) 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 -- 172.21.15.137:0/3365311791 learned_addr learned my addr 172.21.15.137:0/3365311791 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f7174fa2700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170102680 0x7f71700fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 -- 172.21.15.137:0/3365311791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170102680 msgr2=0x7f71700fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170102680 0x7f71700fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 -- 172.21.15.137:0/3365311791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640095f0 con 0x7f7170101d60 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f71757a3700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71680060b0 tx=0x7f716800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716800dda0 con 0x7f7170101d60 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f716800f040 con 0x7f7170101d60 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71701a28a0 con 0x7f7170101d60 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71680145d0 con 0x7f7170101d60 2024-06-15T01:30:13.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.306+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71701a2c50 con 0x7f7170101d60 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.308+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7170108270 con 0x7f7170101d60 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.308+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7168015400 con 0x7f7170101d60 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.309+0000 7f716e7fc700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 0x7f715806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.309+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7168089d10 con 0x7f7170101d60 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.309+0000 7f7174fa2700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 0x7f715806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.309+0000 7f7174fa2700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 0x7f715806d440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164005a90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.314+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7168054c70 con 0x7f7170101d60 2024-06-15T01:30:13.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.587+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f717002cc70 con 0x7f7170101d60 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.587+0000 7f716e7fc700 1 -- 172.21.15.137:0/3365311791 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7168058290 con 0x7f7170101d60 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:30:13.573 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:30:13.574 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 msgr2=0x7f715806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 0x7f715806d440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164005a90 comp rx=0 tx=0).stop 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 msgr2=0x7f71700ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71680060b0 tx=0x7f716800d3b0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 shutdown_connections 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f715806af80 0x7f715806d440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7170101d60 0x7f71700ff9c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 --2- 172.21.15.137:0/3365311791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7170102680 0x7f71700fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 >> 172.21.15.137:0/3365311791 conn(0x7f71700fb550 msgr2=0x7f71701053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 shutdown_connections 2024-06-15T01:30:13.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.589+0000 7f7177a07700 1 -- 172.21.15.137:0/3365311791 wait complete. 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 -- 172.21.15.137:0/2768953563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 msgr2=0x7f06c410ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 --2- 172.21.15.137:0/2768953563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c410ab20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009910 tx=0x7f06ac009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 -- 172.21.15.137:0/2768953563 shutdown_connections 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 --2- 172.21.15.137:0/2768953563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c410ab20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 --2- 172.21.15.137:0/2768953563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 -- 172.21.15.137:0/2768953563 >> 172.21.15.137:0/2768953563 conn(0x7f06c40fb540 msgr2=0x7f06c40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:13.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 -- 172.21.15.137:0/2768953563 shutdown_connections 2024-06-15T01:30:13.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.698+0000 7f06c9557700 1 -- 172.21.15.137:0/2768953563 wait complete. 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 Processor -- start 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 -- start start 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c40ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c41003f0 con 0x7f06c4102630 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c9557700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c4100530 con 0x7f06c4101d10 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46756/0 (socket says 172.21.15.137:46756) 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 -- 172.21.15.137:0/465055610 learned_addr learned my addr 172.21.15.137:0/465055610 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c2ffd700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c40ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 -- 172.21.15.137:0/465055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 msgr2=0x7f06c40ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c40ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c27fc700 1 -- 172.21.15.137:0/465055610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0095f0 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.699+0000 7f06c2ffd700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c40ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06c27fc700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f06ac005f50 tx=0x7f06ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ac01d070 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06ac00f460 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c41007b0 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ac021640 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.700+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c41a2b90 con 0x7f06c4102630 2024-06-15T01:30:13.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.701+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c4108220 con 0x7f06c4102630 2024-06-15T01:30:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.701+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f06ac02b440 con 0x7f06c4102630 2024-06-15T01:30:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.702+0000 7f06bbfff700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 0x7f06b006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.702+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f06ac08b6f0 con 0x7f06c4102630 2024-06-15T01:30:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.702+0000 7f06c2ffd700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 0x7f06b006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.702+0000 7f06c2ffd700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 0x7f06b006d440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0060b0 tx=0x7f06bc0076a0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:13.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.706+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f06ac0566b0 con 0x7f06c4102630 2024-06-15T01:30:13.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:13 smithi142 ceph-mon[82136]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:13 smithi142 ceph-mon[82136]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:13 smithi142 ceph-mon[82136]: pgmap v724: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:13.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:13 smithi142 ceph-mon[82136]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.941+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f06c402cc70 con 0x7f06c4102630 2024-06-15T01:30:13.926 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:13 smithi137 ceph-mon[112295]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.926 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:13 smithi137 ceph-mon[112295]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.926 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:13 smithi137 ceph-mon[112295]: pgmap v724: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:13.926 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:13 smithi137 ceph-mon[112295]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.941+0000 7f06bbfff700 1 -- 172.21.15.137:0/465055610 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f06ac059cd0 con 0x7f06c4102630 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:e31 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:30:13.927 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 22 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:14:29.156585+0000 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:30:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:up {} 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:failed 0,1 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:30:13.929 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{-1:34338} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{-1:34342} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:13.930 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 msgr2=0x7f06b006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 0x7f06b006d440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0060b0 tx=0x7f06bc0076a0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 msgr2=0x7f06c40ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f06ac005f50 tx=0x7f06ac00be20 comp rx=0 tx=0).stop 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 shutdown_connections 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f06b006af80 0x7f06b006d440 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c4101d10 0x7f06c40ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 --2- 172.21.15.137:0/465055610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c4102630 0x7f06c40ffeb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 >> 172.21.15.137:0/465055610 conn(0x7f06c40fb540 msgr2=0x7f06c4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:13.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 shutdown_connections 2024-06-15T01:30:13.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:13.944+0000 7f06c9557700 1 -- 172.21.15.137:0/465055610 wait complete. 2024-06-15T01:30:13.932 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:30:14.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 -- 172.21.15.137:0/1239218209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb01003b0 msgr2=0x7f5fb0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/1239218209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb01003b0 0x7f5fb0100830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009910 tx=0x7f5f9c009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 -- 172.21.15.137:0/1239218209 shutdown_connections 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/1239218209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb01003b0 0x7f5fb0100830 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/1239218209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb0105c30 0x7f5fb0106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 -- 172.21.15.137:0/1239218209 >> 172.21.15.137:0/1239218209 conn(0x7f5fb0074d80 msgr2=0x7f5fb0075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 -- 172.21.15.137:0/1239218209 shutdown_connections 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.050+0000 7f5fb6d38700 1 -- 172.21.15.137:0/1239218209 wait complete. 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 Processor -- start 2024-06-15T01:30:14.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 -- start start 2024-06-15T01:30:14.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 0x7f5fb019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019d180 con 0x7f5fb0105c30 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb6d38700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0196630 con 0x7f5fb01003b0 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46776/0 (socket says 172.21.15.137:46776) 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 -- 172.21.15.137:0/651483005 learned_addr learned my addr 172.21.15.137:0/651483005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb4ad4700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 0x7f5fb019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 -- 172.21.15.137:0/651483005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 msgr2=0x7f5fb019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 0x7f5fb019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5faffff700 1 -- 172.21.15.137:0/651483005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0095f0 con 0x7f5fb0105c30 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.051+0000 7f5fb4ad4700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 0x7f5fb019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:30:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5faffff700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c01d070 con 0x7f5fb0105c30 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb01968b0 con 0x7f5fb0105c30 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f9c0044c0 con 0x7f5fb0105c30 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c0039d0 con 0x7f5fb0105c30 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.052+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb0196d40 con 0x7f5fb0105c30 2024-06-15T01:30:14.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.053+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb004ea90 con 0x7f5fb0105c30 2024-06-15T01:30:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.053+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5f9c0048e0 con 0x7f5fb0105c30 2024-06-15T01:30:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.054+0000 7f5fadffb700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 0x7f5fa006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.054+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5f9c08ba40 con 0x7f5fb0105c30 2024-06-15T01:30:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.054+0000 7f5fb4ad4700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 0x7f5fa006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.054+0000 7f5fb4ad4700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 0x7f5fa006d440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009be0 tx=0x7f5fa4009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:14.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.058+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f9c05a300 con 0x7f5fb0105c30 2024-06-15T01:30:14.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.270+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb0197060 con 0x7f5fa006af80 2024-06-15T01:30:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.272+0000 7f5fadffb700 1 -- 172.21.15.137:0/651483005 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5fb0197060 con 0x7f5fa006af80 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:30:14.257 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:30:14.258 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "16/23 daemons upgraded", 2024-06-15T01:30:14.258 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading mds daemons", 2024-06-15T01:30:14.258 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:30:14.258 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:30:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 msgr2=0x7f5fa006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 0x7f5fa006d440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009be0 tx=0x7f5fa4009260 comp rx=0 tx=0).stop 2024-06-15T01:30:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 msgr2=0x7f5fb019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c00be90 comp rx=0 tx=0).stop 2024-06-15T01:30:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 shutdown_connections 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5fa006af80 0x7f5fa006d440 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb01003b0 0x7f5fb019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 --2- 172.21.15.137:0/651483005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fb0105c30 0x7f5fb019caf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 >> 172.21.15.137:0/651483005 conn(0x7f5fb0074d80 msgr2=0x7f5fb0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 shutdown_connections 2024-06-15T01:30:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.274+0000 7f5fb6d38700 1 -- 172.21.15.137:0/651483005 wait complete. 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- 172.21.15.137:0/678630687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970101770 msgr2=0x7f09701057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 --2- 172.21.15.137:0/678630687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970101770 0x7f09701057c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0964009910 tx=0x7f0964009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- 172.21.15.137:0/678630687 shutdown_connections 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 --2- 172.21.15.137:0/678630687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970101770 0x7f09701057c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 --2- 172.21.15.137:0/678630687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970100dc0 0x7f09701011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- 172.21.15.137:0/678630687 >> 172.21.15.137:0/678630687 conn(0x7f09700fc650 msgr2=0x7f09700fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- 172.21.15.137:0/678630687 shutdown_connections 2024-06-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- 172.21.15.137:0/678630687 wait complete. 2024-06-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 Processor -- start 2024-06-15T01:30:14.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.378+0000 7f09764c8700 1 -- start start 2024-06-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f09764c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f09764c8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 0x7f0970196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f09764c8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701972a0 con 0x7f0970100dc0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f09764c8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970199fd0 con 0x7f0970101770 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46804/0 (socket says 172.21.15.137:46804) 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 -- 172.21.15.137:0/266384906 learned_addr learned my addr 172.21.15.137:0/266384906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096f7fe700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 0x7f0970196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 -- 172.21.15.137:0/266384906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 msgr2=0x7f0970196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 0x7f0970196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 -- 172.21.15.137:0/266384906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640095f0 con 0x7f0970100dc0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096f7fe700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 0x7f0970196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096ffff700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f096000c8a0 tx=0x7f096000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096000cea0 con 0x7f0970100dc0 2024-06-15T01:30:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0960004830 con 0x7f0970100dc0 2024-06-15T01:30:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0960005500 con 0x7f0970100dc0 2024-06-15T01:30:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.379+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097019a2b0 con 0x7f0970100dc0 2024-06-15T01:30:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.380+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097019a800 con 0x7f0970100dc0 2024-06-15T01:30:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.380+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09701091c0 con 0x7f0970100dc0 2024-06-15T01:30:14.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.381+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f09600049a0 con 0x7f0970100dc0 2024-06-15T01:30:14.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.381+0000 7f096d7fa700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 0x7f095806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:14.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.381+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0960089ff0 con 0x7f0970100dc0 2024-06-15T01:30:14.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.381+0000 7f096f7fe700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 0x7f095806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:14.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.382+0000 7f096f7fe700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 0x7f095806d440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0964006010 tx=0x7f096400b540 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:14.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.386+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0960054e80 con 0x7f0970100dc0 2024-06-15T01:30:14.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.656+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f097004ea90 con 0x7f0970100dc0 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.657+0000 7f096d7fa700 1 -- 172.21.15.137:0/266384906 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+789 (secure 0 0 0) 0x7f096001d090 con 0x7f0970100dc0 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 4 mgr modules have recently crashed 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is degraded 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout: fs cephfs is offline because no MDS is active for it. 2024-06-15T01:30:14.642 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:30:14.643 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:30:14.643 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:30:14.643 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:30:14.643 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 msgr2=0x7f095806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 0x7f095806d440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0964006010 tx=0x7f096400b540 comp rx=0 tx=0).stop 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 msgr2=0x7f09701966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f096000c8a0 tx=0x7f096000cc60 comp rx=0 tx=0).stop 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 shutdown_connections 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f095806af80 0x7f095806d440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0970100dc0 0x7f09701966d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 --2- 172.21.15.137:0/266384906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0970101770 0x7f0970196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 >> 172.21.15.137:0/266384906 conn(0x7f09700fc650 msgr2=0x7f09700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:14.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 shutdown_connections 2024-06-15T01:30:14.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:14.659+0000 7f09764c8700 1 -- 172.21.15.137:0/266384906 wait complete. 2024-06-15T01:30:14.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:14 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3365311791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:14.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:14 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/465055610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:30:14.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:14 smithi142 ceph-mon[82136]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:14.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:14 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3365311791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:14.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:14 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/465055610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:30:14.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:14 smithi137 ceph-mon[112295]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:15.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:15 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/266384906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:30:15.809 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:15 smithi142 ceph-mon[82136]: pgmap v725: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:15.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:15 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/266384906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:30:15.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:15 smithi137 ceph-mon[112295]: pgmap v725: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:18.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:18 smithi142 ceph-mon[82136]: pgmap v726: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:18.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:18 smithi137 ceph-mon[112295]: pgmap v726: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:19.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:19 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:19.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:19 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:20.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:20 smithi142 ceph-mon[82136]: pgmap v727: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:20.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:20 smithi137 ceph-mon[112295]: pgmap v727: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:22.266 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:22 smithi137 ceph-mon[112295]: pgmap v728: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:22.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:22 smithi142 ceph-mon[82136]: pgmap v728: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:24.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:24 smithi137 ceph-mon[112295]: pgmap v729: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:24.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:24 smithi142 ceph-mon[82136]: pgmap v729: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:25.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: pgmap v730: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:30:25.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.afplli"}]': finished 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.nsntjq"}]': finished 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.examgd"}]': finished 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.uczriv"}]': finished 2024-06-15T01:30:25.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:25 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: pgmap v730: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.afplli"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.afplli"}]': finished 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.nsntjq"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi137.nsntjq"}]': finished 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.examgd"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.examgd"}]': finished 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.uczriv"}]: dispatch 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.uczriv"}]': finished 2024-06-15T01:30:25.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:25 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-15T01:30:26.146 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all mds 2024-06-15T01:30:26.146 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Upgrade: Setting filesystem cephfs Joinable 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Standby daemon mds.cephfs.smithi137.nsntjq assigned to filesystem cephfs as rank 0 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Standby daemon mds.cephfs.smithi137.afplli assigned to filesystem cephfs as rank 1 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:replay,1=cephfs.smithi137.afplli=up:replay} 2 up:standby 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:26.147 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:26 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all mds 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Upgrade: Setting filesystem cephfs Joinable 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Standby daemon mds.cephfs.smithi137.nsntjq assigned to filesystem cephfs as rank 0 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Standby daemon mds.cephfs.smithi137.afplli assigned to filesystem cephfs as rank 1 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:replay,1=cephfs.smithi137.afplli=up:replay} 2 up:standby 2024-06-15T01:30:26.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:26.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:26.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:26.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:26 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:27.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:27 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all rgw 2024-06-15T01:30:27.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:27 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all rbd-mirror 2024-06-15T01:30:27.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:27 smithi137 ceph-mon[112295]: pgmap v731: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-06-15T01:30:27.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:27 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all rgw 2024-06-15T01:30:27.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:27 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all rbd-mirror 2024-06-15T01:30:27.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:27 smithi142 ceph-mon[82136]: pgmap v731: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-06-15T01:30:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:28 smithi142 ceph-mon[82136]: Upgrade: Updating ceph-exporter.smithi137 (1/2) 2024-06-15T01:30:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:28 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:28 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:30:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:28 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:28.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:28 smithi142 ceph-mon[82136]: Deploying daemon ceph-exporter.smithi137 on smithi137 2024-06-15T01:30:28.459 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:28 smithi137 ceph-mon[112295]: Upgrade: Updating ceph-exporter.smithi137 (1/2) 2024-06-15T01:30:28.459 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:28 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:28.459 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:28 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi137", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:30:28.459 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:28 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:28.459 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:28 smithi137 ceph-mon[112295]: Deploying daemon ceph-exporter.smithi137 on smithi137 2024-06-15T01:30:29.158 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:29 smithi137 ceph-mon[112295]: pgmap v732: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-06-15T01:30:29.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:29 smithi142 ceph-mon[82136]: pgmap v732: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-06-15T01:30:31.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:31 smithi137 ceph-mon[112295]: pgmap v733: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-06-15T01:30:31.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:31 smithi142 ceph-mon[82136]: pgmap v733: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-06-15T01:30:32.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:32 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:resolve 2024-06-15T01:30:32.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:32 smithi137 ceph-mon[112295]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:replay,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:30:32.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:32 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:resolve 2024-06-15T01:30:32.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:32 smithi142 ceph-mon[82136]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:replay,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:30:33.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:33 smithi137 ceph-mon[112295]: pgmap v734: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-06-15T01:30:33.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:33 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:resolve 2024-06-15T01:30:33.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:33 smithi137 ceph-mon[112295]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:resolve,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:30:33.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:33.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:33.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:33 smithi142 ceph-mon[82136]: pgmap v734: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-06-15T01:30:33.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:33 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:resolve 2024-06-15T01:30:33.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:33 smithi142 ceph-mon[82136]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:resolve,1=cephfs.smithi137.afplli=up:resolve} 2 up:standby 2024-06-15T01:30:33.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:33.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: reconnect by client.14550 192.168.0.1:0/345370474 after 0.000999982 2024-06-15T01:30:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: reconnect by client.24309 192.168.0.1:0/2246087572 after 0.00199996 2024-06-15T01:30:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: reconnect by client.14550 192.168.0.1:0/345370474 after 0.00199996 2024-06-15T01:30:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: reconnect by client.24309 192.168.0.1:0/2246087572 after 0.00199996 2024-06-15T01:30:34.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:reconnect 2024-06-15T01:30:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:reconnect 2024-06-15T01:30:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:reconnect,1=cephfs.smithi137.afplli=up:reconnect} 2 up:standby 2024-06-15T01:30:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:34.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: reconnect by client.14550 192.168.0.1:0/345370474 after 0.000999982 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: reconnect by client.24309 192.168.0.1:0/2246087572 after 0.00199996 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: reconnect by client.14550 192.168.0.1:0/345370474 after 0.00199996 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: reconnect by client.24309 192.168.0.1:0/2246087572 after 0.00199996 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:reconnect 2024-06-15T01:30:34.666 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:reconnect 2024-06-15T01:30:34.667 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:reconnect,1=cephfs.smithi137.afplli=up:reconnect} 2 up:standby 2024-06-15T01:30:34.667 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:34.667 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: pgmap v735: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:rejoin 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:rejoin 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:rejoin,1=cephfs.smithi137.afplli=up:rejoin} 2 up:standby 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: daemon mds.cephfs.smithi137.nsntjq is now active in filesystem cephfs as rank 0 2024-06-15T01:30:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: daemon mds.cephfs.smithi137.afplli is now active in filesystem cephfs as rank 1 2024-06-15T01:30:35.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:35.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:30:35.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:35 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: pgmap v735: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:rejoin 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:rejoin 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: fsmap cephfs:2/2 {0=cephfs.smithi137.nsntjq=up:rejoin,1=cephfs.smithi137.afplli=up:rejoin} 2 up:standby 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: daemon mds.cephfs.smithi137.nsntjq is now active in filesystem cephfs as rank 0 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: daemon mds.cephfs.smithi137.afplli is now active in filesystem cephfs as rank 1 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-06-15T01:30:35.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:35 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: Upgrade: Updating ceph-exporter.smithi142 (2/2) 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:active 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:active 2024-06-15T01:30:36.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:36 smithi137 ceph-mon[112295]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:30:36.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: Upgrade: Updating ceph-exporter.smithi142 (2/2) 2024-06-15T01:30:36.724 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-06-15T01:30:36.724 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-15T01:30:36.724 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] up:active 2024-06-15T01:30:36.724 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: mds.? [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] up:active 2024-06-15T01:30:36.724 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:36 smithi142 ceph-mon[82136]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:30:37.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:37 smithi142 ceph-mon[82136]: pgmap v736: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 9 op/s 2024-06-15T01:30:37.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:37 smithi137 ceph-mon[112295]: pgmap v736: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 9 op/s 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: pgmap v737: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 426 B/s wr, 10 op/s 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:39 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: pgmap v737: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 426 B/s wr, 10 op/s 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.589 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:39.590 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:39 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:41.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:41 smithi137 ceph-mon[112295]: pgmap v738: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-06-15T01:30:41.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:41 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:41.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:41 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:41.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:41 smithi142 ceph-mon[82136]: pgmap v738: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-06-15T01:30:41.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:41 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:41.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:41 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:42.914 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:42 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:42.915 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:42 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:42.915 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:42 smithi142 ceph-mon[82136]: pgmap v739: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-06-15T01:30:42.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:42 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:42.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:42 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:42.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:42 smithi137 ceph-mon[112295]: pgmap v739: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 -- 172.21.15.137:0/3958819632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c101770 msgr2=0x7f039c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 --2- 172.21.15.137:0/3958819632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c101770 0x7f039c1057c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f038c009910 tx=0x7f038c009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 -- 172.21.15.137:0/3958819632 shutdown_connections 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 --2- 172.21.15.137:0/3958819632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c101770 0x7f039c1057c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 --2- 172.21.15.137:0/3958819632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c100dc0 0x7f039c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 -- 172.21.15.137:0/3958819632 >> 172.21.15.137:0/3958819632 conn(0x7f039c0fc650 msgr2=0x7f039c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:44.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 -- 172.21.15.137:0/3958819632 shutdown_connections 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.772+0000 7f03a4191700 1 -- 172.21.15.137:0/3958819632 wait complete. 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 Processor -- start 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 -- start start 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c101770 0x7f039c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1972a0 con 0x7f039c100dc0 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a4191700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c199fd0 con 0x7f039c101770 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a1f2d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a1f2d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44922/0 (socket says 172.21.15.137:44922) 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a1f2d700 1 -- 172.21.15.137:0/510509118 learned_addr learned my addr 172.21.15.137:0/510509118 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.773+0000 7f03a172c700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c101770 0x7f039c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a1f2d700 1 -- 172.21.15.137:0/510509118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c101770 msgr2=0x7f039c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a1f2d700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c101770 0x7f039c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a1f2d700 1 -- 172.21.15.137:0/510509118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0095f0 con 0x7f039c100dc0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a1f2d700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f039800ef10 tx=0x7f039800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039800c9f0 con 0x7f039c100dc0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f039800f040 con 0x7f039c100dc0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c19a250 con 0x7f039c100dc0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0398014480 con 0x7f039c100dc0 2024-06-15T01:30:44.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.774+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c19a770 con 0x7f039c100dc0 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.775+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c1091c0 con 0x7f039c100dc0 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.775+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0398007500 con 0x7f039c100dc0 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.776+0000 7f0392ffd700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 0x7f038806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.776+0000 7f03a172c700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 0x7f038806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.776+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f039808a6c0 con 0x7f039c100dc0 2024-06-15T01:30:44.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.776+0000 7f03a172c700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 0x7f038806d440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f038c009fd0 tx=0x7f038c005940 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:44.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.780+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03980554d0 con 0x7f039c100dc0 2024-06-15T01:30:44.943 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: pgmap v740: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 14 KiB/s wr, 13 op/s 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.944 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:44 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:44.999+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f039c19a9b0 con 0x7f038806af80 2024-06-15T01:30:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.000+0000 7f0392ffd700 1 -- 172.21.15.137:0/510509118 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f039c19a9b0 con 0x7f038806af80 2024-06-15T01:30:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 msgr2=0x7f038806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 0x7f038806d440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f038c009fd0 tx=0x7f038c005940 comp rx=0 tx=0).stop 2024-06-15T01:30:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 msgr2=0x7f039c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f039800ef10 tx=0x7f039800c3a0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 shutdown_connections 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f038806af80 0x7f038806d440 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f039c100dc0 0x7f039c1966d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 --2- 172.21.15.137:0/510509118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f039c101770 0x7f039c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 >> 172.21.15.137:0/510509118 conn(0x7f039c0fc650 msgr2=0x7f039c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 shutdown_connections 2024-06-15T01:30:44.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.002+0000 7f03a4191700 1 -- 172.21.15.137:0/510509118 wait complete. 2024-06-15T01:30:44.998 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: pgmap v740: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 14 KiB/s wr, 13 op/s 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:44 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:45.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 -- 172.21.15.137:0/3196263830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 msgr2=0x7f2cd80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3196263830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd80ffec0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009910 tx=0x7f2ccc009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 -- 172.21.15.137:0/3196263830 shutdown_connections 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3196263830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd80ffec0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3196263830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 -- 172.21.15.137:0/3196263830 >> 172.21.15.137:0/3196263830 conn(0x7f2cd80fb530 msgr2=0x7f2cd80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.107+0000 7f2cdca60700 1 -- 172.21.15.137:0/3196263830 shutdown_connections 2024-06-15T01:30:45.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 -- 172.21.15.137:0/3196263830 wait complete. 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 Processor -- start 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 -- start start 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8198140 con 0x7f2cd80ffa40 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.108+0000 7f2cdca60700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81982b0 con 0x7f2cd8105a40 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44934/0 (socket says 172.21.15.137:44934) 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 -- 172.21.15.137:0/3861698855 learned_addr learned my addr 172.21.15.137:0/3861698855 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd5d9b700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 -- 172.21.15.137:0/3861698855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 msgr2=0x7f2cd8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 -- 172.21.15.137:0/3861698855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0095f0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd5d9b700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cd659c700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800d8d0 tx=0x7f2cc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc8009940 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd81a0590 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cc8010460 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc800f5f0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.109+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd81a0ae0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.110+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd8107fd0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.111+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2cc80105d0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.111+0000 7f2cc77fe700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 0x7f2cc006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.111+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2cc808a3e0 con 0x7f2cd80ffa40 2024-06-15T01:30:45.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.111+0000 7f2cd5d9b700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 0x7f2cc006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.112+0000 7f2cd5d9b700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 0x7f2cc006d440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8069680 tx=0x7f2ccc01a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.115+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cc8055210 con 0x7f2cd80ffa40 2024-06-15T01:30:45.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.330+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cd80611d0 con 0x7f2cc006af80 2024-06-15T01:30:45.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.331+0000 7f2cc77fe700 1 -- 172.21.15.137:0/3861698855 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f2cd80611d0 con 0x7f2cc006af80 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 msgr2=0x7f2cc006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 0x7f2cc006d440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8069680 tx=0x7f2ccc01a040 comp rx=0 tx=0).stop 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 msgr2=0x7f2cd8197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800d8d0 tx=0x7f2cc800dc90 comp rx=0 tx=0).stop 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 shutdown_connections 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2cc006af80 0x7f2cc006d440 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cd80ffa40 0x7f2cd8197520 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 --2- 172.21.15.137:0/3861698855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd8105a40 0x7f2cd8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 >> 172.21.15.137:0/3861698855 conn(0x7f2cd80fb530 msgr2=0x7f2cd80fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 shutdown_connections 2024-06-15T01:30:45.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.334+0000 7f2cdca60700 1 -- 172.21.15.137:0/3861698855 wait complete. 2024-06-15T01:30:45.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 -- 172.21.15.137:0/95087977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f60105a40 msgr2=0x7f3f60105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 --2- 172.21.15.137:0/95087977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f60105a40 0x7f3f60105e20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009910 tx=0x7f3f5c009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 -- 172.21.15.137:0/95087977 shutdown_connections 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 --2- 172.21.15.137:0/95087977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f600ffbd0 0x7f3f60100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 --2- 172.21.15.137:0/95087977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f60105a40 0x7f3f60105e20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.439+0000 7f3f67534700 1 -- 172.21.15.137:0/95087977 >> 172.21.15.137:0/95087977 conn(0x7f3f600755a0 msgr2=0x7f3f600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:45.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 -- 172.21.15.137:0/95087977 shutdown_connections 2024-06-15T01:30:45.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 -- 172.21.15.137:0/95087977 wait complete. 2024-06-15T01:30:45.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 Processor -- start 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 -- start start 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f60105a40 0x7f3f60197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60198020 con 0x7f3f600ffbd0 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f67534700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60198160 con 0x7f3f60105a40 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f652d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f652d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44958/0 (socket says 172.21.15.137:44958) 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.440+0000 7f3f652d0700 1 -- 172.21.15.137:0/4266509412 learned_addr learned my addr 172.21.15.137:0/4266509412 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f64acf700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f60105a40 0x7f3f60197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f652d0700 1 -- 172.21.15.137:0/4266509412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f60105a40 msgr2=0x7f3f60197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f652d0700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f60105a40 0x7f3f60197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f652d0700 1 -- 172.21.15.137:0/4266509412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0095f0 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f652d0700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006010 tx=0x7f3f5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c01d070 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f5c005710 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f601a0530 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c00fe20 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.441+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f601a0940 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.442+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f60108040 con 0x7f3f600ffbd0 2024-06-15T01:30:45.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.443+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3f5c003a60 con 0x7f3f600ffbd0 2024-06-15T01:30:45.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.443+0000 7f3f567fc700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 0x7f3f4c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.443+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3f5c08baf0 con 0x7f3f600ffbd0 2024-06-15T01:30:45.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.443+0000 7f3f64acf700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 0x7f3f4c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.444+0000 7f3f64acf700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 0x7f3f4c06d440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f60068cc0 tx=0x7f3f5000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.447+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3f5c056920 con 0x7f3f600ffbd0 2024-06-15T01:30:45.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.650+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f60198350 con 0x7f3f4c06af80 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.660+0000 7f3f567fc700 1 -- 172.21.15.137:0/4266509412 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3f50015070 con 0x7f3f4c06af80 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (21m) 1s ago 31m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (14s) 1s ago 32m 7142k - 18.2.2-1443-g6f4b0956 e7282a3bffba ca1313e53d0b 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (7s) 4s ago 31m 6803k - 18.2.2-1443-g6f4b0956 e7282a3bffba f476b89ec466 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (19m) 1s ago 32m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (19m) 4s ago 30m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (21m) 1s ago 31m 88.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:30:45.645 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (15m) 1s ago 26m 69.4M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (16m) 1s ago 26m 61.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (15m) 4s ago 26m 22.0M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (15m) 4s ago 26m 20.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (23m) 1s ago 33m 502M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (22m) 4s ago 30m 447M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (19m) 1s ago 33m 69.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (19m) 4s ago 30m 51.2M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (21m) 1s ago 32m 15.8M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (21m) 4s ago 30m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:30:45.646 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (19m) 1s ago 30m 194M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (18m) 1s ago 29m 113M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (18m) 1s ago 29m 123M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (17m) 4s ago 28m 123M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (16m) 4s ago 28m 116M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (16m) 4s ago 27m 112M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:30:45.647 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (21m) 1s ago 31m 77.9M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 msgr2=0x7f3f4c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 0x7f3f4c06d440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f60068cc0 tx=0x7f3f5000b410 comp rx=0 tx=0).stop 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 msgr2=0x7f3f601974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006010 tx=0x7f3f5c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 shutdown_connections 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f4c06af80 0x7f3f4c06d440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f600ffbd0 0x7f3f601974c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 --2- 172.21.15.137:0/4266509412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f60105a40 0x7f3f60197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 >> 172.21.15.137:0/4266509412 conn(0x7f3f600755a0 msgr2=0x7f3f600fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:45.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 shutdown_connections 2024-06-15T01:30:45.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.662+0000 7f3f67534700 1 -- 172.21.15.137:0/4266509412 wait complete. 2024-06-15T01:30:45.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 -- 172.21.15.137:0/2664262842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d580ffbd0 msgr2=0x7f1d58100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/2664262842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d580ffbd0 0x7f1d58100050 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009910 tx=0x7f1d4c009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:45.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 -- 172.21.15.137:0/2664262842 shutdown_connections 2024-06-15T01:30:45.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/2664262842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d580ffbd0 0x7f1d58100050 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/2664262842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d58105a40 0x7f1d58105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 -- 172.21.15.137:0/2664262842 >> 172.21.15.137:0/2664262842 conn(0x7f1d580755a0 msgr2=0x7f1d580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:45.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 -- 172.21.15.137:0/2664262842 shutdown_connections 2024-06-15T01:30:45.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 -- 172.21.15.137:0/2664262842 wait complete. 2024-06-15T01:30:45.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.766+0000 7f1d5de1e700 1 Processor -- start 2024-06-15T01:30:45.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d5de1e700 1 -- start start 2024-06-15T01:30:45.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d5de1e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d580ffbd0 0x7f1d58195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d5de1e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d5de1e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d581960d0 con 0x7f1d58105a40 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d5de1e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d58196210 con 0x7f1d580ffbd0 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44986/0 (socket says 172.21.15.137:44986) 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 -- 172.21.15.137:0/3605684845 learned_addr learned my addr 172.21.15.137:0/3605684845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d577fe700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d580ffbd0 0x7f1d58195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 -- 172.21.15.137:0/3605684845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d580ffbd0 msgr2=0x7f1d58195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d580ffbd0 0x7f1d58195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.767+0000 7f1d56ffd700 1 -- 172.21.15.137:0/3605684845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c0095f0 con 0x7f1d58105a40 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d56ffd700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c006010 tx=0x7f1d4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d4c01d070 con 0x7f1d58105a40 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5819a000 con 0x7f1d58105a40 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5819a4f0 con 0x7f1d58105a40 2024-06-15T01:30:45.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d4c005940 con 0x7f1d58105a40 2024-06-15T01:30:45.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.768+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d4c021410 con 0x7f1d58105a40 2024-06-15T01:30:45.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.769+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d58107fd0 con 0x7f1d58105a40 2024-06-15T01:30:45.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.769+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f1d4c003a60 con 0x7f1d58105a40 2024-06-15T01:30:45.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.770+0000 7f1d54ff9700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 0x7f1d4006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:45.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.770+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1d4c08b620 con 0x7f1d58105a40 2024-06-15T01:30:45.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.770+0000 7f1d577fe700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 0x7f1d4006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:45.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.770+0000 7f1d577fe700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 0x7f1d4006d440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800ac50 tx=0x7f1d4800a380 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:45.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:45.774+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1d4c056450 con 0x7f1d58105a40 2024-06-15T01:30:45.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:45 smithi137 ceph-mon[112295]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:45.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:45 smithi137 ceph-mon[112295]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:46.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.069+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1d5804ea90 con 0x7f1d58105a40 2024-06-15T01:30:46.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.069+0000 7f1d54ff9700 1 -- 172.21.15.137:0/3605684845 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1d4c05ee60 con 0x7f1d58105a40 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:30:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:30:46.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:30:46.056 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:30:46.056 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:30:46.056 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:30:46.056 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:30:46.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:45 smithi142 ceph-mon[82136]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:46.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:45 smithi142 ceph-mon[82136]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 msgr2=0x7f1d4006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 0x7f1d4006d440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800ac50 tx=0x7f1d4800a380 comp rx=0 tx=0).stop 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 msgr2=0x7f1d58195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c006010 tx=0x7f1d4c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 shutdown_connections 2024-06-15T01:30:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1d4006af80 0x7f1d4006d440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d580ffbd0 0x7f1d58195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 --2- 172.21.15.137:0/3605684845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d58105a40 0x7f1d58195a80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 >> 172.21.15.137:0/3605684845 conn(0x7f1d580755a0 msgr2=0x7f1d580fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 shutdown_connections 2024-06-15T01:30:46.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.072+0000 7f1d5de1e700 1 -- 172.21.15.137:0/3605684845 wait complete. 2024-06-15T01:30:46.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- 172.21.15.137:0/84139821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 msgr2=0x7f2fe80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/84139821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe80ffec0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009910 tx=0x7f2fd8009c20 comp rx=0 tx=0).stop 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- 172.21.15.137:0/84139821 shutdown_connections 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/84139821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe80ffec0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/84139821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- 172.21.15.137:0/84139821 >> 172.21.15.137:0/84139821 conn(0x7f2fe80fb570 msgr2=0x7f2fe80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- 172.21.15.137:0/84139821 shutdown_connections 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- 172.21.15.137:0/84139821 wait complete. 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 Processor -- start 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.181+0000 7f2fef5a6700 1 -- start start 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fef5a6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fef5a6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fef5a6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8196190 con 0x7f2fe80ffa40 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fef5a6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8199f20 con 0x7f2fe8105a40 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48156/0 (socket says 172.21.15.137:48156) 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 -- 172.21.15.137:0/3570947582 learned_addr learned my addr 172.21.15.137:0/3570947582 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fecb41700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 -- 172.21.15.137:0/3570947582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 msgr2=0x7f2fe8195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 -- 172.21.15.137:0/3570947582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80095f0 con 0x7f2fe80ffa40 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fecb41700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fed342700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d8d0 tx=0x7f2fe400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fe4009940 con 0x7f2fe80ffa40 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2fe4010460 con 0x7f2fe80ffa40 2024-06-15T01:30:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe819a200 con 0x7f2fe80ffa40 2024-06-15T01:30:46.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.182+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fe400f5f0 con 0x7f2fe80ffa40 2024-06-15T01:30:46.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.183+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe819a750 con 0x7f2fe80ffa40 2024-06-15T01:30:46.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.183+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe8107fd0 con 0x7f2fe80ffa40 2024-06-15T01:30:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.184+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2fe40105d0 con 0x7f2fe80ffa40 2024-06-15T01:30:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.184+0000 7f2fde7fc700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 0x7f2fd406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.184+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2fe408a2e0 con 0x7f2fe80ffa40 2024-06-15T01:30:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.184+0000 7f2fecb41700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 0x7f2fd406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.185+0000 7f2fecb41700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 0x7f2fd406d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8005b40 tx=0x7f2fd801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.189+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2fe4055190 con 0x7f2fe80ffa40 2024-06-15T01:30:46.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.439+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2fe819aa80 con 0x7f2fe80ffa40 2024-06-15T01:30:46.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.440+0000 7f2fde7fc700 1 -- 172.21.15.137:0/3570947582 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1864 (secure 0 0 0) 0x7f2fe4010880 con 0x7f2fe80ffa40 2024-06-15T01:30:46.425 INFO:teuthology.orchestra.run.smithi137.stdout:e38 2024-06-15T01:30:46.425 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 38 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:30:35.389785+0000 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:30:46.426 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=34338,1=34342} 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:30:46.427 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:34338} state up:active seq 243 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:34342} state up:active seq 238 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:30:46.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 msgr2=0x7f2fd406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 0x7f2fd406d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8005b40 tx=0x7f2fd801a040 comp rx=0 tx=0).stop 2024-06-15T01:30:46.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 msgr2=0x7f2fe8195570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d8d0 tx=0x7f2fe400dc90 comp rx=0 tx=0).stop 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 shutdown_connections 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2fd406af80 0x7f2fd406d440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe80ffa40 0x7f2fe8195570 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 --2- 172.21.15.137:0/3570947582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fe8105a40 0x7f2fe8195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 >> 172.21.15.137:0/3570947582 conn(0x7f2fe80fb570 msgr2=0x7f2fe80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 shutdown_connections 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.443+0000 7f2fef5a6700 1 -- 172.21.15.137:0/3570947582 wait complete. 2024-06-15T01:30:46.430 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 38 2024-06-15T01:30:46.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 -- 172.21.15.137:0/3953211925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 msgr2=0x7f5d8c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 --2- 172.21.15.137:0/3953211925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c068a50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b0a0 tx=0x7f5d7c00b3b0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 -- 172.21.15.137:0/3953211925 shutdown_connections 2024-06-15T01:30:46.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 --2- 172.21.15.137:0/3953211925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 0x7f5d8c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 --2- 172.21.15.137:0/3953211925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c068a50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 -- 172.21.15.137:0/3953211925 >> 172.21.15.137:0/3953211925 conn(0x7f5d8c0755a0 msgr2=0x7f5d8c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.552+0000 7f5d939f3700 1 -- 172.21.15.137:0/3953211925 shutdown_connections 2024-06-15T01:30:46.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.553+0000 7f5d939f3700 1 -- 172.21.15.137:0/3953211925 wait complete. 2024-06-15T01:30:46.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.553+0000 7f5d939f3700 1 Processor -- start 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.553+0000 7f5d939f3700 1 -- start start 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d939f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d939f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 0x7f5d8c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d939f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c19d100 con 0x7f5d8c068670 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d939f3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c196620 con 0x7f5d8c069020 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48172/0 (socket says 172.21.15.137:48172) 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 -- 172.21.15.137:0/440636541 learned_addr learned my addr 172.21.15.137:0/440636541 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d90f8e700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 0x7f5d8c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 -- 172.21.15.137:0/440636541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 msgr2=0x7f5d8c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 0x7f5d8c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 -- 172.21.15.137:0/440636541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009d00 con 0x7f5d8c068670 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d9178f700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c015040 tx=0x7f5d7c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d7c00e040 con 0x7f5d8c068670 2024-06-15T01:30:46.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1968a0 con 0x7f5d8c068670 2024-06-15T01:30:46.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d7c003de0 con 0x7f5d8c068670 2024-06-15T01:30:46.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.554+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d7c01d7d0 con 0x7f5d8c068670 2024-06-15T01:30:46.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.555+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c196d90 con 0x7f5d8c068670 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.556+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c109130 con 0x7f5d8c068670 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.556+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5d7c004770 con 0x7f5d8c068670 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.557+0000 7f5d8a7fc700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 0x7f5d7406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.557+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5d7c08bcc0 con 0x7f5d8c068670 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.557+0000 7f5d90f8e700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 0x7f5d7406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.557+0000 7f5d90f8e700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 0x7f5d7406d440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c197df0 tx=0x7f5d8000b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.562+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d7c05a580 con 0x7f5d8c068670 2024-06-15T01:30:46.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.807+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d8c197c80 con 0x7f5d7406af80 2024-06-15T01:30:46.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.809+0000 7f5d8a7fc700 1 -- 172.21.15.137:0/440636541 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f5d8c197c80 con 0x7f5d7406af80 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": true, 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "Upgrading all daemon types on all hosts", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [ 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "osd", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph-exporter", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "mon", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "mds", 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr" 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-06-15T01:30:46.794 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": "18/23 daemons upgraded", 2024-06-15T01:30:46.795 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-06-15T01:30:46.795 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:30:46.795 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:30:46.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.811+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 msgr2=0x7f5d7406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.811+0000 7f5d939f3700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 0x7f5d7406d440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c197df0 tx=0x7f5d8000b410 comp rx=0 tx=0).stop 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.811+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 msgr2=0x7f5d8c19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.811+0000 7f5d939f3700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c015040 tx=0x7f5d7c007b10 comp rx=0 tx=0).stop 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 shutdown_connections 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5d7406af80 0x7f5d7406d440 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d8c068670 0x7f5d8c19c5a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 --2- 172.21.15.137:0/440636541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8c069020 0x7f5d8c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 >> 172.21.15.137:0/440636541 conn(0x7f5d8c0755a0 msgr2=0x7f5d8c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 shutdown_connections 2024-06-15T01:30:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.812+0000 7f5d939f3700 1 -- 172.21.15.137:0/440636541 wait complete. 2024-06-15T01:30:46.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 -- 172.21.15.137:0/3136690361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 msgr2=0x7fc754103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 --2- 172.21.15.137:0/3136690361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754103e00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc74400aa20 tx=0x7fc74400ad30 comp rx=0 tx=0).stop 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 -- 172.21.15.137:0/3136690361 shutdown_connections 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 --2- 172.21.15.137:0/3136690361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754103e00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 --2- 172.21.15.137:0/3136690361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 0x7fc7540ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 -- 172.21.15.137:0/3136690361 >> 172.21.15.137:0/3136690361 conn(0x7fc754075e30 msgr2=0x7fc7540fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 -- 172.21.15.137:0/3136690361 shutdown_connections 2024-06-15T01:30:46.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.920+0000 7fc7593d9700 1 -- 172.21.15.137:0/3136690361 wait complete. 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.921+0000 7fc7593d9700 1 Processor -- start 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.921+0000 7fc7593d9700 1 -- start start 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7593d9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 0x7fc754194440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7593d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7593d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754195060 con 0x7fc7540ffdb0 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7593d9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc754198df0 con 0x7fc7540ff400 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48190/0 (socket says 172.21.15.137:48190) 2024-06-15T01:30:46.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 -- 172.21.15.137:0/1631604253 learned_addr learned my addr 172.21.15.137:0/1631604253 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc752ffd700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 0x7fc754194440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 -- 172.21.15.137:0/1631604253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 msgr2=0x7fc754194440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 0x7fc754194440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 -- 172.21.15.137:0/1631604253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74400a700 con 0x7fc7540ffdb0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.922+0000 7fc7527fc700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc74400a3f0 tx=0x7fc74400ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.923+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc744005720 con 0x7fc7540ffdb0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.923+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc744005070 con 0x7fc7540ffdb0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.923+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc754199070 con 0x7fc7540ffdb0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.923+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74401d9a0 con 0x7fc7540ffdb0 2024-06-15T01:30:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.923+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc754199480 con 0x7fc7540ffdb0 2024-06-15T01:30:46.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.924+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc754107800 con 0x7fc7540ffdb0 2024-06-15T01:30:46.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.924+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fc74401dbd0 con 0x7fc7540ffdb0 2024-06-15T01:30:46.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.925+0000 7fc73bfff700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 0x7fc73c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:30:46.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.925+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fc744012070 con 0x7fc7540ffdb0 2024-06-15T01:30:46.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.925+0000 7fc752ffd700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 0x7fc73c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:30:46.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.926+0000 7fc752ffd700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 0x7fc73c075c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc748009c60 tx=0x7fc748009260 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:30:46.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:46.929+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc744056d30 con 0x7fc7540ffdb0 2024-06-15T01:30:47.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:46 smithi142 ceph-mon[82136]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:47.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:46 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3605684845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:46 smithi142 ceph-mon[82136]: pgmap v741: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-06-15T01:30:47.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:46 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3570947582' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:30:47.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.217+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc75404ea90 con 0x7fc7540ffdb0 2024-06-15T01:30:47.202 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:46 smithi137 ceph-mon[112295]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:47.202 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:46 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3605684845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.203 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:46 smithi137 ceph-mon[112295]: pgmap v741: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-06-15T01:30:47.203 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:46 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3570947582' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.217+0000 7fc73bfff700 1 -- 172.21.15.137:0/1631604253 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3938 (secure 0 0 0) 0x7fc74405a350 con 0x7fc7540ffdb0 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN failed to probe daemons or devices; 4 mgr modules have recently crashed 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stdout: host smithi142 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi142/config 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/podman: timeout after 895 seconds 2024-06-15T01:30:47.203 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 -e NODE_NAME=smithi142 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpjpakf3k_:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 inventory --format=json-pretty --filter-for-batch 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout:Traceback (most recent call last): 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: return _run_code(code, main_globals, None, 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: exec(code, run_globals) 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 10889, in 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 10877, in main 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 2576, in _infer_config 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 2604, in _infer_image 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-06-15T01:30:47.204 INFO:teuthology.orchestra.run.smithi137.stdout: File "/tmp/tmpv71d1xxp.cephadm.build/__main__.py", line 2267, in call_throws 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 -e NODE_NAME=smithi142 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpjpakf3k_:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 inventory --format=json-pretty --filter-for-batch 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:30:47.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 msgr2=0x7fc73c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 0x7fc73c075c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc748009c60 tx=0x7fc748009260 comp rx=0 tx=0).stop 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 msgr2=0x7fc754194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc74400a3f0 tx=0x7fc74400ff80 comp rx=0 tx=0).stop 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 shutdown_connections 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fc73c073790 0x7fc73c075c50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7540ff400 0x7fc754194440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 --2- 172.21.15.137:0/1631604253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7540ffdb0 0x7fc754194980 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 >> 172.21.15.137:0/1631604253 conn(0x7fc754075e30 msgr2=0x7fc754103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.219+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 shutdown_connections 2024-06-15T01:30:47.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:30:47.220+0000 7fc7593d9700 1 -- 172.21.15.137:0/1631604253 wait complete. 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1631604253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.964 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.965 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.965 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:47.965 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:47 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1631604253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:48.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:48.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:47 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-06-15T01:30:49.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: pgmap v742: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: Upgrade: Setting filesystem cephfs Joinable 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi137"}]: dispatch 2024-06-15T01:30:49.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi137"}]': finished 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]': finished 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-15T01:30:49.309 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.310 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:49.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: pgmap v742: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-06-15T01:30:49.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: Upgrade: Setting filesystem cephfs Joinable 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: fsmap cephfs:2 {0=cephfs.smithi137.nsntjq=up:active,1=cephfs.smithi137.afplli=up:active} 2 up:standby 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi137"}]: dispatch 2024-06-15T01:30:49.465 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi137"}]': finished 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]': finished 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-06-15T01:30:49.467 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.468 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:30:49.469 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:30:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:30:49.470 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:50.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all ceph-exporter 2024-06-15T01:30:50.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all iscsi 2024-06-15T01:30:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all nfs 2024-06-15T01:30:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Setting container_image for all nvmeof 2024-06-15T01:30:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Finalizing container_image settings 2024-06-15T01:30:50.307 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:50 smithi142 ceph-mon[82136]: Upgrade: Complete! 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all ceph-exporter 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all iscsi 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all nfs 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Setting container_image for all nvmeof 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Finalizing container_image settings 2024-06-15T01:30:50.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:50 smithi137 ceph-mon[112295]: Upgrade: Complete! 2024-06-15T01:30:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:51 smithi142 ceph-mon[82136]: pgmap v743: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-06-15T01:30:51.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:51 smithi137 ceph-mon[112295]: pgmap v743: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-06-15T01:30:53.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:53 smithi142 ceph-mon[82136]: pgmap v744: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s wr, 2 op/s 2024-06-15T01:30:53.387 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:53 smithi137 ceph-mon[112295]: pgmap v744: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s wr, 2 op/s 2024-06-15T01:30:54.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:54 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:54.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:54 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:30:55.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:55 smithi137 ceph-mon[112295]: pgmap v745: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s wr, 2 op/s 2024-06-15T01:30:55.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:55 smithi142 ceph-mon[82136]: pgmap v745: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s wr, 2 op/s 2024-06-15T01:30:57.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:57 smithi137 ceph-mon[112295]: pgmap v746: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:57.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:57 smithi142 ceph-mon[82136]: pgmap v746: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:59.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:30:59 smithi142 ceph-mon[82136]: pgmap v747: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:30:59.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:30:59 smithi137 ceph-mon[112295]: pgmap v747: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:01.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:01 smithi137 ceph-mon[112295]: pgmap v748: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:01.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:01 smithi142 ceph-mon[82136]: pgmap v748: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:03.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:03 smithi137 ceph-mon[112295]: pgmap v749: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:03.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:31:03.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:03.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:03 smithi142 ceph-mon[82136]: pgmap v749: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:03.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:31:03.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:05.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:05 smithi137 ceph-mon[112295]: pgmap v750: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:05.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:05 smithi142 ceph-mon[82136]: pgmap v750: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:07.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:07 smithi137 ceph-mon[112295]: pgmap v751: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:07.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:07 smithi142 ceph-mon[82136]: pgmap v751: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:09.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:09 smithi137 ceph-mon[112295]: pgmap v752: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:09.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:09 smithi142 ceph-mon[82136]: pgmap v752: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:11.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:11 smithi137 ceph-mon[112295]: pgmap v753: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:11.548 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:11 smithi142 ceph-mon[82136]: pgmap v753: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:13.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:13 smithi137 ceph-mon[112295]: pgmap v754: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:13.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:13 smithi142 ceph-mon[82136]: pgmap v754: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:15.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:15 smithi137 ceph-mon[112295]: pgmap v755: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:15.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:15 smithi142 ceph-mon[82136]: pgmap v755: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 -- 172.21.15.137:0/1376332556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774101750 msgr2=0x7f37741057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 --2- 172.21.15.137:0/1376332556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774101750 0x7f37741057a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3768009910 tx=0x7f3768009c20 comp rx=0 tx=0).stop 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 -- 172.21.15.137:0/1376332556 shutdown_connections 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 --2- 172.21.15.137:0/1376332556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774101750 0x7f37741057a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 --2- 172.21.15.137:0/1376332556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774100da0 0x7f3774101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 -- 172.21.15.137:0/1376332556 >> 172.21.15.137:0/1376332556 conn(0x7f37740fc650 msgr2=0x7f37740fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 -- 172.21.15.137:0/1376332556 shutdown_connections 2024-06-15T01:31:17.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.338+0000 7f3778c6b700 1 -- 172.21.15.137:0/1376332556 wait complete. 2024-06-15T01:31:17.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 Processor -- start 2024-06-15T01:31:17.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 -- start start 2024-06-15T01:31:17.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:17.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774101750 0x7f3774196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:17.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3774197290 con 0x7f3774100da0 2024-06-15T01:31:17.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f3778c6b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419b020 con 0x7f3774101750 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.339+0000 7f377259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57564/0 (socket says 172.21.15.137:57564) 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 -- 172.21.15.137:0/3438173771 learned_addr learned my addr 172.21.15.137:0/3438173771 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f3771d9b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774101750 0x7f3774196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 -- 172.21.15.137:0/3438173771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774101750 msgr2=0x7f3774196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774101750 0x7f3774196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 -- 172.21.15.137:0/3438173771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680095f0 con 0x7f3774100da0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f377259c700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f376400c8a0 tx=0x7f376400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37640078e0 con 0x7f3774100da0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f376400f450 con 0x7f3774100da0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377419b300 con 0x7f3774100da0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376400e5e0 con 0x7f3774100da0 2024-06-15T01:31:17.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.340+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377419b850 con 0x7f3774100da0 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.341+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37741091a0 con 0x7f3774100da0 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.342+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f376400f5c0 con 0x7f3774100da0 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.342+0000 7f37637fe700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 0x7f375c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.342+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3764089fa0 con 0x7f3774100da0 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.342+0000 7f3771d9b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 0x7f375c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.343+0000 7f3771d9b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 0x7f375c06d440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3774197c90 tx=0x7f376801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.347+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3764054dd0 con 0x7f3774100da0 2024-06-15T01:31:17.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:17 smithi137 ceph-mon[112295]: pgmap v756: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:17.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.558+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f377419bbc0 con 0x7f375c06af80 2024-06-15T01:31:17.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.559+0000 7f37637fe700 1 -- 172.21.15.137:0/3438173771 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f377419bbc0 con 0x7f375c06af80 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 msgr2=0x7f375c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 0x7f375c06d440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3774197c90 tx=0x7f376801a040 comp rx=0 tx=0).stop 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 msgr2=0x7f3774196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f376400c8a0 tx=0x7f376400cbb0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 shutdown_connections 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f375c06af80 0x7f375c06d440 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3774100da0 0x7f3774196670 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 --2- 172.21.15.137:0/3438173771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3774101750 0x7f3774196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:17.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 >> 172.21.15.137:0/3438173771 conn(0x7f37740fc650 msgr2=0x7f37740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:17.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 shutdown_connections 2024-06-15T01:31:17.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:17.561+0000 7f3778c6b700 1 -- 172.21.15.137:0/3438173771 wait complete. 2024-06-15T01:31:17.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:17 smithi142 ceph-mon[82136]: pgmap v756: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:18.165 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch ps' 2024-06-15T01:31:18.373 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:18.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:18.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:19.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:19 smithi137 ceph-mon[112295]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:19.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:19 smithi137 ceph-mon[112295]: pgmap v757: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:19.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:19 smithi142 ceph-mon[82136]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:19.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:19 smithi142 ceph-mon[82136]: pgmap v757: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 -- 172.21.15.137:0/99133812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ff510 msgr2=0x7fa05c0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 --2- 172.21.15.137:0/99133812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ff510 0x7fa05c0ff8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa058009910 tx=0x7fa058009c20 comp rx=0 tx=0).stop 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 -- 172.21.15.137:0/99133812 shutdown_connections 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 --2- 172.21.15.137:0/99133812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ffec0 0x7fa05c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 --2- 172.21.15.137:0/99133812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ff510 0x7fa05c0ff8f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 -- 172.21.15.137:0/99133812 >> 172.21.15.137:0/99133812 conn(0x7fa05c074d80 msgr2=0x7fa05c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 -- 172.21.15.137:0/99133812 shutdown_connections 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.240+0000 7fa06404e700 1 -- 172.21.15.137:0/99133812 wait complete. 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 Processor -- start 2024-06-15T01:31:20.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 -- start start 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ff510 0x7fa05c196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c197270 con 0x7fa05c0ffec0 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa06404e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c19b000 con 0x7fa05c0ff510 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57582/0 (socket says 172.21.15.137:57582) 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 -- 172.21.15.137:0/2579292208 learned_addr learned my addr 172.21.15.137:0/2579292208 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa061dea700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ff510 0x7fa05c196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 -- 172.21.15.137:0/2579292208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ff510 msgr2=0x7fa05c196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ff510 0x7fa05c196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 -- 172.21.15.137:0/2579292208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580095f0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa0615e9700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c8f0 tx=0x7fa04c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:20.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.241+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c0078a0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.242+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa04c00f450 con 0x7fa05c0ffec0 2024-06-15T01:31:20.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.242+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c19b2e0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.242+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c007ea0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.242+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c19b830 con 0x7fa05c0ffec0 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.243+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c1077d0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.243+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa04c007a00 con 0x7fa05c0ffec0 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.243+0000 7fa052ffd700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 0x7fa04806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.243+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa04c08a2d0 con 0x7fa05c0ffec0 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.243+0000 7fa061dea700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 0x7fa04806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.244+0000 7fa061dea700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 0x7fa04806d440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa058005f50 tx=0x7fa05801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:20.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.248+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa04c055100 con 0x7fa05c0ffec0 2024-06-15T01:31:20.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.452+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa05c19bb80 con 0x7fa04806af80 2024-06-15T01:31:20.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.462+0000 7fa052ffd700 1 -- 172.21.15.137:0/2579292208 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa05c19bb80 con 0x7fa04806af80 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (21m) 36s ago 32m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (49s) 36s ago 32m 7142k - 18.2.2-1443-g6f4b0956 e7282a3bffba ca1313e53d0b 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (42s) 39s ago 31m 6803k - 18.2.2-1443-g6f4b0956 e7282a3bffba f476b89ec466 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (20m) 36s ago 32m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (20m) 39s ago 31m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (21m) 36s ago 32m 88.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (16m) 36s ago 26m 69.4M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (16m) 36s ago 26m 61.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (16m) 39s ago 26m 22.0M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:31:20.446 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (15m) 39s ago 26m 20.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (23m) 36s ago 34m 502M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (22m) 39s ago 31m 447M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (20m) 36s ago 34m 69.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (20m) 39s ago 31m 51.2M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (22m) 36s ago 32m 15.8M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (22m) 39s ago 31m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (19m) 36s ago 30m 194M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (19m) 36s ago 30m 113M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (18m) 36s ago 29m 123M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (18m) 39s ago 29m 123M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (17m) 39s ago 28m 116M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (16m) 39s ago 28m 112M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:31:20.447 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (22m) 36s ago 31m 77.9M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 msgr2=0x7fa04806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 0x7fa04806d440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa058005f50 tx=0x7fa05801a040 comp rx=0 tx=0).stop 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 msgr2=0x7fa05c196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c8f0 tx=0x7fa04c00cc00 comp rx=0 tx=0).stop 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 shutdown_connections 2024-06-15T01:31:20.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa04806af80 0x7fa04806d440 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05c0ff510 0x7fa05c196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 --2- 172.21.15.137:0/2579292208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c0ffec0 0x7fa05c196b90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:20.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 >> 172.21.15.137:0/2579292208 conn(0x7fa05c074d80 msgr2=0x7fa05c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:20.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 shutdown_connections 2024-06-15T01:31:20.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:20.464+0000 7fa06404e700 1 -- 172.21.15.137:0/2579292208 wait complete. 2024-06-15T01:31:21.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch upgrade status' 2024-06-15T01:31:21.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:21 smithi142 ceph-mon[82136]: pgmap v758: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:21.325 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:21.437 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:21 smithi137 ceph-mon[112295]: pgmap v758: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:22.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:22 smithi137 ceph-mon[112295]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:22.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:22 smithi142 ceph-mon[82136]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:23.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/1510793270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 msgr2=0x7fcfe0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/1510793270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0105e20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009910 tx=0x7fcfd0009c20 comp rx=0 tx=0).stop 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/1510793270 shutdown_connections 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/1510793270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 0x7fcfe00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/1510793270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0105e20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/1510793270 >> 172.21.15.137:0/1510793270 conn(0x7fcfe00fb570 msgr2=0x7fcfe00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/1510793270 shutdown_connections 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/1510793270 wait complete. 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 Processor -- start 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.072+0000 7fcfe4cfd700 1 -- start start 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfe4cfd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 0x7fcfe0195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:23.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfe4cfd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfe4cfd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01961a0 con 0x7fcfe0105a40 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfe4cfd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0199f30 con 0x7fcfe00ffa40 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57600/0 (socket says 172.21.15.137:57600) 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 -- 172.21.15.137:0/460330293 learned_addr learned my addr 172.21.15.137:0/460330293 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfde59c700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 0x7fcfe0195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 -- 172.21.15.137:0/460330293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 msgr2=0x7fcfe0195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 0x7fcfe0195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 -- 172.21.15.137:0/460330293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00095f0 con 0x7fcfe0105a40 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfddd9b700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400d8d0 tx=0x7fcfd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.073+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd4009940 con 0x7fcfe0105a40 2024-06-15T01:31:23.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.074+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe019a210 con 0x7fcfe0105a40 2024-06-15T01:31:23.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.074+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfd4010460 con 0x7fcfe0105a40 2024-06-15T01:31:23.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.074+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe019a760 con 0x7fcfe0105a40 2024-06-15T01:31:23.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.074+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd400f5f0 con 0x7fcfe0105a40 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.075+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0108040 con 0x7fcfe0105a40 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.075+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fcfd40105d0 con 0x7fcfe0105a40 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.076+0000 7fcfcf7fe700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 0x7fcfc806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.076+0000 7fcfde59c700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 0x7fcfc806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.076+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fcfd408a430 con 0x7fcfe0105a40 2024-06-15T01:31:23.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.076+0000 7fcfde59c700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 0x7fcfc806d440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd001a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:23.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.080+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfd40551c0 con 0x7fcfe0105a40 2024-06-15T01:31:23.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.292+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfe019ab00 con 0x7fcfc806af80 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.293+0000 7fcfcf7fe700 1 -- 172.21.15.137:0/460330293 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fcfe019ab00 con 0x7fcfc806af80 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "target_image": null, 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "in_progress": false, 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "which": "", 2024-06-15T01:31:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "services_complete": [], 2024-06-15T01:31:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "progress": null, 2024-06-15T01:31:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "message": "", 2024-06-15T01:31:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "is_paused": false 2024-06-15T01:31:23.279 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:31:23.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 msgr2=0x7fcfc806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:23.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 0x7fcfc806d440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd001a040 comp rx=0 tx=0).stop 2024-06-15T01:31:23.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 msgr2=0x7fcfe0195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400d8d0 tx=0x7fcfd400dc90 comp rx=0 tx=0).stop 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 shutdown_connections 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fcfc806af80 0x7fcfc806d440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfe00ffa40 0x7fcfe0195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 --2- 172.21.15.137:0/460330293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfe0105a40 0x7fcfe0195ac0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.295+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 >> 172.21.15.137:0/460330293 conn(0x7fcfe00fb570 msgr2=0x7fcfe00fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.296+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 shutdown_connections 2024-06-15T01:31:23.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:23.296+0000 7fcfe4cfd700 1 -- 172.21.15.137:0/460330293 wait complete. 2024-06-15T01:31:23.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:23 smithi137 ceph-mon[112295]: pgmap v759: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:23.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:23 smithi142 ceph-mon[82136]: pgmap v759: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:23.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph health detail' 2024-06-15T01:31:24.144 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:24.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:24 smithi137 ceph-mon[112295]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:24.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:24 smithi142 ceph-mon[82136]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:31:25.456 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:25 smithi137 ceph-mon[112295]: pgmap v760: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:25.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:25 smithi142 ceph-mon[82136]: pgmap v760: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:26.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 -- 172.21.15.137:0/564952829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c105a40 msgr2=0x7fb32c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 --2- 172.21.15.137:0/564952829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c105a40 0x7fb32c105e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009910 tx=0x7fb31c009c20 comp rx=0 tx=0).stop 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 -- 172.21.15.137:0/564952829 shutdown_connections 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 --2- 172.21.15.137:0/564952829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c0ffa40 0x7fb32c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 --2- 172.21.15.137:0/564952829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c105a40 0x7fb32c105e20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 -- 172.21.15.137:0/564952829 >> 172.21.15.137:0/564952829 conn(0x7fb32c0fb570 msgr2=0x7fb32c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 -- 172.21.15.137:0/564952829 shutdown_connections 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.059+0000 7fb3319e1700 1 -- 172.21.15.137:0/564952829 wait complete. 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 Processor -- start 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 -- start start 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c105a40 0x7fb32c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:26.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19d0f0 con 0x7fb32c0ffa40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb3319e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c1965a0 con 0x7fb32c105a40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40014/0 (socket says 172.21.15.137:40014) 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 -- 172.21.15.137:0/3818046006 learned_addr learned my addr 172.21.15.137:0/3818046006 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32a7fc700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c105a40 0x7fb32c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 -- 172.21.15.137:0/3818046006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c105a40 msgr2=0x7fb32c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c105a40 0x7fb32c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.060+0000 7fb32affd700 1 -- 172.21.15.137:0/3818046006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0095f0 con 0x7fb32c0ffa40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb32affd700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b5c0 tx=0x7fb31c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c01d070 con 0x7fb32c0ffa40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb31c00f460 con 0x7fb32c0ffa40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c196820 con 0x7fb32c0ffa40 2024-06-15T01:31:26.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c021640 con 0x7fb32c0ffa40 2024-06-15T01:31:26.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.061+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c196d10 con 0x7fb32c0ffa40 2024-06-15T01:31:26.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.062+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c109050 con 0x7fb32c0ffa40 2024-06-15T01:31:26.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.062+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb31c00f9b0 con 0x7fb32c0ffa40 2024-06-15T01:31:26.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.063+0000 7fb3309df700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 0x7fb31406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:26.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.063+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb31c08b6c0 con 0x7fb32c0ffa40 2024-06-15T01:31:26.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.063+0000 7fb32a7fc700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 0x7fb31406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:26.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.063+0000 7fb32a7fc700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 0x7fb31406d440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb320006fd0 tx=0x7fb320008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:26.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.067+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb31c059f80 con 0x7fb32c0ffa40 2024-06-15T01:31:26.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.341+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb32c108a30 con 0x7fb32c0ffa40 2024-06-15T01:31:26.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.341+0000 7fb3309df700 1 -- 172.21.15.137:0/3818046006 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+567 (secure 0 0 0) 0x7fb31c059b10 con 0x7fb32c0ffa40 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 4 mgr modules have recently crashed 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] RECENT_MGR_MODULE_CRASH: 4 mgr modules have recently crashed 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi137.nbzbyc on host smithi137 at 2024-06-15T01:07:38.660551Z 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:06:25.273294Z 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:07:48.850867Z 2024-06-15T01:31:26.327 INFO:teuthology.orchestra.run.smithi137.stdout: mgr module k8sevents crashed in daemon mgr.smithi142.dllryb on host smithi142 at 2024-06-15T01:08:43.057142Z 2024-06-15T01:31:26.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.343+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 msgr2=0x7fb31406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.343+0000 7fb3319e1700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 0x7fb31406d440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb320006fd0 tx=0x7fb320008040 comp rx=0 tx=0).stop 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.343+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 msgr2=0x7fb32c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.343+0000 7fb3319e1700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b5c0 tx=0x7fb31c00be20 comp rx=0 tx=0).stop 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 shutdown_connections 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb31406af80 0x7fb31406d440 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32c0ffa40 0x7fb32c19c520 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 --2- 172.21.15.137:0/3818046006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb32c105a40 0x7fb32c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 >> 172.21.15.137:0/3818046006 conn(0x7fb32c0fb570 msgr2=0x7fb32c069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 shutdown_connections 2024-06-15T01:31:26.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:26.344+0000 7fb3319e1700 1 -- 172.21.15.137:0/3818046006 wait complete. 2024-06-15T01:31:26.935 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions' 2024-06-15T01:31:27.140 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:27.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:27 smithi137 ceph-mon[112295]: pgmap v761: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:27.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:27 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3818046006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:31:27.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:27 smithi142 ceph-mon[82136]: pgmap v761: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:27.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:27 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3818046006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-15T01:31:28.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- 172.21.15.137:0/1694610164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 msgr2=0x7f75901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:28.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 --2- 172.21.15.137:0/1694610164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901011a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7584009910 tx=0x7f7584009c20 comp rx=0 tx=0).stop 2024-06-15T01:31:28.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- 172.21.15.137:0/1694610164 shutdown_connections 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 --2- 172.21.15.137:0/1694610164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f75901057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 --2- 172.21.15.137:0/1694610164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901011a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- 172.21.15.137:0/1694610164 >> 172.21.15.137:0/1694610164 conn(0x7f75900fc650 msgr2=0x7f75900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- 172.21.15.137:0/1694610164 shutdown_connections 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- 172.21.15.137:0/1694610164 wait complete. 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 Processor -- start 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 -- start start 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.816+0000 7f7597163700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f7590198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:28.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7597163700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7590199570 con 0x7f7590100dc0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7597163700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019d380 con 0x7f7590101770 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40046/0 (socket says 172.21.15.137:40046) 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 -- 172.21.15.137:0/3548801681 learned_addr learned my addr 172.21.15.137:0/3548801681 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f758ffff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f7590198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 -- 172.21.15.137:0/3548801681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 msgr2=0x7f7590198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f7590198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 -- 172.21.15.137:0/3548801681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840095f0 con 0x7f7590100dc0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f758ffff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f7590198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7594eff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f758400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758401d070 con 0x7f7590100dc0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f758400f460 con 0x7f7590100dc0 2024-06-15T01:31:28.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7584021640 con 0x7f7590100dc0 2024-06-15T01:31:28.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.817+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759019d600 con 0x7f7590100dc0 2024-06-15T01:31:28.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.818+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759019daa0 con 0x7f7590100dc0 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.819+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590104f40 con 0x7f7590100dc0 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.819+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f758400fa20 con 0x7f7590100dc0 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.819+0000 7f758dffb700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 0x7f758006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.819+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f758408b670 con 0x7f7590100dc0 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.820+0000 7f758ffff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 0x7f758006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:31:28.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.820+0000 7f758ffff700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 0x7f758006d440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7590199fc0 tx=0x7f757c008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:31:28.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:28.824+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7584059f30 con 0x7f7590100dc0 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.097+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7590199e00 con 0x7f7590100dc0 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.097+0000 7f758dffb700 1 -- 172.21.15.137:0/3548801681 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7584059ac0 con 0x7f7590100dc0 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:31:29.083 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:31:29.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 msgr2=0x7f758006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:29.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 0x7f758006d440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7590199fc0 tx=0x7f757c008040 comp rx=0 tx=0).stop 2024-06-15T01:31:29.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 msgr2=0x7f75901989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:31:29.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f758400be20 comp rx=0 tx=0).stop 2024-06-15T01:31:29.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 shutdown_connections 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f758006af80 0x7f758006d440 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7590100dc0 0x7f75901989a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 --2- 172.21.15.137:0/3548801681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7590101770 0x7f7590198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 >> 172.21.15.137:0/3548801681 conn(0x7f75900fc650 msgr2=0x7f75900fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 shutdown_connections 2024-06-15T01:31:29.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:31:29.099+0000 7f7597163700 1 -- 172.21.15.137:0/3548801681 wait complete. 2024-06-15T01:31:29.444 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:29 smithi137 ceph-mon[112295]: pgmap v762: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:29.445 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:29 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3548801681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:31:29.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:29 smithi142 ceph-mon[82136]: pgmap v762: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:29.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:29 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3548801681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:31:29.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-06-15T01:31:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:31.412 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:31 smithi137 ceph-mon[112295]: pgmap v763: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:31.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:31 smithi142 ceph-mon[82136]: pgmap v763: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:31.647 INFO:teuthology.orchestra.run.smithi137.stdout:wait for servicemap items w/ changing names to refresh 2024-06-15T01:31:32.320 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'sleep 60' 2024-06-15T01:31:32.529 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:31:33.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:33 smithi137 ceph-mon[112295]: pgmap v764: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:33.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:33.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:33 smithi142 ceph-mon[82136]: pgmap v764: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:33.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:35.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:35 smithi137 ceph-mon[112295]: pgmap v765: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:35.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:35 smithi142 ceph-mon[82136]: pgmap v765: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:37.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:37 smithi137 ceph-mon[112295]: pgmap v766: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:37.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:37 smithi142 ceph-mon[82136]: pgmap v766: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:39.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:39 smithi137 ceph-mon[112295]: pgmap v767: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:39.535 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:39 smithi142 ceph-mon[82136]: pgmap v767: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:41.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:41 smithi137 ceph-mon[112295]: pgmap v768: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:41.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:41 smithi142 ceph-mon[82136]: pgmap v768: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:43.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:43 smithi137 ceph-mon[112295]: pgmap v769: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:43.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:43 smithi142 ceph-mon[82136]: pgmap v769: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:45.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:45 smithi137 ceph-mon[112295]: pgmap v770: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:45.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:45 smithi142 ceph-mon[82136]: pgmap v770: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:47.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:47 smithi142 ceph-mon[82136]: pgmap v771: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:47.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:47 smithi137 ceph-mon[112295]: pgmap v771: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:48.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:48.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:31:49.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:49 smithi142 ceph-mon[82136]: pgmap v772: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:49.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:31:49.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:49 smithi137 ceph-mon[112295]: pgmap v772: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:49.677 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:31:50.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:31:50.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:31:50.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:31:50.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:31:50.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:31:50.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:31:51.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:51 smithi137 ceph-mon[112295]: pgmap v773: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:51.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:51 smithi142 ceph-mon[82136]: pgmap v773: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:53.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:53 smithi137 ceph-mon[112295]: pgmap v774: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:53.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:53 smithi142 ceph-mon[82136]: pgmap v774: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:55.714 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:55 smithi137 ceph-mon[112295]: pgmap v775: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:55.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:55 smithi142 ceph-mon[82136]: pgmap v775: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:57.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:57 smithi137 ceph-mon[112295]: pgmap v776: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:57.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:57 smithi142 ceph-mon[82136]: pgmap v776: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:59.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:31:59 smithi137 ceph-mon[112295]: pgmap v777: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:31:59.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:31:59 smithi142 ceph-mon[82136]: pgmap v777: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:01.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:01 smithi137 ceph-mon[112295]: pgmap v778: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:01.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:01 smithi142 ceph-mon[82136]: pgmap v778: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:03.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:03 smithi137 ceph-mon[112295]: pgmap v779: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:03.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:03.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:03 smithi142 ceph-mon[82136]: pgmap v779: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:03.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:05.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:05 smithi137 ceph-mon[112295]: pgmap v780: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:05.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:05 smithi142 ceph-mon[82136]: pgmap v780: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:07.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:07 smithi137 ceph-mon[112295]: pgmap v781: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:07.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:07 smithi142 ceph-mon[82136]: pgmap v781: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:08.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:08 smithi142 ceph-mon[82136]: pgmap v782: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:08.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:08 smithi137 ceph-mon[112295]: pgmap v782: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:11.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:10 smithi142 ceph-mon[82136]: pgmap v783: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:11.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:10 smithi137 ceph-mon[112295]: pgmap v783: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:13.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:12 smithi142 ceph-mon[82136]: pgmap v784: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:13.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:12 smithi137 ceph-mon[112295]: pgmap v784: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:15.014 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:14 smithi142 ceph-mon[82136]: pgmap v785: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:15.116 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:14 smithi137 ceph-mon[112295]: pgmap v785: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:17.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:16 smithi142 ceph-mon[82136]: pgmap v786: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:17.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:16 smithi137 ceph-mon[112295]: pgmap v786: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:19.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:18 smithi142 ceph-mon[82136]: pgmap v787: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:19.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:19.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:18 smithi137 ceph-mon[112295]: pgmap v787: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:19.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:21.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:20 smithi142 ceph-mon[82136]: pgmap v788: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:21.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:20 smithi137 ceph-mon[112295]: pgmap v788: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:22.984 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:22 smithi137 ceph-mon[112295]: pgmap v789: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:23.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:22 smithi142 ceph-mon[82136]: pgmap v789: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:25.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:24 smithi142 ceph-mon[82136]: pgmap v790: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:25.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:24 smithi137 ceph-mon[112295]: pgmap v790: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:27.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:26 smithi142 ceph-mon[82136]: pgmap v791: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:27.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:26 smithi137 ceph-mon[112295]: pgmap v791: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:29.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:28 smithi142 ceph-mon[82136]: pgmap v792: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:29.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:28 smithi137 ceph-mon[112295]: pgmap v792: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:31.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:30 smithi142 ceph-mon[82136]: pgmap v793: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:31.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:30 smithi137 ceph-mon[112295]: pgmap v793: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:33.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:32 smithi142 ceph-mon[82136]: pgmap v794: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:33.214 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:32 smithi137 ceph-mon[112295]: pgmap v794: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:34.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:34.152 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:34.875 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph orch ps' 2024-06-15T01:32:35.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:34 smithi142 ceph-mon[82136]: pgmap v795: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:32:35.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:34 smithi137 ceph-mon[112295]: pgmap v795: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 -- 172.21.15.137:0/3918917639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 msgr2=0x7f0ac8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/3918917639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8105e20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009910 tx=0x7f0ab8009c20 comp rx=0 tx=0).stop 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 -- 172.21.15.137:0/3918917639 shutdown_connections 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/3918917639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 0x7f0ac8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/3918917639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8105e20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 -- 172.21.15.137:0/3918917639 >> 172.21.15.137:0/3918917639 conn(0x7f0ac80755a0 msgr2=0x7f0ac80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 -- 172.21.15.137:0/3918917639 shutdown_connections 2024-06-15T01:32:36.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.699+0000 7f0ad04e6700 1 -- 172.21.15.137:0/3918917639 wait complete. 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 Processor -- start 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 -- start start 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 0x7f0ac81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8197f90 con 0x7f0ac8105a40 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ad04e6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81980d0 con 0x7f0ac80ffbd0 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0acda81700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0acda81700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45502/0 (socket says 172.21.15.137:45502) 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0acda81700 1 -- 172.21.15.137:0/392364082 learned_addr learned my addr 172.21.15.137:0/392364082 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.700+0000 7f0ace282700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 0x7f0ac81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0acda81700 1 -- 172.21.15.137:0/392364082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 msgr2=0x7f0ac81974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:36.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0acda81700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 0x7f0ac81974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0acda81700 1 -- 172.21.15.137:0/392364082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80095f0 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0acda81700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400c930 tx=0x7f0ac400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4007ad0 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ac400ce80 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81a0530 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4018630 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.701+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac81a09a0 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.702+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8108040 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.702+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f0ac4007c30 con 0x7f0ac8105a40 2024-06-15T01:32:36.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.703+0000 7f0abf7fe700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 0x7f0ab406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:36.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.703+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0ac408a2d0 con 0x7f0ac8105a40 2024-06-15T01:32:36.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.703+0000 7f0ace282700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 0x7f0ab406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:36.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.703+0000 7f0ace282700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 0x7f0ab406d440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8006010 tx=0x7f0ab801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.707+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ac4055180 con 0x7f0ac8105a40 2024-06-15T01:32:36.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.911+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 --> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac802cce0 con 0x7f0ab406af80 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.920+0000 7f0abf7fe700 1 -- 172.21.15.137:0/392364082 <== mgr.24425 v2:172.21.15.137:6800/1246465045 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0ac802cce0 con 0x7f0ab406af80 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:alertmanager.smithi137 smithi137 *:9093,9094 running (23m) 113s ago 33m 27.6M - 0.25.0 c8568f914cd2 ae0bd880f3cf 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi137 smithi137 running (2m) 113s ago 34m 7142k - 18.2.2-1443-g6f4b0956 e7282a3bffba ca1313e53d0b 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-exporter.smithi142 smithi142 running (118s) 115s ago 32m 6803k - 18.2.2-1443-g6f4b0956 e7282a3bffba f476b89ec466 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi137 smithi137 running (21m) 113s ago 33m 6975k - 18.2.2-1443-g6f4b0956 e7282a3bffba b35062dadcef 2024-06-15T01:32:36.905 INFO:teuthology.orchestra.run.smithi137.stdout:crash.smithi142 smithi142 running (21m) 115s ago 32m 6991k - 18.2.2-1443-g6f4b0956 e7282a3bffba 9f4436127f3e 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:grafana.smithi137 smithi137 *:3000 running (22m) 113s ago 33m 88.0M - 9.4.7 954c08fa6188 168bee7ba38e 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.afplli smithi137 running (17m) 113s ago 28m 69.4M - 18.2.2-1443-g6f4b0956 e7282a3bffba 0b03328e16f6 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi137.nsntjq smithi137 running (17m) 113s ago 28m 61.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba d600ba06e632 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.examgd smithi142 running (17m) 115s ago 28m 22.0M - 18.2.2-1443-g6f4b0956 e7282a3bffba 98fcbe192e62 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mds.cephfs.smithi142.uczriv smithi142 running (17m) 115s ago 28m 20.5M - 18.2.2-1443-g6f4b0956 e7282a3bffba e4c1a6515f7e 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi137.nbzbyc smithi137 *:8443,9283,8765 running (25m) 113s ago 35m 502M - 18.2.2-1443-g6f4b0956 e7282a3bffba fc42aa224244 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.smithi142.dllryb smithi142 *:8443,9283,8765 running (23m) 115s ago 32m 447M - 18.2.2-1443-g6f4b0956 e7282a3bffba 667a0b4853b2 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi137 smithi137 running (21m) 113s ago 35m 69.3M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba d12730fea2eb 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:mon.smithi142 smithi142 running (21m) 115s ago 32m 51.2M 2048M 18.2.2-1443-g6f4b0956 e7282a3bffba abb9be3029e9 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi137 smithi137 *:9100 running (23m) 113s ago 33m 15.8M - 1.5.0 0da6a335fe13 f59070a76071 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:node-exporter.smithi142 smithi142 *:9100 running (23m) 115s ago 32m 17.0M - 1.5.0 0da6a335fe13 2e450c607665 2024-06-15T01:32:36.906 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (20m) 113s ago 31m 194M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba e60b459cde44 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi137 running (20m) 113s ago 31m 113M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 5216643ca244 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi137 running (19m) 113s ago 30m 123M 1216M 18.2.2-1443-g6f4b0956 e7282a3bffba 8d1c1b0fd6f6 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi142 running (19m) 115s ago 30m 123M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba 3555c500b38b 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi142 running (18m) 115s ago 30m 116M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba f41525c74c89 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi142 running (18m) 115s ago 29m 112M 2240M 18.2.2-1443-g6f4b0956 e7282a3bffba ab1f879dfaf4 2024-06-15T01:32:36.907 INFO:teuthology.orchestra.run.smithi137.stdout:prometheus.smithi137 smithi137 *:9095 running (23m) 113s ago 33m 77.9M - 2.43.0 a07b618ecd1d 9c62321488e1 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 msgr2=0x7f0ab406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 0x7f0ab406d440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8006010 tx=0x7f0ab801a040 comp rx=0 tx=0).stop 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 msgr2=0x7f0ac8197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400c930 tx=0x7f0ac400cc40 comp rx=0 tx=0).stop 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 shutdown_connections 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f0ab406af80 0x7f0ab406d440 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac80ffbd0 0x7f0ac81974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 --2- 172.21.15.137:0/392364082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac8105a40 0x7f0ac8197a00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 >> 172.21.15.137:0/392364082 conn(0x7f0ac80755a0 msgr2=0x7f0ac80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 shutdown_connections 2024-06-15T01:32:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:36.923+0000 7f0ad04e6700 1 -- 172.21.15.137:0/392364082 wait complete. 2024-06-15T01:32:37.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:36 smithi142 ceph-mon[82136]: pgmap v796: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:37.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:36 smithi137 ceph-mon[112295]: pgmap v796: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:37.559 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions' 2024-06-15T01:32:37.776 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:32:38.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:37 smithi142 ceph-mon[82136]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:32:38.105 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:37 smithi137 ceph-mon[112295]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-15T01:32:39.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:38 smithi142 ceph-mon[82136]: pgmap v797: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:39.097 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:38 smithi137 ceph-mon[112295]: pgmap v797: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/657214661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba41003b0 msgr2=0x7ffba4100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/657214661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba41003b0 0x7ffba4100830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009910 tx=0x7ffb94009c20 comp rx=0 tx=0).stop 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/657214661 shutdown_connections 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/657214661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba41003b0 0x7ffba4100830 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/657214661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba4105c30 0x7ffba4106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/657214661 >> 172.21.15.137:0/657214661 conn(0x7ffba4074d80 msgr2=0x7ffba4075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/657214661 shutdown_connections 2024-06-15T01:32:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.328+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/657214661 wait complete. 2024-06-15T01:32:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.329+0000 7ffbaa2d9700 1 Processor -- start 2024-06-15T01:32:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.329+0000 7ffbaa2d9700 1 -- start start 2024-06-15T01:32:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba41003b0 0x7ffba419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba419d140 con 0x7ffba4105c30 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41965f0 con 0x7ffba41003b0 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45522/0 (socket says 172.21.15.137:45522) 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 -- 172.21.15.137:0/1880673066 learned_addr learned my addr 172.21.15.137:0/1880673066 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba3fff700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba41003b0 0x7ffba419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 -- 172.21.15.137:0/1880673066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba41003b0 msgr2=0x7ffba419c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba41003b0 0x7ffba419c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 -- 172.21.15.137:0/1880673066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb940095f0 con 0x7ffba4105c30 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba37fe700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffb940092e0 tx=0x7ffb9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb9401d070 con 0x7ffba4105c30 2024-06-15T01:32:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4196870 con 0x7ffba4105c30 2024-06-15T01:32:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb94005e00 con 0x7ffba4105c30 2024-06-15T01:32:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb9400fe60 con 0x7ffba4105c30 2024-06-15T01:32:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.330+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba4196d60 con 0x7ffba4105c30 2024-06-15T01:32:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.331+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba404ea90 con 0x7ffba4105c30 2024-06-15T01:32:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.332+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ffb9400f6d0 con 0x7ffba4105c30 2024-06-15T01:32:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.332+0000 7ffba17fa700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 0x7ffb8c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.332+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ffb9408b650 con 0x7ffba4105c30 2024-06-15T01:32:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.333+0000 7ffba3fff700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 0x7ffb8c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.333+0000 7ffba3fff700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 0x7ffb8c06d440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005950 tx=0x7ffb980058e0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:39.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.337+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffb9405a030 con 0x7ffba4105c30 2024-06-15T01:32:39.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.610+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffba4197090 con 0x7ffba4105c30 2024-06-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.610+0000 7ffba17fa700 1 -- 172.21.15.137:0/1880673066 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ffb94059bc0 con 0x7ffba4105c30 2024-06-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-06-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi137.stdout: "mon": { 2024-06-15T01:32:39.595 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr": { 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 2 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "osd": { 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 6 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 4 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "overall": { 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)": 14 2024-06-15T01:32:39.596 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-06-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-06-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.612+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 msgr2=0x7ffb8c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.612+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 0x7ffb8c06d440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005950 tx=0x7ffb980058e0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.612+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 msgr2=0x7ffba419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:39.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.612+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffb940092e0 tx=0x7ffb9400be20 comp rx=0 tx=0).stop 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 shutdown_connections 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ffb8c06af80 0x7ffb8c06d440 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba41003b0 0x7ffba419c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 --2- 172.21.15.137:0/1880673066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4105c30 0x7ffba419cab0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 >> 172.21.15.137:0/1880673066 conn(0x7ffba4074d80 msgr2=0x7ffba4103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 shutdown_connections 2024-06-15T01:32:39.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:39.613+0000 7ffbaa2d9700 1 -- 172.21.15.137:0/1880673066 wait complete. 2024-06-15T01:32:39.931 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:39 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1880673066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:40.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:39 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1880673066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:40.223 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-06-15T01:32:40.433 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:32:40.887 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:40 smithi137 ceph-mon[112295]: pgmap v798: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:41.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:40 smithi142 ceph-mon[82136]: pgmap v798: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:42.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 -- 172.21.15.137:0/1715310994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c0ffa40 msgr2=0x7fed8c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 --2- 172.21.15.137:0/1715310994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c0ffa40 0x7fed8c0ffec0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009910 tx=0x7fed7c009c20 comp rx=0 tx=0).stop 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 -- 172.21.15.137:0/1715310994 shutdown_connections 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 --2- 172.21.15.137:0/1715310994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c0ffa40 0x7fed8c0ffec0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 --2- 172.21.15.137:0/1715310994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c105a40 0x7fed8c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 -- 172.21.15.137:0/1715310994 >> 172.21.15.137:0/1715310994 conn(0x7fed8c0fb570 msgr2=0x7fed8c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 -- 172.21.15.137:0/1715310994 shutdown_connections 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 -- 172.21.15.137:0/1715310994 wait complete. 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.128+0000 7fed944e8700 1 Processor -- start 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed944e8700 1 -- start start 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed944e8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0ffa40 0x7fed8c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:42.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed944e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed944e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c1972e0 con 0x7fed8c105a40 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed944e8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c199fa0 con 0x7fed8c0ffa40 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45540/0 (socket says 172.21.15.137:45540) 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 -- 172.21.15.137:0/1519512614 learned_addr learned my addr 172.21.15.137:0/1519512614 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed92284700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0ffa40 0x7fed8c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 -- 172.21.15.137:0/1519512614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0ffa40 msgr2=0x7fed8c196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0ffa40 0x7fed8c196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 -- 172.21.15.137:0/1519512614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0095f0 con 0x7fed8c105a40 2024-06-15T01:32:42.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.129+0000 7fed91a83700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fed7c006010 tx=0x7fed7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:42.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.130+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed7c01d070 con 0x7fed8c105a40 2024-06-15T01:32:42.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.130+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c19a220 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.130+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed7c00f460 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.130+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed7c021640 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.130+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c19a710 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.131+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c04ea90 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.131+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fed7c00f9e0 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.131+0000 7fed837fe700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 0x7fed7806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.131+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fed7c08b620 con 0x7fed8c105a40 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.132+0000 7fed92284700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 0x7fed7806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:42.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.132+0000 7fed92284700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 0x7fed7806d440 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed88005950 tx=0x7fed8800a300 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:42.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.136+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed7c056450 con 0x7fed8c105a40 2024-06-15T01:32:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.408+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fed8c19a930 con 0x7fed8c105a40 2024-06-15T01:32:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.408+0000 7fed837fe700 1 -- 172.21.15.137:0/1519512614 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fed7c05ee60 con 0x7fed8c105a40 2024-06-15T01:32:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.410+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 msgr2=0x7fed7806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.410+0000 7fed944e8700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 0x7fed7806d440 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed88005950 tx=0x7fed8800a300 comp rx=0 tx=0).stop 2024-06-15T01:32:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.410+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 msgr2=0x7fed8c196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fed7c006010 tx=0x7fed7c00bd70 comp rx=0 tx=0).stop 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 shutdown_connections 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fed7806af80 0x7fed7806d440 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0ffa40 0x7fed8c196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 --2- 172.21.15.137:0/1519512614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c105a40 0x7fed8c196c50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 >> 172.21.15.137:0/1519512614 conn(0x7fed8c0fb570 msgr2=0x7fed8c103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 shutdown_connections 2024-06-15T01:32:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:42.411+0000 7fed944e8700 1 -- 172.21.15.137:0/1519512614 wait complete. 2024-06-15T01:32:42.405 INFO:teuthology.orchestra.run.smithi137.stdout:true 2024-06-15T01:32:42.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:42 smithi137 ceph-mon[112295]: pgmap v799: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:42.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:42 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1519512614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:43.028 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-06-15T01:32:43.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:42 smithi142 ceph-mon[82136]: pgmap v799: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:43.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:42 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1519512614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:43.234 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:32:44.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 -- 172.21.15.137:0/2092256897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 msgr2=0x7f88181057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 --2- 172.21.15.137:0/2092256897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f88181057c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f880c009910 tx=0x7f880c009c20 comp rx=0 tx=0).stop 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 -- 172.21.15.137:0/2092256897 shutdown_connections 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 --2- 172.21.15.137:0/2092256897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f88181057c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 --2- 172.21.15.137:0/2092256897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f88181011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.850+0000 7f881e59a700 1 -- 172.21.15.137:0/2092256897 >> 172.21.15.137:0/2092256897 conn(0x7f88180fc650 msgr2=0x7f88180fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 -- 172.21.15.137:0/2092256897 shutdown_connections 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 -- 172.21.15.137:0/2092256897 wait complete. 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 Processor -- start 2024-06-15T01:32:44.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 -- start start 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f881819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881819f250 con 0x7f8818101770 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.851+0000 7f881e59a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8818198700 con 0x7f8818100dc0 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45550/0 (socket says 172.21.15.137:45550) 2024-06-15T01:32:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 -- 172.21.15.137:0/2948378600 learned_addr learned my addr 172.21.15.137:0/2948378600 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f8817fff700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f881819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 -- 172.21.15.137:0/2948378600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 msgr2=0x7f881819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f881819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 -- 172.21.15.137:0/2948378600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0095f0 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f8817fff700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f881819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88177fe700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f880c0092e0 tx=0x7f880c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880c01d070 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f880c005940 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8818198980 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880c00fe50 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.852+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8818198e70 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.853+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8818104f40 con 0x7f8818101770 2024-06-15T01:32:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.854+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f880c003a60 con 0x7f8818101770 2024-06-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.854+0000 7f88157fa700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 0x7f880006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.854+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f880c08b510 con 0x7f8818101770 2024-06-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.854+0000 7f8817fff700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 0x7f880006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:44.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.855+0000 7f8817fff700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 0x7f880006d440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8808005fd0 tx=0x7f8808005ee0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:44.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:44.859+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f880c059ef0 con 0x7f8818101770 2024-06-15T01:32:44.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:44 smithi137 ceph-mon[112295]: pgmap v800: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:45.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:44 smithi142 ceph-mon[82136]: pgmap v800: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:45.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.130+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f881804ea90 con 0x7f8818101770 2024-06-15T01:32:45.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.131+0000 7f88157fa700 1 -- 172.21.15.137:0/2948378600 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f880c059a80 con 0x7f8818101770 2024-06-15T01:32:45.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 msgr2=0x7f880006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:45.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 0x7f880006d440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8808005fd0 tx=0x7f8808005ee0 comp rx=0 tx=0).stop 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 msgr2=0x7f881819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f880c0092e0 tx=0x7f880c00be20 comp rx=0 tx=0).stop 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 shutdown_connections 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f880006af80 0x7f880006d440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8818100dc0 0x7f881819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 --2- 172.21.15.137:0/2948378600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8818101770 0x7f881819ebc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 >> 172.21.15.137:0/2948378600 conn(0x7f88180fc650 msgr2=0x7f88180fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 shutdown_connections 2024-06-15T01:32:45.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:45.133+0000 7f881e59a700 1 -- 172.21.15.137:0/2948378600 wait complete. 2024-06-15T01:32:45.128 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph version 18.2.2-1443-g6f4b0956 (6f4b09560ed399c3f139402a02f389c2fc8bb2cc) reef (stable)" 2024-06-15T01:32:45.800 DEBUG:teuthology.parallel:result is None 2024-06-15T01:32:45.800 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-15T01:32:45.811 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:32:45.812 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-15T01:32:46.019 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/mon.smithi137/config 2024-06-15T01:32:46.143 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:45 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2948378600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:46.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:45 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2948378600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-15T01:32:47.178 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:46 smithi137 ceph-mon[112295]: pgmap v801: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:47.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:46 smithi142 ceph-mon[82136]: pgmap v801: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 -- 172.21.15.137:0/2698349902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ffee0 msgr2=0x7f13d8105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 --2- 172.21.15.137:0/2698349902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ffee0 0x7f13d8105870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009910 tx=0x7f13c8009c20 comp rx=0 tx=0).stop 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 -- 172.21.15.137:0/2698349902 shutdown_connections 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 --2- 172.21.15.137:0/2698349902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ffee0 0x7f13d8105870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 --2- 172.21.15.137:0/2698349902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ff530 0x7f13d80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 -- 172.21.15.137:0/2698349902 >> 172.21.15.137:0/2698349902 conn(0x7f13d8074df0 msgr2=0x7f13d8075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 -- 172.21.15.137:0/2698349902 shutdown_connections 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.706+0000 7f13dd06f700 1 -- 172.21.15.137:0/2698349902 wait complete. 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 Processor -- start 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 -- start start 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ffee0 0x7f13d819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d819d030 con 0x7f13d80ff530 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13dd06f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8196570 con 0x7f13d80ffee0 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13d6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13d6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49636/0 (socket says 172.21.15.137:49636) 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.707+0000 7f13d6d9d700 1 -- 172.21.15.137:0/1255066088 learned_addr learned my addr 172.21.15.137:0/1255066088 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13d659c700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ffee0 0x7f13d819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:47.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13d6d9d700 1 -- 172.21.15.137:0/1255066088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ffee0 msgr2=0x7f13d819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:47.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13d6d9d700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ffee0 0x7f13d819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13d6d9d700 1 -- 172.21.15.137:0/1255066088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80095f0 con 0x7f13d80ff530 2024-06-15T01:32:47.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13d6d9d700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c930 tx=0x7f13c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0007ad0 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13c0007c30 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8196850 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0018630 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.708+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d8196da0 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.709+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8109150 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.710+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f13c000f450 con 0x7f13d80ff530 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.710+0000 7f13cffff700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.710+0000 7f13d659c700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:47.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.710+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f13c008a5b0 con 0x7f13d80ff530 2024-06-15T01:32:47.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.711+0000 7f13d659c700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13d80fe120 tx=0x7f13c801a040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:47.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.715+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f13c0058e70 con 0x7f13d80ff530 2024-06-15T01:32:47.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.946+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13d804ea90 con 0x7f13d80ff530 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.946+0000 7f13cffff700 1 -- 172.21.15.137:0/1255066088 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1864 (secure 0 0 0) 0x7f13c0058a00 con 0x7f13d80ff530 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:e39 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-06-15T01:32:47.931 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 39 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-06-15T01:04:19.419133+0000 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-06-15T01:30:48.683855+0000 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 60 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 89 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-15T01:32:47.932 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 2 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=34338,1=34342} 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 1 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.nsntjq{0:34338} state up:active seq 243 join_fscid=1 addr [v2:172.21.15.137:6827/208578565,v1:172.21.15.137:6829/208578565] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi137.afplli{1:34342} state up:active seq 238 join_fscid=1 addr [v2:172.21.15.137:6826/3297690656,v1:172.21.15.137:6828/3297690656] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:47.933 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:47.934 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-06-15T01:32:47.934 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:47.934 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.examgd{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6824/1483389069,v1:172.21.15.142:6825/1483389069] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:32:47.934 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.cephfs.smithi142.uczriv{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/3598160234,v1:172.21.15.142:6827/3598160234] compat {c=[1],r=[1],i=[7ff]}] 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 msgr2=0x7f13c406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13d80fe120 tx=0x7f13c801a040 comp rx=0 tx=0).stop 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 msgr2=0x7f13d819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c930 tx=0x7f13c000ccf0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 shutdown_connections 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f13c406af80 0x7f13c406d440 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13d80ff530 0x7f13d819c4f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 --2- 172.21.15.137:0/1255066088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d80ffee0 0x7f13d819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.948+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 >> 172.21.15.137:0/1255066088 conn(0x7f13d8074df0 msgr2=0x7f13d81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.949+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 shutdown_connections 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:47.949+0000 7f13dd06f700 1 -- 172.21.15.137:0/1255066088 wait complete. 2024-06-15T01:32:47.935 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 39 2024-06-15T01:32:48.565 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-06-15T01:32:48.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-15T01:32:48.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- 172.21.15.137:0/3465237593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 msgr2=0x7f1278100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 --2- 172.21.15.137:0/3465237593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f1278100660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f126c009880 tx=0x7f126c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- 172.21.15.137:0/3465237593 shutdown_connections 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 --2- 172.21.15.137:0/3465237593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1278100c30 0x7f1278104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 --2- 172.21.15.137:0/3465237593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f1278100660 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- 172.21.15.137:0/3465237593 >> 172.21.15.137:0/3465237593 conn(0x7f12780fbb20 msgr2=0x7f12780fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- 172.21.15.137:0/3465237593 shutdown_connections 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- 172.21.15.137:0/3465237593 wait complete. 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 Processor -- start 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.696+0000 7f127ebe1640 1 -- start start 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127ebe1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127ebe1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f127819bed0 0x7f1278195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127ebe1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127819c4a0 con 0x7f1278100280 2024-06-15T01:32:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127ebe1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278195f40 con 0x7f127819bed0 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49654/0 (socket says 172.21.15.137:49654) 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 -- 172.21.15.137:0/1972861300 learned_addr learned my addr 172.21.15.137:0/1972861300 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f1277fff640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f127819bed0 0x7f1278195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 -- 172.21.15.137:0/1972861300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f127819bed0 msgr2=0x7f1278195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f127819bed0 0x7f1278195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 -- 172.21.15.137:0/1972861300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009560 con 0x7f1278100280 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127c97d640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f126c005d90 tx=0x7f126c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126c003b40 con 0x7f1278100280 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f126c003ca0 con 0x7f1278100280 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12781961c0 con 0x7f1278100280 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.697+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126c034670 con 0x7f1278100280 2024-06-15T01:32:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.698+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1278196720 con 0x7f1278100280 2024-06-15T01:32:48.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.699+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127804e040 con 0x7f1278100280 2024-06-15T01:32:48.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.699+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f126c03eaf0 con 0x7f1278100280 2024-06-15T01:32:48.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.699+0000 7f1275ffb640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 0x7f126806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.699+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f126c0ace90 con 0x7f1278100280 2024-06-15T01:32:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.700+0000 7f1277fff640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 0x7f126806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.700+0000 7f1277fff640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 0x7f126806d470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1264005fd0 tx=0x7f1264005850 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:48.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.704+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f126c077d40 con 0x7f1278100280 2024-06-15T01:32:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.881+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1278108580 con 0x7f1278100280 2024-06-15T01:32:48.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.883+0000 7f1275ffb640 1 -- 172.21.15.137:0/1972861300 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f126c07b360 con 0x7f1278100280 2024-06-15T01:32:48.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:48.868 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":95,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:15:20.620477+0000","last_up_change":"2024-06-15T01:14:30.413524+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-06-15T01:04:17.764733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-06-15T01:04:18.637750+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6803","nonce":4060871906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6805","nonce":4060871906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6809","nonce":4060871906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6807","nonce":4060871906}]},"public_addr":"172.21.15.137:6803/4060871906","cluster_addr":"172.21.15.137:6805/4060871906","heartbeat_back_addr":"172.21.15.137:6809/4060871906","heartbeat_front_addr":"172.21.15.137:6807/4060871906","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6811","nonce":1628106669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6813","nonce":1628106669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6817","nonce":1628106669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6815","nonce":1628106669}]},"public_addr":"172.21.15.137:6811/1628106669","cluster_addr":"172.21.15.137:6813/1628106669","heartbeat_back_addr":"172.21.15.137:6817/1628106669","heartbeat_front_addr":"172.21.15.137:6815/1628106669","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":90,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6819","nonce":3034759480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6821","nonce":3034759480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6825","nonce":3034759480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6823","nonce":3034759480}]},"public_addr":"172.21.15.137:6819/3034759480","cluster_addr":"172.21.15.137:6821/3034759480","heartbeat_back_addr":"172.21.15.137:6825/3034759480","heartbeat_front_addr":"172.21.15.137:6823/3034759480","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":90,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6801","nonce":3361419228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6803","nonce":3361419228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6807","nonce":3361419228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6805","nonce":3361419228}]},"public_addr":"172.21.15.142:6801/3361419228","cluster_addr":"172.21.15.142:6803/3361419228","heartbeat_back_addr":"172.21.15.142:6807/3361419228","heartbeat_front_addr":"172.21.15.142:6805/3361419228","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":81,"up_thru":90,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6809","nonce":464434813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6811","nonce":464434813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6815","nonce":464434813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6813","nonce":464434813}]},"public_addr":"172.21.15.142:6809/464434813","cluster_addr":"172.21.15.142:6811/464434813","heartbeat_back_addr":"172.21.15.142:6815/464434813","heartbeat_front_addr":"172.21.15.142:6813/464434813","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":90,"up_thru":90,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6817","nonce":1909901146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6819","nonce":1909901146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6823","nonce":1909901146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6821","nonce":1909901146}]},"public_addr":"172.21.15.142:6817/1909901146","cluster_addr":"172.21.15.142:6819/1909901146","heartbeat_back_addr":"172.21.15.142:6823/1909901146","heartbeat_front_addr":"172.21.15.142:6821/1909901146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":3,"to":2},{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":5}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.f","mappings":[{"from":0,"to":2}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.b","mappings":[{"from":1,"to":2}]},{"pgid":"3.10","mappings":[{"from":1,"to":4}]},{"pgid":"3.11","mappings":[{"from":3,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6829/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6828/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6826/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:0/3277128014":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/623606439":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/146013243":"2024-06-16T01:06:34.431086+0000","172.21.15.142:6825/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:0/4288581606":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.142:0/1961055405":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6824/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:6829/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6828/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/388348183":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/72877675":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/944831942":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6801/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/3967735174":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6827/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:6800/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/544255511":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:32:48.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 msgr2=0x7f126806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 0x7f126806d470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1264005fd0 tx=0x7f1264005850 comp rx=0 tx=0).stop 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 msgr2=0x7f127819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f126c005d90 tx=0x7f126c0035f0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 shutdown_connections 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f126806afb0 0x7f126806d470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f127819bed0 0x7f1278195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 --2- 172.21.15.137:0/1972861300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1278100280 0x7f127819b990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 >> 172.21.15.137:0/1972861300 conn(0x7f12780fbb20 msgr2=0x7f12780fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 shutdown_connections 2024-06-15T01:32:48.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:48.885+0000 7f127ebe1640 1 -- 172.21.15.137:0/1972861300 wait complete. 2024-06-15T01:32:48.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-15T01:01:49.347551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-06-15T01:04:17.764733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-06-15T01:04:18.637750+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-15T01:32:48.882 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-15T01:32:48.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/1437672345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 msgr2=0x7f6d3810a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/1437672345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3810a000 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009880 tx=0x7f6d2802bd70 comp rx=0 tx=0).stop 2024-06-15T01:32:48.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/1437672345 shutdown_connections 2024-06-15T01:32:48.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/1437672345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3810a000 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/1437672345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 0x7f6d381015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/1437672345 >> 172.21.15.137:0/1437672345 conn(0x7f6d380faa20 msgr2=0x7f6d380fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/1437672345 shutdown_connections 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.000+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/1437672345 wait complete. 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 Processor -- start 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 -- start start 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 0x7f6d3819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38195a90 con 0x7f6d38101b20 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3fb4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38195c00 con 0x7f6d38101200 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49662/0 (socket says 172.21.15.137:49662) 2024-06-15T01:32:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 -- 172.21.15.137:0/3908751810 learned_addr learned my addr 172.21.15.137:0/3908751810 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d8ea640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 0x7f6d3819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 -- 172.21.15.137:0/3908751810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 msgr2=0x7f6d3819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 0x7f6d3819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.001+0000 7f6d3d0e9640 1 -- 172.21.15.137:0/3908751810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28009560 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d3d0e9640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009230 tx=0x7f6d280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d280039d0 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d2803e450 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d38195e80 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d28034670 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.002+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381963e0 con 0x7f6d38101b20 2024-06-15T01:32:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.003+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804e040 con 0x7f6d38101b20 2024-06-15T01:32:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.003+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6d2803ea10 con 0x7f6d38101b20 2024-06-15T01:32:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.004+0000 7f6d2effd640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 0x7f6d2406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.004+0000 7f6d3d8ea640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 0x7f6d2406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.004+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6d280acf50 con 0x7f6d38101b20 2024-06-15T01:32:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.004+0000 7f6d3d8ea640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 0x7f6d2406d3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005fd0 tx=0x7f6d34005850 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:48.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.008+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6d280e0900 con 0x7f6d38101b20 2024-06-15T01:32:49.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.180+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d3810d1a0 con 0x7f6d38101b20 2024-06-15T01:32:49.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.180+0000 7f6d2effd640 1 -- 172.21.15.137:0/3908751810 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d280430a0 con 0x7f6d38101b20 2024-06-15T01:32:49.165 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 msgr2=0x7f6d2406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 0x7f6d2406d3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005fd0 tx=0x7f6d34005850 comp rx=0 tx=0).stop 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 msgr2=0x7f6d3819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009230 tx=0x7f6d280045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 shutdown_connections 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6d2406aef0 0x7f6d2406d3b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d38101b20 0x7f6d3819bf60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 --2- 172.21.15.137:0/3908751810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38101200 0x7f6d3819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 >> 172.21.15.137:0/3908751810 conn(0x7f6d380faa20 msgr2=0x7f6d38104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 shutdown_connections 2024-06-15T01:32:49.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.182+0000 7f6d3fb4e640 1 -- 172.21.15.137:0/3908751810 wait complete. 2024-06-15T01:32:49.176 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:48 smithi137 ceph-mon[112295]: pgmap v802: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:49.177 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:48 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1255066088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:32:49.177 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:49.178 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-06-15T01:32:49.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 -- 172.21.15.137:0/2976353179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d40ff390 msgr2=0x7f76d4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/2976353179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d40ff390 0x7f76d4104d10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009880 tx=0x7f76c802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 -- 172.21.15.137:0/2976353179 shutdown_connections 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/2976353179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d40ff390 0x7f76d4104d10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/2976353179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d40fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 -- 172.21.15.137:0/2976353179 >> 172.21.15.137:0/2976353179 conn(0x7f76d40741f0 msgr2=0x7f76d4074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 -- 172.21.15.137:0/2976353179 shutdown_connections 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.296+0000 7f76d9fc5640 1 -- 172.21.15.137:0/2976353179 wait complete. 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 Processor -- start 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 -- start start 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419c5e0 con 0x7f76d419c010 2024-06-15T01:32:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d9fc5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4196080 con 0x7f76d40fe9e0 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49688/0 (socket says 172.21.15.137:49688) 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 -- 172.21.15.137:0/3259215411 learned_addr learned my addr 172.21.15.137:0/3259215411 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d37fe640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 -- 172.21.15.137:0/3259215411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 msgr2=0x7f76d419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d2ffd640 1 -- 172.21.15.137:0/3259215411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8009560 con 0x7f76d419c010 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.297+0000 7f76d37fe640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d2ffd640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009230 tx=0x7f76c80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c8003b50 con 0x7f76d419c010 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76c8003cb0 con 0x7f76d419c010 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d4196300 con 0x7f76d419c010 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c8034700 con 0x7f76d419c010 2024-06-15T01:32:49.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.298+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d4196860 con 0x7f76d419c010 2024-06-15T01:32:49.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.299+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d404e040 con 0x7f76d419c010 2024-06-15T01:32:49.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.299+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f76c803eb60 con 0x7f76d419c010 2024-06-15T01:32:49.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.300+0000 7f76d0ff9640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 0x7f76bc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.300+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f76c80ad010 con 0x7f76d419c010 2024-06-15T01:32:49.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.300+0000 7f76d37fe640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 0x7f76bc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.300+0000 7f76d37fe640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 0x7f76bc06d3b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76c40058c0 tx=0x7f76c4005d30 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.304+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f76c8077e40 con 0x7f76d419c010 2024-06-15T01:32:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:48 smithi142 ceph-mon[82136]: pgmap v802: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:48 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1255066088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-15T01:32:49.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:32:49.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.475+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f76d41085f0 con 0x7f76d419c010 2024-06-15T01:32:49.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.476+0000 7f76d0ff9640 1 -- 172.21.15.137:0/3259215411 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f76c807b460 con 0x7f76d419c010 2024-06-15T01:32:49.461 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 msgr2=0x7f76bc06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 0x7f76bc06d3b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76c40058c0 tx=0x7f76c4005d30 comp rx=0 tx=0).stop 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 msgr2=0x7f76d4195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009230 tx=0x7f76c80045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 shutdown_connections 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f76bc06aef0 0x7f76bc06d3b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d419c010 0x7f76d4195b40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 --2- 172.21.15.137:0/3259215411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40fe9e0 0x7f76d419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 >> 172.21.15.137:0/3259215411 conn(0x7f76d40741f0 msgr2=0x7f76d40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 shutdown_connections 2024-06-15T01:32:49.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.478+0000 7f76d9fc5640 1 -- 172.21.15.137:0/3259215411 wait complete. 2024-06-15T01:32:49.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-06-15T01:32:49.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- 172.21.15.137:0/1273751449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 msgr2=0x7fda2c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 --2- 172.21.15.137:0/1273751449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c1006b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda20009880 tx=0x7fda2002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- 172.21.15.137:0/1273751449 shutdown_connections 2024-06-15T01:32:49.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 --2- 172.21.15.137:0/1273751449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c100c80 0x7fda2c104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 --2- 172.21.15.137:0/1273751449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c1006b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- 172.21.15.137:0/1273751449 >> 172.21.15.137:0/1273751449 conn(0x7fda2c0fbb50 msgr2=0x7fda2c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- 172.21.15.137:0/1273751449 shutdown_connections 2024-06-15T01:32:49.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- 172.21.15.137:0/1273751449 wait complete. 2024-06-15T01:32:49.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 Processor -- start 2024-06-15T01:32:49.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- start start 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c19c010 0x7fda2c195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19c5e0 con 0x7fda2c1002d0 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.596+0000 7fda3325f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c196080 con 0x7fda2c19c010 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49710/0 (socket says 172.21.15.137:49710) 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 -- 172.21.15.137:0/885248534 learned_addr learned my addr 172.21.15.137:0/885248534 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda2bfff640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c19c010 0x7fda2c195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 -- 172.21.15.137:0/885248534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c19c010 msgr2=0x7fda2c195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c19c010 0x7fda2c195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 -- 172.21.15.137:0/885248534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009560 con 0x7fda2c1002d0 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda30ffb640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda20005d90 tx=0x7fda200035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda20003b40 con 0x7fda2c1002d0 2024-06-15T01:32:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda20003ca0 con 0x7fda2c1002d0 2024-06-15T01:32:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c196300 con 0x7fda2c1002d0 2024-06-15T01:32:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.597+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda20034670 con 0x7fda2c1002d0 2024-06-15T01:32:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.598+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c196860 con 0x7fda2c1002d0 2024-06-15T01:32:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.599+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c04e040 con 0x7fda2c1002d0 2024-06-15T01:32:49.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.600+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fda2003eaf0 con 0x7fda2c1002d0 2024-06-15T01:32:49.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.600+0000 7fda29ffb640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 0x7fda1c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.600+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fda200ace80 con 0x7fda2c1002d0 2024-06-15T01:32:49.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.600+0000 7fda2bfff640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 0x7fda1c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.601+0000 7fda2bfff640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 0x7fda1c06d3b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda18005d00 tx=0x7fda1800b410 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.604+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fda20077cb0 con 0x7fda2c1002d0 2024-06-15T01:32:49.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.785+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fda2c197030 con 0x7fda2c1002d0 2024-06-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.785+0000 7fda29ffb640 1 -- 172.21.15.137:0/885248534 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fda2007b2d0 con 0x7fda2c1002d0 2024-06-15T01:32:49.770 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 msgr2=0x7fda1c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 0x7fda1c06d3b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda18005d00 tx=0x7fda1800b410 comp rx=0 tx=0).stop 2024-06-15T01:32:49.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 msgr2=0x7fda2c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda20005d90 tx=0x7fda200035f0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 shutdown_connections 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fda1c06aef0 0x7fda1c06d3b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c19c010 0x7fda2c195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 --2- 172.21.15.137:0/885248534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda2c1002d0 0x7fda2c19bad0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 >> 172.21.15.137:0/885248534 conn(0x7fda2c0fbb50 msgr2=0x7fda2c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 shutdown_connections 2024-06-15T01:32:49.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.788+0000 7fda3325f640 1 -- 172.21.15.137:0/885248534 wait complete. 2024-06-15T01:32:49.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-06-15T01:32:49.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.905+0000 7fe827951640 1 -- 172.21.15.137:0/55136754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe820101140 msgr2=0x7fe8200fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.905+0000 7fe827951640 1 --2- 172.21.15.137:0/55136754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe820101140 0x7fe8200fe540 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009c20 tx=0x7fe81c02c1b0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- 172.21.15.137:0/55136754 shutdown_connections 2024-06-15T01:32:49.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 --2- 172.21.15.137:0/55136754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8200fea80 0x7fe8200feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 --2- 172.21.15.137:0/55136754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe820101140 0x7fe8200fe540 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- 172.21.15.137:0/55136754 >> 172.21.15.137:0/55136754 conn(0x7fe8200fa640 msgr2=0x7fe8200fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- 172.21.15.137:0/55136754 shutdown_connections 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- 172.21.15.137:0/55136754 wait complete. 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 Processor -- start 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- start start 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 0x7fe820198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820198bf0 con 0x7fe8200fea80 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.906+0000 7fe827951640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82019c9c0 con 0x7fe820101140 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49724/0 (socket says 172.21.15.137:49724) 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 -- 172.21.15.137:0/557803817 learned_addr learned my addr 172.21.15.137:0/557803817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe824eec640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 0x7fe820198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 -- 172.21.15.137:0/557803817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 msgr2=0x7fe820198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 0x7fe820198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 -- 172.21.15.137:0/557803817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c009900 con 0x7fe8200fea80 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe824eec640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 0x7fe820198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8256ed640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe81c037040 tx=0x7fe81c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe81c034610 con 0x7fe8200fea80 2024-06-15T01:32:49.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82019cc40 con 0x7fe8200fea80 2024-06-15T01:32:49.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe81c003e10 con 0x7fe8200fea80 2024-06-15T01:32:49.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe81c004700 con 0x7fe8200fea80 2024-06-15T01:32:49.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.907+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82019d140 con 0x7fe8200fea80 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.909+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82004e040 con 0x7fe8200fea80 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.909+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe81c03b040 con 0x7fe8200fea80 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.909+0000 7fe8167fc640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 0x7fe80c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.909+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe81c0ad150 con 0x7fe8200fea80 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.909+0000 7fe824eec640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 0x7fe80c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:49.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.910+0000 7fe824eec640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 0x7fe80c06d470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe820199630 tx=0x7fe810005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:49.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:49.915+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe81c0780a0 con 0x7fe8200fea80 2024-06-15T01:32:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.126+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe820199370 con 0x7fe8200fea80 2024-06-15T01:32:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.127+0000 7fe8167fc640 1 -- 172.21.15.137:0/557803817 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4854 (secure 0 0 0) 0x7fe81c07b6c0 con 0x7fe8200fea80 2024-06-15T01:32:50.112 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:50.112 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:48.683855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:active","state_seq":243,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:active","state_seq":238,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:50.112 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 39 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 msgr2=0x7fe80c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 0x7fe80c06d470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe820199630 tx=0x7fe810005f90 comp rx=0 tx=0).stop 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 msgr2=0x7fe820198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe81c037040 tx=0x7fe81c0093d0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 shutdown_connections 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe80c06afb0 0x7fe80c06d470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe820101140 0x7fe820198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 --2- 172.21.15.137:0/557803817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8200fea80 0x7fe820198020 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 >> 172.21.15.137:0/557803817 conn(0x7fe8200fa640 msgr2=0x7fe8200fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 shutdown_connections 2024-06-15T01:32:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.129+0000 7fe827951640 1 -- 172.21.15.137:0/557803817 wait complete. 2024-06-15T01:32:50.124 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:49 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1972861300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:32:50.124 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:49 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3908751810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.124 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:49 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3259215411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.124 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:49 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:32:50.124 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:49 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/885248534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.125 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-06-15T01:32:50.125 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-06-15T01:32:50.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 -- 172.21.15.137:0/3342645941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 msgr2=0x7ff6cc100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3342645941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc100680 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009880 tx=0x7ff6c002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 -- 172.21.15.137:0/3342645941 shutdown_connections 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3342645941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc100c50 0x7ff6cc104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3342645941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc100680 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 -- 172.21.15.137:0/3342645941 >> 172.21.15.137:0/3342645941 conn(0x7ff6cc0fbb20 msgr2=0x7ff6cc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 -- 172.21.15.137:0/3342645941 shutdown_connections 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.241+0000 7ff6d341c640 1 -- 172.21.15.137:0/3342645941 wait complete. 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 Processor -- start 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 -- start start 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc196290 0x7ff6cc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc196890 con 0x7ff6cc1002a0 2024-06-15T01:32:50.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d341c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc196a00 con 0x7ff6cc196290 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49732/0 (socket says 172.21.15.137:49732) 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 -- 172.21.15.137:0/3601699028 learned_addr learned my addr 172.21.15.137:0/3601699028 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d09b7640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc196290 0x7ff6cc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 -- 172.21.15.137:0/3601699028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc196290 msgr2=0x7ff6cc19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc196290 0x7ff6cc19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.242+0000 7ff6d11b8640 1 -- 172.21.15.137:0/3601699028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0009560 con 0x7ff6cc1002a0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6d11b8640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009230 tx=0x7ff6c00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c0003b40 con 0x7ff6cc1002a0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc19ac80 con 0x7ff6cc1002a0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6c0003ca0 con 0x7ff6cc1002a0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c0034670 con 0x7ff6cc1002a0 2024-06-15T01:32:50.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.243+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc19b1e0 con 0x7ff6cc1002a0 2024-06-15T01:32:50.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.244+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc104f40 con 0x7ff6cc1002a0 2024-06-15T01:32:50.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.244+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7ff6c003eaf0 con 0x7ff6cc1002a0 2024-06-15T01:32:50.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.244+0000 7ff6be7fc640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 0x7ff6b806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.245+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff6c00acf60 con 0x7ff6cc1002a0 2024-06-15T01:32:50.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.245+0000 7ff6d09b7640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 0x7ff6b806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.245+0000 7ff6d09b7640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 0x7ff6b806d3b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005fd0 tx=0x7ff6c800b590 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.249+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff6c00e0900 con 0x7ff6cc1002a0 2024-06-15T01:32:50.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:49 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1972861300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:32:50.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:49 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3908751810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:49 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3259215411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:49 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:32:50.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:49 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/885248534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:32:50.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.456+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7ff6cc04e040 con 0x7ff6cc1002a0 2024-06-15T01:32:50.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.457+0000 7ff6be7fc640 1 -- 172.21.15.137:0/3601699028 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v39) v1 ==== 107+0+4844 (secure 0 0 0) 0x7ff6c007b820 con 0x7ff6cc1002a0 2024-06-15T01:32:50.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:50.442 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:06:03.420216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":14496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6827/183082556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":183082556},{"type":"v1","addr":"172.21.15.137:6827","nonce":183082556}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14496":{"gid":14496,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6829/937501599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":937501599},{"type":"v1","addr":"172.21.15.137:6829","nonce":937501599}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:50.442 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 18 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 msgr2=0x7ff6b806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 0x7ff6b806d3b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005fd0 tx=0x7ff6c800b590 comp rx=0 tx=0).stop 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 msgr2=0x7ff6cc195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009230 tx=0x7ff6c00035f0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 shutdown_connections 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7ff6b806aef0 0x7ff6b806d3b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6cc196290 0x7ff6cc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 --2- 172.21.15.137:0/3601699028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6cc1002a0 0x7ff6cc195d50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 >> 172.21.15.137:0/3601699028 conn(0x7ff6cc0fbb20 msgr2=0x7ff6cc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 shutdown_connections 2024-06-15T01:32:50.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.459+0000 7ff6d341c640 1 -- 172.21.15.137:0/3601699028 wait complete. 2024-06-15T01:32:50.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 -- 172.21.15.137:0/2930162176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a880ffb30 msgr2=0x7f2a88103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/2930162176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a880ffb30 0x7f2a88103b70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009880 tx=0x7f2a7002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 -- 172.21.15.137:0/2930162176 shutdown_connections 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/2930162176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a880ffb30 0x7f2a88103b70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/2930162176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 0x7f2a880ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 -- 172.21.15.137:0/2930162176 >> 172.21.15.137:0/2930162176 conn(0x7f2a880faa40 msgr2=0x7f2a880fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 -- 172.21.15.137:0/2930162176 shutdown_connections 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 -- 172.21.15.137:0/2930162176 wait complete. 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.571+0000 7f2a8d41c640 1 Processor -- start 2024-06-15T01:32:50.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a8d41c640 1 -- start start 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a8d41c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 0x7f2a88196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a8d41c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a8d41c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8819fb50 con 0x7f2a88197020 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a8d41c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8819fc90 con 0x7f2a880ff180 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49748/0 (socket says 172.21.15.137:49748) 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 -- 172.21.15.137:0/3369335308 learned_addr learned my addr 172.21.15.137:0/3369335308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a86ffd640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 0x7f2a88196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 -- 172.21.15.137:0/3369335308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 msgr2=0x7f2a88196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 0x7f2a88196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 -- 172.21.15.137:0/3369335308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70009560 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.572+0000 7f2a867fc640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a70036040 tx=0x7f2a70004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.573+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a70003a50 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.573+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8819fe90 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.573+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a70003bb0 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.573+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a70034670 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.573+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881a0370 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.574+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8804e040 con 0x7f2a88197020 2024-06-15T01:32:50.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.574+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f2a70003d20 con 0x7f2a88197020 2024-06-15T01:32:50.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.574+0000 7f2a7ffff640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 0x7f2a7406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.574+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2a700acd90 con 0x7f2a88197020 2024-06-15T01:32:50.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.574+0000 7f2a86ffd640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 0x7f2a7406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.575+0000 7f2a86ffd640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 0x7f2a7406d3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005fd0 tx=0x7f2a80009500 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.579+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2a70077bc0 con 0x7f2a88197020 2024-06-15T01:32:50.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.799+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f2a88107480 con 0x7f2a88197020 2024-06-15T01:32:50.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.800+0000 7f2a7ffff640 1 -- 172.21.15.137:0/3369335308 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f2a7007b1e0 con 0x7f2a88197020 2024-06-15T01:32:50.784 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:50.784 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:07:38.953304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":14496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6827/183082556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":183082556},{"type":"v1","addr":"172.21.15.137:6827","nonce":183082556}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14496":{"gid":14496,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6829/937501599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":937501599},{"type":"v1","addr":"172.21.15.137:6829","nonce":937501599}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:50.785 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 19 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 msgr2=0x7f2a7406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 0x7f2a7406d3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005fd0 tx=0x7f2a80009500 comp rx=0 tx=0).stop 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 msgr2=0x7f2a88197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a70036040 tx=0x7f2a70004570 comp rx=0 tx=0).stop 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 shutdown_connections 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f2a7406aef0 0x7f2a7406d3b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a88197020 0x7f2a88197480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 --2- 172.21.15.137:0/3369335308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a880ff180 0x7f2a88196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 >> 172.21.15.137:0/3369335308 conn(0x7f2a880faa40 msgr2=0x7f2a880fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 shutdown_connections 2024-06-15T01:32:50.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.802+0000 7f2a8d41c640 1 -- 172.21.15.137:0/3369335308 wait complete. 2024-06-15T01:32:50.798 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- 172.21.15.137:0/863548173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 msgr2=0x7f16480ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 --2- 172.21.15.137:0/863548173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f16480ff640 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1638009880 tx=0x7f163802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- 172.21.15.137:0/863548173 shutdown_connections 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 --2- 172.21.15.137:0/863548173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16480ffc10 0x7f1648103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 --2- 172.21.15.137:0/863548173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f16480ff640 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- 172.21.15.137:0/863548173 >> 172.21.15.137:0/863548173 conn(0x7f1648074a50 msgr2=0x7f1648074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- 172.21.15.137:0/863548173 shutdown_connections 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- 172.21.15.137:0/863548173 wait complete. 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 Processor -- start 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.916+0000 7f164e42f640 1 -- start start 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 0x7f1648194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819b4d0 con 0x7f16480ff260 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648194f70 con 0x7f164819af00 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49774/0 (socket says 172.21.15.137:49774) 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 -- 172.21.15.137:0/1958768762 learned_addr learned my addr 172.21.15.137:0/1958768762 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 -- 172.21.15.137:0/1958768762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 msgr2=0x7f1648194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f163ffff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 0x7f1648194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 0x7f1648194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 -- 172.21.15.137:0/1958768762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638009560 con 0x7f16480ff260 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f163ffff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 0x7f1648194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1647fff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1638005d90 tx=0x7f16380035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1638003b40 con 0x7f16480ff260 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1638003ca0 con 0x7f16480ff260 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16481951f0 con 0x7f16480ff260 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1638034670 con 0x7f16480ff260 2024-06-15T01:32:50.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.917+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1648195750 con 0x7f16480ff260 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.918+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164804e040 con 0x7f16480ff260 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.919+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f163803eaf0 con 0x7f16480ff260 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.919+0000 7f1645ffb640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 0x7f163406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.919+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f16380acfa0 con 0x7f16480ff260 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.919+0000 7f163ffff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 0x7f163406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:50.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.920+0000 7f163ffff640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 0x7f163406d3b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1648072e40 tx=0x7f1630005e50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:50.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:50.923+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f16380e0900 con 0x7f16480ff260 2024-06-15T01:32:51.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.130+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f1648107480 con 0x7f16480ff260 2024-06-15T01:32:51.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.130+0000 7f1645ffb640 1 -- 172.21.15.137:0/1958768762 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4844 (secure 0 0 0) 0x7f163807b860 con 0x7f16480ff260 2024-06-15T01:32:51.115 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:51.115 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:10:28.494929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":14496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6827/183082556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":183082556},{"type":"v1","addr":"172.21.15.137:6827","nonce":183082556}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14496":{"gid":14496,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6829/937501599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":937501599},{"type":"v1","addr":"172.21.15.137:6829","nonce":937501599}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:51.115 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 20 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 msgr2=0x7f163406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 0x7f163406d3b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1648072e40 tx=0x7f1630005e50 comp rx=0 tx=0).stop 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 msgr2=0x7f164819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1638005d90 tx=0x7f16380035f0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 shutdown_connections 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f163406aef0 0x7f163406d3b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f164819af00 0x7f1648194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 --2- 172.21.15.137:0/1958768762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16480ff260 0x7f164819a9c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.132+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 >> 172.21.15.137:0/1958768762 conn(0x7f1648074a50 msgr2=0x7f16480fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.133+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 shutdown_connections 2024-06-15T01:32:51.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.133+0000 7f164e42f640 1 -- 172.21.15.137:0/1958768762 wait complete. 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: pgmap v803: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/557803817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3601699028' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-06-15T01:32:51.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:50 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3369335308' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-06-15T01:32:51.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.244+0000 7fdee0cc7640 1 -- 172.21.15.137:0/429734276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc0fe970 msgr2=0x7fdedc0fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.244+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/429734276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc0fe970 0x7fdedc0fed50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009880 tx=0x7fdec802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 -- 172.21.15.137:0/429734276 shutdown_connections 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/429734276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0ff320 0x7fdedc104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/429734276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc0fe970 0x7fdedc0fed50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 -- 172.21.15.137:0/429734276 >> 172.21.15.137:0/429734276 conn(0x7fdedc076620 msgr2=0x7fdedc076a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 -- 172.21.15.137:0/429734276 shutdown_connections 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 -- 172.21.15.137:0/429734276 wait complete. 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 Processor -- start 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.245+0000 7fdee0cc7640 1 -- start start 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0fe970 0x7fdedc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc1968d0 con 0x7fdedc1962d0 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc196a40 con 0x7fdedc0fe970 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49784/0 (socket says 172.21.15.137:49784) 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 -- 172.21.15.137:0/3310555226 learned_addr learned my addr 172.21.15.137:0/3310555226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdeda59c640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0fe970 0x7fdedc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 -- 172.21.15.137:0/3310555226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0fe970 msgr2=0x7fdedc195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0fe970 0x7fdedc195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 -- 172.21.15.137:0/3310555226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8009560 con 0x7fdedc1962d0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded9d9b640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00eb70 tx=0x7fdecc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdecc004b20 con 0x7fdedc1962d0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdecc004c80 con 0x7fdedc1962d0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc19ad30 con 0x7fdedc1962d0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdecc010650 con 0x7fdedc1962d0 2024-06-15T01:32:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.246+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc19b290 con 0x7fdedc1962d0 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.247+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdedc04e040 con 0x7fdedc1962d0 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.248+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fdecc005020 con 0x7fdedc1962d0 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.248+0000 7fded37fe640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.248+0000 7fdeda59c640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.248+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fdecc014070 con 0x7fdedc1962d0 2024-06-15T01:32:51.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.249+0000 7fdeda59c640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 0x7fdec006d3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006010 tx=0x7fdec8005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.253+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdecc08ce60 con 0x7fdedc1962d0 2024-06-15T01:32:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: pgmap v803: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:32:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:32:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:32:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/557803817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-15T01:32:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3601699028' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-06-15T01:32:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:50 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3369335308' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-06-15T01:32:51.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.461+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fdedc1970e0 con 0x7fdedc1962d0 2024-06-15T01:32:51.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.462+0000 7fded37fe640 1 -- 172.21.15.137:0/3310555226 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4843 (secure 0 0 0) 0x7fdecc055a60 con 0x7fdedc1962d0 2024-06-15T01:32:51.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:51.447 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:10:39.500783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":14496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6827/183082556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":183082556},{"type":"v1","addr":"172.21.15.137:6827","nonce":183082556}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14496":{"gid":14496,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6829/937501599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":937501599},{"type":"v1","addr":"172.21.15.137:6829","nonce":937501599}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:51.447 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 21 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 msgr2=0x7fdec006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 0x7fdec006d3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006010 tx=0x7fdec8005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 msgr2=0x7fdedc19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00eb70 tx=0x7fdecc00c6a0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 shutdown_connections 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdec006aef0 0x7fdec006d3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1962d0 0x7fdedc19a730 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 --2- 172.21.15.137:0/3310555226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc0fe970 0x7fdedc195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 >> 172.21.15.137:0/3310555226 conn(0x7fdedc076620 msgr2=0x7fdedc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 shutdown_connections 2024-06-15T01:32:51.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.464+0000 7fdee0cc7640 1 -- 172.21.15.137:0/3310555226 wait complete. 2024-06-15T01:32:51.460 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-06-15T01:32:51.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 -- 172.21.15.137:0/1568595696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd800ffab0 msgr2=0x7fdd800ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1568595696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd800ffab0 0x7fdd800ffe90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009880 tx=0x7fdd7002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 -- 172.21.15.137:0/1568595696 shutdown_connections 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1568595696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd80100460 0x7fdd80103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1568595696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd800ffab0 0x7fdd800ffe90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 -- 172.21.15.137:0/1568595696 >> 172.21.15.137:0/1568595696 conn(0x7fdd80074250 msgr2=0x7fdd80074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 -- 172.21.15.137:0/1568595696 shutdown_connections 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.577+0000 7fdd85f43640 1 -- 172.21.15.137:0/1568595696 wait complete. 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 Processor -- start 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 -- start start 2024-06-15T01:32:51.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd800ffab0 0x7fdd80195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80196880 con 0x7fdd80196280 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd85f43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801969f0 con 0x7fdd800ffab0 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49816/0 (socket says 172.21.15.137:49816) 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 -- 172.21.15.137:0/1007974054 learned_addr learned my addr 172.21.15.137:0/1007974054 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7f7fe640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd800ffab0 0x7fdd80195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 -- 172.21.15.137:0/1007974054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd800ffab0 msgr2=0x7fdd80195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd800ffab0 0x7fdd80195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.578+0000 7fdd7effd640 1 -- 172.21.15.137:0/1007974054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70009560 con 0x7fdd80196280 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd7effd640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c920 tx=0x7fdd7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd74007b80 con 0x7fdd80196280 2024-06-15T01:32:51.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd74007ce0 con 0x7fdd80196280 2024-06-15T01:32:51.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8019ace0 con 0x7fdd80196280 2024-06-15T01:32:51.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7400f450 con 0x7fdd80196280 2024-06-15T01:32:51.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.579+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8019b240 con 0x7fdd80196280 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.580+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8019b520 con 0x7fdd80196280 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.580+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fdd740046c0 con 0x7fdd80196280 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.580+0000 7fdd7cff9640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 0x7fdd6806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.581+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fdd7408a1e0 con 0x7fdd80196280 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.581+0000 7fdd7f7fe640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 0x7fdd6806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.581+0000 7fdd7f7fe640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 0x7fdd6806d3b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006010 tx=0x7fdd70000bc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.585+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd74055010 con 0x7fdd80196280 2024-06-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.791+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fdd80197090 con 0x7fdd80196280 2024-06-15T01:32:51.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.792+0000 7fdd7cff9640 1 -- 172.21.15.137:0/1007974054 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+3264 (secure 0 0 0) 0x7fdd74058630 con 0x7fdd80196280 2024-06-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:51.777 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 22 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 msgr2=0x7fdd6806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 0x7fdd6806d3b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006010 tx=0x7fdd70000bc0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 msgr2=0x7fdd8019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c920 tx=0x7fdd7400cde0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 shutdown_connections 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fdd6806aef0 0x7fdd6806d3b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd80196280 0x7fdd8019a6e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 --2- 172.21.15.137:0/1007974054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd800ffab0 0x7fdd80195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 >> 172.21.15.137:0/1007974054 conn(0x7fdd80074250 msgr2=0x7fdd80103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 shutdown_connections 2024-06-15T01:32:51.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.794+0000 7fdd85f43640 1 -- 172.21.15.137:0/1007974054 wait complete. 2024-06-15T01:32:51.790 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:51.790 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-06-15T01:32:51.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- 172.21.15.137:0/1630985529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd0067f70 msgr2=0x7f5dd0104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 --2- 172.21.15.137:0/1630985529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd0067f70 0x7f5dd0104c70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009880 tx=0x7f5dcc02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- 172.21.15.137:0/1630985529 shutdown_connections 2024-06-15T01:32:51.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 --2- 172.21.15.137:0/1630985529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd0067f70 0x7f5dd0104c70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 --2- 172.21.15.137:0/1630985529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- 172.21.15.137:0/1630985529 >> 172.21.15.137:0/1630985529 conn(0x7f5dd0074a50 msgr2=0x7f5dd0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- 172.21.15.137:0/1630985529 shutdown_connections 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- 172.21.15.137:0/1630985529 wait complete. 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 Processor -- start 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- start start 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0101050 con 0x7f5dd00ff600 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.907+0000 7f5dd797a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd01a1e30 con 0x7f5dd00675c0 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49820/0 (socket says 172.21.15.137:49820) 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 -- 172.21.15.137:0/2818813933 learned_addr learned my addr 172.21.15.137:0/2818813933 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd5716640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 -- 172.21.15.137:0/2818813933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 msgr2=0x7f5dd00ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 -- 172.21.15.137:0/2818813933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc009560 con 0x7f5dd00ff600 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd5716640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd4f15640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc036040 tx=0x7f5dcc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dcc003b50 con 0x7f5dd00ff600 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5dcc003cb0 con 0x7f5dd00ff600 2024-06-15T01:32:51.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd01a1fd0 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dcc034790 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.908+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd01a2500 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.909+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd004e040 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.910+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f5dcc03eb60 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.910+0000 7f5dc67fc640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 0x7f5dbc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.910+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5dcc0acff0 con 0x7f5dd00ff600 2024-06-15T01:32:51.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.910+0000 7f5dd5716640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 0x7f5dbc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:51.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.911+0000 7f5dd5716640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 0x7f5dbc06d3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000abe0 tx=0x7f5dc000a5c0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:51.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:51.915+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5dcc077e20 con 0x7f5dd00ff600 2024-06-15T01:32:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.123+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f5dd0108580 con 0x7f5dd00ff600 2024-06-15T01:32:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.124+0000 7f5dc67fc640 1 -- 172.21.15.137:0/2818813933 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f5dcc09a020 con 0x7f5dd00ff600 2024-06-15T01:32:52.109 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:52.109 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34304,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3864353360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3864353360},{"type":"v1","addr":"172.21.15.137:6828","nonce":3864353360}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34306,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/850529376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":850529376},{"type":"v1","addr":"172.21.15.137:6829","nonce":850529376}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 msgr2=0x7f5dbc06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 0x7f5dbc06d3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000abe0 tx=0x7f5dc000a5c0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 msgr2=0x7f5dd00ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc036040 tx=0x7f5dcc0045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 shutdown_connections 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f5dbc06aef0 0x7f5dbc06d3b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dd00ff600 0x7f5dd00ffa60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 --2- 172.21.15.137:0/2818813933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dd00675c0 0x7f5dd00ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 >> 172.21.15.137:0/2818813933 conn(0x7f5dd0074a50 msgr2=0x7f5dd00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 shutdown_connections 2024-06-15T01:32:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.126+0000 7f5dd797a640 1 -- 172.21.15.137:0/2818813933 wait complete. 2024-06-15T01:32:52.120 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1958768762' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-06-15T01:32:52.120 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3310555226' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-06-15T01:32:52.121 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:51 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1007974054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-06-15T01:32:52.122 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:52.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 -- 172.21.15.137:0/1148606170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e40ffc10 msgr2=0x7fa8e4103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/1148606170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e40ffc10 0x7fa8e4103b70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009880 tx=0x7fa8d402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 -- 172.21.15.137:0/1148606170 shutdown_connections 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/1148606170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e40ffc10 0x7fa8e4103b70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/1148606170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 0x7fa8e40ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 -- 172.21.15.137:0/1148606170 >> 172.21.15.137:0/1148606170 conn(0x7fa8e4074a50 msgr2=0x7fa8e4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 -- 172.21.15.137:0/1148606170 shutdown_connections 2024-06-15T01:32:52.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.241+0000 7fa8eb84a640 1 -- 172.21.15.137:0/1148606170 wait complete. 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.242+0000 7fa8eb84a640 1 Processor -- start 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.242+0000 7fa8eb84a640 1 -- start start 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 0x7fa8e419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e419b520 con 0x7fa8e419af50 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4194fc0 con 0x7fa8e40ff260 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49824/0 (socket says 172.21.15.137:49824) 2024-06-15T01:32:52.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 -- 172.21.15.137:0/3916952792 learned_addr learned my addr 172.21.15.137:0/3916952792 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e95e6640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 0x7fa8e419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 -- 172.21.15.137:0/3916952792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 msgr2=0x7fa8e419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 0x7fa8e419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 -- 172.21.15.137:0/3916952792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009560 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8e8de5640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009230 tx=0x7fa8d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8d4003b50 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa8d4003cb0 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e4195240 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8d4034700 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.243+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e41957a0 con 0x7fa8e419af50 2024-06-15T01:32:52.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.244+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e404e040 con 0x7fa8e419af50 2024-06-15T01:32:52.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.245+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fa8d403eaf0 con 0x7fa8e419af50 2024-06-15T01:32:52.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.245+0000 7fa8da7fc640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 0x7fa8d006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.246+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa8d40acea0 con 0x7fa8e419af50 2024-06-15T01:32:52.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.246+0000 7fa8e95e6640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 0x7fa8d006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.246+0000 7fa8e95e6640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 0x7fa8d006d3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005fd0 tx=0x7fa8e000b390 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.249+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa8d4077cd0 con 0x7fa8e419af50 2024-06-15T01:32:52.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1958768762' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-06-15T01:32:52.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3310555226' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-06-15T01:32:52.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:51 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1007974054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-06-15T01:32:52.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.464+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fa8e4107480 con 0x7fa8e419af50 2024-06-15T01:32:52.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.465+0000 7fa8da7fc640 1 -- 172.21.15.137:0/3916952792 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+4046 (secure 0 0 0) 0x7fa8d407b2f0 con 0x7fa8e419af50 2024-06-15T01:32:52.450 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:52.450 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34304,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3864353360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3864353360},{"type":"v1","addr":"172.21.15.137:6828","nonce":3864353360}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:52.450 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 msgr2=0x7fa8d006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 0x7fa8d006d3b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005fd0 tx=0x7fa8e000b390 comp rx=0 tx=0).stop 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 msgr2=0x7fa8e4194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009230 tx=0x7fa8d40045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 shutdown_connections 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fa8d006aef0 0x7fa8d006d3b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e419af50 0x7fa8e4194a80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 --2- 172.21.15.137:0/3916952792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8e40ff260 0x7fa8e419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 >> 172.21.15.137:0/3916952792 conn(0x7fa8e4074a50 msgr2=0x7fa8e40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 shutdown_connections 2024-06-15T01:32:52.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.467+0000 7fa8eb84a640 1 -- 172.21.15.137:0/3916952792 wait complete. 2024-06-15T01:32:52.462 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:52.463 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 -- 172.21.15.137:0/1284235294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74000ffc10 msgr2=0x7f7400103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1284235294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74000ffc10 0x7f7400103b70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009880 tx=0x7f73e802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 -- 172.21.15.137:0/1284235294 shutdown_connections 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1284235294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74000ffc10 0x7f7400103b70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1284235294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f74000ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 -- 172.21.15.137:0/1284235294 >> 172.21.15.137:0/1284235294 conn(0x7f7400074a50 msgr2=0x7f7400074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 -- 172.21.15.137:0/1284235294 shutdown_connections 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 -- 172.21.15.137:0/1284235294 wait complete. 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.584+0000 7f7406a7d640 1 Processor -- start 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7406a7d640 1 -- start start 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7406a7d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f740019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7406a7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7406a7d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740019b520 con 0x7f740019af50 2024-06-15T01:32:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7406a7d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7400194fc0 con 0x7f74000ff260 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49840/0 (socket says 172.21.15.137:49840) 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 -- 172.21.15.137:0/1210045385 learned_addr learned my addr 172.21.15.137:0/1210045385 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7404819640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f740019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 -- 172.21.15.137:0/1210045385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 msgr2=0x7f740019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f740019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f73fffff640 1 -- 172.21.15.137:0/1210045385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8009560 con 0x7f740019af50 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.585+0000 7f7404819640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f740019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f73fffff640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f73e802c220 tx=0x7f73e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e8003a50 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f73e802de90 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7400195240 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73e8034440 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.586+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74001957a0 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.587+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740004e040 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.588+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f73e803e450 con 0x7f740019af50 2024-06-15T01:32:52.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.588+0000 7f73fdffb640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 0x7f73ec06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.588+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f73e80ace70 con 0x7f740019af50 2024-06-15T01:32:52.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.588+0000 7f7404819640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 0x7f73ec06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.589+0000 7f7404819640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 0x7f73ec06d3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009b50 tx=0x7f73f0005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.592+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f73e8077ca0 con 0x7f740019af50 2024-06-15T01:32:52.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.800+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f7400107480 con 0x7f740019af50 2024-06-15T01:32:52.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.800+0000 7f73fdffb640 1 -- 172.21.15.137:0/1210045385 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f73e807b2c0 con 0x7f740019af50 2024-06-15T01:32:52.786 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:52.786 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34304,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3864353360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3864353360},{"type":"v1","addr":"172.21.15.137:6828","nonce":3864353360}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:52.786 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 25 2024-06-15T01:32:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.802+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 msgr2=0x7f73ec06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.802+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 0x7f73ec06d3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009b50 tx=0x7f73f0005f90 comp rx=0 tx=0).stop 2024-06-15T01:32:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 msgr2=0x7f7400194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f73e802c220 tx=0x7f73e80045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 shutdown_connections 2024-06-15T01:32:52.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f73ec06aef0 0x7f73ec06d3b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f740019af50 0x7f7400194a80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 --2- 172.21.15.137:0/1210045385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74000ff260 0x7f740019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 >> 172.21.15.137:0/1210045385 conn(0x7f7400074a50 msgr2=0x7f74000fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 shutdown_connections 2024-06-15T01:32:52.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.803+0000 7f7406a7d640 1 -- 172.21.15.137:0/1210045385 wait complete. 2024-06-15T01:32:52.800 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:52.800 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-06-15T01:32:52.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 -- 172.21.15.137:0/1406906019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ffc10 msgr2=0x7f4c98103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/1406906019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ffc10 0x7f4c98103b70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009880 tx=0x7f4c8402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 -- 172.21.15.137:0/1406906019 shutdown_connections 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/1406906019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ffc10 0x7f4c98103b70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/1406906019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c980ff260 0x7f4c980ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 -- 172.21.15.137:0/1406906019 >> 172.21.15.137:0/1406906019 conn(0x7f4c98074a50 msgr2=0x7f4c98074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 -- 172.21.15.137:0/1406906019 shutdown_connections 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.917+0000 7f4c9efc5640 1 -- 172.21.15.137:0/1406906019 wait complete. 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 Processor -- start 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 -- start start 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c98197080 0x7f4c981974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819fc40 con 0x7f4c980ff260 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9efc5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819fd80 con 0x7f4c98197080 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49866/0 (socket says 172.21.15.137:49866) 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 -- 172.21.15.137:0/2590191764 learned_addr learned my addr 172.21.15.137:0/2590191764 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c97fff640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c98197080 0x7f4c981974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 -- 172.21.15.137:0/2590191764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c98197080 msgr2=0x7f4c981974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c98197080 0x7f4c981974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.918+0000 7f4c9cd61640 1 -- 172.21.15.137:0/2590191764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84009560 con 0x7f4c980ff260 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c9cd61640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00d720 tx=0x7f4c8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8c0051f0 con 0x7f4c980ff260 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9819ff80 con 0x7f4c980ff260 2024-06-15T01:32:52.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c8c009820 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8c004d40 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.919+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c981a0400 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.920+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9804e040 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.920+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4c8c009990 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.921+0000 7f4c95ffb640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 0x7f4c8806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.921+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4c8c089e10 con 0x7f4c980ff260 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.921+0000 7f4c97fff640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 0x7f4c8806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.921+0000 7f4c97fff640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 0x7f4c8806d3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84000c00 tx=0x7f4c84005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:52.925+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4c8c054cf0 con 0x7f4c980ff260 2024-06-15T01:32:53.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.132+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f4c98107480 con 0x7f4c980ff260 2024-06-15T01:32:53.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.132+0000 7f4c95ffb640 1 -- 172.21.15.137:0/2590191764 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4043 (secure 0 0 0) 0x7f4c8c009c40 con 0x7f4c980ff260 2024-06-15T01:32:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:53.117 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 26 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.134+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 msgr2=0x7f4c8806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.134+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 0x7f4c8806d3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84000c00 tx=0x7f4c84005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.134+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 msgr2=0x7f4c98196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.134+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00d720 tx=0x7f4c8c00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 shutdown_connections 2024-06-15T01:32:53.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4c8806aef0 0x7f4c8806d3b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c98197080 0x7f4c981974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 --2- 172.21.15.137:0/2590191764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c980ff260 0x7f4c98196b40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 >> 172.21.15.137:0/2590191764 conn(0x7f4c98074a50 msgr2=0x7f4c980fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 shutdown_connections 2024-06-15T01:32:53.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.135+0000 7f4c9efc5640 1 -- 172.21.15.137:0/2590191764 wait complete. 2024-06-15T01:32:53.129 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:52 smithi137 ceph-mon[112295]: pgmap v804: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:53.129 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:52 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2818813933' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-06-15T01:32:53.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:52 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3916952792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-06-15T01:32:53.130 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:52 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1210045385' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-06-15T01:32:53.131 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:53.131 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-06-15T01:32:53.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.247+0000 7f7edf8b9640 1 -- 172.21.15.137:0/1989925975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ffc10 msgr2=0x7f7ed8103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.247+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/1989925975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ffc10 0x7f7ed8103b70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009880 tx=0x7f7ed402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- 172.21.15.137:0/1989925975 shutdown_connections 2024-06-15T01:32:53.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/1989925975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ffc10 0x7f7ed8103b70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/1989925975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed80ff260 0x7f7ed80ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- 172.21.15.137:0/1989925975 >> 172.21.15.137:0/1989925975 conn(0x7f7ed8074a50 msgr2=0x7f7ed8074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- 172.21.15.137:0/1989925975 shutdown_connections 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- 172.21.15.137:0/1989925975 wait complete. 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 Processor -- start 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- start start 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed8196270 0x7f7ed819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8196870 con 0x7f7ed80ff260 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edf8b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed81969e0 con 0x7f7ed8196270 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.248+0000 7f7edd655640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49878/0 (socket says 172.21.15.137:49878) 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 -- 172.21.15.137:0/2469736768 learned_addr learned my addr 172.21.15.137:0/2469736768 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edce54640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed8196270 0x7f7ed819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 -- 172.21.15.137:0/2469736768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed8196270 msgr2=0x7f7ed819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed8196270 0x7f7ed819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 -- 172.21.15.137:0/2469736768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009560 con 0x7f7ed80ff260 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edd655640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e9a0 tx=0x7f7ec800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec800cd50 con 0x7f7ed80ff260 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed819acd0 con 0x7f7ed80ff260 2024-06-15T01:32:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ec800ceb0 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed819b230 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.249+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec8018600 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.250+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed804e040 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.251+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7ec8018760 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.251+0000 7f7ece7fc640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 0x7f7ec406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.251+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7ec8014070 con 0x7f7ed80ff260 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.251+0000 7f7edce54640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 0x7f7ec406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.252+0000 7f7edce54640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 0x7f7ec406d3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4000c00 tx=0x7f7ed4005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.255+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7ec8055880 con 0x7f7ed80ff260 2024-06-15T01:32:53.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:52 smithi142 ceph-mon[82136]: pgmap v804: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:53.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:52 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2818813933' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-06-15T01:32:53.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:52 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3916952792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-06-15T01:32:53.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:52 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1210045385' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-06-15T01:32:53.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.466+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f7ed80ffc60 con 0x7f7ed80ff260 2024-06-15T01:32:53.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.466+0000 7f7ece7fc640 1 -- 172.21.15.137:0/2469736768 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f7ec8058ea0 con 0x7f7ed80ff260 2024-06-15T01:32:53.451 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:53.451 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24277,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/2666411319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2666411319},{"type":"v1","addr":"172.21.15.142:6825","nonce":2666411319}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:53.451 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 msgr2=0x7f7ec406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 0x7f7ec406d3b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4000c00 tx=0x7f7ed4005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 msgr2=0x7f7ed8195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e9a0 tx=0x7f7ec800eda0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 shutdown_connections 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f7ec406aef0 0x7f7ec406d3b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed8196270 0x7f7ed819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 --2- 172.21.15.137:0/2469736768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ed80ff260 0x7f7ed8195d30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 >> 172.21.15.137:0/2469736768 conn(0x7f7ed8074a50 msgr2=0x7f7ed8103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 shutdown_connections 2024-06-15T01:32:53.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.468+0000 7f7edf8b9640 1 -- 172.21.15.137:0/2469736768 wait complete. 2024-06-15T01:32:53.464 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:53.464 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-06-15T01:32:53.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 -- 172.21.15.137:0/2981858998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 msgr2=0x7f6a8c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2981858998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c0ff580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009880 tx=0x7f6a7c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 -- 172.21.15.137:0/2981858998 shutdown_connections 2024-06-15T01:32:53.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2981858998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a8c0ffb50 0x7f6a8c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2981858998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c0ff580 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 -- 172.21.15.137:0/2981858998 >> 172.21.15.137:0/2981858998 conn(0x7f6a8c0faa20 msgr2=0x7f6a8c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 -- 172.21.15.137:0/2981858998 shutdown_connections 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.581+0000 7f6a91f63640 1 -- 172.21.15.137:0/2981858998 wait complete. 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 Processor -- start 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 -- start start 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a8c1970f0 0x7f6a8c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c19fc20 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a91f63640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c19fd60 con 0x7f6a8c1970f0 2024-06-15T01:32:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49886/0 (socket says 172.21.15.137:49886) 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 -- 172.21.15.137:0/2121087564 learned_addr learned my addr 172.21.15.137:0/2121087564 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 -- 172.21.15.137:0/2121087564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a8c1970f0 msgr2=0x7f6a8c197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a8c1970f0 0x7f6a8c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 -- 172.21.15.137:0/2121087564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009560 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a8b7fe640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009230 tx=0x7f6a7c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.582+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a7c003cf0 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.583+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a7c003e50 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.583+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c19ff00 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.583+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a7c034790 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.583+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c1a0380 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.584+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c04e040 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.584+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f6a7c03ebf0 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.584+0000 7f6a897fa640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 0x7f6a7806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.584+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6a7c0acfd0 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.584+0000 7f6a83fff640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 0x7f6a7806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.585+0000 7f6a83fff640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 0x7f6a7806d3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0697f0 tx=0x7f6a74005c70 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.588+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6a7c077e80 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.799+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f6a8c1074a0 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.799+0000 7f6a897fa640 1 -- 172.21.15.137:0/2121087564 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4043 (secure 0 0 0) 0x7f6a7c07b4a0 con 0x7f6a8c0ff1a0 2024-06-15T01:32:53.784 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:53.784 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:53.784 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 28 2024-06-15T01:32:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 msgr2=0x7f6a7806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 0x7f6a7806d3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0697f0 tx=0x7f6a74005c70 comp rx=0 tx=0).stop 2024-06-15T01:32:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 msgr2=0x7f6a8c196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009230 tx=0x7f6a7c003820 comp rx=0 tx=0).stop 2024-06-15T01:32:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 shutdown_connections 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f6a7806aef0 0x7f6a7806d3b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a8c1970f0 0x7f6a8c197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 --2- 172.21.15.137:0/2121087564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8c0ff1a0 0x7f6a8c196bb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 >> 172.21.15.137:0/2121087564 conn(0x7f6a8c0faa20 msgr2=0x7f6a8c0fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 shutdown_connections 2024-06-15T01:32:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.802+0000 7f6a91f63640 1 -- 172.21.15.137:0/2121087564 wait complete. 2024-06-15T01:32:53.797 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:53.797 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.914+0000 7fd49bf3c640 1 -- 172.21.15.137:0/695750179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd494067f70 msgr2=0x7fd494104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.914+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/695750179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd494067f70 0x7fd494104c70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd490009880 tx=0x7fd49002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- 172.21.15.137:0/695750179 shutdown_connections 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/695750179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd494067f70 0x7fd494104c70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/695750179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 0x7fd4940679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- 172.21.15.137:0/695750179 >> 172.21.15.137:0/695750179 conn(0x7fd494074a50 msgr2=0x7fd494074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- 172.21.15.137:0/695750179 shutdown_connections 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- 172.21.15.137:0/695750179 wait complete. 2024-06-15T01:32:53.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 Processor -- start 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- start start 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 0x7fd494195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4941968a0 con 0x7fd4941962a0 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.915+0000 7fd49bf3c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd494196a10 con 0x7fd4940675c0 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49892/0 (socket says 172.21.15.137:49892) 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 -- 172.21.15.137:0/1271466090 learned_addr learned my addr 172.21.15.137:0/1271466090 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd499cd8640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 0x7fd494195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 -- 172.21.15.137:0/1271466090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 msgr2=0x7fd494195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 0x7fd494195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 -- 172.21.15.137:0/1271466090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490009560 con 0x7fd4941962a0 2024-06-15T01:32:53.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd4994d7640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd490036040 tx=0x7fd4900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd490003b50 con 0x7fd4941962a0 2024-06-15T01:32:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd490003cb0 con 0x7fd4941962a0 2024-06-15T01:32:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49419ad00 con 0x7fd4941962a0 2024-06-15T01:32:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd490034790 con 0x7fd4941962a0 2024-06-15T01:32:53.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.916+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49419b260 con 0x7fd4941962a0 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.917+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49404e040 con 0x7fd4941962a0 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.918+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fd49003eb60 con 0x7fd4941962a0 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.918+0000 7fd486ffd640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 0x7fd48006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.918+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd4900acfd0 con 0x7fd4941962a0 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.918+0000 7fd499cd8640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 0x7fd48006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:53.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.919+0000 7fd499cd8640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 0x7fd48006d3b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd488009890 tx=0x7fd488008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:53.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:53.922+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd490077e00 con 0x7fd4941962a0 2024-06-15T01:32:54.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.129+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fd494108580 con 0x7fd4941962a0 2024-06-15T01:32:54.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.130+0000 7fd486ffd640 1 -- 172.21.15.137:0/1271466090 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+4825 (secure 0 0 0) 0x7fd49007b420 con 0x7fd4941962a0 2024-06-15T01:32:54.115 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:54.115 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24273,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.142:6827/201053718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":201053718},{"type":"v1","addr":"172.21.15.142:6827","nonce":201053718}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:54.115 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 msgr2=0x7fd48006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 0x7fd48006d3b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd488009890 tx=0x7fd488008040 comp rx=0 tx=0).stop 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 msgr2=0x7fd49419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd490036040 tx=0x7fd4900045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 shutdown_connections 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fd48006aef0 0x7fd48006d3b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4941962a0 0x7fd49419a700 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 --2- 172.21.15.137:0/1271466090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4940675c0 0x7fd494195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 >> 172.21.15.137:0/1271466090 conn(0x7fd494074a50 msgr2=0x7fd4940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 shutdown_connections 2024-06-15T01:32:54.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.132+0000 7fd49bf3c640 1 -- 172.21.15.137:0/1271466090 wait complete. 2024-06-15T01:32:54.126 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:53 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2590191764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-06-15T01:32:54.126 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:53 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2469736768' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-06-15T01:32:54.126 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:53 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2121087564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-06-15T01:32:54.127 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:54.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-06-15T01:32:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 -- 172.21.15.137:0/500998186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 msgr2=0x7f49d0100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 --2- 172.21.15.137:0/500998186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0100680 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009880 tx=0x7f49cc02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 -- 172.21.15.137:0/500998186 shutdown_connections 2024-06-15T01:32:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 --2- 172.21.15.137:0/500998186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0100c50 0x7f49d0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 --2- 172.21.15.137:0/500998186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0100680 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 -- 172.21.15.137:0/500998186 >> 172.21.15.137:0/500998186 conn(0x7f49d00fbb20 msgr2=0x7f49d00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 -- 172.21.15.137:0/500998186 shutdown_connections 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.244+0000 7f49d76c4640 1 -- 172.21.15.137:0/500998186 wait complete. 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 Processor -- start 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 -- start start 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0196290 0x7f49d019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0196800 con 0x7f49d01002a0 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d76c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0196970 con 0x7f49d0196290 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49918/0 (socket says 172.21.15.137:49918) 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 -- 172.21.15.137:0/2615689318 learned_addr learned my addr 172.21.15.137:0/2615689318 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d4c5f640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0196290 0x7f49d019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 -- 172.21.15.137:0/2615689318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0196290 msgr2=0x7f49d019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0196290 0x7f49d019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 -- 172.21.15.137:0/2615689318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc009560 con 0x7f49d01002a0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.245+0000 7f49d5460640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009230 tx=0x7f49cc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.246+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc003cf0 con 0x7f49d01002a0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.246+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d019ac90 con 0x7f49d01002a0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.246+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49cc003e50 con 0x7f49d01002a0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.246+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49cc034790 con 0x7f49d01002a0 2024-06-15T01:32:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.246+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d019b1a0 con 0x7f49d01002a0 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.247+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d004e040 con 0x7f49d01002a0 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.247+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f49cc03ebf0 con 0x7f49d01002a0 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.247+0000 7f49c67fc640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 0x7f49bc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.248+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f49cc0ad070 con 0x7f49d01002a0 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.248+0000 7f49d4c5f640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 0x7f49bc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.248+0000 7f49d4c5f640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 0x7f49bc06d3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f49c0005e10 tx=0x7f49c0005da0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.252+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49cc077ea0 con 0x7f49d01002a0 2024-06-15T01:32:54.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:53 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2590191764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-06-15T01:32:54.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:53 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2469736768' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-06-15T01:32:54.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:53 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2121087564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-06-15T01:32:54.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.456+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f49d01085a0 con 0x7f49d01002a0 2024-06-15T01:32:54.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.457+0000 7f49c67fc640 1 -- 172.21.15.137:0/2615689318 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+4046 (secure 0 0 0) 0x7f49cc07b4c0 con 0x7f49d01002a0 2024-06-15T01:32:54.442 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:54.442 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:54.442 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 30 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 msgr2=0x7f49bc06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 0x7f49bc06d3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f49c0005e10 tx=0x7f49c0005da0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 msgr2=0x7f49d0195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009230 tx=0x7f49cc003820 comp rx=0 tx=0).stop 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 shutdown_connections 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f49bc06aef0 0x7f49bc06d3b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d0196290 0x7f49d019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 --2- 172.21.15.137:0/2615689318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d01002a0 0x7f49d0195d50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 >> 172.21.15.137:0/2615689318 conn(0x7f49d00fbb20 msgr2=0x7f49d00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 shutdown_connections 2024-06-15T01:32:54.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.459+0000 7f49d76c4640 1 -- 172.21.15.137:0/2615689318 wait complete. 2024-06-15T01:32:54.454 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:54.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-06-15T01:32:54.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 -- 172.21.15.137:0/756782422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ffb60 msgr2=0x7fbe68103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 --2- 172.21.15.137:0/756782422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ffb60 0x7fbe68103ba0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009880 tx=0x7fbe5c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 -- 172.21.15.137:0/756782422 shutdown_connections 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 --2- 172.21.15.137:0/756782422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ffb60 0x7fbe68103ba0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 --2- 172.21.15.137:0/756782422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe680ff1b0 0x7fbe680ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 -- 172.21.15.137:0/756782422 >> 172.21.15.137:0/756782422 conn(0x7fbe680faa30 msgr2=0x7fbe680fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 -- 172.21.15.137:0/756782422 shutdown_connections 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.570+0000 7fbe6da87640 1 -- 172.21.15.137:0/756782422 wait complete. 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 Processor -- start 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 -- start start 2024-06-15T01:32:54.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68197370 0x7fbe681977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819fea0 con 0x7fbe680ff1b0 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe6da87640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819ffe0 con 0x7fbe68197370 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49936/0 (socket says 172.21.15.137:49936) 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.571+0000 7fbe677fe640 1 -- 172.21.15.137:0/110041960 learned_addr learned my addr 172.21.15.137:0/110041960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe66ffd640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68197370 0x7fbe681977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe677fe640 1 -- 172.21.15.137:0/110041960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68197370 msgr2=0x7fbe681977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe677fe640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68197370 0x7fbe681977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe677fe640 1 -- 172.21.15.137:0/110041960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c009560 con 0x7fbe680ff1b0 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe677fe640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d720 tx=0x7fbe5800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe580051f0 con 0x7fbe680ff1b0 2024-06-15T01:32:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe681a0240 con 0x7fbe680ff1b0 2024-06-15T01:32:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe58009820 con 0x7fbe680ff1b0 2024-06-15T01:32:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe58004d40 con 0x7fbe680ff1b0 2024-06-15T01:32:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.572+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe681a0750 con 0x7fbe680ff1b0 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.573+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6804e040 con 0x7fbe680ff1b0 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.573+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbe58009990 con 0x7fbe680ff1b0 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.574+0000 7fbe64ff9640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 0x7fbe5006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.574+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbe58089f40 con 0x7fbe680ff1b0 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.574+0000 7fbe66ffd640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 0x7fbe5006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.574+0000 7fbe66ffd640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 0x7fbe5006d3b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02c7c0 tx=0x7fbe5c005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.578+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe580bd900 con 0x7fbe680ff1b0 2024-06-15T01:32:54.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.784+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fbe681074b0 con 0x7fbe680ff1b0 2024-06-15T01:32:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.784+0000 7fbe64ff9640 1 -- 172.21.15.137:0/110041960 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+4828 (secure 0 0 0) 0x7fbe58020090 con 0x7fbe680ff1b0 2024-06-15T01:32:54.770 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:54.770 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:14:29.156585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:54.772 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 31 2024-06-15T01:32:54.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 msgr2=0x7fbe5006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 0x7fbe5006d3b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02c7c0 tx=0x7fbe5c005fd0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 msgr2=0x7fbe68196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d720 tx=0x7fbe5800dbe0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 shutdown_connections 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbe5006aef0 0x7fbe5006d3b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe68197370 0x7fbe681977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 --2- 172.21.15.137:0/110041960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe680ff1b0 0x7fbe68196e30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 >> 172.21.15.137:0/110041960 conn(0x7fbe680faa30 msgr2=0x7fbe680fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 shutdown_connections 2024-06-15T01:32:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.786+0000 7fbe6da87640 1 -- 172.21.15.137:0/110041960 wait complete. 2024-06-15T01:32:54.782 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-06-15T01:32:54.783 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 -- 172.21.15.137:0/454125144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18080ff1b0 msgr2=0x7f18080ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 --2- 172.21.15.137:0/454125144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18080ff1b0 0x7f18080ff590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009c20 tx=0x7f17f802c1b0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 -- 172.21.15.137:0/454125144 shutdown_connections 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 --2- 172.21.15.137:0/454125144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ffb60 0x7f1808103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 --2- 172.21.15.137:0/454125144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18080ff1b0 0x7f18080ff590 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 -- 172.21.15.137:0/454125144 >> 172.21.15.137:0/454125144 conn(0x7f18080fa9f0 msgr2=0x7f18080fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:54.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 -- 172.21.15.137:0/454125144 shutdown_connections 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.899+0000 7f180e07d640 1 -- 172.21.15.137:0/454125144 wait complete. 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 Processor -- start 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 -- start start 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ff1b0 0x7f1808194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1808195800 con 0x7f1808195200 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f180e07d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1808195940 con 0x7f18080ff1b0 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49968/0 (socket says 172.21.15.137:49968) 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f1806ffd640 1 -- 172.21.15.137:0/366170336 learned_addr learned my addr 172.21.15.137:0/366170336 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.900+0000 7f18077fe640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ff1b0 0x7f1808194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1806ffd640 1 -- 172.21.15.137:0/366170336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ff1b0 msgr2=0x7f1808194cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1806ffd640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ff1b0 0x7f1808194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1806ffd640 1 -- 172.21.15.137:0/366170336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8009900 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1806ffd640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b3d0 tx=0x7f17fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc011800 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17fc011e00 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1808198bb0 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc00f2e0 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.901+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1808199110 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.902+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180804e040 con 0x7f1808195200 2024-06-15T01:32:54.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.902+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f17fc010380 con 0x7f1808195200 2024-06-15T01:32:54.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.903+0000 7f1804ff9640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 0x7f17f006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:54.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.903+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f17fc08a100 con 0x7f1808195200 2024-06-15T01:32:54.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.903+0000 7f18077fe640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 0x7f17f006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:54.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.903+0000 7f18077fe640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 0x7f17f006d3b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006010 tx=0x7f17f8009800 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:54.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:54.907+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f17fc054f30 con 0x7f1808195200 2024-06-15T01:32:55.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.113+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f18081074b0 con 0x7f1808195200 2024-06-15T01:32:55.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.114+0000 7f1804ff9640 1 -- 172.21.15.137:0/366170336 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+4827 (secure 0 0 0) 0x7f17fc058550 con 0x7f1808195200 2024-06-15T01:32:55.098 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:55.098 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34342,"name":"cephfs.smithi137.afplli","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:25.478074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:55.099 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 32 2024-06-15T01:32:55.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 msgr2=0x7f17f006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 0x7f17f006d3b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f17f8006010 tx=0x7f17f8009800 comp rx=0 tx=0).stop 2024-06-15T01:32:55.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 msgr2=0x7f18081985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b3d0 tx=0x7f17fc00b890 comp rx=0 tx=0).stop 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 shutdown_connections 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f17f006aef0 0x7f17f006d3b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1808195200 0x7f18081985b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 --2- 172.21.15.137:0/366170336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18080ff1b0 0x7f1808194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 >> 172.21.15.137:0/366170336 conn(0x7f18080fa9f0 msgr2=0x7f18080fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 shutdown_connections 2024-06-15T01:32:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.116+0000 7f180e07d640 1 -- 172.21.15.137:0/366170336 wait complete. 2024-06-15T01:32:55.110 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:54 smithi137 ceph-mon[112295]: pgmap v805: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:55.110 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:54 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1271466090' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-06-15T01:32:55.110 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:54 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2615689318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-06-15T01:32:55.110 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:54 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/110041960' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-06-15T01:32:55.111 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-06-15T01:32:55.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- 172.21.15.137:0/254746144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f300ff170 msgr2=0x7f3f300ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 --2- 172.21.15.137:0/254746144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f300ff170 0x7f3f300ff550 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009880 tx=0x7f3f2002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- 172.21.15.137:0/254746144 shutdown_connections 2024-06-15T01:32:55.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 --2- 172.21.15.137:0/254746144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ffb20 0x7f3f30103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 --2- 172.21.15.137:0/254746144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f300ff170 0x7f3f300ff550 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- 172.21.15.137:0/254746144 >> 172.21.15.137:0/254746144 conn(0x7f3f300fa9f0 msgr2=0x7f3f300fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- 172.21.15.137:0/254746144 shutdown_connections 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- 172.21.15.137:0/254746144 wait complete. 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 Processor -- start 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.228+0000 7f3f38681640 1 -- start start 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ff170 0x7f3f3019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019b410 con 0x7f3f3019aed0 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019b550 con 0x7f3f300ff170 2024-06-15T01:32:55.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49990/0 (socket says 172.21.15.137:49990) 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 -- 172.21.15.137:0/2823400217 learned_addr learned my addr 172.21.15.137:0/2823400217 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 -- 172.21.15.137:0/2823400217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ff170 msgr2=0x7f3f3019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f3641d640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ff170 0x7f3f3019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ff170 0x7f3f3019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 -- 172.21.15.137:0/2823400217 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009560 con 0x7f3f3019aed0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f35c1c640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c8f0 tx=0x7f3f2c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2c004410 con 0x7f3f3019aed0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f2c007c40 con 0x7f3f3019aed0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f30195000 con 0x7f3f3019aed0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2c00f450 con 0x7f3f3019aed0 2024-06-15T01:32:55.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.229+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f30195560 con 0x7f3f3019aed0 2024-06-15T01:32:55.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.230+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3004e040 con 0x7f3f3019aed0 2024-06-15T01:32:55.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.231+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f3f2c0077a0 con 0x7f3f3019aed0 2024-06-15T01:32:55.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.231+0000 7f3f277fe640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 0x7f3f1c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.231+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3f2c08a2a0 con 0x7f3f3019aed0 2024-06-15T01:32:55.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.231+0000 7f3f3641d640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 0x7f3f1c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.231+0000 7f3f3641d640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 0x7f3f1c06d3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002c7c0 tx=0x7f3f20005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.235+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3f2c055150 con 0x7f3f3019aed0 2024-06-15T01:32:55.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:54 smithi142 ceph-mon[82136]: pgmap v805: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:55.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:54 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1271466090' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-06-15T01:32:55.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:54 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2615689318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-06-15T01:32:55.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:54 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/110041960' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-06-15T01:32:55.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.440+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f3f301074a0 con 0x7f3f3019aed0 2024-06-15T01:32:55.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.441+0000 7f3f277fe640 1 -- 172.21.15.137:0/2823400217 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+4850 (secure 0 0 0) 0x7f3f2c058770 con 0x7f3f3019aed0 2024-06-15T01:32:55.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:55.425 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:25.484170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:55.426 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 33 2024-06-15T01:32:55.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 msgr2=0x7f3f1c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 0x7f3f1c06d3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002c7c0 tx=0x7f3f20005fd0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 msgr2=0x7f3f30194a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c8f0 tx=0x7f3f2c00cdb0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 shutdown_connections 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f3f1c06aef0 0x7f3f1c06d3b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f3019aed0 0x7f3f30194a00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 --2- 172.21.15.137:0/2823400217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f300ff170 0x7f3f3019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 >> 172.21.15.137:0/2823400217 conn(0x7f3f300fa9f0 msgr2=0x7f3f300fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 shutdown_connections 2024-06-15T01:32:55.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.443+0000 7f3f38681640 1 -- 172.21.15.137:0/2823400217 wait complete. 2024-06-15T01:32:55.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- 172.21.15.137:0/3356104993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d00ffc10 msgr2=0x7f16d0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 --2- 172.21.15.137:0/3356104993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d00ffc10 0x7f16d0103b70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009880 tx=0x7f16b802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- 172.21.15.137:0/3356104993 shutdown_connections 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 --2- 172.21.15.137:0/3356104993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d00ffc10 0x7f16d0103b70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 --2- 172.21.15.137:0/3356104993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 0x7f16d00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- 172.21.15.137:0/3356104993 >> 172.21.15.137:0/3356104993 conn(0x7f16d0074a50 msgr2=0x7f16d0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- 172.21.15.137:0/3356104993 shutdown_connections 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- 172.21.15.137:0/3356104993 wait complete. 2024-06-15T01:32:55.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 Processor -- start 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- start start 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 0x7f16d0194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0195860 con 0x7f16d0195260 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.556+0000 7f16d5b47640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d01959d0 con 0x7f16d00ff260 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50014/0 (socket says 172.21.15.137:50014) 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 -- 172.21.15.137:0/1097970079 learned_addr learned my addr 172.21.15.137:0/1097970079 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16cf7fe640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 0x7f16d0194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 -- 172.21.15.137:0/1097970079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 msgr2=0x7f16d0194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 0x7f16d0194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 -- 172.21.15.137:0/1097970079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009560 con 0x7f16d0195260 2024-06-15T01:32:55.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16c7fff640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f16b8036040 tx=0x7f16b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8003b50 con 0x7f16d0195260 2024-06-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16b8003cb0 con 0x7f16d0195260 2024-06-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d0199cc0 con 0x7f16d0195260 2024-06-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8034790 con 0x7f16d0195260 2024-06-15T01:32:55.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.557+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d019a220 con 0x7f16d0195260 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.558+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d004e040 con 0x7f16d0195260 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.559+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f16b803eb60 con 0x7f16d0195260 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.559+0000 7f16cd7fa640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 0x7f16bc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.559+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f16b80aced0 con 0x7f16d0195260 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.559+0000 7f16cf7fe640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 0x7f16bc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.560+0000 7f16cf7fe640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 0x7f16bc06d3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009890 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.564+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f16b8077d80 con 0x7f16d0195260 2024-06-15T01:32:55.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.768+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f16d00ffc10 con 0x7f16d0195260 2024-06-15T01:32:55.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.769+0000 7f16cd7fa640 1 -- 172.21.15.137:0/1097970079 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+4853 (secure 0 0 0) 0x7f16b807b3a0 con 0x7f16d0195260 2024-06-15T01:32:55.753 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:55.753 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:31.367368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:resolve","state_seq":235,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:55.754 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 34 2024-06-15T01:32:55.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 msgr2=0x7f16bc06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 0x7f16bc06d3b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009890 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2024-06-15T01:32:55.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 msgr2=0x7f16d01996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f16b8036040 tx=0x7f16b80045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 shutdown_connections 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f16bc06aef0 0x7f16bc06d3b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d0195260 0x7f16d01996c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 --2- 172.21.15.137:0/1097970079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16d00ff260 0x7f16d0194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 >> 172.21.15.137:0/1097970079 conn(0x7f16d0074a50 msgr2=0x7f16d0102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 shutdown_connections 2024-06-15T01:32:55.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.771+0000 7f16d5b47640 1 -- 172.21.15.137:0/1097970079 wait complete. 2024-06-15T01:32:55.766 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-06-15T01:32:55.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/1933747870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 msgr2=0x7fb7a00ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/1933747870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a00ff580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb798009880 tx=0x7fb79802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/1933747870 shutdown_connections 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/1933747870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00ffb50 0x7fb7a0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/1933747870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a00ff580 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/1933747870 >> 172.21.15.137:0/1933747870 conn(0x7fb7a00faa20 msgr2=0x7fb7a00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/1933747870 shutdown_connections 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/1933747870 wait complete. 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 Processor -- start 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- start start 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a019c020 0x7fb7a0195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a019c5f0 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.898+0000 7fb7a6d1d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0196090 con 0x7fb7a019c020 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50032/0 (socket says 172.21.15.137:50032) 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 -- 172.21.15.137:0/2909761145 learned_addr learned my addr 172.21.15.137:0/2909761145 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb79ffff640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a019c020 0x7fb7a0195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 -- 172.21.15.137:0/2909761145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a019c020 msgr2=0x7fb7a0195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a019c020 0x7fb7a0195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 -- 172.21.15.137:0/2909761145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009560 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.899+0000 7fb7a4ab9640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb798005d90 tx=0x7fb7980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.900+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb798003b10 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.900+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb798003c70 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.900+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb798034670 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.900+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a0196310 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.900+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a0196870 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.901+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a004e040 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.902+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb79803ea80 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.903+0000 7fb79dffb640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 0x7fb78c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.903+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb7980acfc0 con 0x7fb7a00ff1a0 2024-06-15T01:32:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.903+0000 7fb79ffff640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 0x7fb78c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.903+0000 7fb79ffff640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 0x7fb78c06d3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00697f0 tx=0x7fb788005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:55.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:55.907+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb7980e0900 con 0x7fb7a00ff1a0 2024-06-15T01:32:56.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.130+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fb7a0196ff0 con 0x7fb7a00ff1a0 2024-06-15T01:32:56.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.131+0000 7fb79dffb640 1 -- 172.21.15.137:0/2909761145 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+4856 (secure 0 0 0) 0x7fb798077d50 con 0x7fb7a00ff1a0 2024-06-15T01:32:56.116 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:56.116 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:32.372623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:resolve","state_seq":240,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:resolve","state_seq":235,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:56.116 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 35 2024-06-15T01:32:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 msgr2=0x7fb78c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 0x7fb78c06d3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00697f0 tx=0x7fb788005f90 comp rx=0 tx=0).stop 2024-06-15T01:32:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 msgr2=0x7fb7a019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb798005d90 tx=0x7fb7980045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 shutdown_connections 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb78c06aef0 0x7fb78c06d3b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a019c020 0x7fb7a0195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 --2- 172.21.15.137:0/2909761145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7a00ff1a0 0x7fb7a019bae0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 >> 172.21.15.137:0/2909761145 conn(0x7fb7a00faa20 msgr2=0x7fb7a01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 shutdown_connections 2024-06-15T01:32:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.133+0000 7fb7a6d1d640 1 -- 172.21.15.137:0/2909761145 wait complete. 2024-06-15T01:32:56.127 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/366170336' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-06-15T01:32:56.128 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2823400217' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-06-15T01:32:56.128 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:55 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1097970079' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-06-15T01:32:56.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-06-15T01:32:56.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 -- 172.21.15.137:0/559330949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 msgr2=0x7f4c000feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 --2- 172.21.15.137:0/559330949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c000feb60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009880 tx=0x7f4bf402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 -- 172.21.15.137:0/559330949 shutdown_connections 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 --2- 172.21.15.137:0/559330949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c000feb60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 --2- 172.21.15.137:0/559330949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 0x7f4c00104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 -- 172.21.15.137:0/559330949 >> 172.21.15.137:0/559330949 conn(0x7f4c00075250 msgr2=0x7f4c00075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 -- 172.21.15.137:0/559330949 shutdown_connections 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.244+0000 7f4c081c5640 1 -- 172.21.15.137:0/559330949 wait complete. 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 Processor -- start 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 -- start start 2024-06-15T01:32:56.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 0x7f4c00199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c00193840 con 0x7f4c000fe720 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c001939b0 con 0x7f4c001046b0 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57632/0 (socket says 172.21.15.137:57632) 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 -- 172.21.15.137:0/3733400495 learned_addr learned my addr 172.21.15.137:0/3733400495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 -- 172.21.15.137:0/3733400495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 msgr2=0x7f4c00199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05760640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 0x7f4c00199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 0x7f4c00199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 -- 172.21.15.137:0/3733400495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009560 con 0x7f4c000fe720 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c05f61640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00ed50 tx=0x7f4bfc00a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bfc0084e0 con 0x7f4c000fe720 2024-06-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bfc008ae0 con 0x7f4c000fe720 2024-06-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c00193c90 con 0x7f4c000fe720 2024-06-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.245+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bfc017c00 con 0x7f4c000fe720 2024-06-15T01:32:56.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.246+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c001941f0 con 0x7f4c000fe720 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.247+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0004e040 con 0x7f4c000fe720 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.247+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f4bfc008c50 con 0x7f4c000fe720 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.247+0000 7f4bf2ffd640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 0x7f4bec06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.247+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4bfc00ee40 con 0x7f4c000fe720 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.247+0000 7f4c05760640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 0x7f4bec06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.248+0000 7f4c05760640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 0x7f4bec06d3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402c7c0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.252+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4bfc056640 con 0x7f4c000fe720 2024-06-15T01:32:56.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/366170336' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-06-15T01:32:56.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2823400217' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-06-15T01:32:56.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:55 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1097970079' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-06-15T01:32:56.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.459+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f4c00106ba0 con 0x7f4c000fe720 2024-06-15T01:32:56.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.459+0000 7f4bf2ffd640 1 -- 172.21.15.137:0/3733400495 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+4860 (secure 0 0 0) 0x7f4bfc059c60 con 0x7f4c000fe720 2024-06-15T01:32:56.444 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:56.444 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:33.377467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":241,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":236,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:56.444 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 36 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 msgr2=0x7f4bec06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 0x7f4bec06d3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402c7c0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 msgr2=0x7f4c001997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00ed50 tx=0x7f4bfc00a720 comp rx=0 tx=0).stop 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 shutdown_connections 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f4bec06aef0 0x7f4bec06d3b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c001046b0 0x7f4c00199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 --2- 172.21.15.137:0/3733400495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c000fe720 0x7f4c001997d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 >> 172.21.15.137:0/3733400495 conn(0x7f4c00075250 msgr2=0x7f4c000fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 shutdown_connections 2024-06-15T01:32:56.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.461+0000 7f4c081c5640 1 -- 172.21.15.137:0/3733400495 wait complete. 2024-06-15T01:32:56.457 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-06-15T01:32:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 -- 172.21.15.137:0/2808221959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 msgr2=0x7f347c10a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 --2- 172.21.15.137:0/2808221959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c10a000 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3470009880 tx=0x7f347002bd70 comp rx=0 tx=0).stop 2024-06-15T01:32:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 -- 172.21.15.137:0/2808221959 shutdown_connections 2024-06-15T01:32:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 --2- 172.21.15.137:0/2808221959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c10a000 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 --2- 172.21.15.137:0/2808221959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f347c101200 0x7f347c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 -- 172.21.15.137:0/2808221959 >> 172.21.15.137:0/2808221959 conn(0x7f347c0faa20 msgr2=0x7f347c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 -- 172.21.15.137:0/2808221959 shutdown_connections 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.572+0000 7f3482608640 1 -- 172.21.15.137:0/2808221959 wait complete. 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 Processor -- start 2024-06-15T01:32:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 -- start start 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f347c101200 0x7f347c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c195a90 con 0x7f347c101b20 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f3482608640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c195c00 con 0x7f347c101200 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57660/0 (socket says 172.21.15.137:57660) 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 -- 172.21.15.137:0/1795565067 learned_addr learned my addr 172.21.15.137:0/1795565067 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 -- 172.21.15.137:0/1795565067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f347c101200 msgr2=0x7f347c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f347c101200 0x7f347c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.573+0000 7f347b7fe640 1 -- 172.21.15.137:0/1795565067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009560 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f347b7fe640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3470009230 tx=0x7f34700045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34700039d0 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f347002c710 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3470034440 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c195e80 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.574+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c196390 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.575+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c04e040 con 0x7f347c101b20 2024-06-15T01:32:56.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.575+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f347003e830 con 0x7f347c101b20 2024-06-15T01:32:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.576+0000 7f34797fa640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 0x7f346406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.576+0000 7f347bfff640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 0x7f346406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.576+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f34700acdb0 con 0x7f347c101b20 2024-06-15T01:32:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.576+0000 7f347bfff640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 0x7f346406d3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f346c0058c0 tx=0x7f346c005850 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.580+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3470077be0 con 0x7f347c101b20 2024-06-15T01:32:56.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.789+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f347c10d1a0 con 0x7f347c101b20 2024-06-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.789+0000 7f34797fa640 1 -- 172.21.15.137:0/1795565067 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+4854 (secure 0 0 0) 0x7f347007b200 con 0x7f347c101b20 2024-06-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:34.382945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":242,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":237,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:56.774 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 37 2024-06-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 msgr2=0x7f346406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 0x7f346406d3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f346c0058c0 tx=0x7f346c005850 comp rx=0 tx=0).stop 2024-06-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 msgr2=0x7f347c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3470009230 tx=0x7f34700045c0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 shutdown_connections 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f346406aef0 0x7f346406d3b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f347c101b20 0x7f347c19bf60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 --2- 172.21.15.137:0/1795565067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f347c101200 0x7f347c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 >> 172.21.15.137:0/1795565067 conn(0x7f347c0faa20 msgr2=0x7f347c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 shutdown_connections 2024-06-15T01:32:56.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.792+0000 7f3482608640 1 -- 172.21.15.137:0/1795565067 wait complete. 2024-06-15T01:32:56.788 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-06-15T01:32:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 -- 172.21.15.137:0/1358389642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4100450 msgr2=0x7f18d4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 --2- 172.21.15.137:0/1358389642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4100450 0x7f18d4103b90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009880 tx=0x7f18c402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 -- 172.21.15.137:0/1358389642 shutdown_connections 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 --2- 172.21.15.137:0/1358389642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4100450 0x7f18d4103b90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 --2- 172.21.15.137:0/1358389642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18d40ffaa0 0x7f18d40ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 -- 172.21.15.137:0/1358389642 >> 172.21.15.137:0/1358389642 conn(0x7f18d4075250 msgr2=0x7f18d4075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 -- 172.21.15.137:0/1358389642 shutdown_connections 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.904+0000 7f18db673640 1 -- 172.21.15.137:0/1358389642 wait complete. 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 Processor -- start 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 -- start start 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18d4101170 0x7f18d41015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41a1f60 con 0x7f18d40ffaa0 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18db673640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41a20a0 con 0x7f18d4101170 2024-06-15T01:32:56.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57676/0 (socket says 172.21.15.137:57676) 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 -- 172.21.15.137:0/642836016 learned_addr learned my addr 172.21.15.137:0/642836016 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 -- 172.21.15.137:0/642836016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18d4101170 msgr2=0x7f18d41015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18d4101170 0x7f18d41015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 -- 172.21.15.137:0/642836016 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4009560 con 0x7f18d40ffaa0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.905+0000 7f18d940f640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f18d000d720 tx=0x7f18d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.906+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18d00051f0 con 0x7f18d40ffaa0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.906+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18d0009820 con 0x7f18d40ffaa0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.906+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d41a2240 con 0x7f18d40ffaa0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.906+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18d0004d40 con 0x7f18d40ffaa0 2024-06-15T01:32:56.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.906+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41a27a0 con 0x7f18d40ffaa0 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.907+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d404e040 con 0x7f18d40ffaa0 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.907+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f18d0009990 con 0x7f18d40ffaa0 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.907+0000 7f18ca7fc640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 0x7f18c006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.907+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f18d0089eb0 con 0x7f18d40ffaa0 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.908+0000 7f18d8c0e640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 0x7f18c006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:56.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.908+0000 7f18d8c0e640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 0x7f18c006d3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f18c4036040 tx=0x7f18c4005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:56.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:56.912+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f18d0054d60 con 0x7f18d40ffaa0 2024-06-15T01:32:57.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.121+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f18d4107470 con 0x7f18d40ffaa0 2024-06-15T01:32:57.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.121+0000 7f18ca7fc640 1 -- 172.21.15.137:0/642836016 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+4854 (secure 0 0 0) 0x7f18d0017090 con 0x7f18d40ffaa0 2024-06-15T01:32:57.106 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:57.106 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi142.examgd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6825/1483389069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1483389069},{"type":"v1","addr":"172.21.15.142:6825","nonce":1483389069}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi142.uczriv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3598160234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3598160234},{"type":"v1","addr":"172.21.15.142:6827","nonce":3598160234}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-15T01:04:19.419133+0000","modified":"2024-06-15T01:30:35.389785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34338,"mds_1":34342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34338":{"gid":34338,"name":"cephfs.smithi137.nsntjq","rank":0,"incarnation":33,"state":"up:active","state_seq":243,"addr":"172.21.15.137:6829/208578565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6827","nonce":208578565},{"type":"v1","addr":"172.21.15.137:6829","nonce":208578565}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34342":{"gid":34342,"name":"cephfs.smithi137.afplli","rank":1,"incarnation":33,"state":"up:active","state_seq":238,"addr":"172.21.15.137:6828/3297690656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3297690656},{"type":"v1","addr":"172.21.15.137:6828","nonce":3297690656}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-06-15T01:32:57.106 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 38 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 msgr2=0x7f18c006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 0x7f18c006d3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f18c4036040 tx=0x7f18c4005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 msgr2=0x7f18d4100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f18d000d720 tx=0x7f18d000dbe0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 shutdown_connections 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f18c006aef0 0x7f18c006d3b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18d4101170 0x7f18d41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 --2- 172.21.15.137:0/642836016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d40ffaa0 0x7f18d4100c30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 >> 172.21.15.137:0/642836016 conn(0x7f18d4075250 msgr2=0x7f18d41030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 shutdown_connections 2024-06-15T01:32:57.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.123+0000 7f18db673640 1 -- 172.21.15.137:0/642836016 wait complete. 2024-06-15T01:32:57.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:56 smithi137 ceph-mon[112295]: pgmap v806: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:57.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:56 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2909761145' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-06-15T01:32:57.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:56 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3733400495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-06-15T01:32:57.118 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:56 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1795565067' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-06-15T01:32:57.119 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-06-15T01:32:57.130 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-06-15T01:32:57.130 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:32:57.131 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-15T01:32:57.149 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:32:57.149 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-15T01:32:57.206 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-15T01:32:57.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:56 smithi142 ceph-mon[82136]: pgmap v806: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:57.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:56 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2909761145' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-06-15T01:32:57.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:56 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3733400495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-06-15T01:32:57.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:56 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1795565067' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 -- 172.21.15.137:0/1811634362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 msgr2=0x7fe7880ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 --2- 172.21.15.137:0/1811634362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe7880ff440 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe770009880 tx=0x7fe77002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 -- 172.21.15.137:0/1811634362 shutdown_connections 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 --2- 172.21.15.137:0/1811634362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe7880ff440 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 --2- 172.21.15.137:0/1811634362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 0x7fe788104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 -- 172.21.15.137:0/1811634362 >> 172.21.15.137:0/1811634362 conn(0x7fe788075b40 msgr2=0x7fe788075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.361+0000 7fe78d48f640 1 -- 172.21.15.137:0/1811634362 shutdown_connections 2024-06-15T01:32:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 -- 172.21.15.137:0/1811634362 wait complete. 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 Processor -- start 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 -- start start 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 0x7fe788197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881976b0 con 0x7fe7880fefe0 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe78d48f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881977f0 con 0x7fe788103f20 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57706/0 (socket says 172.21.15.137:57706) 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe786ffd640 1 -- 172.21.15.137:0/4291006397 learned_addr learned my addr 172.21.15.137:0/4291006397 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.362+0000 7fe77ffff640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 0x7fe788197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe786ffd640 1 -- 172.21.15.137:0/4291006397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 msgr2=0x7fe788197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe786ffd640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 0x7fe788197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe786ffd640 1 -- 172.21.15.137:0/4291006397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009560 con 0x7fe7880fefe0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe786ffd640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe77800d6f0 tx=0x7fe77800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe778009a30 con 0x7fe7880fefe0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78819fbc0 con 0x7fe7880fefe0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7780054c0 con 0x7fe7880fefe0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe778004db0 con 0x7fe7880fefe0 2024-06-15T01:32:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.363+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881a0090 con 0x7fe7880fefe0 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.364+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78804e040 con 0x7fe7880fefe0 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.364+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe778004f30 con 0x7fe7880fefe0 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.365+0000 7fe784ff9640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 0x7fe77406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.365+0000 7fe77ffff640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 0x7fe77406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.365+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe778089dc0 con 0x7fe7880fefe0 2024-06-15T01:32:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.366+0000 7fe77ffff640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 0x7fe77406d410 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe770005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.369+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe778054ca0 con 0x7fe7880fefe0 2024-06-15T01:32:57.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.541+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe78810c2b0 con 0x7fe7880fefe0 2024-06-15T01:32:57.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.542+0000 7fe784ff9640 1 -- 172.21.15.137:0/4291006397 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16801 (secure 0 0 0) 0x7fe7780582c0 con 0x7fe7880fefe0 2024-06-15T01:32:57.527 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:57.527 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":95,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:15:20.620477+0000","last_up_change":"2024-06-15T01:14:30.413524+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-06-15T01:04:17.764733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-06-15T01:04:18.637750+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6803","nonce":4060871906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6805","nonce":4060871906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6809","nonce":4060871906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6807","nonce":4060871906}]},"public_addr":"172.21.15.137:6803/4060871906","cluster_addr":"172.21.15.137:6805/4060871906","heartbeat_back_addr":"172.21.15.137:6809/4060871906","heartbeat_front_addr":"172.21.15.137:6807/4060871906","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6811","nonce":1628106669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6813","nonce":1628106669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6817","nonce":1628106669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6815","nonce":1628106669}]},"public_addr":"172.21.15.137:6811/1628106669","cluster_addr":"172.21.15.137:6813/1628106669","heartbeat_back_addr":"172.21.15.137:6817/1628106669","heartbeat_front_addr":"172.21.15.137:6815/1628106669","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":90,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6819","nonce":3034759480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6821","nonce":3034759480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6825","nonce":3034759480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6823","nonce":3034759480}]},"public_addr":"172.21.15.137:6819/3034759480","cluster_addr":"172.21.15.137:6821/3034759480","heartbeat_back_addr":"172.21.15.137:6825/3034759480","heartbeat_front_addr":"172.21.15.137:6823/3034759480","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":90,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6801","nonce":3361419228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6803","nonce":3361419228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6807","nonce":3361419228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6805","nonce":3361419228}]},"public_addr":"172.21.15.142:6801/3361419228","cluster_addr":"172.21.15.142:6803/3361419228","heartbeat_back_addr":"172.21.15.142:6807/3361419228","heartbeat_front_addr":"172.21.15.142:6805/3361419228","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":81,"up_thru":90,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6809","nonce":464434813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6811","nonce":464434813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6815","nonce":464434813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6813","nonce":464434813}]},"public_addr":"172.21.15.142:6809/464434813","cluster_addr":"172.21.15.142:6811/464434813","heartbeat_back_addr":"172.21.15.142:6815/464434813","heartbeat_front_addr":"172.21.15.142:6813/464434813","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":90,"up_thru":90,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6817","nonce":1909901146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6819","nonce":1909901146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6823","nonce":1909901146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6821","nonce":1909901146}]},"public_addr":"172.21.15.142:6817/1909901146","cluster_addr":"172.21.15.142:6819/1909901146","heartbeat_back_addr":"172.21.15.142:6823/1909901146","heartbeat_front_addr":"172.21.15.142:6821/1909901146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":3,"to":2},{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":5}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.f","mappings":[{"from":0,"to":2}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.b","mappings":[{"from":1,"to":2}]},{"pgid":"3.10","mappings":[{"from":1,"to":4}]},{"pgid":"3.11","mappings":[{"from":3,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6829/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6828/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6826/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:0/3277128014":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/623606439":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/146013243":"2024-06-16T01:06:34.431086+0000","172.21.15.142:6825/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:0/4288581606":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.142:0/1961055405":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6824/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:6829/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6828/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/388348183":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/72877675":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/944831942":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6801/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/3967735174":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6827/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:6800/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/544255511":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 msgr2=0x7fe77406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 0x7fe77406d410 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe770005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 msgr2=0x7fe788196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe77800d6f0 tx=0x7fe77800dbb0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 shutdown_connections 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe77406af50 0x7fe77406d410 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe788103f20 0x7fe788197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 --2- 172.21.15.137:0/4291006397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7880fefe0 0x7fe788196b50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 >> 172.21.15.137:0/4291006397 conn(0x7fe788075b40 msgr2=0x7fe7880fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 shutdown_connections 2024-06-15T01:32:57.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.544+0000 7fe78d48f640 1 -- 172.21.15.137:0/4291006397 wait complete. 2024-06-15T01:32:57.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-15T01:01:49.347551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-06-15T01:04:17.764733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-06-15T01:04:18.637750+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-15T01:32:57.541 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- 172.21.15.137:0/3221436912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 msgr2=0x7fbc6c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 --2- 172.21.15.137:0/3221436912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c0ff640 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009880 tx=0x7fbc5c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- 172.21.15.137:0/3221436912 shutdown_connections 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 --2- 172.21.15.137:0/3221436912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c0ffc10 0x7fbc6c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 --2- 172.21.15.137:0/3221436912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c0ff640 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- 172.21.15.137:0/3221436912 >> 172.21.15.137:0/3221436912 conn(0x7fbc6c074a50 msgr2=0x7fbc6c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- 172.21.15.137:0/3221436912 shutdown_connections 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- 172.21.15.137:0/3221436912 wait complete. 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 Processor -- start 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.659+0000 7fbc729f0640 1 -- start start 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c1961f0 0x7fbc6c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c1967f0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c196930 con 0x7fbc6c1961f0 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57732/0 (socket says 172.21.15.137:57732) 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 -- 172.21.15.137:0/1049374487 learned_addr learned my addr 172.21.15.137:0/1049374487 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc63fff640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c1961f0 0x7fbc6c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 -- 172.21.15.137:0/1049374487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c1961f0 msgr2=0x7fbc6c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c1961f0 0x7fbc6c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 -- 172.21.15.137:0/1049374487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c009560 con 0x7fbc6c0ff260 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc6bfff640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009230 tx=0x7fbc5c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5c003cf0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c199ba0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc5c003e50 con 0x7fbc6c0ff260 2024-06-15T01:32:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5c034790 con 0x7fbc6c0ff260 2024-06-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.660+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c19a100 con 0x7fbc6c0ff260 2024-06-15T01:32:57.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.661+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c19a3e0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.662+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fbc5c03ebf0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.662+0000 7fbc69ffb640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 0x7fbc5806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.662+0000 7fbc63fff640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 0x7fbc5806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.663+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbc5c0ad0a0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.663+0000 7fbc63fff640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 0x7fbc5806d3b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbc540058c0 tx=0x7fbc5400a4c0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.667+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbc5c077ed0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.838+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc6c196fc0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.838+0000 7fbc69ffb640 1 -- 172.21.15.137:0/1049374487 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc5c07b4f0 con 0x7fbc6c0ff260 2024-06-15T01:32:57.823 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 msgr2=0x7fbc5806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 0x7fbc5806d3b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbc540058c0 tx=0x7fbc5400a4c0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 msgr2=0x7fbc6c195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009230 tx=0x7fbc5c003820 comp rx=0 tx=0).stop 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 shutdown_connections 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fbc5806aef0 0x7fbc5806d3b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc6c1961f0 0x7fbc6c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 --2- 172.21.15.137:0/1049374487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc6c0ff260 0x7fbc6c195cb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 >> 172.21.15.137:0/1049374487 conn(0x7fbc6c074a50 msgr2=0x7fbc6c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 shutdown_connections 2024-06-15T01:32:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.840+0000 7fbc729f0640 1 -- 172.21.15.137:0/1049374487 wait complete. 2024-06-15T01:32:57.835 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-06-15T01:32:57.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- 172.21.15.137:0/2028883805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 msgr2=0x7f1fd81015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/2028883805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd81015e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009c20 tx=0x7f1fc802c230 comp rx=0 tx=0).stop 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- 172.21.15.137:0/2028883805 shutdown_connections 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/2028883805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 0x7f1fd810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/2028883805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd81015e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- 172.21.15.137:0/2028883805 >> 172.21.15.137:0/2028883805 conn(0x7f1fd80faa20 msgr2=0x7f1fd80fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- 172.21.15.137:0/2028883805 shutdown_connections 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- 172.21.15.137:0/2028883805 wait complete. 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 Processor -- start 2024-06-15T01:32:57.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.954+0000 7f1fddfd3640 1 -- start start 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 0x7f1fd819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8195ae0 con 0x7f1fd8101200 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8195c50 con 0x7f1fd8101b20 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57752/0 (socket says 172.21.15.137:57752) 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 -- 172.21.15.137:0/3949716645 learned_addr learned my addr 172.21.15.137:0/3949716645 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd6ffd640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 0x7f1fd819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 -- 172.21.15.137:0/3949716645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 msgr2=0x7f1fd819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 0x7f1fd819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 -- 172.21.15.137:0/3949716645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8009900 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd77fe640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009d40 tx=0x7f1fc8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc8003b40 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1fc8032d50 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd8195ed0 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc8004650 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.955+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd81963d0 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.956+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd804e040 con 0x7f1fd8101200 2024-06-15T01:32:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.957+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f1fc803b040 con 0x7f1fd8101200 2024-06-15T01:32:57.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.957+0000 7f1fd4ff9640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 0x7f1fc006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:57.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.957+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1fc80acfc0 con 0x7f1fd8101200 2024-06-15T01:32:57.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.957+0000 7f1fd6ffd640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 0x7f1fc006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:57.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.958+0000 7f1fd6ffd640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 0x7f1fc006d3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80fc040 tx=0x7f1fcc005f90 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:57.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:57.962+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1fc8077f90 con 0x7f1fd8101200 2024-06-15T01:32:58.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.134+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1fd8107670 con 0x7f1fd8101200 2024-06-15T01:32:58.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.135+0000 7f1fd4ff9640 1 -- 172.21.15.137:0/3949716645 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1fd8107670 con 0x7f1fd8101200 2024-06-15T01:32:58.119 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:58.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.136+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 msgr2=0x7f1fc006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.136+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 0x7f1fc006d3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80fc040 tx=0x7f1fcc005f90 comp rx=0 tx=0).stop 2024-06-15T01:32:58.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 msgr2=0x7f1fd819ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009d40 tx=0x7f1fc8034670 comp rx=0 tx=0).stop 2024-06-15T01:32:58.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 shutdown_connections 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f1fc006aef0 0x7f1fc006d3b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd8101b20 0x7f1fd819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 --2- 172.21.15.137:0/3949716645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fd8101200 0x7f1fd819ba70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 >> 172.21.15.137:0/3949716645 conn(0x7f1fd80faa20 msgr2=0x7f1fd8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 shutdown_connections 2024-06-15T01:32:58.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.137+0000 7f1fddfd3640 1 -- 172.21.15.137:0/3949716645 wait complete. 2024-06-15T01:32:58.131 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:57 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/642836016' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-06-15T01:32:58.131 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:57 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4291006397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:32:58.131 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:57 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1049374487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:32:58.132 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-06-15T01:32:58.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- 172.21.15.137:0/684854431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe488100450 msgr2=0x7fe488103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/684854431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe488100450 0x7fe488103b90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe474009880 tx=0x7fe47402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- 172.21.15.137:0/684854431 shutdown_connections 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/684854431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe488100450 0x7fe488103b90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/684854431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4880ffaa0 0x7fe4880ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- 172.21.15.137:0/684854431 >> 172.21.15.137:0/684854431 conn(0x7fe488075250 msgr2=0x7fe488075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- 172.21.15.137:0/684854431 shutdown_connections 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- 172.21.15.137:0/684854431 wait complete. 2024-06-15T01:32:58.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 Processor -- start 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.250+0000 7fe48f0c7640 1 -- start start 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48f0c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48f0c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48819af30 0x7fe488194a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48f0c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48819b500 con 0x7fe4880ffaa0 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48f0c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488194fa0 con 0x7fe48819af30 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57764/0 (socket says 172.21.15.137:57764) 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 -- 172.21.15.137:0/2559807497 learned_addr learned my addr 172.21.15.137:0/2559807497 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe487fff640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48819af30 0x7fe488194a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 -- 172.21.15.137:0/2559807497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48819af30 msgr2=0x7fe488194a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48819af30 0x7fe488194a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 -- 172.21.15.137:0/2559807497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474009560 con 0x7fe4880ffaa0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48ce63640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e970 tx=0x7fe47c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c00cc90 con 0x7fe4880ffaa0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe47c00cdf0 con 0x7fe4880ffaa0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c010430 con 0x7fe4880ffaa0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.251+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe488195280 con 0x7fe4880ffaa0 2024-06-15T01:32:58.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.252+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4881957e0 con 0x7fe4880ffaa0 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.253+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48804e040 con 0x7fe4880ffaa0 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.253+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe47c0046c0 con 0x7fe4880ffaa0 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.254+0000 7fe485ffb640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 0x7fe47806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.254+0000 7fe487fff640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 0x7fe47806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.254+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe47c014070 con 0x7fe4880ffaa0 2024-06-15T01:32:58.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.254+0000 7fe487fff640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 0x7fe47806d3b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe474009230 tx=0x7fe474005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.258+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe47c055660 con 0x7fe4880ffaa0 2024-06-15T01:32:58.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:57 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/642836016' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-06-15T01:32:58.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:57 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4291006397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:32:58.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:57 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1049374487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:32:58.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.432+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe488195f70 con 0x7fe4880ffaa0 2024-06-15T01:32:58.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.432+0000 7fe485ffb640 1 -- 172.21.15.137:0/2559807497 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe47c058c80 con 0x7fe4880ffaa0 2024-06-15T01:32:58.417 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 msgr2=0x7fe47806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 0x7fe47806d3b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe474009230 tx=0x7fe474005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 msgr2=0x7fe48819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e970 tx=0x7fe47c00ed70 comp rx=0 tx=0).stop 2024-06-15T01:32:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 shutdown_connections 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47806aef0 0x7fe47806d3b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48819af30 0x7fe488194a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 --2- 172.21.15.137:0/2559807497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4880ffaa0 0x7fe48819a9f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.434+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 >> 172.21.15.137:0/2559807497 conn(0x7fe488075250 msgr2=0x7fe4880fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.435+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 shutdown_connections 2024-06-15T01:32:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.435+0000 7fe48f0c7640 1 -- 172.21.15.137:0/2559807497 wait complete. 2024-06-15T01:32:58.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-06-15T01:32:58.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 -- 172.21.15.137:0/2640604194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 msgr2=0x7f70540ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2640604194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f70540ffc90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f703c009880 tx=0x7f703c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 -- 172.21.15.137:0/2640604194 shutdown_connections 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2640604194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f70540ffc90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2640604194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 0x7f7054105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 -- 172.21.15.137:0/2640604194 >> 172.21.15.137:0/2640604194 conn(0x7f70540752f0 msgr2=0x7f7054075700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 -- 172.21.15.137:0/2640604194 shutdown_connections 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.546+0000 7f7059ce5640 1 -- 172.21.15.137:0/2640604194 wait complete. 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 Processor -- start 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 -- start start 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 0x7f7054196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054197480 con 0x7f70540ff830 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7059ce5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541975f0 con 0x7f7054105180 2024-06-15T01:32:58.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57790/0 (socket says 172.21.15.137:57790) 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 -- 172.21.15.137:0/2616604706 learned_addr learned my addr 172.21.15.137:0/2616604706 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f7052ffd640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 0x7f7054196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 -- 172.21.15.137:0/2616604706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 msgr2=0x7f7054196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 0x7f7054196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.547+0000 7f70537fe640 1 -- 172.21.15.137:0/2616604706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009560 con 0x7f70540ff830 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f70537fe640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f704400d6f0 tx=0x7f704400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7044009a30 con 0x7f70540ff830 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70440054c0 con 0x7f70540ff830 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705419f9e0 con 0x7f70540ff830 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7044004db0 con 0x7f70540ff830 2024-06-15T01:32:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.548+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705419fe60 con 0x7f70540ff830 2024-06-15T01:32:58.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.549+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705404e040 con 0x7f70540ff830 2024-06-15T01:32:58.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.549+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f7044004f30 con 0x7f70540ff830 2024-06-15T01:32:58.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.550+0000 7f7050ff9640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.550+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f7044089f00 con 0x7f70540ff830 2024-06-15T01:32:58.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.550+0000 7f7052ffd640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.550+0000 7f7052ffd640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 0x7f704006d420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f70540fef80 tx=0x7f703c000bc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.554+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7044054d30 con 0x7f70540ff830 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.738+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f705410c3a0 con 0x7f70540ff830 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.738+0000 7f7050ff9640 1 -- 172.21.15.137:0/2616604706 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v95) v1 ==== 81+0+2099 (secure 0 0 0) 0x7f7044017850 con 0x7f70540ff830 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6829/937501599 2024-06-16T01:14:29.156226+0000 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6828/937501599 2024-06-16T01:14:29.156226+0000 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/2929246681 2024-06-16T00:59:33.029395+0000 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6826/183082556 2024-06-16T01:14:29.156215+0000 2024-06-15T01:32:58.723 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3277128014 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.724 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/623606439 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.725 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/146013243 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.725 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6825/2333305414 2024-06-16T01:04:33.393618+0000 2024-06-15T01:32:58.725 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/4288581606 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/3423141192 2024-06-16T00:57:46.426899+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/778536834 2024-06-16T00:59:33.029395+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/1961055405 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6824/2333305414 2024-06-16T01:04:33.393618+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6829/2339029036 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6828/2339029036 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/388348183 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2983067162 2024-06-16T00:57:46.426899+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/4018707027 2024-06-16T00:58:18.710601+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3386774887 2024-06-16T00:58:18.710601+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1189222759 2024-06-16T00:57:46.426899+0000 2024-06-15T01:32:58.726 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/72877675 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/944831942 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/3216905676 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/3423141192 2024-06-16T00:57:46.426899+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/4018707027 2024-06-16T00:58:18.710601+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/3967735174 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6827/183082556 2024-06-16T01:14:29.156215+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/3216905676 2024-06-16T01:06:34.431086+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/941695676 2024-06-16T00:58:18.710601+0000 2024-06-15T01:32:58.727 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1762265425 2024-06-16T00:59:33.029395+0000 2024-06-15T01:32:58.728 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/578055110 2024-06-16T00:57:46.426899+0000 2024-06-15T01:32:58.728 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3247562455 2024-06-16T00:59:33.029395+0000 2024-06-15T01:32:58.728 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1946521916 2024-06-16T00:58:18.710601+0000 2024-06-15T01:32:58.728 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/544255511 2024-06-16T01:07:47.641756+0000 2024-06-15T01:32:58.728 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/2929246681 2024-06-16T00:59:33.029395+0000 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:listed 35 entries 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 msgr2=0x7f704006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 0x7f704006d420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f70540fef80 tx=0x7f703c000bc0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 msgr2=0x7f7054196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f704400d6f0 tx=0x7f704400dbb0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 shutdown_connections 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f704006af60 0x7f704006d420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7054105180 0x7f7054196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 --2- 172.21.15.137:0/2616604706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff830 0x7f7054196970 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 >> 172.21.15.137:0/2616604706 conn(0x7f70540752f0 msgr2=0x7f70540fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 shutdown_connections 2024-06-15T01:32:58.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.740+0000 7f7059ce5640 1 -- 172.21.15.137:0/2616604706 wait complete. 2024-06-15T01:32:58.736 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-06-15T01:32:58.736 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-15T01:32:58.753 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 -- 172.21.15.137:0/3787288146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffb30 msgr2=0x7f23a4103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3787288146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffb30 0x7f23a4103b70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f238c009880 tx=0x7f238c02bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 -- 172.21.15.137:0/3787288146 shutdown_connections 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3787288146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffb30 0x7f23a4103b70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3787288146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 0x7f23a40ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 -- 172.21.15.137:0/3787288146 >> 172.21.15.137:0/3787288146 conn(0x7f23a40fa9e0 msgr2=0x7f23a40fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 -- 172.21.15.137:0/3787288146 shutdown_connections 2024-06-15T01:32:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.909+0000 7f23aa03a640 1 -- 172.21.15.137:0/3787288146 wait complete. 2024-06-15T01:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 Processor -- start 2024-06-15T01:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 -- start start 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 0x7f23a419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a419c4c0 con 0x7f23a419bef0 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23aa03a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4195f60 con 0x7f23a40ff180 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57804/0 (socket says 172.21.15.137:57804) 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 -- 172.21.15.137:0/3300562282 learned_addr learned my addr 172.21.15.137:0/3300562282 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23a37fe640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 0x7f23a419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 -- 172.21.15.137:0/3300562282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 msgr2=0x7f23a419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 0x7f23a419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 -- 172.21.15.137:0/3300562282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c009560 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f239bfff640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f238c009230 tx=0x7f238c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.910+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f238c003b50 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.911+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f238c003cb0 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.911+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f238c034700 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.911+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a41961e0 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.911+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a4196740 con 0x7f23a419bef0 2024-06-15T01:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.911+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a404e040 con 0x7f23a419bef0 2024-06-15T01:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.912+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f238c03eaf0 con 0x7f23a419bef0 2024-06-15T01:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.912+0000 7f23a17fa640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 0x7f239006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.913+0000 7f23a37fe640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 0x7f239006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.913+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f238c0acf50 con 0x7f23a419bef0 2024-06-15T01:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.913+0000 7f23a37fe640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 0x7f239006d470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2394005fd0 tx=0x7f2394005850 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:58.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:58.917+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f238c0e0900 con 0x7f23a419bef0 2024-06-15T01:32:59.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.087+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a40ffb30 con 0x7f23a419bef0 2024-06-15T01:32:59.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.089+0000 7f23a17fa640 1 -- 172.21.15.137:0/3300562282 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f238c077d80 con 0x7f23a419bef0 2024-06-15T01:32:59.073 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-06-15T01:32:59.073 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":95,"fsid":"225ca7a2-2ab2-11ef-bc9d-c7b262605968","created":"2024-06-15T00:57:12.444834+0000","modified":"2024-06-15T01:15:20.620477+0000","last_up_change":"2024-06-15T01:14:30.413524+0000","last_in_change":"2024-06-15T01:02:43.914451+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-15T01:01:49.347551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-06-15T01:04:17.764733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-06-15T01:04:18.637750+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a69222b-cedc-4834-9d6d-0a1801c6a9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6803","nonce":4060871906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6805","nonce":4060871906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6809","nonce":4060871906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4060871906},{"type":"v1","addr":"172.21.15.137:6807","nonce":4060871906}]},"public_addr":"172.21.15.137:6803/4060871906","cluster_addr":"172.21.15.137:6805/4060871906","heartbeat_back_addr":"172.21.15.137:6809/4060871906","heartbeat_front_addr":"172.21.15.137:6807/4060871906","state":["exists","up"]},{"osd":1,"uuid":"6842e406-422f-463c-94cf-42233a80ccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6811","nonce":1628106669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6813","nonce":1628106669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6817","nonce":1628106669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1628106669},{"type":"v1","addr":"172.21.15.137:6815","nonce":1628106669}]},"public_addr":"172.21.15.137:6811/1628106669","cluster_addr":"172.21.15.137:6813/1628106669","heartbeat_back_addr":"172.21.15.137:6817/1628106669","heartbeat_front_addr":"172.21.15.137:6815/1628106669","state":["exists","up"]},{"osd":2,"uuid":"fe07b1ac-bc4d-47a6-a991-2a6b1ece03fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":90,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6819","nonce":3034759480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6821","nonce":3034759480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6825","nonce":3034759480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3034759480},{"type":"v1","addr":"172.21.15.137:6823","nonce":3034759480}]},"public_addr":"172.21.15.137:6819/3034759480","cluster_addr":"172.21.15.137:6821/3034759480","heartbeat_back_addr":"172.21.15.137:6825/3034759480","heartbeat_front_addr":"172.21.15.137:6823/3034759480","state":["exists","up"]},{"osd":3,"uuid":"edcfe413-a142-4887-ac04-50c3ea15fb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":90,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6801","nonce":3361419228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6803","nonce":3361419228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6807","nonce":3361419228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3361419228},{"type":"v1","addr":"172.21.15.142:6805","nonce":3361419228}]},"public_addr":"172.21.15.142:6801/3361419228","cluster_addr":"172.21.15.142:6803/3361419228","heartbeat_back_addr":"172.21.15.142:6807/3361419228","heartbeat_front_addr":"172.21.15.142:6805/3361419228","state":["exists","up"]},{"osd":4,"uuid":"6fe44512-dd8e-468e-9a24-33bf9b288d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":81,"up_thru":90,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6809","nonce":464434813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6811","nonce":464434813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6815","nonce":464434813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":464434813},{"type":"v1","addr":"172.21.15.142:6813","nonce":464434813}]},"public_addr":"172.21.15.142:6809/464434813","cluster_addr":"172.21.15.142:6811/464434813","heartbeat_back_addr":"172.21.15.142:6815/464434813","heartbeat_front_addr":"172.21.15.142:6813/464434813","state":["exists","up"]},{"osd":5,"uuid":"d3ec5d99-d399-439d-a909-709c9eff7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":85,"up_from":90,"up_thru":90,"down_at":86,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6817","nonce":1909901146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6819","nonce":1909901146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6823","nonce":1909901146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1909901146},{"type":"v1","addr":"172.21.15.142:6821","nonce":1909901146}]},"public_addr":"172.21.15.142:6817/1909901146","cluster_addr":"172.21.15.142:6819/1909901146","heartbeat_back_addr":"172.21.15.142:6823/1909901146","heartbeat_front_addr":"172.21.15.142:6821/1909901146","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:00:48.199270+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:17.994082+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:01:45.640378+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:10.070984+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:02:35.943542+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-06-15T01:03:01.104678+0000","dead_epoch":85}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":3,"to":2},{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":5}]},{"pgid":"2.9","mappings":[{"from":0,"to":2}]},{"pgid":"2.f","mappings":[{"from":0,"to":2}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.b","mappings":[{"from":1,"to":2}]},{"pgid":"3.10","mappings":[{"from":1,"to":4}]},{"pgid":"3.11","mappings":[{"from":3,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6829/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6828/937501599":"2024-06-16T01:14:29.156226+0000","172.21.15.137:6801/2929246681":"2024-06-16T00:59:33.029395+0000","172.21.15.137:6826/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:0/3277128014":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/623606439":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/146013243":"2024-06-16T01:06:34.431086+0000","172.21.15.142:6825/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:0/4288581606":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/778536834":"2024-06-16T00:59:33.029395+0000","172.21.15.142:0/1961055405":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6824/2333305414":"2024-06-16T01:04:33.393618+0000","172.21.15.142:6829/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.142:6828/2339029036":"2024-06-16T01:07:47.641756+0000","172.21.15.137:0/388348183":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/2983067162":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6801/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/3386774887":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1189222759":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/72877675":"2024-06-16T01:06:34.431086+0000","172.21.15.142:0/944831942":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6801/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:6801/3423141192":"2024-06-16T00:57:46.426899+0000","172.21.15.137:6800/4018707027":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/3967735174":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6827/183082556":"2024-06-16T01:14:29.156215+0000","172.21.15.137:6800/3216905676":"2024-06-16T01:06:34.431086+0000","172.21.15.137:0/941695676":"2024-06-16T00:58:18.710601+0000","172.21.15.137:0/1762265425":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/578055110":"2024-06-16T00:57:46.426899+0000","172.21.15.137:0/3247562455":"2024-06-16T00:59:33.029395+0000","172.21.15.137:0/1946521916":"2024-06-16T00:58:18.710601+0000","172.21.15.142:0/544255511":"2024-06-16T01:07:47.641756+0000","172.21.15.137:6800/2929246681":"2024-06-16T00:59:33.029395+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-15T01:32:59.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 msgr2=0x7f239006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 0x7f239006d470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2394005fd0 tx=0x7f2394005850 comp rx=0 tx=0).stop 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 msgr2=0x7f23a4195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f238c009230 tx=0x7f238c0045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 shutdown_connections 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f239006afb0 0x7f239006d470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a419bef0 0x7f23a4195a20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 --2- 172.21.15.137:0/3300562282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a40ff180 0x7f23a419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 >> 172.21.15.137:0/3300562282 conn(0x7f23a40fa9e0 msgr2=0x7f23a40fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 shutdown_connections 2024-06-15T01:32:59.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.091+0000 7f23aa03a640 1 -- 172.21.15.137:0/3300562282 wait complete. 2024-06-15T01:32:59.085 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:58 smithi137 ceph-mon[112295]: pgmap v807: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:59.085 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:58 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3949716645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.085 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:58 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2559807497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.085 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:58 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/2616604706' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-15T01:32:59.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-15T01:01:49.347551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-06-15T01:04:17.764733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-06-15T01:04:18.637750+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-15T01:32:59.087 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- 172.21.15.137:0/1453722662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ffc10 msgr2=0x7f89b0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 --2- 172.21.15.137:0/1453722662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ffc10 0x7f89b0103b70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009880 tx=0x7f89a002bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- 172.21.15.137:0/1453722662 shutdown_connections 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 --2- 172.21.15.137:0/1453722662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ffc10 0x7f89b0103b70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 --2- 172.21.15.137:0/1453722662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b00ff260 0x7f89b00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- 172.21.15.137:0/1453722662 >> 172.21.15.137:0/1453722662 conn(0x7f89b0074a50 msgr2=0x7f89b0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- 172.21.15.137:0/1453722662 shutdown_connections 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- 172.21.15.137:0/1453722662 wait complete. 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 Processor -- start 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- start start 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 0x7f89b01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b019fc40 con 0x7f89b00ff260 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.207+0000 7f89b7786640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b019fd80 con 0x7f89b0197080 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57832/0 (socket says 172.21.15.137:57832) 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 -- 172.21.15.137:0/4062009877 learned_addr learned my addr 172.21.15.137:0/4062009877 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b4d21640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 0x7f89b01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 -- 172.21.15.137:0/4062009877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 msgr2=0x7f89b01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 0x7f89b01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 -- 172.21.15.137:0/4062009877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a0009560 con 0x7f89b00ff260 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b4d21640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 0x7f89b01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b5522640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00d720 tx=0x7f89ac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ac0051f0 con 0x7f89b00ff260 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89ac009820 con 0x7f89b00ff260 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b019ff80 con 0x7f89b00ff260 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ac004d40 con 0x7f89b00ff260 2024-06-15T01:32:59.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.208+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b01a0400 con 0x7f89b00ff260 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.209+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b004e040 con 0x7f89b00ff260 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.210+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7f89ac009990 con 0x7f89b00ff260 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.210+0000 7f89a67fc640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 0x7f899c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.210+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f89ac089ee0 con 0x7f89b00ff260 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.210+0000 7f89b4d21640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 0x7f899c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.211+0000 7f89b4d21640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 0x7f899c06d3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a0005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.214+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f89ac054d10 con 0x7f89b00ff260 2024-06-15T01:32:59.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:58 smithi142 ceph-mon[82136]: pgmap v807: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:32:59.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:58 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3949716645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:58 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2559807497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:58 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/2616604706' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-15T01:32:59.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.385+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89b00ffcb0 con 0x7f89b00ff260 2024-06-15T01:32:59.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.385+0000 7f89a67fc640 1 -- 172.21.15.137:0/4062009877 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89ac0200a0 con 0x7f89b00ff260 2024-06-15T01:32:59.370 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-06-15T01:32:59.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 msgr2=0x7f899c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 0x7f899c06d3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a0005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:59.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 msgr2=0x7f89b0196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00d720 tx=0x7f89ac00dbe0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 shutdown_connections 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7f899c06aef0 0x7f899c06d3b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89b0197080 0x7f89b01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 --2- 172.21.15.137:0/4062009877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b00ff260 0x7f89b0196b40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 >> 172.21.15.137:0/4062009877 conn(0x7f89b0074a50 msgr2=0x7f89b00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 shutdown_connections 2024-06-15T01:32:59.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.388+0000 7f89b7786640 1 -- 172.21.15.137:0/4062009877 wait complete. 2024-06-15T01:32:59.383 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- 172.21.15.137:0/2792445357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe490100c50 msgr2=0x7fe490104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 --2- 172.21.15.137:0/2792445357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe490100c50 0x7fe490104c90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe478009880 tx=0x7fe47802bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- 172.21.15.137:0/2792445357 shutdown_connections 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 --2- 172.21.15.137:0/2792445357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe490100c50 0x7fe490104c90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 --2- 172.21.15.137:0/2792445357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4901002a0 0x7fe490100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- 172.21.15.137:0/2792445357 >> 172.21.15.137:0/2792445357 conn(0x7fe4900fbb20 msgr2=0x7fe4900fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- 172.21.15.137:0/2792445357 shutdown_connections 2024-06-15T01:32:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- 172.21.15.137:0/2792445357 wait complete. 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 Processor -- start 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.501+0000 7fe495054640 1 -- start start 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe495054640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe495054640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe490196280 0x7fe49019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe495054640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490196880 con 0x7fe4901002a0 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe495054640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901969f0 con 0x7fe490196280 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57852/0 (socket says 172.21.15.137:57852) 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 -- 172.21.15.137:0/3432375577 learned_addr learned my addr 172.21.15.137:0/3432375577 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe487fff640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe490196280 0x7fe49019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 -- 172.21.15.137:0/3432375577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe490196280 msgr2=0x7fe49019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe490196280 0x7fe49019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 -- 172.21.15.137:0/3432375577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009560 con 0x7fe4901002a0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48ed9d640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe48000e9a0 tx=0x7fe48000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48000cd50 con 0x7fe4901002a0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe48000ceb0 con 0x7fe4901002a0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49019ace0 con 0x7fe4901002a0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.502+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe480010430 con 0x7fe4901002a0 2024-06-15T01:32:59.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.503+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49019b240 con 0x7fe4901002a0 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.503+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49004e040 con 0x7fe4901002a0 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.504+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fe480005020 con 0x7fe4901002a0 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.504+0000 7fe48cd99640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 0x7fe47c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.505+0000 7fe487fff640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 0x7fe47c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.505+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe480014070 con 0x7fe4901002a0 2024-06-15T01:32:59.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.505+0000 7fe487fff640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 0x7fe47c06d3b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005d50 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.509+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe480055720 con 0x7fe4901002a0 2024-06-15T01:32:59.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.678+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe490197090 con 0x7fe4901002a0 2024-06-15T01:32:59.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.679+0000 7fe48cd99640 1 -- 172.21.15.137:0/3432375577 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe480058d40 con 0x7fe4901002a0 2024-06-15T01:32:59.663 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:59.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 msgr2=0x7fe47c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 0x7fe47c06d3b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005d50 comp rx=0 tx=0).stop 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 msgr2=0x7fe490195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe48000e9a0 tx=0x7fe48000eda0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 shutdown_connections 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fe47c06aef0 0x7fe47c06d3b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe490196280 0x7fe49019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 --2- 172.21.15.137:0/3432375577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4901002a0 0x7fe490195d40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 >> 172.21.15.137:0/3432375577 conn(0x7fe4900fbb20 msgr2=0x7fe4900fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 shutdown_connections 2024-06-15T01:32:59.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.681+0000 7fe495054640 1 -- 172.21.15.137:0/3432375577 wait complete. 2024-06-15T01:32:59.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-06-15T01:32:59.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.794+0000 7fee4c155640 1 -- 172.21.15.137:0/238139043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee440ff390 msgr2=0x7fee44104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.794+0000 7fee4c155640 1 --2- 172.21.15.137:0/238139043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee440ff390 0x7fee44104d10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fee34009880 tx=0x7fee3402bce0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- 172.21.15.137:0/238139043 shutdown_connections 2024-06-15T01:32:59.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 --2- 172.21.15.137:0/238139043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee440ff390 0x7fee44104d10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 --2- 172.21.15.137:0/238139043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee440fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- 172.21.15.137:0/238139043 >> 172.21.15.137:0/238139043 conn(0x7fee440741f0 msgr2=0x7fee44074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- 172.21.15.137:0/238139043 shutdown_connections 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- 172.21.15.137:0/238139043 wait complete. 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 Processor -- start 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- start start 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee44195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee441968b0 con 0x7fee441962b0 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee4c155640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee44196a20 con 0x7fee440fe9e0 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.795+0000 7fee496f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57872/0 (socket says 172.21.15.137:57872) 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 -- 172.21.15.137:0/1149344548 learned_addr learned my addr 172.21.15.137:0/1149344548 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee49ef1640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee44195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 -- 172.21.15.137:0/1149344548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 msgr2=0x7fee44195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee44195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 -- 172.21.15.137:0/1149344548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34009560 con 0x7fee441962b0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee49ef1640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee44195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee496f0640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fee34036040 tx=0x7fee340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee34003b50 con 0x7fee441962b0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee34003cb0 con 0x7fee441962b0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4419ad10 con 0x7fee441962b0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee34034790 con 0x7fee441962b0 2024-06-15T01:32:59.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.796+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4419b270 con 0x7fee441962b0 2024-06-15T01:32:59.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.797+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4404e040 con 0x7fee441962b0 2024-06-15T01:32:59.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.798+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fee3403eb60 con 0x7fee441962b0 2024-06-15T01:32:59.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.798+0000 7fee3affd640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 0x7fee3006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:32:59.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.798+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fee340acea0 con 0x7fee441962b0 2024-06-15T01:32:59.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.798+0000 7fee49ef1640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 0x7fee3006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:32:59.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.799+0000 7fee49ef1640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 0x7fee3006d3b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee40008040 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:32:59.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.802+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fee34077d50 con 0x7fee441962b0 2024-06-15T01:32:59.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.974+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fee441085f0 con 0x7fee441962b0 2024-06-15T01:32:59.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.974+0000 7fee3affd640 1 -- 172.21.15.137:0/1149344548 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fee34043020 con 0x7fee441962b0 2024-06-15T01:32:59.959 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-06-15T01:32:59.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 msgr2=0x7fee3006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 0x7fee3006d3b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee40008040 comp rx=0 tx=0).stop 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 msgr2=0x7fee4419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fee34036040 tx=0x7fee340045e0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 shutdown_connections 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fee3006aef0 0x7fee3006d3b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee441962b0 0x7fee4419a710 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 --2- 172.21.15.137:0/1149344548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee440fe9e0 0x7fee44195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 >> 172.21.15.137:0/1149344548 conn(0x7fee440741f0 msgr2=0x7fee440fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 shutdown_connections 2024-06-15T01:32:59.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:32:59.976+0000 7fee4c155640 1 -- 172.21.15.137:0/1149344548 wait complete. 2024-06-15T01:32:59.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:59 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3300562282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:32:59.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:59 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/4062009877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.963 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:32:59 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/3432375577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:32:59.971 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-06-15T01:33:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 -- 172.21.15.137:0/1950688954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4140ff170 msgr2=0x7fb4140ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:33:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 --2- 172.21.15.137:0/1950688954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4140ff170 0x7fb4140ff550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb400009880 tx=0x7fb40002bce0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 -- 172.21.15.137:0/1950688954 shutdown_connections 2024-06-15T01:33:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 --2- 172.21.15.137:0/1950688954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ffb20 0x7fb414103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 --2- 172.21.15.137:0/1950688954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4140ff170 0x7fb4140ff550 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 -- 172.21.15.137:0/1950688954 >> 172.21.15.137:0/1950688954 conn(0x7fb4140fa9d0 msgr2=0x7fb4140fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:33:00.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 -- 172.21.15.137:0/1950688954 shutdown_connections 2024-06-15T01:33:00.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.128+0000 7fb41c800640 1 -- 172.21.15.137:0/1950688954 wait complete. 2024-06-15T01:33:00.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 Processor -- start 2024-06-15T01:33:00.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 -- start start 2024-06-15T01:33:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ff170 0x7fb41419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:33:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:33:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41419c4f0 con 0x7fb41419bfb0 2024-06-15T01:33:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41419c630 con 0x7fb4140ff170 2024-06-15T01:33:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57898/0 (socket says 172.21.15.137:57898) 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 -- 172.21.15.137:0/596579136 learned_addr learned my addr 172.21.15.137:0/596579136 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb41a59c640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ff170 0x7fb41419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 -- 172.21.15.137:0/596579136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ff170 msgr2=0x7fb41419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ff170 0x7fb41419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 -- 172.21.15.137:0/596579136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009560 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.129+0000 7fb419d9b640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c8f0 tx=0x7fb40800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.130+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb408004410 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.130+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb408007c40 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.130+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4141960e0 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.130+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb40800f450 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.130+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb414196640 con 0x7fb41419bfb0 2024-06-15T01:33:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.131+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41404e040 con 0x7fb41419bfb0 2024-06-15T01:33:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.131+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 32) v1 ==== 89472+0+0 (secure 0 0 0) 0x7fb4080077a0 con 0x7fb41419bfb0 2024-06-15T01:33:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.131+0000 7fb40f7fe640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 0x7fb40406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-15T01:33:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.132+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb40808a2a0 con 0x7fb41419bfb0 2024-06-15T01:33:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.132+0000 7fb41a59c640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 0x7fb40406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-15T01:33:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.132+0000 7fb41a59c640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 0x7fb40406d470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb400006010 tx=0x7fb400000bc0 comp rx=0 tx=0).ready entity=mgr.24425 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-15T01:33:00.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.136+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb408055150 con 0x7fb41419bfb0 2024-06-15T01:33:00.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.320+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4140ffb20 con 0x7fb41419bfb0 2024-06-15T01:33:00.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.321+0000 7fb40f7fe640 1 -- 172.21.15.137:0/596579136 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v95) v1 ==== 81+0+2099 (secure 0 0 0) 0x7fb408058770 con 0x7fb41419bfb0 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6829/937501599 2024-06-16T01:14:29.156226+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6828/937501599 2024-06-16T01:14:29.156226+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/2929246681 2024-06-16T00:59:33.029395+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6826/183082556 2024-06-16T01:14:29.156215+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3277128014 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/623606439 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/146013243 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6825/2333305414 2024-06-16T01:04:33.393618+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/4288581606 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/3423141192 2024-06-16T00:57:46.426899+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/778536834 2024-06-16T00:59:33.029395+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/1961055405 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6824/2333305414 2024-06-16T01:04:33.393618+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6829/2339029036 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:6828/2339029036 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/388348183 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2983067162 2024-06-16T00:57:46.426899+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/4018707027 2024-06-16T00:58:18.710601+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3386774887 2024-06-16T00:58:18.710601+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1189222759 2024-06-16T00:57:46.426899+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/72877675 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/944831942 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/3216905676 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/3423141192 2024-06-16T00:57:46.426899+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/4018707027 2024-06-16T00:58:18.710601+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/3967735174 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6827/183082556 2024-06-16T01:14:29.156215+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/3216905676 2024-06-16T01:06:34.431086+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/941695676 2024-06-16T00:58:18.710601+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1762265425 2024-06-16T00:59:33.029395+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/578055110 2024-06-16T00:57:46.426899+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3247562455 2024-06-16T00:59:33.029395+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1946521916 2024-06-16T00:58:18.710601+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.142:0/544255511 2024-06-16T01:07:47.641756+0000 2024-06-15T01:33:00.308 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/2929246681 2024-06-16T00:59:33.029395+0000 2024-06-15T01:33:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:listed 35 entries 2024-06-15T01:33:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 msgr2=0x7fb40406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:33:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 0x7fb40406d470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb400006010 tx=0x7fb400000bc0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 msgr2=0x7fb414195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c8f0 tx=0x7fb40800cdb0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 shutdown_connections 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:6800/1246465045,v1:172.21.15.137:6801/1246465045] conn(0x7fb40406afb0 0x7fb40406d470 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb41419bfb0 0x7fb414195ae0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 --2- 172.21.15.137:0/596579136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4140ff170 0x7fb41419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 >> 172.21.15.137:0/596579136 conn(0x7fb4140fa9d0 msgr2=0x7fb4141033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 shutdown_connections 2024-06-15T01:33:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-06-15T01:33:00.323+0000 7fb41c800640 1 -- 172.21.15.137:0/596579136 wait complete. 2024-06-15T01:33:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:59 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3300562282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-15T01:33:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:59 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/4062009877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-15T01:33:00.311 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:32:59 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/3432375577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-06-15T01:33:00.319 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi137.front.sepia.ceph.com... 2024-06-15T01:33:00.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-15T01:33:00.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-06-15T01:33:00.344 INFO:teuthology.orchestra.run:waiting for 300 2024-06-15T01:33:01.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:00 smithi142 ceph-mon[82136]: pgmap v808: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:01.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:00 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/1149344548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:33:01.057 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:00 smithi142 ceph-mon[82136]: from='client.? 172.21.15.137:0/596579136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-15T01:33:01.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:00 smithi137 ceph-mon[112295]: pgmap v808: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:01.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:00 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/1149344548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-06-15T01:33:01.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:00 smithi137 ceph-mon[112295]: from='client.? 172.21.15.137:0/596579136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-15T01:33:03.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:02 smithi137 ceph-mon[112295]: pgmap v809: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:03.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:02 smithi142 ceph-mon[82136]: pgmap v809: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:04.217 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:04.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:05.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:04 smithi137 ceph-mon[112295]: pgmap v810: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:05.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:04 smithi142 ceph-mon[82136]: pgmap v810: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:07.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:06 smithi137 ceph-mon[112295]: pgmap v811: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:07.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:06 smithi142 ceph-mon[82136]: pgmap v811: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:09.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:08 smithi137 ceph-mon[112295]: pgmap v812: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:09.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:08 smithi142 ceph-mon[82136]: pgmap v812: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:11.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:10 smithi142 ceph-mon[82136]: pgmap v813: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:11.128 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:10 smithi137 ceph-mon[112295]: pgmap v813: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:13.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:12 smithi137 ceph-mon[112295]: pgmap v814: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:13.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:12 smithi142 ceph-mon[82136]: pgmap v814: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:15.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:14 smithi137 ceph-mon[112295]: pgmap v815: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:15.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:14 smithi142 ceph-mon[82136]: pgmap v815: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:17.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:16 smithi137 ceph-mon[112295]: pgmap v816: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:17.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:16 smithi142 ceph-mon[82136]: pgmap v816: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:19.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:18 smithi137 ceph-mon[112295]: pgmap v817: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:19.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:19.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:18 smithi142 ceph-mon[82136]: pgmap v817: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:19.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:21.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:20 smithi137 ceph-mon[112295]: pgmap v818: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:21.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:20 smithi142 ceph-mon[82136]: pgmap v818: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:23.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:22 smithi137 ceph-mon[112295]: pgmap v819: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:23.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:22 smithi142 ceph-mon[82136]: pgmap v819: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:25.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:24 smithi137 ceph-mon[112295]: pgmap v820: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:25.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:24 smithi142 ceph-mon[82136]: pgmap v820: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:27.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:26 smithi137 ceph-mon[112295]: pgmap v821: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:27.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:26 smithi142 ceph-mon[82136]: pgmap v821: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:29.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:28 smithi137 ceph-mon[112295]: pgmap v822: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:29.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:28 smithi142 ceph-mon[82136]: pgmap v822: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:31.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:30 smithi137 ceph-mon[112295]: pgmap v823: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:31.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:30 smithi142 ceph-mon[82136]: pgmap v823: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:33.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:32 smithi137 ceph-mon[112295]: pgmap v824: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:33.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:32 smithi142 ceph-mon[82136]: pgmap v824: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:34.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:34.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:35.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:34 smithi137 ceph-mon[112295]: pgmap v825: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:35.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:34 smithi142 ceph-mon[82136]: pgmap v825: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:37.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:36 smithi137 ceph-mon[112295]: pgmap v826: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:37.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:36 smithi142 ceph-mon[82136]: pgmap v826: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:39.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:38 smithi137 ceph-mon[112295]: pgmap v827: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:39.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:38 smithi142 ceph-mon[82136]: pgmap v827: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:41.186 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:40 smithi142 ceph-mon[82136]: pgmap v828: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:41.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:40 smithi137 ceph-mon[112295]: pgmap v828: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:43.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:42 smithi137 ceph-mon[112295]: pgmap v829: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:43.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:42 smithi142 ceph-mon[82136]: pgmap v829: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:45.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:44 smithi137 ceph-mon[112295]: pgmap v830: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:45.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:44 smithi142 ceph-mon[82136]: pgmap v830: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:47.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:46 smithi137 ceph-mon[112295]: pgmap v831: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:47.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:46 smithi142 ceph-mon[82136]: pgmap v831: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:49.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:48 smithi137 ceph-mon[112295]: pgmap v832: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:49.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:48 smithi142 ceph-mon[82136]: pgmap v832: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:33:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:50 smithi137 ceph-mon[112295]: pgmap v833: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:33:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:33:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:33:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:50 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:33:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:50 smithi142 ceph-mon[82136]: pgmap v833: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:33:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:33:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:33:51.306 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:50 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:33:53.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:52 smithi137 ceph-mon[112295]: pgmap v834: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:52 smithi142 ceph-mon[82136]: pgmap v834: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:55.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:54 smithi137 ceph-mon[112295]: pgmap v835: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:55.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:54 smithi142 ceph-mon[82136]: pgmap v835: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:57.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:56 smithi137 ceph-mon[112295]: pgmap v836: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:57.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:56 smithi142 ceph-mon[82136]: pgmap v836: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:59.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:33:58 smithi137 ceph-mon[112295]: pgmap v837: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:33:59.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:33:58 smithi142 ceph-mon[82136]: pgmap v837: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:01.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:00 smithi137 ceph-mon[112295]: pgmap v838: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:01.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:00 smithi142 ceph-mon[82136]: pgmap v838: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:03.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:02 smithi137 ceph-mon[112295]: pgmap v839: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:03.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:02 smithi142 ceph-mon[82136]: pgmap v839: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:04.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:04.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:05.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:04 smithi137 ceph-mon[112295]: pgmap v840: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:05.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:04 smithi142 ceph-mon[82136]: pgmap v840: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:07.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:06 smithi142 ceph-mon[82136]: pgmap v841: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:07.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:06 smithi137 ceph-mon[112295]: pgmap v841: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:09.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:08 smithi142 ceph-mon[82136]: pgmap v842: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:09.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:08 smithi137 ceph-mon[112295]: pgmap v842: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:11.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:10 smithi142 ceph-mon[82136]: pgmap v843: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:11.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:10 smithi137 ceph-mon[112295]: pgmap v843: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:13.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:12 smithi142 ceph-mon[82136]: pgmap v844: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:13.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:12 smithi137 ceph-mon[112295]: pgmap v844: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:15.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:14 smithi142 ceph-mon[82136]: pgmap v845: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:15.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:14 smithi137 ceph-mon[112295]: pgmap v845: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:17.284 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:16 smithi137 ceph-mon[112295]: pgmap v846: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:17.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:16 smithi142 ceph-mon[82136]: pgmap v846: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:19.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:18 smithi142 ceph-mon[82136]: pgmap v847: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:19.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:19.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:18 smithi137 ceph-mon[112295]: pgmap v847: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:19.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:21.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:21 smithi142 ceph-mon[82136]: pgmap v848: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:21.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:21 smithi137 ceph-mon[112295]: pgmap v848: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:23.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:23 smithi142 ceph-mon[82136]: pgmap v849: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:23.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:23 smithi137 ceph-mon[112295]: pgmap v849: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:25.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:25 smithi142 ceph-mon[82136]: pgmap v850: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:25.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:25 smithi137 ceph-mon[112295]: pgmap v850: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:27.055 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:27 smithi142 ceph-mon[82136]: pgmap v851: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:27.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:27 smithi137 ceph-mon[112295]: pgmap v851: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:29.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:29 smithi142 ceph-mon[82136]: pgmap v852: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:29.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:29 smithi137 ceph-mon[112295]: pgmap v852: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:31.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:31 smithi142 ceph-mon[82136]: pgmap v853: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:31.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:31 smithi137 ceph-mon[112295]: pgmap v853: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:33.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:33 smithi142 ceph-mon[82136]: pgmap v854: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:33.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:33 smithi137 ceph-mon[112295]: pgmap v854: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:34.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:34 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:34.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:34 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:35.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:35 smithi142 ceph-mon[82136]: pgmap v855: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:35.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:35 smithi137 ceph-mon[112295]: pgmap v855: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:37.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:37 smithi142 ceph-mon[82136]: pgmap v856: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:37.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:37 smithi137 ceph-mon[112295]: pgmap v856: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:39.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:39 smithi142 ceph-mon[82136]: pgmap v857: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:39.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:39 smithi137 ceph-mon[112295]: pgmap v857: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:41.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:41 smithi142 ceph-mon[82136]: pgmap v858: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:41.466 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:41 smithi137 ceph-mon[112295]: pgmap v858: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:43.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:43 smithi142 ceph-mon[82136]: pgmap v859: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:43.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:43 smithi137 ceph-mon[112295]: pgmap v859: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:45.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:45 smithi142 ceph-mon[82136]: pgmap v860: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:45.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:45 smithi137 ceph-mon[112295]: pgmap v860: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:47.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:47 smithi142 ceph-mon[82136]: pgmap v861: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:47.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:47 smithi137 ceph-mon[112295]: pgmap v861: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:48.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:48.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:34:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:49 smithi142 ceph-mon[82136]: pgmap v862: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:49.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:49 smithi137 ceph-mon[112295]: pgmap v862: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:51 smithi142 ceph-mon[82136]: pgmap v863: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:34:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:34:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:34:51.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:34:51.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:51 smithi137 ceph-mon[112295]: pgmap v863: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:51.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:34:51.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:34:51.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:34:51.464 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:34:53.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:53 smithi137 ceph-mon[112295]: pgmap v864: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:53 smithi142 ceph-mon[82136]: pgmap v864: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:55.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:55 smithi142 ceph-mon[82136]: pgmap v865: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:55.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:55 smithi137 ceph-mon[112295]: pgmap v865: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:57.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:57 smithi142 ceph-mon[82136]: pgmap v866: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:57.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:57 smithi137 ceph-mon[112295]: pgmap v866: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:59.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:34:59 smithi142 ceph-mon[82136]: pgmap v867: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:34:59.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:34:59 smithi137 ceph-mon[112295]: pgmap v867: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:01.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:01 smithi137 ceph-mon[112295]: pgmap v868: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:01.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:01 smithi142 ceph-mon[82136]: pgmap v868: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:03.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:03 smithi137 ceph-mon[112295]: pgmap v869: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:03.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:03.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:03 smithi142 ceph-mon[82136]: pgmap v869: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:03.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:05.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:05 smithi137 ceph-mon[112295]: pgmap v870: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:05.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:05 smithi142 ceph-mon[82136]: pgmap v870: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:07.351 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:07 smithi142 ceph-mon[82136]: pgmap v871: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:07.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:07 smithi137 ceph-mon[112295]: pgmap v871: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:09.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:09 smithi137 ceph-mon[112295]: pgmap v872: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:09.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:09 smithi142 ceph-mon[82136]: pgmap v872: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:11.357 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:11 smithi142 ceph-mon[82136]: pgmap v873: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:11.462 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:11 smithi137 ceph-mon[112295]: pgmap v873: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:13.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:13 smithi137 ceph-mon[112295]: pgmap v874: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:13.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:13 smithi142 ceph-mon[82136]: pgmap v874: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:15.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:15 smithi137 ceph-mon[112295]: pgmap v875: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:15.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:15 smithi142 ceph-mon[82136]: pgmap v875: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:17.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:17 smithi137 ceph-mon[112295]: pgmap v876: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:17.489 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:17 smithi142 ceph-mon[82136]: pgmap v876: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:18.357 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:18.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:19.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:19 smithi137 ceph-mon[112295]: pgmap v877: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:19.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:19 smithi142 ceph-mon[82136]: pgmap v877: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:21.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:21 smithi137 ceph-mon[112295]: pgmap v878: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:21.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:21 smithi142 ceph-mon[82136]: pgmap v878: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:23.446 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:23 smithi137 ceph-mon[112295]: pgmap v879: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:23.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:23 smithi142 ceph-mon[82136]: pgmap v879: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:25.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:25 smithi137 ceph-mon[112295]: pgmap v880: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:25.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:25 smithi142 ceph-mon[82136]: pgmap v880: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:27.462 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:27 smithi137 ceph-mon[112295]: pgmap v881: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:27.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:27 smithi142 ceph-mon[82136]: pgmap v881: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:29.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:29 smithi137 ceph-mon[112295]: pgmap v882: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:29.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:29 smithi142 ceph-mon[82136]: pgmap v882: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:31.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:31 smithi137 ceph-mon[112295]: pgmap v883: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:31.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:31 smithi142 ceph-mon[82136]: pgmap v883: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:33.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:33 smithi137 ceph-mon[112295]: pgmap v884: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:33.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:33.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:33 smithi142 ceph-mon[82136]: pgmap v884: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:33.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:35.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:35 smithi137 ceph-mon[112295]: pgmap v885: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:35.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:35 smithi142 ceph-mon[82136]: pgmap v885: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:37.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:37 smithi137 ceph-mon[112295]: pgmap v886: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:37.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:37 smithi142 ceph-mon[82136]: pgmap v886: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:39.462 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:39 smithi137 ceph-mon[112295]: pgmap v887: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:39.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:39 smithi142 ceph-mon[82136]: pgmap v887: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:41.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:41 smithi137 ceph-mon[112295]: pgmap v888: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:41.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:41 smithi142 ceph-mon[82136]: pgmap v888: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:43.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:43 smithi137 ceph-mon[112295]: pgmap v889: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:43.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:43 smithi142 ceph-mon[82136]: pgmap v889: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:45.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:45 smithi137 ceph-mon[112295]: pgmap v890: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:45.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:45 smithi142 ceph-mon[82136]: pgmap v890: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:47.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:47 smithi137 ceph-mon[112295]: pgmap v891: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:47.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:47 smithi142 ceph-mon[82136]: pgmap v891: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:48.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:48.462 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:35:49.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:49 smithi137 ceph-mon[112295]: pgmap v892: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:49.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:49 smithi142 ceph-mon[82136]: pgmap v892: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:51.462 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:51 smithi137 ceph-mon[112295]: pgmap v893: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:51.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:35:51.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:51 smithi142 ceph-mon[82136]: pgmap v893: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:51.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:35:52.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:35:52.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:35:52.463 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:35:52.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:35:52.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:35:52.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:35:53.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:53 smithi137 ceph-mon[112295]: pgmap v894: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:53.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:53 smithi142 ceph-mon[82136]: pgmap v894: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:55.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:55 smithi137 ceph-mon[112295]: pgmap v895: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:55.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:55 smithi142 ceph-mon[82136]: pgmap v895: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:57.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:57 smithi137 ceph-mon[112295]: pgmap v896: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:57.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:57 smithi142 ceph-mon[82136]: pgmap v896: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:59.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:35:59 smithi137 ceph-mon[112295]: pgmap v897: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:35:59.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:35:59 smithi142 ceph-mon[82136]: pgmap v897: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:01.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:01 smithi137 ceph-mon[112295]: pgmap v898: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:01.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:01 smithi142 ceph-mon[82136]: pgmap v898: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:03.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:03 smithi137 ceph-mon[112295]: pgmap v899: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:03.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:03.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:03 smithi142 ceph-mon[82136]: pgmap v899: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:03.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:05.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:05 smithi137 ceph-mon[112295]: pgmap v900: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:05.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:05 smithi142 ceph-mon[82136]: pgmap v900: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:07.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:07 smithi137 ceph-mon[112295]: pgmap v901: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:07.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:07 smithi142 ceph-mon[82136]: pgmap v901: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:09.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:09 smithi137 ceph-mon[112295]: pgmap v902: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:09.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:09 smithi142 ceph-mon[82136]: pgmap v902: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:11.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:11 smithi137 ceph-mon[112295]: pgmap v903: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:11.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:11 smithi142 ceph-mon[82136]: pgmap v903: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:13.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:13 smithi142 ceph-mon[82136]: pgmap v904: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:13.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:13 smithi137 ceph-mon[112295]: pgmap v904: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:15.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:15 smithi137 ceph-mon[112295]: pgmap v905: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:15.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:15 smithi142 ceph-mon[82136]: pgmap v905: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:17.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:17 smithi137 ceph-mon[112295]: pgmap v906: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:17.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:17 smithi142 ceph-mon[82136]: pgmap v906: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:18.558 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:18.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:19.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:19 smithi137 ceph-mon[112295]: pgmap v907: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:19.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:19 smithi142 ceph-mon[82136]: pgmap v907: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:21.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:21 smithi137 ceph-mon[112295]: pgmap v908: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:21.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:21 smithi142 ceph-mon[82136]: pgmap v908: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:23.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:23 smithi137 ceph-mon[112295]: pgmap v909: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:23.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:23 smithi142 ceph-mon[82136]: pgmap v909: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:25.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:25 smithi137 ceph-mon[112295]: pgmap v910: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:25.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:25 smithi142 ceph-mon[82136]: pgmap v910: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:27.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:27 smithi137 ceph-mon[112295]: pgmap v911: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:27.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:27 smithi142 ceph-mon[82136]: pgmap v911: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:29.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:29 smithi137 ceph-mon[112295]: pgmap v912: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:29.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:29 smithi142 ceph-mon[82136]: pgmap v912: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:31.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:31 smithi137 ceph-mon[112295]: pgmap v913: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:31.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:31 smithi142 ceph-mon[82136]: pgmap v913: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:33.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:33 smithi137 ceph-mon[112295]: pgmap v914: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:33.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:33.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:33 smithi142 ceph-mon[82136]: pgmap v914: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:33.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:35.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:35 smithi137 ceph-mon[112295]: pgmap v915: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:35.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:35 smithi142 ceph-mon[82136]: pgmap v915: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:37.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:37 smithi137 ceph-mon[112295]: pgmap v916: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:37.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:37 smithi142 ceph-mon[82136]: pgmap v916: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:39.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:39 smithi137 ceph-mon[112295]: pgmap v917: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:39.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:39 smithi142 ceph-mon[82136]: pgmap v917: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:41.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:41 smithi137 ceph-mon[112295]: pgmap v918: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:41.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:41 smithi142 ceph-mon[82136]: pgmap v918: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:43.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:43 smithi137 ceph-mon[112295]: pgmap v919: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:43.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:43 smithi142 ceph-mon[82136]: pgmap v919: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:45.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:45 smithi137 ceph-mon[112295]: pgmap v920: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:45.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:45 smithi142 ceph-mon[82136]: pgmap v920: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:47.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:47 smithi137 ceph-mon[112295]: pgmap v921: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:47.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:47 smithi142 ceph-mon[82136]: pgmap v921: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:48.711 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:48.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:36:49.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:49 smithi137 ceph-mon[112295]: pgmap v922: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:49.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:49 smithi142 ceph-mon[82136]: pgmap v922: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:50.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:50 smithi142 ceph-mon[82136]: pgmap v923: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:50.962 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:50 smithi137 ceph-mon[112295]: pgmap v923: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:51.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:51 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:36:51.962 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:51 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:36:52.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:52 smithi137 ceph-mon[112295]: pgmap v924: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:52.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:36:52.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:36:52.713 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:36:52.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:52 smithi142 ceph-mon[82136]: pgmap v924: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:52.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:36:52.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:36:52.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:36:55.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:54 smithi137 ceph-mon[112295]: pgmap v925: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:55.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:54 smithi142 ceph-mon[82136]: pgmap v925: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:57.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:56 smithi137 ceph-mon[112295]: pgmap v926: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:57.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:56 smithi142 ceph-mon[82136]: pgmap v926: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:59.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:36:58 smithi137 ceph-mon[112295]: pgmap v927: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:36:59.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:36:58 smithi142 ceph-mon[82136]: pgmap v927: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:01.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:00 smithi137 ceph-mon[112295]: pgmap v928: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:01.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:00 smithi142 ceph-mon[82136]: pgmap v928: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:03.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:02 smithi137 ceph-mon[112295]: pgmap v929: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:03.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:02 smithi142 ceph-mon[82136]: pgmap v929: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:04.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:03 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:04.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:03 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:05.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:04 smithi137 ceph-mon[112295]: pgmap v930: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:05.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:04 smithi142 ceph-mon[82136]: pgmap v930: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:07.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:06 smithi137 ceph-mon[112295]: pgmap v931: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:07.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:06 smithi142 ceph-mon[82136]: pgmap v931: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:08.962 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:08 smithi137 ceph-mon[112295]: pgmap v932: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:08.985 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:08 smithi142 ceph-mon[82136]: pgmap v932: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:11.107 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:10 smithi142 ceph-mon[82136]: pgmap v933: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:11.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:10 smithi137 ceph-mon[112295]: pgmap v933: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:13.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:12 smithi137 ceph-mon[112295]: pgmap v934: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:13.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:12 smithi142 ceph-mon[82136]: pgmap v934: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:15.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:14 smithi137 ceph-mon[112295]: pgmap v935: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:15.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:14 smithi142 ceph-mon[82136]: pgmap v935: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:17.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:16 smithi137 ceph-mon[112295]: pgmap v936: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:17.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:16 smithi142 ceph-mon[82136]: pgmap v936: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:19.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:18 smithi142 ceph-mon[82136]: pgmap v937: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:19.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:18 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:19.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:18 smithi137 ceph-mon[112295]: pgmap v937: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:19.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:18 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:21.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:20 smithi137 ceph-mon[112295]: pgmap v938: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:21.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:20 smithi142 ceph-mon[82136]: pgmap v938: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:23.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:22 smithi137 ceph-mon[112295]: pgmap v939: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:23.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:22 smithi142 ceph-mon[82136]: pgmap v939: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:25.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:24 smithi137 ceph-mon[112295]: pgmap v940: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:25.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:24 smithi142 ceph-mon[82136]: pgmap v940: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:27.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:26 smithi137 ceph-mon[112295]: pgmap v941: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:27.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:26 smithi142 ceph-mon[82136]: pgmap v941: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:29.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:28 smithi137 ceph-mon[112295]: pgmap v942: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:29.261 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:28 smithi142 ceph-mon[82136]: pgmap v942: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:30.962 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:30 smithi137 ceph-mon[112295]: pgmap v943: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:31.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:30 smithi142 ceph-mon[82136]: pgmap v943: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:33.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:32 smithi137 ceph-mon[112295]: pgmap v944: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:33.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:32 smithi142 ceph-mon[82136]: pgmap v944: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:34.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:33 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:34.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:33 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:35.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:34 smithi137 ceph-mon[112295]: pgmap v945: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:35.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:34 smithi142 ceph-mon[82136]: pgmap v945: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:37.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:36 smithi137 ceph-mon[112295]: pgmap v946: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:37.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:36 smithi142 ceph-mon[82136]: pgmap v946: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:39.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:38 smithi137 ceph-mon[112295]: pgmap v947: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:39.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:38 smithi142 ceph-mon[82136]: pgmap v947: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:41.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:40 smithi137 ceph-mon[112295]: pgmap v948: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:40 smithi142 ceph-mon[82136]: pgmap v948: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:43.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:42 smithi137 ceph-mon[112295]: pgmap v949: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:43.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:42 smithi142 ceph-mon[82136]: pgmap v949: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:45.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:44 smithi137 ceph-mon[112295]: pgmap v950: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:45.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:44 smithi142 ceph-mon[82136]: pgmap v950: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:47.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:46 smithi137 ceph-mon[112295]: pgmap v951: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:47.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:46 smithi142 ceph-mon[82136]: pgmap v951: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:49.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:48 smithi137 ceph-mon[112295]: pgmap v952: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:49.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:48 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:49.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:48 smithi142 ceph-mon[82136]: pgmap v952: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:49.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:48 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-15T01:37:51.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:50 smithi137 ceph-mon[112295]: pgmap v953: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:51.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:50 smithi142 ceph-mon[82136]: pgmap v953: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:53.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:52 smithi137 ceph-mon[112295]: pgmap v954: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:53.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:37:53.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:37:53.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:37:53.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:52 smithi137 ceph-mon[112295]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:37:53.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:52 smithi142 ceph-mon[82136]: pgmap v954: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-15T01:37:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-15T01:37:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-15T01:37:53.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:52 smithi142 ceph-mon[82136]: from='mgr.24425 172.21.15.137:0/633545969' entity='mgr.smithi137.nbzbyc' 2024-06-15T01:37:55.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:54 smithi137 ceph-mon[112295]: pgmap v955: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:55.305 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:54 smithi142 ceph-mon[82136]: pgmap v955: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:57.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:56 smithi137 ceph-mon[112295]: pgmap v956: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:57.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:56 smithi142 ceph-mon[82136]: pgmap v956: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:59.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:58 smithi137 ceph-mon[112295]: pgmap v957: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:59.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:37:58 smithi142 ceph-mon[82136]: pgmap v957: 65 pgs: 65 active+clean; 262 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-06-15T01:37:59.418 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-06-15T01:37:59.418 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-06-15T01:37:59.420 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-15T01:37:59.431 INFO:tasks.cephadm:Teardown begin 2024-06-15T01:37:59.432 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-06-15T01:37:59.434 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-15T01:37:59.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-15T01:37:59.501 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-15T01:37:59.501 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-15T01:37:59.525 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-15T01:37:59.558 INFO:tasks.cephadm:Stopping all daemons... 2024-06-15T01:37:59.559 INFO:tasks.cephadm.mon.smithi137:Stopping mon.smithi137... 2024-06-15T01:37:59.559 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137 2024-06-15T01:37:59.712 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:59 smithi137 systemd[1]: Stopping Ceph mon.smithi137 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:00.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:59 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:37:59.891+0000 7f9ac784c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi137 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:00.212 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:59 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137[112291]: 2024-06-15T01:37:59.891+0000 7f9ac784c640 -1 mon.smithi137@0(leader) e2 *** Got Signal Terminated *** 2024-06-15T01:38:00.213 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:37:59 smithi137 podman[159726]: 2024-06-15 01:37:59.943434534 +0000 UTC m=+0.170128273 container died d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-15T01:38:00.668 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:38:00 smithi137 podman[159726]: 2024-06-15 01:38:00.40893956 +0000 UTC m=+0.635633297 container cleanup d12730fea2eba131d12ad56d2cf33e1b21b36fefb3cf8579da87acd25303b1c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS) 2024-06-15T01:38:00.668 INFO:journalctl@ceph.mon.smithi137.smithi137.stdout:Jun 15 01:38:00 smithi137 bash[159726]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi137 2024-06-15T01:38:00.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi137.service' 2024-06-15T01:38:00.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:00.873 INFO:tasks.cephadm.mon.smithi137:Stopped mon.smithi137 2024-06-15T01:38:00.873 INFO:tasks.cephadm.mon.smithi142:Stopping mon.smithi142... 2024-06-15T01:38:00.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142 2024-06-15T01:38:01.204 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:00 smithi142 systemd[1]: Stopping Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:01.488 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142[82132]: 2024-06-15T01:38:01.221+0000 7fab9e37d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi142 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:01.488 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142[82132]: 2024-06-15T01:38:01.221+0000 7fab9e37d640 -1 mon.smithi142@1(peon) e2 *** Got Signal Terminated *** 2024-06-15T01:38:01.488 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 podman[117317]: 2024-06-15 01:38:01.252495863 +0000 UTC m=+0.156588716 container died abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-06-15T01:38:01.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 podman[117317]: 2024-06-15 01:38:01.688338836 +0000 UTC m=+0.592431680 container cleanup abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2) 2024-06-15T01:38:01.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 bash[117317]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142 2024-06-15T01:38:02.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service' 2024-06-15T01:38:02.160 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:01 smithi142 podman[117330]: 2024-06-15 01:38:01.923078023 +0000 UTC m=+0.669805990 container remove abb9be3029e946ac5b72c997df5559a74a6487203b3cf46d335b91546cbfca84 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-mon-smithi142, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-15T01:38:02.160 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:02 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service: Deactivated successfully. 2024-06-15T01:38:02.160 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:02 smithi142 systemd[1]: Stopped Ceph mon.smithi142 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968. 2024-06-15T01:38:02.160 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Jun 15 01:38:02 smithi142 systemd[1]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@mon.smithi142.service: Consumed 11.490s CPU time. 2024-06-15T01:38:02.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:02.191 INFO:tasks.cephadm.mon.smithi142:Stopped mon.smithi142 2024-06-15T01:38:02.191 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-15T01:38:02.191 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.0 2024-06-15T01:38:02.521 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:02 smithi137 systemd[1]: Stopping Ceph osd.0 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:02.962 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:02 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:38:02.537+0000 7f0baf8df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:02.963 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:02 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:38:02.537+0000 7f0baf8df640 -1 osd.0 95 *** Got signal Terminated *** 2024-06-15T01:38:02.963 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:02 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0[121330]: 2024-06-15T01:38:02.537+0000 7f0baf8df640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:08.140 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:07 smithi137 podman[159861]: 2024-06-15 01:38:07.873427162 +0000 UTC m=+5.459831707 container died e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:08.462 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:08 smithi137 podman[159861]: 2024-06-15 01:38:08.246939643 +0000 UTC m=+5.833344190 container cleanup e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS) 2024-06-15T01:38:08.462 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:08 smithi137 bash[159861]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0 2024-06-15T01:38:08.859 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:08 smithi137 podman[159877]: 2024-06-15 01:38:08.523527343 +0000 UTC m=+0.644636847 container remove e60b459cde443af838c9c509c70f1b5125fea3e07e03d4a3b1ea73e2af826fbb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.35.3) 2024-06-15T01:38:09.168 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:08 smithi137 podman[159965]: 2024-06-15 01:38:08.779445644 +0000 UTC m=+0.017225678 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:09.168 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:08 smithi137 podman[159965]: 2024-06-15 01:38:08.907983241 +0000 UTC m=+0.145763270 container create 8d0afcf3442e18e58dcb31bce13ee4a0efd5782b0f646b69c00d13ddd079ca9f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:38:09.168 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:09 smithi137 podman[159965]: 2024-06-15 01:38:09.185726846 +0000 UTC m=+0.423506893 container init 8d0afcf3442e18e58dcb31bce13ee4a0efd5782b0f646b69c00d13ddd079ca9f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-15T01:38:09.422 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:09 smithi137 podman[159965]: 2024-06-15 01:38:09.189472135 +0000 UTC m=+0.427252177 container start 8d0afcf3442e18e58dcb31bce13ee4a0efd5782b0f646b69c00d13ddd079ca9f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=) 2024-06-15T01:38:09.422 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:09 smithi137 podman[159965]: 2024-06-15 01:38:09.260509896 +0000 UTC m=+0.498289935 container attach 8d0afcf3442e18e58dcb31bce13ee4a0efd5782b0f646b69c00d13ddd079ca9f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:09.712 INFO:journalctl@ceph.osd.0.smithi137.stdout:Jun 15 01:38:09 smithi137 podman[159965]: 2024-06-15 01:38:09.44042027 +0000 UTC m=+0.678200299 container died 8d0afcf3442e18e58dcb31bce13ee4a0efd5782b0f646b69c00d13ddd079ca9f (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:10.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.0.service' 2024-06-15T01:38:10.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:10.070 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-15T01:38:10.070 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-15T01:38:10.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.1 2024-06-15T01:38:10.300 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:10 smithi137 systemd[1]: Stopping Ceph osd.1 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:10.712 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:38:10.460+0000 7f5ee66ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:10.713 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:38:10.460+0000 7f5ee66ce640 -1 osd.1 95 *** Got signal Terminated *** 2024-06-15T01:38:10.713 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:10 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1[126691]: 2024-06-15T01:38:10.460+0000 7f5ee66ce640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:16.213 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:15 smithi137 podman[160069]: 2024-06-15 01:38:15.975799755 +0000 UTC m=+5.637826120 container died 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-15T01:38:16.526 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:16 smithi137 podman[160069]: 2024-06-15 01:38:16.32579056 +0000 UTC m=+5.987816926 container cleanup 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:16.527 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:16 smithi137 bash[160069]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1 2024-06-15T01:38:16.935 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:16 smithi137 podman[160095]: 2024-06-15 01:38:16.543801749 +0000 UTC m=+0.562289972 container remove 5216643ca244ce7838121f0c14ca24dc12af0899dc0d8fb98680dee01893793e (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-15T01:38:17.213 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:16 smithi137 podman[160186]: 2024-06-15 01:38:16.854983015 +0000 UTC m=+0.017419228 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:17.213 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:16 smithi137 podman[160186]: 2024-06-15 01:38:16.953021386 +0000 UTC m=+0.115457591 container create 0df4ad67716063823e612461daee8593d7b624ef7cefd8400686b6accf64abaf (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2) 2024-06-15T01:38:17.489 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:17 smithi137 podman[160186]: 2024-06-15 01:38:17.247591638 +0000 UTC m=+0.410027854 container init 0df4ad67716063823e612461daee8593d7b624ef7cefd8400686b6accf64abaf (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:17.490 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:17 smithi137 podman[160186]: 2024-06-15 01:38:17.251942744 +0000 UTC m=+0.414378949 container start 0df4ad67716063823e612461daee8593d7b624ef7cefd8400686b6accf64abaf (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:17.490 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:17 smithi137 podman[160186]: 2024-06-15 01:38:17.33084876 +0000 UTC m=+0.493284972 container attach 0df4ad67716063823e612461daee8593d7b624ef7cefd8400686b6accf64abaf (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True) 2024-06-15T01:38:17.842 INFO:journalctl@ceph.osd.1.smithi137.stdout:Jun 15 01:38:17 smithi137 podman[160186]: 2024-06-15 01:38:17.506832725 +0000 UTC m=+0.669268936 container died 0df4ad67716063823e612461daee8593d7b624ef7cefd8400686b6accf64abaf (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:18.167 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.1.service' 2024-06-15T01:38:18.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:18.208 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-15T01:38:18.208 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-15T01:38:18.208 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.2 2024-06-15T01:38:18.462 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:18 smithi137 systemd[1]: Stopping Ceph osd.2 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:18.962 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:38:18.578+0000 7f1e00408640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:18.963 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:38:18.578+0000 7f1e00408640 -1 osd.2 95 *** Got signal Terminated *** 2024-06-15T01:38:18.963 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:18 smithi137 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2[132447]: 2024-06-15T01:38:18.578+0000 7f1e00408640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:24.322 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:24 smithi137 podman[160289]: 2024-06-15 01:38:24.034406149 +0000 UTC m=+5.584867009 container died 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:24.647 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:24 smithi137 podman[160289]: 2024-06-15 01:38:24.420897762 +0000 UTC m=+5.971358621 container cleanup 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux ) 2024-06-15T01:38:24.647 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:24 smithi137 bash[160289]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2 2024-06-15T01:38:24.963 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:24 smithi137 podman[160320]: 2024-06-15 01:38:24.663881108 +0000 UTC m=+0.629990734 container remove 8d1c1b0fd6f640536487899526238cf347163cd44ecf0eb43e754cabb3fcc5b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-15T01:38:25.316 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:24 smithi137 podman[160406]: 2024-06-15 01:38:24.898713297 +0000 UTC m=+0.017785184 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:25.317 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:25 smithi137 podman[160406]: 2024-06-15 01:38:25.023105265 +0000 UTC m=+0.142177135 container create dae466c4ebb99341800bbe8a24d19c8810235d15b185841e169703367c0db216 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:25.317 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:25 smithi137 podman[160406]: 2024-06-15 01:38:25.242128005 +0000 UTC m=+0.361199889 container init dae466c4ebb99341800bbe8a24d19c8810235d15b185841e169703367c0db216 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-15T01:38:25.317 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:25 smithi137 podman[160406]: 2024-06-15 01:38:25.245605217 +0000 UTC m=+0.364677087 container start dae466c4ebb99341800bbe8a24d19c8810235d15b185841e169703367c0db216 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3) 2024-06-15T01:38:25.317 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:25 smithi137 podman[160406]: 2024-06-15 01:38:25.33371176 +0000 UTC m=+0.452783635 container attach dae466c4ebb99341800bbe8a24d19c8810235d15b185841e169703367c0db216 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:25.713 INFO:journalctl@ceph.osd.2.smithi137.stdout:Jun 15 01:38:25 smithi137 podman[160406]: 2024-06-15 01:38:25.504292464 +0000 UTC m=+0.623364335 container died dae466c4ebb99341800bbe8a24d19c8810235d15b185841e169703367c0db216 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:26.078 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.2.service' 2024-06-15T01:38:26.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:26.118 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-15T01:38:26.118 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-15T01:38:26.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.3 2024-06-15T01:38:26.305 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:26 smithi142 systemd[1]: Stopping Ceph osd.3 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:26.805 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:26 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:26.440+0000 7fcfc9262640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:26.805 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:26 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:26.440+0000 7fcfc9262640 -1 osd.3 95 *** Got signal Terminated *** 2024-06-15T01:38:26.805 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:26 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:26.440+0000 7fcfc9262640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:28.305 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:28 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:28.007+0000 7fcfc205e640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.088515+0000 front 2024-06-15T01:38:05.088448+0000 (oldest deadline 2024-06-15T01:38:27.988313+0000) 2024-06-15T01:38:29.241 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:28 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:28.986+0000 7fcfc205e640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.088515+0000 front 2024-06-15T01:38:05.088448+0000 (oldest deadline 2024-06-15T01:38:27.988313+0000) 2024-06-15T01:38:29.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:29 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:29.258+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:30.305 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:29 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:29.993+0000 7fcfc205e640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.088515+0000 front 2024-06-15T01:38:05.088448+0000 (oldest deadline 2024-06-15T01:38:27.988313+0000) 2024-06-15T01:38:30.305 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:30 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:30.210+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:31.305 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:31 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:31.109+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:31.306 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:30 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3[90435]: 2024-06-15T01:38:30.981+0000 7fcfc205e640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.088515+0000 front 2024-06-15T01:38:05.088448+0000 (oldest deadline 2024-06-15T01:38:27.988313+0000) 2024-06-15T01:38:31.306 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:31 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:31.228+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:32.250 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:31 smithi142 podman[117456]: 2024-06-15 01:38:31.999194214 +0000 UTC m=+5.662491174 container died 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-15T01:38:32.250 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:32 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:32.215+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:32.250 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:32 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:32.153+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:32.517 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:32 smithi142 podman[117456]: 2024-06-15 01:38:32.340702912 +0000 UTC m=+6.003999883 container cleanup 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-15T01:38:32.517 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:32 smithi142 bash[117456]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3 2024-06-15T01:38:32.805 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:32 smithi142 podman[117474]: 2024-06-15 01:38:32.533532386 +0000 UTC m=+0.526661273 container remove 3555c500b38bc9128443abd407d03eb481e3c4ab675e97643e4d2ed51796c2fe (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-06-15T01:38:33.204 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:33 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:33.188+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:33.204 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:33 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:33.120+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:33.204 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:32 smithi142 podman[117561]: 2024-06-15 01:38:32.859749549 +0000 UTC m=+0.016841108 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:33.204 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:32 smithi142 podman[117561]: 2024-06-15 01:38:32.99278081 +0000 UTC m=+0.149872367 container create ef62050ccd924095ba7ca250edf683756dfc3590a9b3eebe09b938ed7bff1afb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-15T01:38:33.205 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:33 smithi142 podman[117561]: 2024-06-15 01:38:33.220832281 +0000 UTC m=+0.377923848 container init ef62050ccd924095ba7ca250edf683756dfc3590a9b3eebe09b938ed7bff1afb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:33.467 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:33 smithi142 podman[117561]: 2024-06-15 01:38:33.224700254 +0000 UTC m=+0.381791818 container start ef62050ccd924095ba7ca250edf683756dfc3590a9b3eebe09b938ed7bff1afb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-15T01:38:33.467 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:33 smithi142 podman[117561]: 2024-06-15 01:38:33.321629464 +0000 UTC m=+0.478721022 container attach ef62050ccd924095ba7ca250edf683756dfc3590a9b3eebe09b938ed7bff1afb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-06-15T01:38:33.748 INFO:journalctl@ceph.osd.3.smithi142.stdout:Jun 15 01:38:33 smithi142 podman[117561]: 2024-06-15 01:38:33.484204396 +0000 UTC m=+0.641295957 container died ef62050ccd924095ba7ca250edf683756dfc3590a9b3eebe09b938ed7bff1afb (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:34.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.3.service' 2024-06-15T01:38:34.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:34.117 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-15T01:38:34.117 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-15T01:38:34.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.4 2024-06-15T01:38:34.339 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:34 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:34.157+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:34.340 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:34 smithi142 systemd[1]: Stopping Ceph osd.4 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:34.340 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:34 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:34.230+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:34.805 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:34 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:34.494+0000 7f2c96b42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:34.805 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:34 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:34.494+0000 7f2c96b42640 -1 osd.4 95 *** Got signal Terminated *** 2024-06-15T01:38:34.805 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:34 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:34.494+0000 7f2c96b42640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:35.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:35 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:35.223+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:35.555 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:35 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:35.192+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:36.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:36 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:36.229+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:36.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:36 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:36.182+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:37.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:37 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:37.180+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:37.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:37 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:37.217+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:38.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:38 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:38.190+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:38.556 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:38 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:38.190+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:11.529620+0000 front 2024-06-15T01:38:11.529642+0000 (oldest deadline 2024-06-15T01:38:37.429414+0000) 2024-06-15T01:38:38.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:38 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:38.194+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:38.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:38 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:38.194+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:39.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:39 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:39.157+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:06.328688+0000 front 2024-06-15T01:38:06.328643+0000 (oldest deadline 2024-06-15T01:38:29.228655+0000) 2024-06-15T01:38:39.556 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:39 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4[95451]: 2024-06-15T01:38:39.157+0000 7f2c8f93e640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:11.529620+0000 front 2024-06-15T01:38:11.529642+0000 (oldest deadline 2024-06-15T01:38:37.429414+0000) 2024-06-15T01:38:39.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:39 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:39.149+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:39.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:39 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:39.149+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:40.269 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:39 smithi142 podman[117667]: 2024-06-15 01:38:39.988097915 +0000 UTC m=+5.621370342 container died f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-15T01:38:40.270 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:40 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:40.197+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:40.270 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:40 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:40.197+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:40.555 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:40 smithi142 podman[117667]: 2024-06-15 01:38:40.402014236 +0000 UTC m=+6.035286662 container cleanup f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:40.556 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:40 smithi142 bash[117667]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4 2024-06-15T01:38:41.055 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:40 smithi142 podman[117694]: 2024-06-15 01:38:40.670180187 +0000 UTC m=+0.680959924 container remove f41525c74c8909e9970b5bb6a974f57bd57bf2f1069e40c6333b1f459823d8b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.35.3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-06-15T01:38:41.340 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:41 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:41.218+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:41.340 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:41 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:41.218+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:41.340 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:40.987647882 +0000 UTC m=+0.017343481 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:41.340 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:41.095931087 +0000 UTC m=+0.125626672 container create 45c4ad8d4c782fbb33fee8665f817364b884eb5181792d9318c40f4d8e7d7d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.35.3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-15T01:38:41.340 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:41.356569781 +0000 UTC m=+0.386265371 container init 45c4ad8d4c782fbb33fee8665f817364b884eb5181792d9318c40f4d8e7d7d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux ) 2024-06-15T01:38:41.598 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:41.359878222 +0000 UTC m=+0.389573809 container start 45c4ad8d4c782fbb33fee8665f817364b884eb5181792d9318c40f4d8e7d7d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True) 2024-06-15T01:38:41.599 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:41.456636456 +0000 UTC m=+0.486332043 container attach 45c4ad8d4c782fbb33fee8665f817364b884eb5181792d9318c40f4d8e7d7d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-15T01:38:41.892 INFO:journalctl@ceph.osd.4.smithi142.stdout:Jun 15 01:38:41 smithi142 podman[117784]: 2024-06-15 01:38:41.614895838 +0000 UTC m=+0.644591427 container died 45c4ad8d4c782fbb33fee8665f817364b884eb5181792d9318c40f4d8e7d7d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-4-deactivate, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-15T01:38:42.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.4.service' 2024-06-15T01:38:42.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:42.244 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-15T01:38:42.244 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-15T01:38:42.244 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.5 2024-06-15T01:38:42.555 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:42.182+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:42.555 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:42.182+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:42.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 systemd[1]: Stopping Ceph osd.5 for 225ca7a2-2ab2-11ef-bc9d-c7b262605968... 2024-06-15T01:38:43.055 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:42.603+0000 7f57e39cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-15T01:38:43.056 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:42.603+0000 7f57e39cf640 -1 osd.5 95 *** Got signal Terminated *** 2024-06-15T01:38:43.056 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:42 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:42.603+0000 7f57e39cf640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-15T01:38:43.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:43 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:43.190+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:43.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:43 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:43.190+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:44.555 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:44 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:44.234+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:44.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:44 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:44.234+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:45.555 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:45.227+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:45.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:45.227+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:45.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:45 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:45.227+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6822 osd.2 since back 2024-06-15T01:38:22.089484+0000 front 2024-06-15T01:38:22.089525+0000 (oldest deadline 2024-06-15T01:38:44.389378+0000) 2024-06-15T01:38:46.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:46 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:46.211+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:46.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:46 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:46.211+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:46.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:46 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:46.211+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6822 osd.2 since back 2024-06-15T01:38:22.089484+0000 front 2024-06-15T01:38:22.089525+0000 (oldest deadline 2024-06-15T01:38:44.389378+0000) 2024-06-15T01:38:47.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:47 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:47.200+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-06-15T01:38:05.787195+0000 front 2024-06-15T01:38:05.787181+0000 (oldest deadline 2024-06-15T01:38:30.487077+0000) 2024-06-15T01:38:47.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:47 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:47.200+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6814 osd.1 since back 2024-06-15T01:38:12.188141+0000 front 2024-06-15T01:38:12.188096+0000 (oldest deadline 2024-06-15T01:38:38.088058+0000) 2024-06-15T01:38:47.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:47 smithi142 ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5[100314]: 2024-06-15T01:38:47.200+0000 7f57dc7cb640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.137:6822 osd.2 since back 2024-06-15T01:38:22.089484+0000 front 2024-06-15T01:38:22.089525+0000 (oldest deadline 2024-06-15T01:38:44.389378+0000) 2024-06-15T01:38:48.229 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:47 smithi142 podman[117887]: 2024-06-15 01:38:47.974246854 +0000 UTC m=+5.485031488 container died ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:48.229 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:48 smithi142 podman[117887]: 2024-06-15 01:38:48.245438718 +0000 UTC m=+5.756223348 container cleanup ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:48.496 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:48 smithi142 bash[117887]: ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5 2024-06-15T01:38:48.806 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:48 smithi142 podman[117906]: 2024-06-15 01:38:48.513211865 +0000 UTC m=+0.533764464 container remove ab1f879dfaf483538564a43a147e3b26a75c1464c58ae36a2f87f20154f15466 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5, io.buildah.version=1.35.3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095) 2024-06-15T01:38:49.150 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:48 smithi142 podman[117996]: 2024-06-15 01:38:48.772569882 +0000 UTC m=+0.017170365 image pull e7282a3bffba6450e7bbc61351625ccbdc874ba3abbccb32d6c2163005000667 quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0 2024-06-15T01:38:49.150 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:48 smithi142 podman[117996]: 2024-06-15 01:38:48.880732272 +0000 UTC m=+0.125332786 container create f5f54af46e1797f4d9ab05c1b77d5169ef67d40581938f05c85c67ce9c1e0578 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-15T01:38:49.150 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:49 smithi142 podman[117996]: 2024-06-15 01:38:49.099440569 +0000 UTC m=+0.344041048 container init f5f54af46e1797f4d9ab05c1b77d5169ef67d40581938f05c85c67ce9c1e0578 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.35.3) 2024-06-15T01:38:49.150 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:49 smithi142 podman[117996]: 2024-06-15 01:38:49.102711374 +0000 UTC m=+0.347311847 container start f5f54af46e1797f4d9ab05c1b77d5169ef67d40581938f05c85c67ce9c1e0578 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-15T01:38:49.150 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:49 smithi142 podman[117996]: 2024-06-15 01:38:49.166199603 +0000 UTC m=+0.410800076 container attach f5f54af46e1797f4d9ab05c1b77d5169ef67d40581938f05c85c67ce9c1e0578 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-15T01:38:49.556 INFO:journalctl@ceph.osd.5.smithi142.stdout:Jun 15 01:38:49 smithi142 podman[117996]: 2024-06-15 01:38:49.364668883 +0000 UTC m=+0.609269355 container died f5f54af46e1797f4d9ab05c1b77d5169ef67d40581938f05c85c67ce9c1e0578 (image=quay.ceph.io/ceph-ci/ceph@sha256:86fcceac5e55a93f3e03f800f2d89bf9c8b6526203c40d1d5c5822005a1b39f0, name=ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-06-08-0902-reef-6f4b095, ceph=True, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-15T01:38:49.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-225ca7a2-2ab2-11ef-bc9d-c7b262605968@osd.5.service' 2024-06-15T01:38:50.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-15T01:38:50.024 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-15T01:38:50.024 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-15T01:38:50.181 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:38:54.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[77226]: fuse finished with error 0 and tester_r 0 2024-06-15T01:39:18.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-15T01:39:18.455 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:39:33.539 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-15T01:39:33.574 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-15T01:39:33.602 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-15T01:39:33.604 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi137/crash 2024-06-15T01:39:33.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash -- . 2024-06-15T01:39:33.641 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-15T01:39:33.642 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-06-15T01:39:33.644 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi142/crash 2024-06-15T01:39:33.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash -- . 2024-06-15T01:39:33.668 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-15T01:39:33.669 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-06-15T01:39:33.670 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-15T01:39:33.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-06-15T01:39:33.915 INFO:teuthology.orchestra.run.smithi137.stderr:grep: /var/log/ceph/225ca7a2-2ab2-11ef-bc9d-c7b262605968/ceph.log: No such file or directory 2024-06-15T01:39:33.918 INFO:tasks.cephadm:Compressing logs... 2024-06-15T01:39:33.918 DEBUG:teuthology.orchestra.run.smithi137:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-15T01:39:33.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-15T01:39:33.942 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-15T01:39:33.944 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-15T01:39:51.567 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-06-15T01:42:02.714 INFO:tasks.cephadm:Archiving logs... 2024-06-15T01:42:02.715 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi137/log 2024-06-15T01:42:02.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-15T01:42:20.296 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi142/log 2024-06-15T01:42:20.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-15T01:42:24.873 INFO:tasks.cephadm:Removing cluster... 2024-06-15T01:42:24.873 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 --force 2024-06-15T01:42:25.029 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:42:25.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 225ca7a2-2ab2-11ef-bc9d-c7b262605968 --force 2024-06-15T01:42:25.642 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 225ca7a2-2ab2-11ef-bc9d-c7b262605968 2024-06-15T01:42:25.962 INFO:tasks.cephadm:Removing cephadm ... 2024-06-15T01:42:25.962 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-15T01:42:25.980 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-15T01:42:25.996 INFO:tasks.cephadm:Teardown complete 2024-06-15T01:42:25.996 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-15T01:42:26.007 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-06-15T01:42:26.008 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-15T01:42:26.008 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-15T01:42:26.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-15T01:42:26.115 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-15T01:42:26.125 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-15T01:42:26.125 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-15T01:42:26.128 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-15T01:42:26.141 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-06-15T01:42:26.142 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-06-15T01:42:26.145 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-15T01:42:26.145 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-06-15T01:42:26.145 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +535us[ +538us] +/- 53ms 2024-06-15T01:42:26.146 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 15 -231us[ -229us] +/- 56ms 2024-06-15T01:42:26.146 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -545us[ -543us] +/- 67ms 2024-06-15T01:42:26.146 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T01:42:26.147 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-15T01:42:26.147 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-06-15T01:42:26.147 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +572us[ +518us] +/- 53ms 2024-06-15T01:42:26.148 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -217us[ -271us] +/- 56ms 2024-06-15T01:42:26.148 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -492us[ -492us] +/- 67ms 2024-06-15T01:42:26.148 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-15T01:42:26.148 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-15T01:42:26.176 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-15T01:42:26.176 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-15T01:42:26.435 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-15T01:42:26.521 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-15T01:42:26.594 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-15T01:42:26.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:42:26.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-15T01:42:26.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:42:26.651 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-15T01:42:26.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-15T01:42:26.754 INFO:teuthology.task.internal:Duration was 3126.203966 seconds 2024-06-15T01:42:26.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-15T01:42:26.846 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-15T01:42:26.847 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-15T01:42:26.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-15T01:42:26.887 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-15T01:42:26.888 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-15T01:42:27.352 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-15T01:42:27.352 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-06-15T01:42:27.352 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-15T01:42:27.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-06-15T01:42:27.374 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-15T01:42:27.419 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-15T01:42:27.419 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-15T01:42:27.422 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-15T01:42:27.486 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-15T01:42:27.486 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-15T01:42:27.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-15T01:42:28.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-15T01:42:28.397 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-15T01:42:28.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-15T01:42:28.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-15T01:42:28.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-15T01:42:28.462 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-15T01:42:28.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-15T01:42:28.485 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-06-15T01:42:28.515 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-06-15T01:42:28.545 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-15T01:42:28.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:42:28.563 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-15T01:42:28.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:42:28.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-15T01:42:28.612 INFO:teuthology.task.internal:Transferring archived files... 2024-06-15T01:42:28.613 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi137 2024-06-15T01:42:28.613 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-15T01:42:28.664 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-06-14_18:29:32-orch-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7756143/remote/smithi142 2024-06-15T01:42:28.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-15T01:42:28.703 INFO:teuthology.task.internal:Removing archive directory... 2024-06-15T01:42:28.703 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-15T01:42:28.706 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-15T01:42:28.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-15T01:42:28.765 INFO:teuthology.task.internal:Not uploading archives. 2024-06-15T01:42:28.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-15T01:42:28.808 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-15T01:42:28.808 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-15T01:42:28.811 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-15T01:42:28.823 INFO:teuthology.orchestra.run.smithi137.stdout: 265818 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 15 01:42 /home/ubuntu/cephtest 2024-06-15T01:42:28.823 INFO:teuthology.orchestra.run.smithi137.stdout: 556309 4 d--------- 2 ubuntu ubuntu 4096 Jun 15 01:04 /home/ubuntu/cephtest/mnt.0 2024-06-15T01:42:28.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-15T01:42:28.824 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-06-15T01:42:28.825 INFO:teuthology.orchestra.run.smithi137.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-06-15T01:42:28.825 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_cbodley_ceph_e3daf20513a97cf7004cd9e7bfdba8f5a015acbc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi137 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-06-15T01:42:28.826 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-15T01:42:28.865 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-15T01:42:28.924 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-06-15T01:42:28.927 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3126.203966140747 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-06-15T01:42:28.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-15T01:42:29.013 INFO:teuthology.run:FAIL